Latest / kill switch / will blind gamers ever play Zelda like I do? (part 2)
Transcript
- 0:10Speaker 1: Here we go.
- 0:13Speaker 2: This is a racing game called for it's a motorsport.
- 0:18Speaker 2: Aside from the sound effects from the driving, you might
- 0:20Speaker 2: also notice those beaps. Brandon Cole is completely blind, and
- 0:28Speaker 2: this is how he plays the game.
- 0:31Speaker 1: Going to the last turn.
- 0:34Speaker 2: Decide that the beeps are on, and also the pitch
- 0:36Speaker 2: and the intensity gives an information on everything from where
- 0:39Speaker 2: he is on the track, whether or not he's too
- 0:41Speaker 2: close to the wall, to how sharp the next turn is.
- 0:45Speaker 1: I'm so glad I was able to do that turn.
- 0:46Speaker 1: That's actually a very difficult turn.
- 0:48Speaker 3: Oh that's why thirty year actually every count fifteen point
- 0:52Speaker 3: two six five seconds.
- 0:54Speaker 2: Brandon Cole passed away last year, but he wasn't just
- 0:57Speaker 2: a gamer. He was also a consultant on this game.
- 1:00Speaker 2: And this is him demonstrating the game on his YouTube channel.
- 1:04Speaker 1: But just rewind that rewinding. I want to show you this.
- 1:10Speaker 4: Rewind done.
- 1:11Speaker 3: I want to show you that because the blind drivings,
- 1:12Speaker 3: sounds play while you're rewinding.
- 1:14Speaker 2: If we're talking about racing games, there is a before
- 1:17Speaker 2: Fortsa and an after Fortza. Before this game came out,
- 1:21Speaker 2: it wasn't possible really for a blind person to play
- 1:23Speaker 2: racing games.
- 1:25Speaker 4: But now they can.
- 1:26Speaker 3: Hey, other blind person, I offer you control. I offer
- 1:31Speaker 3: you the opportunity to get into a vehicle of your choice,
- 1:36Speaker 3: pick the track of your choice, and once you're on
- 1:39Speaker 3: that track, the ability to drive it yourself, to take
- 1:42Speaker 3: those turns at high speeds yourself.
- 1:45Speaker 1: This is your road.
- 1:46Speaker 4: Now, Welcome to Forza.
- 1:49Speaker 2: Forts of the game was put out by Microsoft, but
- 1:52Speaker 2: the technology that made those blind accessibility features possible actually
- 1:56Speaker 2: came from a small research lab in New York led
- 1:58Speaker 2: by a professor named brad Ian Smith.
- 2:01Speaker 4: When we first.
- 2:02Speaker 2: Spoke, it was early mid twenty twenty, I think, and
- 2:07Speaker 2: you were telling me about this experimental thing that your
- 2:12Speaker 2: lab had come up with where blind people could play
- 2:16Speaker 2: a racing game, and you were very confident you were
- 2:18Speaker 2: going to get in the game. I don't know if
- 2:21Speaker 2: I was as confident.
- 2:23Speaker 5: Because it sounded great, but I maybe I was just
- 2:26Speaker 5: pessimistic about the good front. But yeah, but you know,
- 2:29Speaker 5: I mean I think it was more than like, Okay,
- 2:31Speaker 5: this sounds amazing, but the step is for a game
- 2:35Speaker 5: company needs to adopt this thing.
- 2:38Speaker 4: But it happened.
- 2:40Speaker 2: Brian is an assistant professor of computer science at Columbia
- 2:42Speaker 2: University and he's the head of a lab called Seal
- 2:45Speaker 2: that helped make the first blind accessible Triple A racing game.
- 2:50Speaker 6: What we're most known for is our work in assiste
- 2:52Speaker 6: of technology, in particular technology for people who are blind
- 2:56Speaker 6: or low vision.
- 2:56Speaker 2: Their work goes beyond Fortza, and it goes beyond video games.
- 3:01Speaker 6: Video games are but one example of a visual experience.
- 3:05Speaker 6: By figuring out how we can enable that fundamental ability
- 3:11Speaker 6: to perceive an image or a screen, we can actually
- 3:15Speaker 6: unlock not only access to video games, but all sorts
- 3:19Speaker 6: of visual experiences in daily life.
- 3:27Speaker 2: I'm afraid from Kaleidoscope and iHeart podcasts.
- 3:34Speaker 4: This is kill Switch. I'm Dexor Thomas.
- 3:40Speaker 1: I'm sall right, good guy.
- 4:15Speaker 2: How did you become interested in how blind people play
- 4:19Speaker 2: video games?
- 4:20Speaker 1: It all started with a YouTube video that I actually
- 4:23Speaker 1: encountered while I was a PhD student. It was a
- 4:26Speaker 1: video of a blind person by the name of Terry Garrett.
- 4:31Speaker 1: He was playing a game.
- 4:33Speaker 6: Called The Legend of Zelda Ocher Enough Time, which is
- 4:35Speaker 6: a classic Zelda adventure game that I remember playing as
- 4:38Speaker 6: a kid as well. Yeah, the interesting thing about Terry
- 4:41Speaker 6: Garrett was that he wasn't blind his entire life, like
- 4:44Speaker 6: he actually grew up as a little kid playing that
- 4:47Speaker 6: game and then later lost his vision, and as soon
- 4:50Speaker 6: as he lost his vision, he almost got shut out
- 4:53Speaker 6: of video games completely.
- 4:55Speaker 1: For years, the only things he could play was some
- 4:57Speaker 1: games that he could find online would were called audio games,
- 5:01Speaker 1: which were games specifically designed for people who are blind
- 5:04Speaker 1: that are audio only.
- 5:06Speaker 6: You know, in a lot of ways, they're very distilled
- 5:08Speaker 6: experiences compared to what sited players would be able to play.
- 5:12Speaker 1: So one day Terry decided to pick up Zelda again
- 5:15Speaker 1: and say, though, I want to play this game.
- 5:17Speaker 6: And I'm going to do it no matter what. And
- 5:20Speaker 6: it took a whole bunch of trial and error. He'd
- 5:22Speaker 6: do a whole bunch of like hacks that he figured out. So,
- 5:25Speaker 6: for example, he would walk around a room and whack
- 5:27Speaker 6: the walls with his sword again and again and again
- 5:30Speaker 6: until he heard a different kind of clinking noise which
- 5:32Speaker 6: told him like, oh, that's the door instead of the
- 5:35Speaker 6: rest of the wall, and so that's how I might
- 5:37Speaker 6: get into this room.
- 5:38Speaker 1: There he'd memorize a whole bunch of details, and.
- 5:42Speaker 6: That really landed for me because like it showed me,
- 5:45Speaker 6: here's something that i'd take for granted. These video game experiences, right,
- 5:49Speaker 6: and there are a lot of people who go through
- 5:51Speaker 6: so much effort just to have some semblance of that experience.
- 5:56Speaker 1: Like I was you know, I was floored.
- 5:57Speaker 6: I was amazed, But at the same time I felt pain.
- 6:02Speaker 6: I felt like, wow, like this is not a good
- 6:06Speaker 6: state of affairs right now. We've got to do better
- 6:09Speaker 6: than this.
- 6:09Speaker 1: This isn't right.
- 6:10Speaker 6: And that motivated me as like, Okay, here's a place
- 6:13Speaker 6: where just rethinking.
- 6:15Speaker 1: How we're designing these games, these computer systems.
- 6:18Speaker 6: Really can really enable people to have new types of
- 6:22Speaker 6: meaningful experiences.
- 6:23Speaker 2: Yeah, you know what, We've gotten to the point where
- 6:28Speaker 2: we've legitimately got cars that drive themselves. And when there's
- 6:34Speaker 2: a kid who wants to play a videome, or an
- 6:35Speaker 2: adult for that matter, who wants to play a video
- 6:37Speaker 2: game who can't see the screen, we sort of shrug
- 6:40Speaker 2: our shoulders, I think, collectively, and say, you're asking a
- 6:43Speaker 2: little bit too much, buddy.
- 6:45Speaker 1: Yeah, that's right.
- 6:46Speaker 2: And you said no, you said no, I don't think
- 6:48Speaker 2: you are that's right. There are more games out now
- 6:52Speaker 2: that have accessibility features for blind and low vision players, like,
- 6:55Speaker 2: for example, there's SPORTSA and there's also the last of
- 6:58Speaker 2: Us too, which we mentioned in the last episode. But
- 7:00Speaker 2: that's a handful of games out of well a lot
- 7:03Speaker 2: of video games, and it's probably not fair to compare
- 7:06Speaker 2: the big blockbuster games to the smaller independent games, but
- 7:09Speaker 2: they do have something in common. Most games, especially the
- 7:13Speaker 2: blockbuster ones, the Triple A games what we call them,
- 7:16Speaker 2: are a lot further behind than they could be in
- 7:18Speaker 2: terms of blind accessibility and accessibility in general.
- 7:22Speaker 6: Most mainstream games, the kinds of games you'd buy on
- 7:25Speaker 6: Steam or Xbox, PlayStation, Nintendo, are barely accessible to people
- 7:31Speaker 6: with disabilities. That includes people who are blind to low vision,
- 7:35Speaker 6: people who are deaf or hard of hearing, people with
- 7:38Speaker 6: motor impairments, people who have a hard time like mashing
- 7:41Speaker 6: buttons or pressing too many buttons too quickly.
- 7:44Speaker 1: In sequence, that's the current state of things.
- 7:47Speaker 2: Game companies are starting to take accessibility more seriously. Some
- 7:51Speaker 2: of this stuff seems kind of obvious, like the ability
- 7:53Speaker 2: to make text larger, or the ability to switch to
- 7:56Speaker 2: a color blind mode, or an option to read out
- 7:59Speaker 2: the text on the screen. But in order for a
- 8:01Speaker 2: blind or a low vision player to be able to
- 8:04Speaker 2: truly experience a video game, and anything close to the
- 8:07Speaker 2: way that say, I experience it, that game needs to
- 8:10Speaker 2: be built with that player's experience in mind from the start.
- 8:13Speaker 2: But unfortunately, the industry standard right now is to essentially
- 8:17Speaker 2: dumb down the game for that player.
- 8:20Speaker 6: Oftentimes, the basic approach to accessibility is just to make
- 8:26Speaker 6: the game a little bit easier that actually take away
- 8:29Speaker 6: agency and choice from the player. So, for example, they're
- 8:33Speaker 6: video games that have accessibility features that allow players to
- 8:37Speaker 6: skip different sections or find items faster, but they do
- 8:41Speaker 6: so in a way that almost plays the game for
- 8:44Speaker 6: the player, like, oh, you can't really see the path ahead,
- 8:48Speaker 6: you can't really see the image.
- 8:49Speaker 1: Well, we'll press this button and I'll just warp you there.
- 8:53Speaker 6: And that form of accessibility does not offer players the
- 8:58Speaker 6: same type of experience, and that's the more challenging type
- 9:01Speaker 6: of accessibility problem.
- 9:02Speaker 2: To solve this type of accessibility of making the game
- 9:06Speaker 2: easier to play and taking away some of the agency
- 9:09Speaker 2: of the player. You can even see that in the
- 9:11Speaker 2: Last of Us, and this is the game that's arguably
- 9:13Speaker 2: gotten the most positive attention for their accessibility features.
- 9:17Speaker 7: What The Last of Us does is you can hold
- 9:19Speaker 7: a button to automatically move your character to the next
- 9:23Speaker 7: waypoint to the next object of interest.
- 9:26Speaker 2: Vishnudnair is a research engineer and he was a PhD
- 9:29Speaker 2: student in Brian's lab, which started as just the two
- 9:32Speaker 2: of them working on video game accessibility technology.
- 9:35Speaker 7: A lot of the players that we talked to called
- 9:37Speaker 7: it the golden path. That's all it does. We refer
- 9:40Speaker 7: to it as guidance. These are guidance based systems and
- 9:45Speaker 7: that's all they do. It's like you're taking their hand
- 9:47Speaker 7: and dragging them wherever, and that's not great for independence.
- 9:53Speaker 7: And it's that independence and autonomy, that ability to make
- 9:57Speaker 7: decisions on what I want to do, Which do I
- 9:59Speaker 7: want to turn that corner and maybe find a treasure
- 10:02Speaker 7: chest or if I turn that corner, maybe I'll encounter
- 10:06Speaker 7: a very powerful enemy and die. You know, making those
- 10:09Speaker 7: decisions and facing those consequences is like the most important
- 10:12Speaker 7: part of having fun, and being guided doesn't allow you
- 10:16Speaker 7: to have those experiences.
- 10:20Speaker 2: This Golden Path will basically walk you from point to
- 10:23Speaker 2: point in the game, basically just getting into the next
- 10:25Speaker 2: spot to keep the story moving. It is well intentioned,
- 10:28Speaker 2: but it doesn't give blind players the same experience as
- 10:31Speaker 2: a sided player. This is gonna sound weird, but it
- 10:35Speaker 2: turns out that letting a blind player drive a race
- 10:37Speaker 2: car is relatively pretty easy. What's hard is letting that
- 10:41Speaker 2: blind player get out the race car and just walk around.
- 10:45Speaker 4: But Brian and Vishnu think they got to figure it out.
- 10:48Speaker 2: That's after the break. So let's go back to Terry Garrett.
- 10:58Speaker 2: That's the blind gamer who be Zelda sixty four. The
- 11:01Speaker 2: way he navigated the game was by going up to
- 11:04Speaker 2: a wall and hitting it with his sword, and based
- 11:06Speaker 2: on the sounds that he heard as he went down
- 11:08Speaker 2: the wall hitting it, he could tell if that was
- 11:11Speaker 2: just part of a normal wall, or maybe a door
- 11:13Speaker 2: or a secret passageway. This was a really clever workaround
- 11:17Speaker 2: for a game that was not meant to be blind accessible. Nowadays,
- 11:21Speaker 2: we have games that are developed with blind accessibility in mind.
- 11:24Speaker 2: They incorporate what Vishnu might call that golden path. So
- 11:28Speaker 2: what Terry was having to do was definitely tedious, and
- 11:31Speaker 2: a golden path approach is way more efficient. That sounds great,
- 11:35Speaker 2: but let's step back for a second. Does more efficiency
- 11:39Speaker 2: actually equal fun?
- 11:43Speaker 6: You can imagine a game like Zelda right where we
- 11:46Speaker 6: have the main character Link and around Link might be
- 11:50Speaker 6: a ladder to a second floor, and then a door
- 11:53Speaker 6: into another room, and then a treasure chest or some
- 11:56Speaker 6: bushes that you can cut for rupees the game's currency,
- 11:59Speaker 6: et cetera.
- 12:00Speaker 1: Objects around link at any given time, and.
- 12:03Speaker 6: As a cited person, we can just move the camera around,
- 12:06Speaker 6: look around, see what those things are, figure out which
- 12:08Speaker 6: way we want to go, what we want to do next.
- 12:11Speaker 1: But as a blind person, that experience is very different.
- 12:15Speaker 6: Even with some of the cutting edge accessibility features that
- 12:19Speaker 6: are out there in games right now, the way that
- 12:21Speaker 6: that experience would play out is the blind player might
- 12:24Speaker 6: like press a button to just open a menu that
- 12:27Speaker 6: reads out the objects around them, so it might say door, ladder,
- 12:32Speaker 6: treasure chest, bush, whatever, like that's it. It's just a
- 12:36Speaker 6: linear menu, and oftentimes that menu kind of waters down
- 12:41Speaker 6: the experience of looking around and exploring. So, for example,
- 12:45Speaker 6: if the whole point is to find like a small
- 12:47Speaker 6: key hidden somewhere in some nook or cranny that might
- 12:50Speaker 6: unlock a door on another side of the dungeon, if
- 12:53Speaker 6: you pulled up the menu and the menu just said
- 12:55Speaker 6: small key, it kind of spoil the.
- 12:57Speaker 2: Surprise, right, yeah, yeah, you not playing a game. It's
- 13:01Speaker 2: like you're on a guided tour.
- 13:03Speaker 6: Exactly exactly, and so that experience of like exploration and
- 13:09Speaker 6: discovery is missing from those current accessibility tools.
- 13:14Speaker 2: But how do you define that experience? So this is
- 13:17Speaker 2: gonna sound totally unhinged, but hear me out. There's this
- 13:20Speaker 2: YouTuber who has this series where he goes to what
- 13:22Speaker 2: he calls unremarkable and odd places in Zelda, like places
- 13:26Speaker 2: where you just go and there's a couple of rocks
- 13:29Speaker 2: near a river or something. Has nothing to do with
- 13:31Speaker 2: the progressing of the game. There's no mission critical item there.
- 13:34Speaker 2: It's just a river and some rocks. And one of
- 13:38Speaker 2: his videos has three hundred thousand views, and people in
- 13:40Speaker 2: the comments are saying, yeah, I remember that spot.
- 13:42Speaker 4: I love that spot. I like hanging out at that spot.
- 13:45Speaker 2: So how do we give blind people the closest thing
- 13:49Speaker 2: to getting a game and exploring, not accomplishing things, not
- 13:53Speaker 2: checking stuff off of a checklist, but playing the game
- 13:57Speaker 2: on your own terms. Brian in his last are taking
- 14:00Speaker 2: this question very seriously, and they published two papers on
- 14:03Speaker 2: applications that facilitate precisely this ability. One application is called
- 14:09Speaker 2: navstick and the other it's called Surveyor.
- 14:13Speaker 6: So both Navstick and Surveyor are about making three D
- 14:17Speaker 6: adventure games more meaningfully accessible to blind players. So what
- 14:23Speaker 6: vish news work was really about was what's important to
- 14:27Speaker 6: the experience of exploration and discovery? When cited people are
- 14:31Speaker 6: playing Zelda? What do they do at one moment in
- 14:34Speaker 6: time the next moment of time, How are they having fun?
- 14:37Speaker 6: What choices are they making at any moment in time,
- 14:40Speaker 6: and like, what information are they exploring to figure that out?
- 14:44Speaker 6: And now, how can we make that process of exploration
- 14:48Speaker 6: and discovery more meaningfully accessible?
- 14:51Speaker 2: The first step was figuring out how a blind player
- 14:53Speaker 2: could explore their immediate surroundings so that they could understand
- 14:57Speaker 2: what choices they had without just doing something for them
- 15:00Speaker 2: or telling them where the thing that they were looking
- 15:02Speaker 2: for was.
- 15:03Speaker 7: We were like, okay, a person kind of rotates their
- 15:07Speaker 7: head all around them to look around themselves and get
- 15:10Speaker 7: a map of what's imediately around them. How do we
- 15:13Speaker 7: give that ability to someone with blindness? And so that
- 15:16Speaker 7: was where navstick came along, where it was really simple.
- 15:19Speaker 7: We just repurposed the right thumbstick on the standard game controller,
- 15:24Speaker 7: and all you have to do is just point the
- 15:26Speaker 7: right thumbstick in some direction and whatever lies in that direction.
- 15:30Speaker 7: The game's going to tell you that was it.
- 15:33Speaker 2: So I tried navstick and it's really intuitive. Basically, you
- 15:37Speaker 2: move your right thumbstick around and whatever you pointed at,
- 15:40Speaker 2: a voice plays and tells you.
- 15:42Speaker 4: What it is. Clean, Red Dirk Chocolate, Chocolate, Chippickies Better.
- 15:47Speaker 2: This is a clip of Vishnu demonstrating navstick in a
- 15:50Speaker 2: virtual grocery store setting. You can hear the sound of
- 15:53Speaker 2: the voice moving around depending on where the item is
- 15:56Speaker 2: in relationship to you.
- 15:57Speaker 7: Christians Coffee Grounds, Dirk Chuck. We actually created an entire
- 16:01Speaker 7: video game to test this, and everyone loved it. What
- 16:06Speaker 7: was amazing was that, you know, we had we put
- 16:08Speaker 7: in all these levels. All these levels had like different
- 16:10Speaker 7: trees and enemies that they had to shoot using a
- 16:12Speaker 7: laser gun. Every participant completed every level of that game,
- 16:18Speaker 7: no matter how complex it got. And what we found
- 16:21Speaker 7: was that everyone crapped at different strategies to get through
- 16:24Speaker 7: those levels. Maybe they went this way to the left
- 16:27Speaker 7: and defeated a few enemies to get to the finish line,
- 16:30Speaker 7: or maybe they just barrel straight through. They were able
- 16:32Speaker 7: to make those decisions and everyone said that, oh, this
- 16:35Speaker 7: is like just so much more fun than using like
- 16:38Speaker 7: a menu for instance, and just being told where to go,
- 16:41Speaker 7: I was able to like actually feel like I was
- 16:44Speaker 7: playing the game.
- 16:46Speaker 2: This is a pretty simple application, but it's very powerful.
- 16:49Speaker 2: So I've tried doing the thing where you close your
- 16:52Speaker 2: eyes and make a mental map of your surroundings and listen, man,
- 16:55Speaker 2: I suck at it. This is something that blind people
- 16:58Speaker 2: are usually very good at because because they have to be,
- 17:01Speaker 2: but it can even for them get kind of taxing.
- 17:05Speaker 2: Navstick means that you don't have to memorize a brand
- 17:07Speaker 2: new area when you first go to it. You can
- 17:09Speaker 2: check where you're at right then. But for a full
- 17:13Speaker 2: three D game, especially a big game, napstick is just
- 17:16Speaker 2: one part of that equation.
- 17:18Speaker 7: The question beyond navstick is how do we allow a
- 17:23Speaker 7: blind user to experience a larger area. You have this
- 17:28Speaker 7: entire dungeon that you have to explore. How do you
- 17:32Speaker 7: handle that? How do you break away from that guidance
- 17:35Speaker 7: paradigm where it's like you're leading a person by the
- 17:38Speaker 7: hand and leading them to the treasure chest. In game
- 17:41Speaker 7: design theory, discovery is one of the main elements of
- 17:44Speaker 7: video games that are important for a good experience, and
- 17:48Speaker 7: so how do we facilitate that, and so we create
- 17:51Speaker 7: this entire system Surveyor where they would use navstick, but
- 17:56Speaker 7: in addition to navstick, we help keep a law of
- 18:00Speaker 7: where they've been. Basically, if you were to take an
- 18:04Speaker 7: overhead view of a dungeon and you color in the
- 18:09Speaker 7: places where the person has been, the point is you
- 18:13Speaker 7: want to bring a user to the edge of the
- 18:16Speaker 7: area those borders between where they have explored and where
- 18:20Speaker 7: they haven't explored. That's what we did with Surveyor, where
- 18:23Speaker 7: effectively it would say, okay, about fifty feet away, there
- 18:28Speaker 7: is an unexplored area, and if you select this, you'll
- 18:32Speaker 7: be guided there. And once you reach that unknown the
- 18:35Speaker 7: guidance cuts off and you're left using knapstick to look
- 18:40Speaker 7: around that area.
- 18:42Speaker 2: Oh so, it's kind of like fast traveling a game. Exactly,
- 18:45Speaker 2: I've already explored this part. Don't make me run across
- 18:49Speaker 2: a field for twenty minutes. I've already been here, I've
- 18:52Speaker 2: completed this area of the game. Let me get back
- 18:53Speaker 2: to the town. Cool, and it takes you to the
- 18:56Speaker 2: edge of where you've been, drops you off there and says, okay, figure.
- 19:00Speaker 7: It out exactly exactly, and that's what promotes that sense
- 19:03Speaker 7: of autonomy. That's what promotes that sense of discovery. What
- 19:07Speaker 7: we're trying to do is, Okay, we don't need to
- 19:09Speaker 7: confine the player anymore, right if we can just open
- 19:14Speaker 7: it up, just take something like Tiers of the Kingdom
- 19:18Speaker 7: and put a player with surveyor in it. Can they
- 19:21Speaker 7: just explore that area, these cities, these little dungeons and
- 19:25Speaker 7: all that.
- 19:26Speaker 2: Zelda Breath of the Wild and its sequel, Tiers of
- 19:29Speaker 2: the Kingdom are two of the biggest open world games
- 19:32Speaker 2: that are out right now. They're really popular, and it's
- 19:35Speaker 2: not just because they're good, but because they're really approachable.
- 19:38Speaker 2: I have friends who don't play a whole lot of
- 19:41Speaker 2: video games, but you know, they'll turn on the switch,
- 19:43Speaker 2: They'll jump in Zelda, They'll go run around in a forest,
- 19:45Speaker 2: not even fighting enemies, just having fun. Zelda Tears of
- 19:49Speaker 2: the Kingdom. You think you could put this onto a
- 19:53Speaker 2: Zelda Breath of the Wild type game and people be
- 19:55Speaker 2: able to play it.
- 19:57Speaker 7: Perhaps, I mean, they are obviously going to be some
- 19:59Speaker 7: design decisions that to be made, But in my defense presentation,
- 20:02Speaker 7: I literally use Tears of the Kingdom as an example.
- 20:06Speaker 2: Again, blind people can play some video games now and
- 20:09Speaker 2: feel that sense of satisfaction of accomplishing a task, like
- 20:13Speaker 2: finally beating that last boss. But that's not all a
- 20:15Speaker 2: video game is. I mean the freedom to just run around, explore,
- 20:18Speaker 2: hang out, that isn't there for a blind player. But
- 20:22Speaker 2: now with these tools that Vishio and Brian have developed,
- 20:25Speaker 2: a blind players, say like Ross Minor from last episode,
- 20:28Speaker 2: could probably do precisely that, just go wander around without
- 20:31Speaker 2: assistance or special modifications. That potentially opens things up a lot.
- 20:38Speaker 7: Yeah, that opens a lot of social opportunities as well.
- 20:43Speaker 7: If we can open up the same experiences to blind
- 20:46Speaker 7: players as side of players have, then we can foster
- 20:50Speaker 7: better connections at the end of the day.
- 20:53Speaker 2: I mean that totally makes sense. I mean, you know,
- 20:55Speaker 2: if I found out somebody played Earthbound, we're immediately friends immediately,
- 21:00Speaker 2: you know what I mean.
- 21:00Speaker 4: Or same thing with really.
- 21:03Speaker 2: Anything, music, movies, books, whatever, You've experienced, the same thing,
- 21:08Speaker 2: you read, the same stuff, you listen to the same music.
- 21:11Speaker 2: Obviously video games figure into that pretty heavily too, right,
- 21:16Speaker 2: So you may think this all sounds really exciting, but
- 21:18Speaker 2: you're wondering when this stuff's gonna come out, or you
- 21:21Speaker 2: might think that this is an awful lot of effort,
- 21:23Speaker 2: and you're wondering why these brilliant researchers are wasting their
- 21:27Speaker 2: time on video games. Well, I got two answers for you.
- 21:30Speaker 2: That's after the break. The tools that Brian and Vishnu
- 21:45Speaker 2: were talking about are pretty incredible, but they're also really
- 21:49Speaker 2: intuitive and simple. It kind of makes you wonder why
- 21:52Speaker 2: stuff like this is so rare. Well, A big part
- 21:55Speaker 2: of the problem is that game companies are really siloed.
- 21:58Speaker 2: They don't talk to each other, and they definitely aren't
- 22:00Speaker 2: willing to share their secrets with each other. I've had
- 22:05Speaker 2: numerous people tell me who develop games, either at a
- 22:07Speaker 2: big company or even especially if they're an independent developer,
- 22:11Speaker 2: They'll say, look, I would love to make my game accessible.
- 22:15Speaker 2: It would take me six months. I'd have to stop
- 22:17Speaker 2: everything I'm doing and I'd have to figure it out somehow.
- 22:20Speaker 2: And realistically speaking, that six months and I actually can't
- 22:24Speaker 2: afford to do that.
- 22:25Speaker 6: Yeah, And I think that the reason behind that is,
- 22:29Speaker 6: like every studio has to build these things from scratch.
- 22:33Speaker 1: There aren't enough of.
- 22:35Speaker 6: These like accessibility widgets that are available like just kind
- 22:39Speaker 6: of as a package, ready to go that you can
- 22:41Speaker 6: plug in.
- 22:42Speaker 2: So that's exactly what Brian and Vishnu had been working
- 22:45Speaker 2: on within the lab.
- 22:46Speaker 6: We're actually creating a plug in for Unreal Engine to
- 22:49Speaker 6: make it easier for any game built on Unreal Engine
- 22:53Speaker 6: to adopt napstick, and then we'll later add survey or
- 22:57Speaker 6: functionality to that.
- 23:00Speaker 2: Reel is a game engine that a lot of Triple
- 23:02Speaker 2: A games, that's those big blockbuster games are built on.
- 23:05Speaker 4: So an engine is.
- 23:07Speaker 2: Kind of like the base program that a lot of
- 23:09Speaker 2: game developers build their game on top of. So, for example,
- 23:12Speaker 2: if you're making an adventure game where you walk around,
- 23:15Speaker 2: if you had to program in the concept of gravity
- 23:18Speaker 2: or the concept of a ground that you walk on,
- 23:21Speaker 2: you'd never get anything done. So you get an engine
- 23:24Speaker 2: like Unreal that has a lot of the basic stuff
- 23:26Speaker 2: built in, and if you need additional features, you get
- 23:29Speaker 2: a plug in. So there are plugins that'll help you
- 23:32Speaker 2: make realistic trees or anime an ocean or a river,
- 23:35Speaker 2: or make realistic sound effects. And what Brian is hoping
- 23:38Speaker 2: to do is to make a plug in that any
- 23:41Speaker 2: game developer who's using Unreal could just bring into their
- 23:44Speaker 2: engine and add the navstick or the surveyor features.
- 23:48Speaker 1: So we are going to put as a law. We're
- 23:49Speaker 1: going to publish it on Unreal Engine marketplace so.
- 23:52Speaker 6: That developers can see the possibilities and see exactly like
- 23:56Speaker 6: how to implement and incorporate these things into the game.
- 24:00Speaker 4: How much would this cost for a developer.
- 24:02Speaker 1: It would be minimal. I don't expect it to cost
- 24:05Speaker 1: more than say, like one hundred bucks or something like that.
- 24:08Speaker 4: Okay.
- 24:09Speaker 2: Yeah, And the ability to throw that in and potentially
- 24:12Speaker 2: have some more people by your game who would never
- 24:15Speaker 2: have picked it up is kind of cool.
- 24:17Speaker 6: Absolutely, Yeah. And I would love to, you know, for like,
- 24:20Speaker 6: especially like independent developers and things like that. You know,
- 24:23Speaker 6: I'd love to have a program where they can they
- 24:27Speaker 6: get incorporated into their game without having to pay. This
- 24:30Speaker 6: is something I'm still working with the university on.
- 24:34Speaker 2: The hope is that these plugins make it so obviously
- 24:36Speaker 2: easy to add in accessibility features that developers see it
- 24:40Speaker 2: and they say, you know what, let me just download
- 24:42Speaker 2: this plug in and just do it, and they'll be
- 24:44Speaker 2: able to get started pretty soon.
- 24:46Speaker 6: We're actually hoping to launch that by the end of
- 24:48Speaker 6: this calendar year twenty twenty five.
- 24:51Speaker 4: Wow.
- 24:52Speaker 2: I really do hope that game developers check this out
- 24:54Speaker 2: because I'd love to see what kind of games come
- 24:56Speaker 2: out in the next few years that use this technology,
- 24:59Speaker 2: because this isn't just a niche feature. It really can't
- 25:02Speaker 2: open up games for an entire community of people who've
- 25:04Speaker 2: been shut out of video games, and that can open
- 25:08Speaker 2: up new ways for everyone to connect. I remember early Quarantine,
- 25:13Speaker 2: my niece was in a different state, and when we
- 25:17Speaker 2: wanted to communicate, she didn't call me. She said, Yo,
- 25:21Speaker 2: let's get on Fortnite, and that's how we talked. We
- 25:24Speaker 2: got in Fortnite and we ran around and build stuff,
- 25:29Speaker 2: and you know, she showed me some weird little structure
- 25:31Speaker 2: that she'd built and her friends are running around in there,
- 25:34Speaker 2: and like, that's how we talked. And so video games
- 25:37Speaker 2: now are, for better or worse, where people spend time
- 25:42Speaker 2: with each other.
- 25:44Speaker 1: If they're immersive, right, it's a form of immersion.
- 25:47Speaker 6: And so I think that, like you know, any tool
- 25:50Speaker 6: that can can enable access to video games could also
- 25:53Speaker 6: enable access to a whole host of immersive experiences as well,
- 25:57Speaker 6: like being able to like tour apartments, for example, without
- 26:00Speaker 6: having to visit that apartment or two or a different city.
- 26:04Speaker 7: There's potential to bring nastick something like nastick into the
- 26:08Speaker 7: real world as well. Really yeah, navigation through the real
- 26:12Speaker 7: world using you know, you have a blind person outside
- 26:16Speaker 7: or in a building, you know, pointing in some direction,
- 26:20Speaker 7: maybe with your camera or something else, and having it
- 26:23Speaker 7: say what what's around you.
- 26:26Speaker 2: This is a pretty interesting idea because it could be
- 26:28Speaker 2: an augmented version of what blind and low vision users
- 26:31Speaker 2: have found in the meta smart glasses. If you haven't
- 26:34Speaker 2: seen our episode on that, you can check that out.
- 26:36Speaker 2: But instead of AI as most people think of it,
- 26:39Speaker 2: this would be layering a video game kind of on
- 26:42Speaker 2: top of reality.
- 26:47Speaker 1: An example that I like to give is New York's
- 26:50Speaker 1: Pen Station.
- 26:51Speaker 6: It's the busiest train station in the world, and it's
- 26:55Speaker 6: massively complex to someone who's never been there before, with
- 26:58Speaker 6: multiple railroads and subway lines and you know, et cetera.
- 27:01Speaker 1: It's huge.
- 27:03Speaker 6: And imagine being able to like swipe by like tapping
- 27:07Speaker 6: the side of an earbut and exploring kind of circularly
- 27:12Speaker 6: and being able to hear spatial audio of like what's
- 27:14Speaker 6: around you at Penn Station.
- 27:16Speaker 1: Oh, the ticket office is here, that's track twelve, that's
- 27:19Speaker 1: track thirteen.
- 27:20Speaker 6: Here's the McDonald's, or here's the shake shack, right we
- 27:23Speaker 6: have this physical world, we just have to create a
- 27:26Speaker 6: virtual representation of it first.
- 27:28Speaker 1: Let's say like a video game version, a digital version.
- 27:31Speaker 6: Oh so that like now, when we're in the physical world,
- 27:35Speaker 6: if the system knows where you are, it's the same
- 27:37Speaker 6: as napstick. You're exploring that environment, but you just happen
- 27:41Speaker 6: to be physically inside the place as well.
- 27:43Speaker 2: The video game stuff that I'm messing with and that
- 27:46Speaker 2: in the future gamers will be able to use ends
- 27:49Speaker 2: up kind of being like a trial run for using
- 27:52Speaker 2: this in real life at a train station or a
- 27:55Speaker 2: grocery store.
- 27:56Speaker 6: Yeah, that's a great way of thinking about it. Oftentimes
- 27:59Speaker 6: we think of the games that we prototype as being
- 28:04Speaker 6: almost like testing grounds, Right, that's our little our peatrie
- 28:08Speaker 6: dish really to test our systems. But it's just a
- 28:12Speaker 6: fake version of what might be a real world environment
- 28:16Speaker 6: as well.
- 28:18Speaker 2: If you told someone in twenty fifteen that they wouldn't
- 28:20Speaker 2: be able to walk to their mailbox without being recorded,
- 28:23Speaker 2: they would think that you were describing as science fiction dystopia.
- 28:27Speaker 2: If you say that now in twenty twenty five, they'd say, well, yeah,
- 28:29Speaker 2: you're just describing the basic function of my neighbor's ring
- 28:32Speaker 2: camera we now expect technology to keep expanding, so what
- 28:37Speaker 2: if we started expecting accessibility to advance also. I mean
- 28:41Speaker 2: five years ago, I thought Brian was nuts for thinking
- 28:45Speaker 2: that Ross Minor would be able to play a racing game.
- 28:47Speaker 4: And then this year I.
- 28:48Speaker 2: Walk into Ross's apartment and He's got a driving wheel
- 28:50Speaker 2: on his desk.
- 28:51Speaker 4: To play Fortza.
- 28:52Speaker 2: But this stuff doesn't just happen. It's the industry, it's
- 28:56Speaker 2: the universities, and it's the gamers. I'm not saying you
- 28:59Speaker 2: need to write a strongly worded email to Nintendo, but
- 29:03Speaker 2: you do have influence over things, and they do affect everyone.
- 29:07Speaker 2: I mean, look, I personally do not need Knavstick and
- 29:10Speaker 2: I don't need Survey Air, but I am more likely
- 29:12Speaker 2: to buy a game if I know that it means
- 29:14Speaker 2: that I might get to play co op with Ross.
- 29:17Speaker 2: The stuff that people like Brian and Vishnu and Ross
- 29:19Speaker 2: build is gonna make games more interesting for everyone. I mean,
- 29:23Speaker 2: who knows, Maybe someone listening to this right now, maybe
- 29:26Speaker 2: you could end up contributing something to that process. And
- 29:34Speaker 2: thank you for listening to kill Switch. This was a
- 29:36Speaker 2: fun one to do, and hey, if you got any
- 29:38Speaker 2: ideas for the next one, you can hit us on
- 29:40Speaker 2: email at kill Switch at Kaleidoscope dot NYC, or we're
- 29:44Speaker 2: on Instagram at kill Switch Pod. And if you listen
- 29:48Speaker 2: all the way through our first double episode, hopefully that
- 29:50Speaker 2: means you liked it. So you know, get that phone
- 29:53Speaker 2: out and please leave us a review. I know people
- 29:56Speaker 2: say it all the time, but it really helps other
- 29:58Speaker 2: people find the show, which he'll us keep doing our thing.
- 30:02Speaker 2: Killswitch is hosted by Me Dexter Thomas. It's produced by
- 30:05Speaker 2: Sena Ozaki, Darluck Potts, and Julia Nutter. A theme song
- 30:09Speaker 2: is by me and Kyle Murdoch and Kyle also mixed
- 30:12Speaker 2: the show from Kaleidoscope. Our executive producers are Ozma Lashin,
- 30:16Speaker 2: Mangesh Hatigador, and Kate Osmore from iHeart Our executive producers
- 30:21Speaker 2: are Katrina Norvil and Nikki Etour. Oh one more thing.
- 30:24Speaker 2: Oh this is for all my game developers out there
- 30:26Speaker 2: who are listening to this. You might have been a
- 30:28Speaker 2: little disappointed to hear that they're making these plugins for
- 30:31Speaker 2: Unreal Engine, because look, Unreal is great. A lot of
- 30:34Speaker 2: blockbuster games are built on it, but it's just that
- 30:36Speaker 2: it's kind of for the big time games. You know,
- 30:38Speaker 2: why would you leave the independent game developers out, the
- 30:40Speaker 2: ones who you know, use different software, cheaper development software
- 30:43Speaker 2: like say Unity. Well, I asked Brian about that.
- 30:47Speaker 6: I wanted to follow the model that happened with Fortsa,
- 30:50Speaker 6: where like a major developer had this feature incorporated into
- 30:55Speaker 6: their game that got noticed by the whole industry. That's
- 30:59Speaker 6: an important and first domino to fall. And so we
- 31:02Speaker 6: do want to make a Unity equivalent of the plug
- 31:05Speaker 6: in as well. But we chose Unreal first for that reason,
- 31:08Speaker 6: so that we could work more closely with some of
- 31:12Speaker 6: the bigger developers, work through all the kinks with their
- 31:15Speaker 6: larger teams they would have more resources, and then figure
- 31:19Speaker 6: out how to tailor both that Unreal Engine plug in
- 31:22Speaker 6: and how to do things right on Unity for the
- 31:25Speaker 6: smaller indie developers the first time.
- 31:28Speaker 2: So there's your answer, and you know I get the
- 31:30Speaker 2: approach anyway, Catch on the next one, good guy,