Latest / Bad Decisions Podcast / #31 Building a Successful VR Business with Alex Harvey | Bad Decisions Podcast
Transcript
- 0:00There is no other medium after that that replicates the human
- 0:03experience. The fact that you managed to
- 0:05convince police officers to get in an office room, have 203035
- 0:09people, we blew up a car but we found some secret tunnels
- 0:13underneath the town that connects the castle.
- 0:16It's basically a way to capture the real world and reconstruct
- 0:19it in the most efficient and photorealistic manner.
- 0:22It was like me walking around inside a video because
- 0:25everything was photorealistic textures.
- 0:26How do we create the 3D photogrammetry scenes in VR?
- 0:31Every company on the planet should have a box and a camera
- 0:35so they can film their training scenarios.
- 0:37They can film their procedures. You're literally on Polycam
- 0:40scanning different places and telling people how to make money
- 0:43on Sketchpad. What is up, beautiful people?
- 0:46Welcome back to another episode of the Bad Decisions Podcast,
- 0:49where we give you guys a backstage pass into the minds of
- 0:52creatives, entrepreneurs, and visionaries.
- 0:56And our today's guest is someone who creates reality in virtual
- 1:00reality. He's the Co founder of River Auk
- 1:03based VR production company that specializes in immersive
- 1:07training. They've trained numerous police
- 1:09officer and firefighters in near to live fire and crime scene
- 1:14environment. Everybody, this is Alex James
- 1:16Harvey. How's it going Alex?
- 1:18Good to see you both. Amazing TA.
- 1:20It's a nice sunny Sunday in the UK.
- 1:23Oh my God, this is the first time you're recording a podcast
- 1:25on a Sunday. So thank you for joining us on a
- 1:28Sunday on the podcast. I think it's literally, it is
- 1:31really, I'm impressed that it's Sunday.
- 1:33I'm impressed that you two are so keen on a Sunday and I'm
- 1:36definitely feeling that keenness as well.
- 1:38I love. I love a Sunday and I love
- 1:40scanning talks. Yeah, you know what?
- 1:42We got a lot of conversations to cover today, Alex, and I'm just
- 1:46going to tell everybody we're going to be talking VRARXR and
- 1:50photogrammetry nerves and freaking Gaussian splatting,
- 1:54'cause that's what we've been all about the past few weeks.
- 1:57And I know you guys are excited about it.
- 1:59And and you, Alex, on your own time.
- 2:02I mean, you're a busy man. You work on a lot of cool
- 2:04things. At River, and we're going to
- 2:07talk about that because you work with the government
- 2:09institutions, you provide training for many people around
- 2:12the world, not just in the UK but outside of the UK.
- 2:15But in your free time, you do a lot of cool shit too.
- 2:19You're literally on Polycam scanning different places and
- 2:22telling people how to make money on Sketchfab.
- 2:24And then you're at SpaceX watching rocket launches.
- 2:28And I'm like, what does this guy do?
- 2:30And that's actually how we met. We met online, I think through
- 2:33Polycam I there. No, it was.
- 2:34I think I I watched him going to the Star Starship.
- 2:37Yeah. I think, I think it was Polycam.
- 2:39I I think it was through Polycam because you posted a lot about
- 2:43Polycam. We posted about Polycam before,
- 2:45so I think that's how we stumbled upon each other's
- 2:47profiles. But you do a lot of crazy shit.
- 2:50We're going to get into all of it.
- 2:51But first of all, before we get deep into it, what were you
- 2:53doing at SpaceX 'cause that video looked super cool if
- 2:57you're watching the Rockets. Yeah, watching the Rockets take
- 3:01off or which video? Because last time we actually
- 3:03spoke, the first time I spoke to you, I was in a pool swimming
- 3:07around in Houston, and I had my ray ban stories on and I was
- 3:10filming videos on my glasses. And then we started talking on
- 3:13Instagram. That's the first time we met.
- 3:15I was in Houston, and on that trip I was like 5 hours away
- 3:19from Boca Chica. So I thought, let's just take
- 3:22the time before the show. I was there for a show AVR show
- 3:25with Pico. But before the show, let's drive
- 3:28down to Boca Chica. See some rockets.
- 3:30So I I could not believe that I could just drive into Starbase.
- 3:33I drove into Starbase and I'm getting shivers now as we talk
- 3:36about it, because I'm just, I'm just looking at rockets in front
- 3:40of me. I couldn't believe that you get
- 3:41so close to them, but I was literally, I don't know, 10
- 3:44metres from a Starship. And I was, I was lost for words.
- 3:48But I was there literally just to have a little look.
- 3:51And I was hoping that one of those two days there might have
- 3:54been the launch of the next, the next launch of the Starship.
- 3:58But they cancelled it and it's still yet to take off.
- 4:01But before that I've been a lot of time in in Florida and I've
- 4:05seen three rocket launches of the Falcon 9, which you can
- 4:09probably see a Falcon Nine behind me on the here.
- 4:11This is a 3D printing Falcon 9. I see that right now actually,
- 4:15if you can scan. It You didn't, right?
- 4:18You didn't scan it. I didn't scan it, but I did use
- 4:21I did use Polycam when I was in Boca Chica and I did a 360 photo
- 4:26with inside Polycam. But drones are very not allowed
- 4:30around there. So I didn't take off the drone.
- 4:33It failed for me, but maybe I was too excited.
- 4:35I didn't do a video either for a bit of a splat.
- 4:38I didn't do that. So that's that's my fail.
- 4:40I've got two guys going again on Sunday to Houston for a week.
- 4:44I've had to cancel this trip, but they're gonna go for me.
- 4:48Maybe they'll go down and do something in Houston again.
- 4:51Dude, it's crazy. Are you doing this out of
- 4:53passion or is this related to anything you guys are doing at
- 4:57River? No, this is pure passion.
- 5:00Although we are working with Tesla, so we've got Riverlink
- 5:03kits that are in Tesla factories and we're teaching the
- 5:06firefighters about hazards and stuff using Riverlink.
- 5:10And we've been to Fremont factory in California and I've
- 5:14done a tour of the cars there in 3D180.
- 5:17But everything else like you said about Polycam and scanning
- 5:19is pure passion pure. I just use my phone all the time
- 5:24to scan stuff and put it online. I was, I was at Code Masters for
- 5:29eight years and I was making game trailers.
- 5:32So I was the video editor for Code Masters to make their car
- 5:35trailers, flashpoint trailers and I was the cameraman inside
- 5:41the game world. So I would drive the game on
- 5:44APC, smash some cars together, then watch the replay of that
- 5:49crash and then join the replay, detach the camera and then I can
- 5:53move the camera around like a drone in the game world.
- 5:56And that was before drones. So this was like 2013, then 2014
- 6:00I bought a drone and started doing real world car trailers.
- 6:04The transferable skill was amazing, to be able to be
- 6:07videoing inside a game world with my thumbs and then replay
- 6:12time, and then I was in the real world chasing cars around with a
- 6:15drone. So that's how that happened.
- 6:18But yeah, sorry, I'm going to forget the questions.
- 6:20No, no, no, no, no. I so I was just asking about
- 6:23your passion there. But since you brought it up and
- 6:25for everybody listening and watching right now, what do you
- 6:28guys do at River is really crazy.
- 6:29Me and Farrad have been watching you guys for a while, but can
- 6:31you tell everybody what that's like?
- 6:33Because VR training is a term that, for a lot of people, might
- 6:37make sense if they played virtual reality.
- 6:39Of the games before, if they've experienced 360 video before and
- 6:42360 content, but for a lot of people they might not actually
- 6:45know what that is like. They might be expecting some low
- 6:47Poly experience. But clearly based on what me and
- 6:51Farrad have seen, you guys do something different.
- 6:54I mean through not just the UK but other countries, you guys
- 6:57must have done something crazy impressive.
- 6:59So can you tell everybody what Riverlink is actually because
- 7:02that's one of the products you guys have and what River does in
- 7:05general. I guess I can tell you what
- 7:07Riverlink is, but to if we were going to go back in time and do
- 7:11it in order then I would be talking about another product
- 7:14first because we actually made fire investigation product
- 7:18before we made River Link and we realised that it was 2 ahead of
- 7:21its time. So then we made River Link but,
- 7:25but maybe I should touch on that.
- 7:27I mean, I left Code Masters and I tried virtual reality and I
- 7:31was amazed with it when you know the Vive.
- 7:33I was trying Vive. That was my first sort of
- 7:35experience inside VR. And then Gear VR came out and we
- 7:38were looking at 360 videos inside Gear VR with a phone
- 7:42clipped into a plastic case and that was all.
- 7:45That was all fine. But when I was inside virtual
- 7:48reality, I don't know if you ever tried AVR experience called
- 7:51Realities dot IO and that was inside Steam store.
- 7:56It was in the steam store and you could walk around a
- 7:59photogrammetry room inside virtual reality and I was amazed
- 8:03that you could actually walk around this bit.
- 8:05It was like being inside a video, It was like me walking
- 8:08around inside a video because everything was photorealistic
- 8:10textures. So before River Link came river
- 8:14investigate because we found that we could photogrammetry
- 8:17scan the inside of burnt containers for the fire
- 8:20investigators. So we started by hiring a a a
- 8:27piece of land in the UK and we put four containers, those big
- 8:30metal containers on this land and we got fire investigators
- 8:34round and we dressed the scenes so that they were like a front
- 8:38room or a bedroom or a kitchen. And then we set fire to them
- 8:42with the cause of fire being the plug or you know the PC.
- 8:46So we burnt these containers in the real world and then we got
- 8:49the fire service in to put the fire out.
- 8:51And then we took out each individual item 1 by 1 and
- 8:54photogrammetry scanned each item each each container had about 70
- 8:59items inside it. And then we put all that back
- 9:02together in Unity and we made an interactive fire investigation
- 9:06scene for them. And then we did it seven times.
- 9:09Now we have 9 scenarios and we sell it as a fire investigation
- 9:13software multiplayer, but it was the first thing we made.
- 9:17But the market for it is very niche.
- 9:19It's firefighters that do investigation and that's not
- 9:22every firefighter. So although it's being used now,
- 9:25it was made four years ago, five years ago, like when I sent you
- 9:29that video of the this morning I sent you a video of an explosion
- 9:32in Leicestershire. That was the same sort of time
- 9:35we made that. So that was photogrammetry and
- 9:38that was first. And then came Riverlink.
- 9:40Obviously River Investigate is still there now being used, but
- 9:44then we moved on to 360 Video and realised that the
- 9:47investigation scenes are not for everyone.
- 9:49It's very sort of hard to get people into that environment if
- 9:53they need APC and a Vive and Lighthouse sensors and all these
- 9:57things. So we moved back towards 360
- 10:00video that was much more accessible for people to get
- 10:03into. And how that started back in the
- 10:06day was we were doing videos for Thompson Holidays.
- 10:10I think they're called TUI now. They're a holiday company in the
- 10:12UK and they wanted to show what it's like to be on their planes,
- 10:17to see the extra legroom seats on their planes and they wanted
- 10:20to see what it's like on the cruise ships to video on the
- 10:23cruise ships in 360 video. So it was a horrible task that
- 10:27we, me and two other guys had to go on a cruise ship to Barbados.
- 10:31To shoot some 3 60 video. So that must have been
- 10:36extremely. Difficult.
- 10:38Yeah, it was really hard and we were in the hot tubs with the
- 10:43the cameras we had. Actually back then it was the
- 10:45GoPro Omni. So we were using the GoPro with
- 10:49a load of cables coming off it and we were stitching the soft.
- 10:52We had the stitching software to put it all together, but what we
- 10:55made for them turned into Riverlink.
- 10:57So because they wanted to synchronise the video with four
- 11:01people in a holiday shop, they wanted to show it to four people
- 11:04at the same same time in a headset.
- 11:06So we got Gear vRS with phones clipped in and a laptop and a
- 11:10router, and we made a piece of software and back then it was
- 11:13called VR Sync, but then it subsequently changed to
- 11:17Riverlink. But that was the first
- 11:20incarnation, if you like, of that software that now is used
- 11:24by everyone all over the world in these boxes.
- 11:27It's a synchronised video. So it's basically Riverlink is a
- 11:30box of headsets and it can be 510-1520, however many you want.
- 11:35And we build the router into the box so that there's no need for
- 11:39them to connect to their Wi-Fi because that just causes a world
- 11:42of pain. And then we load the videos, the
- 11:45MP4 videos onto the headset and onto the laptop.
- 11:49So there's no streaming of any data.
- 11:51The video is on each device and it's just sharing the XYZZ of of
- 11:56where the person's looking. So I realised that we just
- 12:00needed products. We if we want to have a company
- 12:03then we need to have products to be able to sustain that company.
- 12:06And Riverlink and River Investigate are the first two
- 12:09products that we have made, and they're still going now.
- 12:12And now I've just been walking around the office here and
- 12:15there's like hundreds of big plastic boxes everywhere, which
- 12:19is people that are building Riverlink kits and shipping them
- 12:22out to people because the camera quality is so cool now that we
- 12:26teach people how to shoot their own 360 video.
- 12:29So I call Riverlink Everyone's stage one VR.
- 12:32It is. Every company on the planet
- 12:35should have a box and a camera so they can film their training
- 12:39scenarios, they can film their procedures, and we've just hit
- 12:43blue light services 1st. But now we're we're into all
- 12:46types of environments, like every warehouse that wants to
- 12:50show the procedures of leaving to do a fire escape.
- 12:54You know which which route do you leave the building to go on
- 12:57a fire escape or inductions. If people are joining the
- 12:59company, they also can film their inductions themselves
- 13:03because it's in their language, it's in contextual, it's in
- 13:07their place. So it's really good for them to
- 13:10just shoot their own video, load it onto the box and have a
- 13:13library of content that's relevant to them.
- 13:15And that's what Riverlink is. It's very simple because it's
- 13:18just synchronized 3D360 video. That's very interesting.
- 13:23That's super interesting. But I want to take it back for
- 13:26us to to the part that you set those containers on fire.
- 13:29So was that an R&D process for you and your team or was that
- 13:34something because? How would you get the idea?
- 13:36Where did that idea came from? Yeah, and emergency services.
- 13:40Like, who would think, you know, you can think of training for
- 13:43video games, you know, there's so many.
- 13:44Other things that. Average people would go for, but
- 13:47you went ahead to this extremely niche market.
- 13:50How did you end up there? We literally started by I saw
- 13:54photogrammetry inside the Vive inside realities dot IO and I
- 14:00thought, this is cool, but can we do it with video?
- 14:02So what I wanted to do was have like stop motion of we want to
- 14:07test it to see if we could get video grammetry into VR because
- 14:12I could see that we could do photogrammetry.
- 14:14So this was before there was even a river team assembled.
- 14:17It was me tinkering with technology and we actually got
- 14:21this guy and we put him inside. We put this guy inside a rig and
- 14:27we put Gopros all around him and we just moved his arms and we
- 14:31tried to do video grammetry to see if we could do animation.
- 14:34This was 2015, two 1016. And so my my idea was that we
- 14:40wanted to do video grammetry and then put it into VR.
- 14:43But when I got people together they were saying to me this is
- 14:46far too advanced, like you can't be doing this in 2016, just rain
- 14:51it back a bit and do something different.
- 14:53So the thing that it got rained back to was we had a friend
- 14:56called Paul Spate who was in Leicestershire Fire and Rescue
- 14:59Service and he allowed us to have access to their world and
- 15:04we would film 360 videos for him of procedures happening and
- 15:08firefighters doing stuff. And that was cool.
- 15:11That's what birthed the sort of River Link content for the fire
- 15:13service. But while we were there, he said
- 15:15to me, come over here and look what the guys are doing in fire
- 15:18investigation. And they were burning containers
- 15:21and putting five people into the container.
- 15:24But as the people went in, the instructor said to them, don't
- 15:28touch anything. You're not allowed to touch
- 15:29anything because we've got another five people coming in
- 15:32later on today and we can't have you touch it.
- 15:34So my brain just went, oh, let's just scan everything, put it
- 15:39into VR and now you can have as many firefighters as you want
- 15:42coming through and picking things up.
- 15:44Because in the real world, when they go and investigate a crime,
- 15:47they need to pick up the cup and look underneath it.
- 15:50They need to see if it's burnt underneath and you can go on
- 15:53YouTube and search all this. You'll see the fire
- 15:55investigation stuff. It's amazing now because they
- 16:00don't have to breathe in all that carcinogens into their
- 16:03lungs, they don't have to burn the environment.
- 16:07So as soon as the firefighters, it's a really, really
- 16:10interesting process of how we sell that to them because once
- 16:12they see it and know about it, it's almost illegal for them to
- 16:17then go back to doing the old ways of training.
- 16:21And so that sells really well once they realise what's
- 16:24available. And I still see people now
- 16:25online burning containers at the back of fire stations just to
- 16:30investigate crimes. And the bonus for me is that
- 16:33photogrammetry was a static scene.
- 16:37There was a static scene with no movement, so it was really good
- 16:40for VR. It was less, it was less painful
- 16:43with the processors and the textures because at first they
- 16:48said to us that we couldn't do it and there was too many
- 16:51textures on all of the items. So it's a painful process that
- 16:55I'm hoping is going to get better soon.
- 16:56But that decimation of mesh and texture and simplification down
- 17:01to get models that are OK in VR used to be such a pain in my ass
- 17:07because it was the biggest blocker.
- 17:09But now software's coming out. That's knowing that this surface
- 17:13is flat and it's going to make it 4 polygons instead of 4
- 17:16million. And that's where things are
- 17:19heading. So it's a lot easier now, but
- 17:21hopefully that gives you a bit of a story about Riverlink and
- 17:24River Investigate. That is very well put.
- 17:28And you know, before we get into it, I just want to say one
- 17:32thing. Your business model is just
- 17:35about to explode in 2023-2024, 2025 because dude,
- 17:39photogrammetry and all these technologies are becoming so
- 17:43widely accessible and so easily. Doable because right?
- 17:49Just like you mentioned, it's getting the right textures right
- 17:52to look good, especially if they're burnt.
- 17:54They have a lot of detail, intricate details, man.
- 17:56Right now you can get that with a fucking iPhone and and get it
- 17:59to work really well. And so for you the process
- 18:03becomes a lot easier and what you are doing, especially with
- 18:06river language is really interesting and I want to make
- 18:09sure everybody noticed. So you do have the services that
- 18:12you provide to all these institutions where you guys go
- 18:15in and do the production yourself.
- 18:17But there's also services where you guys are giving them the
- 18:19capability to use a 360 camera, record their own trainings and
- 18:24then get that box in anytime they want.
- 18:27They want to train 1000 people in a in a day.
- 18:29If they have the numbers, they can and they don't have to set
- 18:32up anything but a room with your river links and they can put on
- 18:36the headsets. And the coolest part was we saw
- 18:39this. On your website there's a
- 18:40laptop. The tutor gets to control the
- 18:44laptop right through. Draw stuff.
- 18:46Use the talk or whatever to draw stuff and then talk to the guys
- 18:50who are within the VR experience.
- 18:52I mean, this just seems so obvious when you've already made
- 18:56it. It's like, why didn't this go
- 18:58for everything else? You know what I mean?
- 19:00Why? Why isn't everybody using this
- 19:02already? I think that it's just a bit of
- 19:04a road. It's a bit of a road, isn't it?
- 19:06It's a bit of a journey that everyone needs to work it out
- 19:08themselves on the path, and I think it's easier for you to to
- 19:11understand how that's really beneficial.
- 19:15But when you are an organization that is so used to iPads and
- 19:18phones and they've only just got used to drones, VR and 360 video
- 19:24is like a step too far for quite a lot of them.
- 19:26So we have to do demos to them. We have to put the headset on
- 19:29people's heads before they understand it.
- 19:32But you're right, we do self shot content or professional
- 19:35content. But we normally do the
- 19:37professional content as their first step and then we give them
- 19:40a camera and teach them how to do it.
- 19:42For example, I flew into Washington two years ago and we
- 19:46worked for Department of Homeland Security.
- 19:49We flew into Dulles Airport and we shot during COVID some bag
- 19:54searching, some patrol check things in the airport in 363-D
- 20:00video. And then it was also how they
- 20:03used their airport scanners, teach people how to use the
- 20:06scanners and that was self shot by us for them and then we gave
- 20:10them cameras so they could do contextual.
- 20:13Photos and videos in all the airports.
- 20:15And we just found out that you can train security personnel
- 20:19really good if you've got 5 videos that teach them all the
- 20:21basics. But then there's baggage
- 20:24handlers, airport staff like coffee staff, every there's
- 20:28everyone in that airport that also sees it and then wants to
- 20:32buy it and use it for their area.
- 20:34So it just explodes once you show one person because they
- 20:38they all use their creativity to go.
- 20:41We could use this for us. We can go and film our own
- 20:44content. And when you tell them that this
- 20:46camera is going to give them the perspective of their head,
- 20:50they're just, you know, it blows their mind.
- 20:52But I think you are really far ahead with your thinking.
- 20:56I like to say, you know humbly that we were very far ahead with
- 21:01our thinking and it's only just catching up to the masses.
- 21:04I see Riverlink as being like a printer in every office, every
- 21:09office, or every warehouse or every place would normally buy a
- 21:12printer. You'll eventually buy a
- 21:14Riverlink kit for your well-being training to make to
- 21:18do your to do your training on personnel for inductions.
- 21:22It will be just something that's in an office that you use and
- 21:25you'll just expect it. And you know, right now the
- 21:28Riverlink is using, I believe you guys are using Pico
- 21:31headsets, is that right? Yeah.
- 21:32We've got the new box can take Oculus as well as Pico.
- 21:37It's just a a box of eight. We call that the Fusion box
- 21:40because now we're seeing the merge of the headsets because
- 21:44the Pico headsets are three degrees of freedom and a lot of
- 21:48the Oculus stuff, as soon as Oculus killed Oculus Go as a
- 21:52headset, everyone thought the video was dead.
- 21:55And I was like my my my hands were in my hair, my head was in
- 21:59my hands. I couldn't believe they did it
- 22:01because video and 3D 180 video is stunning when you get it done
- 22:06right in a headset. So I was a bit disappointed when
- 22:09they killed the GO. So we just went with Pico
- 22:11because they were three Doff headsets.
- 22:13But now all the headsets are converging into one headset
- 22:16which is going to be doing 6 Doff and three Doff in the
- 22:19headset and everyone will eventually forget about having a
- 22:22stand alone 3 Doff headset. They'll be able to do both in
- 22:25them. I mean, the reason I even
- 22:27brought this up is because when you mentioned that every office
- 22:31is going to have one, I mean one of the biggest barriers to entry
- 22:35for virtual reality has always been the form factor of virtual
- 22:39reality headsets. A lot of people, they don't even
- 22:41want to put the headsets on. But you can imagine because you
- 22:45have the Ray Ban glasses yourself.
- 22:46I mean, we haven't even tried them, but I've I've heard so
- 22:48many good reviews from people, you can just imagine how all of
- 22:52these are going to connect together eventually down the
- 22:54line, when the form factor gets smaller, when it becomes more
- 22:56like glasses, that's when you're going to see, you know, River
- 23:00Link with literally glasses. Now that is going to give you
- 23:03the exact same experience. That is going to be the part
- 23:06where I. There you go.
- 23:07That's what I'm talking about. That's it.
- 23:11Are you recording this podcast right now?
- 23:13Now from now. It will be recorded.
- 23:17I am recording you now, but these are the old ones.
- 23:19I'm still waiting for the new ones.
- 23:21When are you going to get them? I've ordered them.
- 23:24I'm just waiting for them to arrive.
- 23:25But yeah, this is what I was filming inside the pool when we
- 23:29first spoke. But you're right, the form
- 23:31factor is going to, they're just playing it from both ends at the
- 23:35moment, as in they've got this and they've got this and
- 23:39eventually there's going to be something in the middle.
- 23:42And I just think you're right, whether you're thinking and I
- 23:45think Mark Zuckerberg mentioned it quite a lot, that they're
- 23:48trying the smallest form factor and the biggest form factor and
- 23:51you've got the Vision Pro coming out, which is obviously quite
- 23:53big as well, But eventually it will be glasses and eventually
- 23:57then it'll probably be contact lenses.
- 23:58So the reason I'm so excited about VR and have been for five
- 24:03years is because it's the last medium.
- 24:05There is no other medium after that that replicates the human
- 24:09experience. That's why I'm so excited.
- 24:11It makes me shiver again because you you both understand it.
- 24:16And what frustrates me in the olden days was when you have VR,
- 24:20sometimes there's a need for it to be cartoon like or stylised.
- 24:24But most people want to be inside a virtual world that
- 24:27looks photorealistic. I don't want to be in a world
- 24:29that looks like The Simpsons. I want to be in Game of Thrones
- 24:33on the wall with Jon Snow looking over and like, I can do
- 24:37that now. So that's where I'm trying to
- 24:39get to. Yeah, you mentioned something
- 24:41which was really important when you so of course you have this
- 24:44mindset and mentality about that.
- 24:46VR is the future. Education in VR is the future,
- 24:50but how do you go about when you meet local authorities or
- 24:54government? Institutions.
- 24:55Is it the hardest to convince? How do you convince these
- 24:59institution with your vision and with the trainings that you
- 25:02provide in VR? They are probably not tech
- 25:05savvy. They are not.
- 25:06Young people, you have to convince people who are probably
- 25:09middle-aged to adults and probably they never were VR
- 25:13before in their life. What does the convincing look
- 25:15like? The IT was interesting because
- 25:18before we got to the the photogrammetry fire
- 25:21investigation scene, we had a demo scene that was
- 25:24photogrammetry inside a burnt room.
- 25:27And we just, we scanned it and we put it into VR and we had a
- 25:31body that was on the floor and you can actually see this on
- 25:34YouTube as well. You walk over to this doorway
- 25:37and you make them look out of the doorway into this alleyway
- 25:41and they see a body on the floor.
- 25:43And that was the moment for so many people in the Blue Light
- 25:46services. We would just go to them and do
- 25:48a demo and show them that demo. We called it the Cora Sweet demo
- 25:52just because that was the name of the building.
- 25:54But as soon as they felt the feeling they wanted to put their
- 25:56hands on the wall, they were sold.
- 25:59And after that point, they're they're sold with VR.
- 26:02So it is an emotional feeling if you can get someone in VR to
- 26:07feel like they're going to cry or feel like they're going to
- 26:10fall over. Like Richie's Plank experience,
- 26:13if you show that in VR, you're walking along and it feels
- 26:16really like you can't step onto this.
- 26:18Plank. The one.
- 26:19You're at the top of the skyscraper, right?
- 26:20And you feel like you're going to fall.
- 26:22That's it, That's it. You we literally showed that,
- 26:26that demo in London. Some people show that to the
- 26:29firefighters, then they call us to come and do a demo to them
- 26:32because these are people that are like they're massively
- 26:35against VR or technology. They've got a pre, pre set
- 26:38mindset that is I don't agree with it.
- 26:41It needs to be real and as soon as they have a go on what we had
- 26:44as our demo Cora suite and you give them that emotional
- 26:47connection, they literally change their opinion on the
- 26:50spot. I've seen so many people change
- 26:53their opinion and then it opens a bit of a can of worms
- 26:57sometimes because then they want to have a photorealistic scene
- 27:01for their world, but then we tell them the price of it and
- 27:04then they don't want it too much.
- 27:06Matter. Like, can you build this entire
- 27:08building and have this entire gaming exchange?
- 27:11I'm like, bro, you asked me to make a video game.
- 27:12We know that there's a moment they see the possibility now
- 27:15they think, OK, anything can be done.
- 27:17Well, yes. But come on, it's going to take
- 27:19some time. Yeah, that's the part they don't
- 27:21understand as well. So how do you deal with that
- 27:24then? As soon as they have that, yeah,
- 27:27they have that, they have that moment where they're like, I
- 27:29love it, I want it now. And we say, OK, just rain it
- 27:32back a bit. We tell them that is stage two
- 27:35and stage one is Riverlink. And so we say it's the same
- 27:38headset. You're just going to rewind time
- 27:40a little bit. You're going to get buy in from
- 27:42your organization first by buying a 360 camera and if they
- 27:46can film their environments or their people, doing procedures
- 27:50with a 360 camera might cost them a grand or £1500 to get
- 27:54going. Then they buy a headset.
- 27:57Then they can buy a river link kit.
- 27:58So we do that. It's the wow factor of a
- 28:00photogrammetry scene scaled back to 360 video.
- 28:04And if they've got enough budget and a little bit more expertise
- 28:08in their industry, then we tell them about 3D180 footage and how
- 28:13if they buy a Canon R5 with the Jewel fisheye lens, if they can
- 28:18do that, they can have a little bit more immersive content.
- 28:21And I mean, as soon as Vision Pro comes out and starts filming
- 28:24with these two cameras and doing some crazy something with the
- 28:29IPD, if as soon as that happens and people can shoot immersive
- 28:33content on their mobile phones, 3D 180, then people will
- 28:37understand more. But yeah, that's how we do it.
- 28:39We do photogrammetry, Wow factor, scale it back to 360
- 28:42content. I think there is an important
- 28:45point which Alex mentioned is sparking an emotion if you want
- 28:48to convince someone and this is not only about VRI.
- 28:51Think if you are in sales and marketing, if you're trying to
- 28:54sell any product. Film movies, the best movies are
- 28:58the ones that spark an emotion. Literally the first scene you
- 29:01get in there, you already feel something.
- 29:03You either feel sad or you feel happy.
- 29:05You know the moment you invoke an emotion, people are going to
- 29:07remember you. And you create that.
- 29:10And especially in a situation, like you said, if you're meeting
- 29:12with 10 extremely serious officers, I cannot imagine what
- 29:17that's like. So I just want to say one thing.
- 29:19What you're doing is truly exceptional because, you know,
- 29:22with virtual reality, with augmented reality, all of these
- 29:26spaces where a lot of people have, like you mentioned, a
- 29:29premeditated mindset where they're like, you know what, no,
- 29:33we, we're not going to do that. That's not for us.
- 29:35For you to be able to actually utilize that industry and create
- 29:39a truly meaningful relationship with institutions like the
- 29:42firefighters, the police officers, and go ahead and
- 29:45create a product that is not just helping them, but actually
- 29:50replacing what they're doing in a very meaningful way.
- 29:54So not only their solution scalable because like you
- 29:57mentioned you can have many virtual reality headsets you can
- 30:00have. The experience done one time and
- 30:03then taught to thousands of students and at the same time
- 30:07it's cost efficient and ultimately it's safer because I
- 30:11mean we weren't aware of this, we didn't think about this the
- 30:121st place. But when you mentioned when you
- 30:14burn something, right, it's dangerous for the students that
- 30:17get in there every single time. And you have all of these points
- 30:21added when you are presenting, I assume to them to to to show
- 30:25them the product. So it's really exceptional what
- 30:28you're doing. And I want to ask you about the
- 30:30development side of things, because a lot of the people who
- 30:33watch and listen to this podcast are either artists or
- 30:35developers. So what do you guys actually
- 30:38use? If you're if we're talking about
- 30:39the 3D side of things, right, because you guys do
- 30:41photogrammetry, What do you guys use in terms of your software to
- 30:45create some of these worlds if they're not just 360 videos?
- 30:50I guess it's really good to always.
- 30:51I bark on about it a lot, 3 doff or 6 doff there, That's where
- 30:55the split is for the company. It's basically Riverlink or it's
- 30:59a photogrammetry immersive scene.
- 31:01And I think what you're talking about then is the software and
- 31:04how do we create the 3D photogrammetry scenes in VR?
- 31:09It's very, you know, the 360 stuff is just video editing and
- 31:11it's that's an easier side of things, which I did come from as
- 31:15well. But yeah, the photogrammetry
- 31:17side of things, and that is for fire investigators that we do
- 31:21that. The majority is for fire
- 31:23investigators. But the reason I think we
- 31:26managed to sell that quite a bit is because we were doing it for
- 31:29the government in the UK and the US We were doing
- 31:33counterterrorism crime scenes and we were photogrammetry,
- 31:36scanning bombs going off and explosions for the government.
- 31:40And we can never show people these scenes because they're so
- 31:44top secret and all that. I hate Ndas, but we can never
- 31:47show these scenes. But that's where we learn most
- 31:50of our skills because we had to do the 77 bombings in London.
- 31:55We went and scanned the items from that bomb, those bomb
- 31:58blasts, and we recreated it in virtual reality and made it into
- 32:02a virtual scene for the government.
- 32:04But with that process of doing that, I think, I mean, I think
- 32:07you mean what software and you know the main software that the
- 32:11guys use here is Reality Capture and that's by a company called
- 32:15Capture and Reality. And we have a partnership with
- 32:18them because we use their software all the time.
- 32:22But that is thousands of photos of of lots of objects from
- 32:26different scenes. And we had a rig that would also
- 32:29do some of that if the object was like delicate and falling
- 32:32apart. But I always break that down
- 32:35into inside out and outside in. There's either an object on a
- 32:39turntable that you're going around, or there you're inside a
- 32:43room and you've got to do the whole of every wall and every
- 32:45ceiling. That process, as you're aware,
- 32:48is getting so much easier now because things like Polycam, but
- 32:53back in the day it was literally just, I say back in the day like
- 32:56I was. Like five years ago, yeah, five
- 33:01years ago we were doing that with, you know, 4000 photos
- 33:05inside a burnt room. And then we were taking out
- 33:08object by objects and doing maybe 3 or 400 photos of each
- 33:11one. But then it was such a pain
- 33:13because you had to decimate everything.
- 33:15So the art team at River just went through hundreds and
- 33:19hundreds of objects where they had to simplify the mesh, redo
- 33:23the textures, then make it like VR ready.
- 33:26And they had a certain limit of objects that they could put in.
- 33:29And were those objects going to be interactable or not?
- 33:32They were all like they were concerns for that team because
- 33:35it had to be at least 90 frames per second.
- 33:38It had to be good enough inside a headset that you didn't pick
- 33:42something up and drop it and it'll float to the floor like it
- 33:45has to drop and things have to move around.
- 33:48It's amazing for me now because things are getting easier, that
- 33:52workflow is getting easier, which means the creativity can
- 33:55explode more. And I've just got, you know, 10
- 33:59more ideas that I'm working on. That river investigate was
- 34:02definitely created four years ago, and there's a lot more
- 34:06things happening now with different stuff.
- 34:08Yeah, because. Of that, I just want to ask if
- 34:11you guys are building AVR experience again.
- 34:14If it's a video, I understand. But if it's a 3D experience, you
- 34:17guys use Unity or Unreal for that as well in your pipeline.
- 34:21Yeah, it's good that you mentioned the two because we've
- 34:24always done Unity. I've I've never done anything in
- 34:27Unreal. So I know I saw your video.
- 34:30Come on. By the way, I'm just joking.
- 34:34Just disclaimer, we have nothing against Unity, just we use
- 34:38Unreal Engine all the time and that's why I I wanted to
- 34:41mention. I mean, Unity did have their
- 34:42whole. Shenanigans happened recently,
- 34:44but I I I was just curious because a lot of the virtual
- 34:47reality development is done on the Unity side as well.
- 34:50So that's why I was curious to see what you guys use.
- 34:53I was laughing at myself because I saw your video recently when I
- 34:56was scrolling through your reels where you did a joke at the
- 34:59start and said this on Unity and then you said only joking.
- 35:02Unreal. So I love that video but and
- 35:07I've I've always I've always wanted to use Unreal for stuff
- 35:10because but it's basically driven by my CTO.
- 35:13His main focus was Unity and so he decided to do all of the
- 35:18stuff that we were doing in Unity and then it continued and
- 35:21continued continued. We made River Link in Unity and
- 35:24we did River investigate and we've also done River CSI in
- 35:27Unity. So we've got the.
- 35:30CSI fucking cool. And you just you just breezed
- 35:34back Fire. What is River CSI?
- 35:36It wasn't on your website. Sounds like the TV shows on your
- 35:39website. Yeah, good.
- 35:40I've it's good that you've been checking the site.
- 35:42But so CSI is the new product we've been working on that is
- 35:46basically a sandbox crime scene that allows you inside virtual
- 35:50reality inside a headset. You can in a stand alone
- 35:54headset. You can go into a crime house
- 35:58and you can lay a body on the floor, move the arm, drip some
- 36:01blood, place a gun on the floor, press save, call your friends in
- 36:06and do a crime scene investigation in VR in a
- 36:09headset. So that is River CSI.
- 36:11You guys are fucking making games right now.
- 36:13This is. This is totally a video game.
- 36:15Super interesting, but that that came out of that was born out of
- 36:19me visiting Derby crime house. So in Derby, the university
- 36:24there they have a whole house, an actual 4 bedroomed house that
- 36:28has cameras in every room. And the forensic students walk
- 36:33in to a crime scene that's been pre made.
- 36:36And the the students are watched by the instructors on cameras to
- 36:40see if they're doing fingerprints properly, to see if
- 36:42they're noticing the blood splatter on the walls.
- 36:45And basically all we've done is lift that and stick it into VR
- 36:49and allow them to have different weapons, different houses,
- 36:52different bodies. Are you going to use that for
- 36:54training police officers or is that something that you guys
- 36:57want to work with local police authorities with that?
- 37:00Yeah, it's for training police officers for forensic
- 37:03investigation. And it's also for universities
- 37:06and schools, because there's a lot of students that are going
- 37:09through that process now. They don't have to use a crime
- 37:12house. They could all just be at home
- 37:13in the crime scene doing the investigation.
- 37:16Everything's the same. You can pull down a camera, take
- 37:19photos, you can lay evidence markers, you can do dusting for
- 37:23fingerprints. It's all the things you need to
- 37:25do in a crime scene. But eventually I want to be able
- 37:28to do an immersive documentaries inside that.
- 37:30So you'd watch half of a 2D documentary on Netflix, then you
- 37:35put the headset on and go into the crime scene, investigate
- 37:37yourself. All right.
- 37:39I think bad Decisions and River needs to work together and
- 37:42things, because we do. I mean, all we're about is
- 37:44immersive storytelling, right? I mean, I don't know if you saw
- 37:47the Egyptian documentary. We created literally the same
- 37:49concept. We've never been there,
- 37:50recreated the entire tomb based off of photos and vlogs we found
- 37:54online. And all we wanted to do was give
- 37:57people an experience of what it's like to be in King Tut's
- 37:59Tomb because we couldn't be there ourselves.
- 38:01So we were like, you know what? This is the only way we can do
- 38:03it now. And we were surprised as to how
- 38:05good it looks. And when we, as a test, put that
- 38:08entire scene into VR, 'cause you can do that with Unity and
- 38:11Unreal Engine, that's the beautiful part.
- 38:13And when you were inside a tomb, I remember I put the headset on
- 38:16Farhad's head 1st, and I was just controlling the computer.
- 38:19And Farhad was like, Oh my God, wow, this is so cool.
- 38:22I'm like, OK, stop overacting. Like, OK, I get it.
- 38:24We've seen this tomb for the past four months.
- 38:26We've been working on it. And then I put the headset on.
- 38:29It's a completely different experience, especially now in
- 38:322023 when you have dynamic lighting and shadows working in
- 38:36real time. Just everything looks gorgeous
- 38:38and I I totally understand. And now I want to say something
- 38:41as well. So I come from a video gaming
- 38:43background. I was in the esports world and I
- 38:45used to stream this game called Phasmophobia.
- 38:48It was a horror game that was extremely famous.
- 38:52What would happen is you would start as four guys or girls in a
- 38:55truck outside of a haunted house and all of you in that truck had
- 39:01tools that you would pick up and there was limited tools.
- 39:03So you had flashlights, fingerprint scanners, you know,
- 39:06different cameras. And then you would go inside the
- 39:09house and you would look for the ghost.
- 39:11And I remember that game was extremely fun because.
- 39:15First of all, you're working in teams and you were judging each
- 39:18other's work. So if someone had the camera you
- 39:21would tell them yo, you're not looking the right direction, you
- 39:23have to look to the other side. If I had the flashlight, I had
- 39:26to look towards a certain, you know, angle and that is
- 39:29literally what you're describing right now.
- 39:31But instead of a game that is just made for fun, it has real
- 39:35life use cases and I remember that game.
- 39:37Was so realistic. So I can only imagine if you put
- 39:40that in VR and I I I assumed there was AVR version to it that
- 39:44I didn't play. But it makes perfect sense that
- 39:46you would go on and do River CSI because you can totally have the
- 39:50exact same tools with the exact same look, work the exact same
- 39:54way and the people who are using these tools from their home,
- 39:59these are students who might not even have money yet to go into
- 40:02university. They can actually get your
- 40:05application, your software, and start training for the real
- 40:08world. So when they go and want to get
- 40:10hired, they already know how to do the job.
- 40:13Like this is crazy. It is.
- 40:15Totally. And it it's amazing that you
- 40:17understand it in that detail because there's a lot of people
- 40:20that we have to, like you said, go through a process of them
- 40:22understanding what that means for them.
- 40:25If I say River CSI, they don't imagine that it is the same as
- 40:29their crime house. But it's very it is different to
- 40:34the CSI, is very different to investigate river, investigate.
- 40:38And because the fire investigators always have in one
- 40:41burnt room, there is one causation of fire.
- 40:44So it might be the plug that has set on fire, and there's always
- 40:47that it can't change. But the crime scene people want
- 40:51things to be different. They want dynamic scenes.
- 40:53They want to be able to place the cup down, place the knife
- 40:57down in different locations. And that was the reason it had
- 41:00to be a different piece of software, because the fire
- 41:03scenes have a specific causation and the crime scenes need to
- 41:06have dynamic environments and change things around.
- 41:09But I love it when you can go in and pick up the.
- 41:12Persons hand and place it differently on the sofa and then
- 41:15drip some blood on them. And this is all going to be in
- 41:18mixed reality as well. So imagine the whole of imagine
- 41:21the whole crime scene, crime scene investigation inside a
- 41:25headset inside a house. Now imagine no house, but it's
- 41:29in your house. Now you can lay the body on your
- 41:32sofa. That's some freaky shit.
- 41:35You got like a dead body on your dining table and you're like
- 41:39doing. Much of it.
- 41:40And your mom walk in and they're like, what are you doing?
- 41:42Nothing. Just inspecting a dead body
- 41:44here. Don't mind me.
- 41:47This is this is getting really next level.
- 41:49But you guys have made so many experiences and I want to ask
- 41:54you, what is the most exciting and the coolest experience that
- 41:57you guys ever made? We saw the police chase that was
- 42:00quite interesting, but which one do you think is the best that
- 42:04you guys have made so far? I wonder if it needs to be split
- 42:06into 360 video, 3D3D video and then a room scale scene because
- 42:11there's so many differences, yeah.
- 42:13We can do that, yeah. Yeah.
- 42:14Let's look at both categories. Okay, so if we're gonna say
- 42:173D183D180 video that we've made. Was definitely inside the Tesla
- 42:23Fremont factory, being this close to the machines and
- 42:27putting the camera this close to machines and then showing it to
- 42:30the tester employees the next day.
- 42:33And they actually said to me, we don't even get this close.
- 42:36We literally got the, we literally got the camera closer
- 42:39than you can get on a tour of the factory.
- 42:41So you one, you can never go in the factory anyway and have a
- 42:44tour, but two, you can get closer than you can on a tour.
- 42:47So that was probably the 3D. 3D180 that was probably one of
- 42:51the best for that. The the 360 car chase was a good
- 42:55one, but we also did riot police training in London, so we were
- 43:01in amongst horses and riot police and Shields.
- 43:05That was also good for filming. But I think probably the the
- 43:09most exciting 3D scene as in photogrammetry scene was
- 43:13probably we blew up a car, we blew up.
- 43:18We very. Loaded explosions into a car in
- 43:22the government and we we scanned the whole scene post blast as a
- 43:27photogrammetry scene. I've never experienced that sort
- 43:30of thing. That again, we can't really talk
- 43:32about it. It was a private scene for the
- 43:34government. But we don't get, we don't get
- 43:36to show it. Dude, can we have bad decisions,
- 43:39invited to some of these cool experience or fuck sake.
- 43:42I mean, blowing up a car seems so cool.
- 43:44I would love to see that. Were you there?
- 43:47So you were present at the time of the explosion?
- 43:49Yeah, it was. There was about four or five of
- 43:51us there. We've got a video of it.
- 43:53Yeah. There's 3D video, 360 video and
- 43:55a photogrammetry scene. And that's how my brain thinks
- 43:57about stuff. So I always think of. 2D
- 44:01documentary, 360 video 3D180 and then a photogrammetry scene.
- 44:05And that connection of those four pieces is what I call Step
- 44:09Inside. And that's my, I don't know, we
- 44:12apply it to people's projects that we can do, but some
- 44:16people's projects we can't do. And so the the most exciting one
- 44:20where we did combine all of those was a project in Warwick,
- 44:24which is my. Hometown where there's a castle
- 44:26and a church and it's I love scanning things there.
- 44:29That's where I'm thinking I want, I want to speak to Netflix
- 44:32more. I'm always pestering Netflix
- 44:34because they have AVR app on the store and it is literally you
- 44:39sit on a sofa and you watch their 2D content on a screen in
- 44:43front of them. Now I want that content to just
- 44:45be a wrap around 360 and then a room scale experience like you
- 44:49can do it now. But we need to maybe clip this
- 44:51bit out for Instagram, Netflix, to watch this podcast, need to
- 44:56understand that you and we can make an immersive photogrammetry
- 45:02scene. All we need is access to one of
- 45:04their IP. And I'm using this as a platform
- 45:06to shout at Netflix. Because if you know.
- 45:10Have you seen the staircase on Netflix where it's a crime
- 45:13documentary? Or maybe Game of Thrones?
- 45:16You've obviously seen Game of Thrones.
- 45:18You could watch, you could watch the 2D Game of Thrones inside
- 45:22AVR headset and then at any point in time you can just go to
- 45:26360 or 3D 180. You could be behind the scenes
- 45:29watching it be made in 3D180. Then you could be stood on the
- 45:33wall with Jon Snow looking at Jon Snow and looking over the
- 45:37wall and seeing White Walkers charging at the wall like.
- 45:40This is all possible two years ago.
- 45:43They don't even know about it now.
- 45:45I want to do it tomorrow. You know what?
- 45:48I love that. And I think that is slowly
- 45:51becoming more and more apparent to people.
- 45:53And that is why we create the content we do, because we what
- 45:57we do with our content is we always try to find use cases for
- 46:00it. Right.
- 46:01When we found out about Gaussian splatting, which we're going to
- 46:03get into in a moment, which is for anybody that doesn't know
- 46:06about this term, which you should buy now because we had
- 46:09seven. Videos about it, about it for
- 46:12like the longest time, but it's basically a way to capture the
- 46:15real world and reconstruct it in the most efficient and photo
- 46:18realistic manner. And it is extremely efficient to
- 46:22view at the same time. So just one of those is a good
- 46:26example because we tested it multiple times with different
- 46:30scans and then we started taking it to movies.
- 46:33We we literally took a movie from 43 years ago.
- 46:36You saw The Shining and we took that camera that was one shot
- 46:40moving to the right side. And then now we have other
- 46:43camera angles 43 years later that we can use.
- 46:47Not that I want to change what the movie was, you know, I don't
- 46:50want to change the movie. The movie is perfect.
- 46:52But I'm just saying, dude, this is possible with the movie made
- 46:5543 years ago. Imagine what's possible with
- 46:56something that was shot yesterday.
- 46:58Imagine. That they put, they put in every
- 47:00single movie scene. They say they tell the director,
- 47:02hey, make sure there's the 360 camera in the in the scene so
- 47:05you guys are recording everything, even if it's a if,
- 47:07it's even if it's a low budget film and they don't even have a
- 47:10360 camera. Every location you go to, that
- 47:13is going to be valuable to your audience, the hardcore audience,
- 47:17just do, just ask someone with their iPhone, Just go ahead and,
- 47:19you know, scan a 360 and it's done.
- 47:21It's literally that easy. And then you can share this
- 47:24content with all the hardcore fans who usually go and get the
- 47:27books, go, you know, go get all. The Fisher Jerk, The
- 47:30collectibles from that. They would love to be able to go
- 47:33inside each scene in 3D. Even if it's walking around the
- 47:36set, walking around the three-dimensional photogrammetry
- 47:39set of a film, it's just be behind the scenes and you could
- 47:42have a little look around. But what's more frustrating,
- 47:44I'll give you a bit of info that's even more frustrating
- 47:47than I heard a year ago, is that most films that are shot, and
- 47:51especially the new season, OK, one of the seasons of Game of
- 47:55Thrones they did this on. I can't talk too much about it.
- 47:59They. Take a photogrammetry rig and
- 48:02they put each of the cast into the rig.
- 48:04Every time they do a shoot, they T pose the cast and take a
- 48:08photogrammetry scan of them in costume.
- 48:11And they only use it for continuity and a little bit of
- 48:16VFX. That blew my mind because they
- 48:19also LIDAR scan every set, so they they already photo
- 48:23everything. They have it all, but they don't
- 48:25use it for anything immersive. They only use it for VFX and
- 48:29green screen stuff to fix. Basically to fix errors,
- 48:32Essentially, right? To fix errors that might have
- 48:35happened during the short they want to change something and
- 48:38like on a minuscule scale, not to use it to some extent of
- 48:42actually promotional material. To go above and beyond and and
- 48:46give the fans what they could actually want.
- 48:48Yeah, imagine. Imagine chucking that at me and
- 48:51you and saying OK, you've got a month to go and make something
- 48:54in VR. With all this content, you could
- 48:57come up with something ground breaking within like 2 weeks and
- 49:00that's that's the power. Yeah, that's the power of it.
- 49:04That's the power of it these days.
- 49:05And people don't understand what's possible with stuff
- 49:08they've already been scanning. And it's so cool watching your
- 49:12shining shot, because it is. You're making camera angles that
- 49:16they never even created. But, but most of the modern
- 49:19films now have all of this data and they just don't use it for
- 49:23anything like this. Yes, and I hope that changes.
- 49:26And you know what, you mentioned that about Game of Thrones.
- 49:28I remember precisely what it was like being in that world, those
- 49:32couple of years where Game of Thrones, especially season four
- 49:34or five, those those perfect times every weekend, the entire
- 49:38world was watching the series. So imagine you drop something
- 49:41like that at that time. Everybody would go on.
- 49:44Everybody's going to freaking get it.
- 49:45Just jumping in on that. Imagine that if you're watching
- 49:48Game of Thrones now, like a new season's come out.
- 49:51You're sat on your sofa, you look to your right, you've got a
- 49:53Quest 3 sat next to you just on the sofa and you've got Netflix
- 49:57on your big screen. You've just watched the episode
- 49:59and right at the end a little QR code pops up on the screen.
- 50:03It says put on your VR headset. Now you put on the headset and
- 50:07what if the pass through camera scans that QR code and dumps you
- 50:10into Winterfell? And you're just there in the
- 50:14castle, photogrammetry straight away, like everyone would just
- 50:17be on the floor. They wouldn't understand what's
- 50:19happening, but it's possible today.
- 50:22Well, isn't that, well, at least something that the Apple Vision
- 50:26Pro is sort of promising? I mean, one of their key
- 50:30features, I guess in the advertisement, in the
- 50:33commercial, in the keynote that they had was, I believe was it a
- 50:36kid sitting in The Mandalorian? I I forgot what it was.
- 50:40But it was essentially showing you that you can now go ahead
- 50:42and immerse yourself. Yeah, immerse yourself in TV
- 50:45shows. I'm really looking forward to
- 50:47see what Apple Vision Pro has to offer.
- 50:49Of course, all most of these things that they were offering,
- 50:52It really depends on the creators.
- 50:54And by by creators, I mean Disney and all that because I
- 50:57don't think they were making them in house.
- 50:58I don't think Apple's making these experiences.
- 51:00I believe it's the other companies that have to create
- 51:02these experiences, these shows, and make sure that Apple has
- 51:05enough. Just like when the iPhone came
- 51:07out, you had people creating Doodle Jump and people creating
- 51:10the lighter application or the beer app that you would drink.
- 51:13So they need to have all these people creating for them, which
- 51:16I'm hoping it will happen, but it will be a slow process.
- 51:18But I assume, and I hope it will, What is your take on it?
- 51:21I mean, you're in the virtual reality world.
- 51:23You're clearly very well, worse in that world.
- 51:27I want to know what's your opinion on the Meta Quest 3:00
- 51:30and also Apple Vision Pro. Where do you think all of that
- 51:32is going? I mean, we've tried the pass
- 51:35through on the Method Quest 3 and it's stunning compared to
- 51:38the Quest Two and compared to all the other bits, but I'm
- 51:41imagining that the Vision Pro is going to wipe the floor with all
- 51:44of them. I was in Cupertino crying when
- 51:48they launched the headset. Literally makes me emotional
- 51:53now. The fact that we've been in VR
- 51:56for so long waiting for Apple. I've been in VR so long waiting
- 51:59for Apple to drop something, that I was there.
- 52:02I was only in the visitor center.
- 52:04I was in the visitor center watching it on a laptop with my
- 52:07team and I was just crying because.
- 52:10The validation that that gives the world is crazy.
- 52:13Like I no longer and the Facebook doing glasses and Apple
- 52:17doing a headset. Like I never have to fight about
- 52:21it ever again. Whereas I've been trying to
- 52:24convince people that this is the next step, this is the next
- 52:26medium, It's the last medium. I've been doing it for so long.
- 52:30It was so emotional. So I think it's going to be
- 52:32great. It's going to be expensive.
- 52:34People are going to moan about it being expensive, but they
- 52:37already are, yeah. When you put it on your.
- 52:40Your head. And there is a ground baking
- 52:42experience, like when you first did pinch and zoom on an iPhone,
- 52:45or when I first did eye tracking inside a Vario headset.
- 52:50I felt magic. It felt like magic.
- 52:52And if they're gaze to click and the pastor is so stunning.
- 52:59People are going to find it as magic.
- 53:01And when those experiences start coming that start making things
- 53:04burst out of your TV onto your front room floor, then people
- 53:08are going to understand why that's going to be the next
- 53:10thing and you're going to want to have step inside everything.
- 53:15That's very true. You know, the concept of the
- 53:17validation of Apple, that's an actual thing.
- 53:20Whenever Apple does something, everybody else follows.
- 53:23And everybody thinks that. You know what?
- 53:24It's time. You know, it's time for this.
- 53:26That's what they did since the beginning.
- 53:28That's what they're doing now. And you know, when Apple comes
- 53:31into a new market, you know, they've done their research,
- 53:34they've waited many years to make sure this is the right next
- 53:39thing. And clearly virtual reality is
- 53:41that. And I, I mean, Farrad, are we
- 53:44going to get one? Probably because Farrad's like
- 53:47the financial manager of bad decisions.
- 53:49I need to ask Farrad, are we going to?
- 53:52We have to think it's coming out in February, right?
- 53:54Is it February? I think I know a bit about that.
- 53:57Yeah, but you could make an experience.
- 53:59I'm sure that would make you get a dev kit.
- 54:02I'm sure you could do something and they could send you a dev
- 54:05kit. We met with the guys at Apple
- 54:06the day after launch and we still haven't got a dev kit.
- 54:09But even just even just the shining thing.
- 54:12Like imagine that in Vision Pro. They need to know the things
- 54:16that you're doing. Your videos and your content are
- 54:19absolutely what's needed because the attention span of people
- 54:23wanting to find out about splats and nerve and PG photogrammetry.
- 54:28It's really short. People need to just have your
- 54:30videos fed to them. I love going through your feed
- 54:33and just having like, an explanation of like a nerf in
- 54:37such an articulated, passionate way is just really powerful.
- 54:41So I think let's get Apple to see your videos and then we can
- 54:45both. We will.
- 54:47Love that. Thank you, Alex.
- 54:49I appreciate that, man. I appreciate that.
- 54:51There there is one concept that we want to discuss with you and
- 54:53because you are very much into photogrammetry but what we see
- 54:57in the videos you guys. Guys, 3D model certain items
- 55:01that you guys have in your scene.
- 55:02What is your go to strategy when you're choosing between
- 55:06photogrammetry or modeling? So when do you choose to
- 55:09photogrammetry an object or where do you say, Oh no, this
- 55:12is, we have to model it and put it in the scene.
- 55:15I always say if it's if it's super complex like a tree and a
- 55:21load of leaves and grass. It is quicker to use that as a
- 55:25scan photogrammetry scan than it is to model, so it's about the
- 55:29speed. If you can go and get that
- 55:31specific thing really quickly with your phone, I suggest doing
- 55:34that. But also if it's for example an
- 55:37artifact that definitely needs to be scanned and can't be
- 55:40recreated, then I would say scan it.
- 55:43If things are like I need to make a ball or a square.
- 55:47Or an item that's gonna just be 1 shape, then it's probably
- 55:50quicker to model it. I assume something that's
- 55:52interactive, because if you have like buttons that need to be
- 55:55pushed in one way or another, I think that's when you kind of
- 55:59have to go from modeling rather than photogrammetry.
- 56:02But what about different methods of scanning?
- 56:05Cuz we were discussing this with Farhad, I mean you had your
- 56:08traditional way of doing photogrammetry, which by
- 56:10traditional I don't mean old, I just mean it's been there for a
- 56:14while now and it's been there for a while.
- 56:16Because it works and it works perfectly and you have
- 56:18applications that are as accessible as Polycam to
- 56:22applications that are professional grade, like reality
- 56:25capture, where you can, you know, throw in thousands of
- 56:28images and and get the perfect model out.
- 56:31But of course, photogrammity has its own advantages and
- 56:33disadvantages, which is why we've been experimenting with a
- 56:36lot of nerves and now recently Gaussian splatting.
- 56:39How do you see that changing the business model for you
- 56:43specifically? Because with Gaussian spots now
- 56:45you can get plants to absolute perfection, right?
- 56:48You can view the the scene that is extremely complex in hundreds
- 56:53of FPS, and all these things should come into play when
- 56:56you're taking it to VR as well. Very experimental now, but how
- 56:59do you see that changing your model?
- 57:01I feel like we're gonna in the same way we have a video camera
- 57:05for people to make their own 360 content for Riverlink.
- 57:09These nerves and splats are going to allow people to make
- 57:12their own photogrammetry scenes. It just happens that we used to
- 57:16do it for them, but it's going to come to the point when we
- 57:18just provide the software. We just provide the world for
- 57:21them to import their things into.
- 57:24We've actually made our own scanning app that we use
- 57:27internally to do our own phone scans quickly.
- 57:31Yeah, it's the same. We're using the Apple objects
- 57:35thing in the background to make our own models.
- 57:38We found that to be quicker and a bit more accurate.
- 57:41But yeah, I think the splat stuff we tried nerf to VR quite
- 57:45a bit and we tested that, but it was very intensive on like
- 57:51viewing things inside it. So I've got to say that in house
- 57:55we haven't actually tried much splats to VR.
- 57:58Like it hasn't been a thing that they've got enough time to work
- 58:01on yet. But as soon as that becomes a
- 58:03thing. Imagine someone going through a
- 58:06road traffic collision and videoing the scene with their
- 58:09phone, and now uploading that to the cloud, and then someone back
- 58:13to office. The headquarters of the police
- 58:16can now put a headset on and walk around the scene.
- 58:18It's just going to make it more accessible, yes.
- 58:21And one other aspect of it for me and Farrad has been the the
- 58:26how quick it is to capture a Gaussian splat.
- 58:29Because with photogrammetry you need to, you know, be extremely
- 58:33careful, the number of photos matter and the detail matters.
- 58:37Whereas with Gaussian splats, what's really amazing is you
- 58:40might only have 10 seconds to take that shot, maybe the crime
- 58:44scene or whatever scene. Especially in emergency
- 58:46services, you might have the person there for 10 seconds and
- 58:50then you know what? Maybe they have to alter the
- 58:51scene. They have to change something.
- 58:53Or maybe it's actually someone's life they're saving.
- 58:56Someone else is taking a shot right before all of that is
- 58:59happening. For me.
- 59:01A Gaussian splat is really easy to do in that scenario because
- 59:05literally a 10 second video for us has resulted in amazing
- 59:09scans, whereas with nurse and Photogrammity, we couldn't get
- 59:12that prior to it. Of course, again, it has its
- 59:14limitations. We tried actually bringing it to
- 59:17VR fire. You want to tell I mean.
- 59:19I think we were trying it for the past two weeks.
- 59:21For the 1st 10 days we couldn't see anything with our right eye.
- 59:25Yeah, that was a bug. Everything that you could see
- 59:27was from the left one, so and then the problem is solved now.
- 59:31But I would say this is just two months after the paper is
- 59:34released. So this is very, very new.
- 59:36Yeah, I think the application will come, I will say in a
- 59:39couple of months, but the, the quality that we can get with
- 59:43Gaussians even in VR is quite crazy.
- 59:45Yeah, I mean it's it's efficient to view as well and that's a
- 59:47plus. I agree with you when you talk
- 59:49about nerves because they are, they can become quite intensive,
- 59:52especially if you put them through VR with Gaussian splats.
- 59:56And I see this only speaking from us using it in Unreal
- 1:00:00Engine. And Unreal Engine only has one
- 1:00:02plugin at the moment, which is developed by a Japanese guy
- 1:00:05which we've gotten in contact with.
- 1:00:07But it's hard to speak to him because he's a one man trying to
- 1:00:11do all of these at the same time.
- 1:00:13So I understand that he's extremely busy.
- 1:00:15He's constantly updating the plugin where the plugin had a
- 1:00:18bug with the VR experience. Now he's fixed that.
- 1:00:21It works. It looks, it looks freaking
- 1:00:23stunning. We went to the scene for
- 1:00:25Inglorious Bastards and that was the coolest moment.
- 1:00:28We cropped the scene, put a point light in the middle where
- 1:00:31the two characters are there. That's the opening shot as well,
- 1:00:34right? Yeah, yeah.
- 1:00:34And I'm literally just behind their head and I'm just
- 1:00:37watching. I'm like okay, this is
- 1:00:39phenomenal. But it still needs some work
- 1:00:42because that plug in is using Niagara particles.
- 1:00:45There are slight flickerings here and there, but generally
- 1:00:48speaking it works fast, it works efficiently and it's photo
- 1:00:51realistic and I can imagine this just getting better and better.
- 1:00:55And Speaking of this, I want to move on to one more topic that I
- 1:00:59wanted to discuss. You posted about this too Lex
- 1:01:02Friedman podcast with Mark Zuckerberg with Kodak avatars
- 1:01:07that was done on a Quest Pro headset.
- 1:01:09For anybody that doesn't know what we're talking about, please
- 1:01:12go and watch this. It's basically Lex Friedman and
- 1:01:15Mark Zuckerberg in different rooms, different parts of the
- 1:01:18world I imagine as well. And they had the Quest Pro
- 1:01:21headset on and they were in this dark room, but they could
- 1:01:24technically see each other. They had photo realistic avatars
- 1:01:27that looked probably 99% accurate to their own look with
- 1:01:31their nuances of facial emotions, and it was the
- 1:01:36craziest thing. It was exactly something you
- 1:01:38would see in a cyberpunk film. But it's now possible it's just
- 1:01:42not something that people can use because as Mark mentioned,
- 1:01:46you still need the professional Rick to do the capturing of the
- 1:01:49data. But once that is done, anybody
- 1:01:52with a Quest Pro headset can actually get into that
- 1:01:54experience. How do you think that's going to
- 1:01:56change the world, especially for everybody and for you as well?
- 1:02:00Because again, you have students, 25 people sometimes
- 1:02:04who are getting into this experience.
- 1:02:05Now imagine all of these students are seeing each other
- 1:02:08in that experience. What do you think about that?
- 1:02:11Because you posted about it, I wanted to get your thoughts.
- 1:02:13Yeah, it was another video that made me cry like that.
- 1:02:17You cry a lot. Man, honestly, I do.
- 1:02:21I literally cry a lot. It is an emotional thing that I
- 1:02:24am connected to that I was so pleased to see because if you
- 1:02:29watch the the Oculus releases of their, well, I forget what
- 1:02:34they're called, their Connect conference.
- 1:02:35Every year for the last two years you see progress in this
- 1:02:39space where you see their avatars.
- 1:02:41Being developed over the years and then you see them putting
- 1:02:44cameras onto headsets, looking at mouths and then you know
- 1:02:47what's coming and then if you if you well, when we saw that it
- 1:02:51was stunning. But then imagine forwarding that
- 1:02:54to what Apple are going to do. You saw their trailer where they
- 1:02:57showed you FaceTime with Vision Pro.
- 1:02:59That's not what FaceTime is going to be like.
- 1:03:01Like FaceTime eventually is going to be headset to headset
- 1:03:04in the same way and that's why Meta dropped that.
- 1:03:07Meta did that so they could beat.
- 1:03:10Apple to the punch, like basically did that, showing that
- 1:03:13they're a little bit ahead with stuff like that.
- 1:03:16But Apple are trying to push the fact that FaceTime from FaceTime
- 1:03:21from an iPad to a headset will be really cool.
- 1:03:24That'll look nice, but the future will be headset to
- 1:03:26headset. And then I imagine that everyone
- 1:03:29using a phone will be able to scan their face.
- 1:03:32You know you'll be able to do that yourself.
- 1:03:34You can do that quite well now. We were doing that three years
- 1:03:38ago and putting people into rigs and we did snooker players for a
- 1:03:41snooker game. We scanned their heads using
- 1:03:44photogrammetry and although Meta say it's a big process.
- 1:03:49I think it's only a big process at the moment because they have
- 1:03:52to put themselves into a big rig and do lots of facial, different
- 1:03:56facial movements and capture that in all the different types
- 1:03:59of lighting. But yeah, it was done on Quest
- 1:04:02Pro. There was a cable plugged in so
- 1:04:04you could see that they were probably tailored to APC.
- 1:04:07It wasn't stand alone, but I imagine it will be stand alone
- 1:04:10at some point. Everything is just converging at
- 1:04:13the moment to this. Like people use the word
- 1:04:17metaverse, but everyone's going to want their own version of the
- 1:04:20Metaverse for their own person and their own brand.
- 1:04:23It's converging to some amazing point to the moment that is.
- 1:04:27It is an emotional thing for me. I think if we can scan
- 1:04:31everything and make anything immersive, then it's going to
- 1:04:34be, it's going to be the last medium like there's going to be
- 1:04:37nothing else that the until we get to brain chips.
- 1:04:40Yeah, I was gonna say that neural link is probably the next
- 1:04:43medium after that. Right.
- 1:04:44So shall we say the next podcast that we do together probably
- 1:04:47would be with a headset using PC.
- 1:04:50So we are putting the Apple, Apple Vision Pros on and then we
- 1:04:54are gonna do a podcast together. Probably, definitely.
- 1:04:58Yeah, let's do that. I imagine that would be amazing.
- 1:05:00We've done it already inside Quest and I've actually done one
- 1:05:03in the back of a car in Texas. I was connected to the Wi-Fi in
- 1:05:07a headset and I was doing AVR thing with Inside Engage VR.
- 1:05:11That was a really that was a couple of years ago.
- 1:05:13So if we can do a Vision Pro One, that'd be amazing.
- 1:05:16Or a Meta Quest 3/1. I would freaking love that.
- 1:05:20I have one last question for me. I don't know about Farhad.
- 1:05:23I want to know if there's anything in the virtual reality
- 1:05:26or extended reality world that you've heard about that you
- 1:05:29might have seen that we haven't talked about.
- 1:05:32Is there anything interesting that has gauged your interest
- 1:05:35maybe at a conference, at an event that you've been to?
- 1:05:38I mean, we did, we did talk about, you know, the the Vision
- 1:05:40Pro, the the Quest 3. But is there anything else that
- 1:05:43is going on that maybe we haven't heard of yet?
- 1:05:46There was a. There was a.
- 1:05:48The podcast recently by Robert Scoble where he was interviewing
- 1:05:51a guy from Rewind dot AI. That was a really cool piece of
- 1:05:55tech where they have a little pendant around your neck and an
- 1:05:58app that connects to your screens so that you can
- 1:06:01basically rewind time. So I could say in a week's time,
- 1:06:04tell me the top points we talked about on the Bad Decisions
- 1:06:07podcast and it just goes and gives me a nice little list of
- 1:06:10everything. That's one thing, but the other
- 1:06:12thing is that someone mentioned about Imagine the whole of
- 1:06:17YouTube. Has immersive experiences in the
- 1:06:20same way you're doing The Shining.
- 1:06:22Imagine every video that gets run through an algorithm that
- 1:06:26then gives you a 3D interactive version of that video that is
- 1:06:31going to be. Bonkers.
- 1:06:33You know, I don't know how that's going to fit in with
- 1:06:35stuff because there's going to be a lot.
- 1:06:36It's going to make sense because think about it like AI, if you,
- 1:06:40I mean we haven't had the time to watch this, but we've heard
- 1:06:42about it, we've read about it. Adobe just had the conference
- 1:06:45which was AI, AI, AI and they talked about all of their
- 1:06:48different tools that are adding AI will be able to generate
- 1:06:52missing detail. And it's going to do that so
- 1:06:55well that all of the 2D videos that you just said that we we
- 1:06:59might not even know how to fix it.
- 1:07:01Well, AI is going to do the rest.
- 1:07:02I imagine that's what's going to happen, right.
- 1:07:04You're going to take something shining.
- 1:07:06Sorry, I. Was going to say your shining
- 1:07:08video then where you can only go left and right and up and down a
- 1:07:11little bit because of the shot was only that way you're going
- 1:07:14to be able to go behind the house.
- 1:07:16There you go, Yes, because it's going to find out that that's
- 1:07:19the texture of the house, then it's going to rebuild it,
- 1:07:21because if it's already doing that with Photoshop images, I
- 1:07:25assume that with videos it's going to be possible.
- 1:07:28I think videos is actually possible now because that's what
- 1:07:30they are. Now, in painting with you, we
- 1:07:32should watch Adobe Max. Yeah, we should watch.
- 1:07:33Did you see that one? The Adobe Max conference?
- 1:07:36Not yet. You must find the same thing,
- 1:07:38that there is too much to listen to at the moment.
- 1:07:40There's too much. Which is why your video should
- 1:07:44be watched more, because they're nice bite sized videos.
- 1:07:46Thank you. No, it means a lot.
- 1:07:47Farah, do you have any? No, I think it was amazing
- 1:07:50talking to Alex and we we wanted to have this conversation for
- 1:07:53the longest time. I think we we started texting
- 1:07:56each other. It was three or four months ago
- 1:07:58and thank you very much for making this happen.
- 1:08:01And we cannot wait for River CSI.
- 1:08:04I personally cannot wait for River CSI.
- 1:08:06Please put videos of that when the demo comes out.
- 1:08:08No more private and NDA. No, don't sign any NDA.
- 1:08:14Get some demo out, Get some demo out.
- 1:08:16We want to see it because you can turn that into a video game.
- 1:08:19Actually, video games, OK, video games are what what video games
- 1:08:24are, essentially, they're interactive movies, in my
- 1:08:26opinion, right? They they're movies you can
- 1:08:27interact with. There's storytelling involved
- 1:08:29and as long as you're creating that experience and you make it
- 1:08:34fun for because OK, fun means different things for different
- 1:08:38people as well. For someone fun is looking at a
- 1:08:40dead body and inspecting why that person is dead, right?
- 1:08:43That is fun for them. They can do that every single
- 1:08:45day of their life. Not to say it's fun to see a
- 1:08:47dead person, but it's fun to figure out puzzles.
- 1:08:50That's what I'm saying. And so if you can make that in
- 1:08:53Unreal Engine, I'm just pointing you towards that software and
- 1:08:56it's photorealistic and it's I'm just kidding if it's if it's
- 1:08:59made in Unreal Engine or Unity and it's photorealistic and it's
- 1:09:03you know a a puzzle that people have to solve that sounds like a
- 1:09:06game to me already And so I I would I would like to try that
- 1:09:09experience. I'm just.
- 1:09:10Saying let's just make sure Netflix and also probably Black
- 1:09:14Mirror. Let's make sure Black Mirror and
- 1:09:15Netflix see this sort of thing. I I I think that would be
- 1:09:19awesome. Well, Alex, thank you so much.
- 1:09:21We appreciate you giving us your Sunday and hope to catch up with
- 1:09:24you soon again. It's amazing.
- 1:09:26The best Sunday podcast I've ever done.
- 1:09:29Thank you so much Alex. Thanks to everyone who is
- 1:09:31watching and listening. See you guys next week.
- 1:09:33Ciao.