Latest / The Occult Rejects / The Unreasonable Effectiveness of AI in Computer Vision with Ashish Part 1

Transcript

The Unreasonable Effectiveness of AI in Computer Vision with Ashish Part 1 · The Occult Rejects

• 0:00
  Speaker 1: Welcome to the Occult Green Jacks. Tonight, we've got a
• 0:04
  Speaker 1: very special guest, Ashih, who is going to tell us
• 0:07
  Speaker 1: a lot of really deep intelligent stuff about AI and
• 0:12
  Speaker 1: all all the rest of the stuff. He'll get into it,
• 0:15
  Speaker 1: but first let's go to Nick. Nick, how you doing
• 0:18
  Speaker 1: what you got coming out? Tell us about your herbal
• 0:21
  Speaker 1: video that just popped out?
• 0:22
  Speaker 2: Oh, thank you, thank you?
• 0:23
  Speaker 3: Yeah that it seems like I seemed to be getting
• 0:26
  Speaker 3: like you even mentioned good reviews are right, ab, we'll
• 0:27
  Speaker 3: getting some good comments.
• 0:29
  Speaker 2: It's a different way of looking at it.
• 0:31
  Speaker 3: I do cover the like kind of the occult angle
• 0:33
  Speaker 3: of the witchcraft angle, maybe even like a little bit
• 0:36
  Speaker 3: of history, like I forgot what it might have been,
• 0:37
  Speaker 3: peppermin or something, spearman something I was mentioning, like even
• 0:40
  Speaker 3: I think in Greek Greek history of Greece they'd wear
• 0:43
  Speaker 3: it around their heads, is like to help them study
• 0:46
  Speaker 3: or something when they.
• 0:46
  Speaker 2: Took tests, some random shit like that.
• 0:48
  Speaker 3: Like and I'm throwing all that stuff in there, and
• 0:50
  Speaker 3: then I'll go into how it actually affects your twelve
• 0:53
  Speaker 3: cranial nerves and when you saw it seeing like when
• 0:56
  Speaker 3: it's actually going to you your your you know your
• 0:58
  Speaker 3: vegas system and your uh you're glad uh if it
• 1:04
  Speaker 3: basically like the some effect like up to three at
• 1:06
  Speaker 3: a time, and that gets pretty interesting when it starts
• 1:09
  Speaker 3: doing that. You know your trigeminal, your vegas system, and
• 1:13
  Speaker 3: maybe one or two one other one.
• 1:15
  Speaker 2: But uh yeah, long story short.
• 1:18
  Speaker 3: I think it's interesting for people to see like what
• 1:20
  Speaker 3: some of these things actually do when you just inhale them,
• 1:24
  Speaker 3: what kind of effects it's gonna have on your body
• 1:26
  Speaker 3: and how you're gonna feel afterwards, and some of them
• 1:29
  Speaker 3: that I even show, Uh, if you decide to just
• 1:32
  Speaker 3: all of a sudden going crazy dosages with it, you'll
• 1:35
  Speaker 3: notice it starts to shut your system down. And I
• 1:38
  Speaker 3: start to wonder about that with maybe shamanistic rituals.
• 1:41
  Speaker 2: You know, that's why you were fumigating that stuff, you know.
• 1:45
  Speaker 3: So it's just interesting to see the kind of science
• 1:47
  Speaker 3: with these things because you start to wonder, did they
• 1:49
  Speaker 3: know this back then, because like the effects that you'll see,
• 1:53
  Speaker 3: like that would be in like Scott Cunningham's book about
• 1:55
  Speaker 3: what these herbs are gonna do for you, you can say, well,
• 1:57
  Speaker 3: that's what it's doing to you fucking physically, just from
• 1:59
  Speaker 3: hill or putting it on your skin, you know, so
• 2:03
  Speaker 3: you almost wonder like were they were still getting it twisted?
• 2:06
  Speaker 2: You know what I'm saying.
• 2:07
  Speaker 3: It is still more science unfortunately with that stuff, and
• 2:09
  Speaker 3: somehow it got woo wooed.
• 2:11
  Speaker 2: You know.
• 2:11
  Speaker 3: I don't know did we lose did we lose the original?
• 2:14
  Speaker 3: I guess a cult cult idea behind it, something like that.
• 2:17
  Speaker 3: So check it out. It is getting good reviews. I
• 2:20
  Speaker 3: was a little surprised that actually has as many comments
• 2:22
  Speaker 3: as it does now. So, uh yeah, it's got about
• 2:24
  Speaker 3: three parts one drop today. We got a few other
• 2:27
  Speaker 3: ones that will be coming out. I got part two
• 2:29
  Speaker 3: of the Colors coming out. I should have Johannes Kepler
• 2:34
  Speaker 3: dropping in like two weeks. I should be editing that
• 2:36
  Speaker 3: up and that will be out. And I got into
• 2:38
  Speaker 3: my park that should be dropping any within the week,
• 2:40
  Speaker 3: I'm hoping. So yeah, we got a bunch of shit
• 2:43
  Speaker 3: coming out. I have a bunch of stuff edited. I
• 2:45
  Speaker 3: just haven't actually wrapped it all up yet. But uh yeah,
• 2:48
  Speaker 3: then we do have some stuff going forward. We'll be
• 2:50
  Speaker 3: covering the Stargate project next week. Again, it's gonna be
• 2:54
  Speaker 3: a big group of the whole cult reject team we'll
• 2:56
  Speaker 3: be here covering more of the Stargate or Gateway project,
• 2:59
  Speaker 3: PaperWorks whatever was Yeah, so that should be interesting.
• 3:02
  Speaker 2: So we got a lot of stuff coming.
• 3:04
  Speaker 1: Thank you all right, sounds awesome, Ahi, Ish tell the
• 3:08
  Speaker 1: people about what you've been through, tell them about who
• 3:11
  Speaker 1: you are and what you do.
• 3:13
  Speaker 4: Hi. Good to be back on guys. Thanks for having
• 3:16
  Speaker 4: me in case you don't know me from the last
• 3:19
  Speaker 4: times I was on here. My background, my name's Ashiesh.
• 3:23
  Speaker 4: My background is in artificial intelligence and in finance. I
• 3:27
  Speaker 4: spent ten years working in finance as a trader for
• 3:30
  Speaker 4: Edward Jones Investments, and part of my responsibilities were covering
• 3:33
  Speaker 4: tech at the time. After that, I transitioned full time
• 3:36
  Speaker 4: into AI and I was specifically trying to apply AI
• 3:40
  Speaker 4: to computer vision. And the presentation I want to give
• 3:43
  Speaker 4: today is related to that because AI is a hot
• 3:45
  Speaker 4: topic and I think one of the things that's happening,
• 3:49
  Speaker 4: just like every other subject we've seen, there's a massive
• 3:51
  Speaker 4: amount of distortion that's happening in AI, And what I
• 3:55
  Speaker 4: want to do is I want to help people. So
• 3:58
  Speaker 4: a lot of people think that AI is like a
• 4:00
  Speaker 4: super advanced subject that's impossible to tackle or understand unless
• 4:05
  Speaker 4: you have some kind of like background in computer science,
• 4:10
  Speaker 4: and to be honest, it's not really necessary. I hope
• 4:13
  Speaker 4: I can make this accessible for people to understand real AI.
• 4:17
  Speaker 4: And I think it's really important that people understand this
• 4:20
  Speaker 4: because there's so many profound things that are actually true
• 4:25
  Speaker 4: about AI that are not really publicly discussed. And so
• 4:30
  Speaker 4: one of the big topics that is kind of a
• 4:33
  Speaker 4: big deal with AI and a lot of people who
• 4:35
  Speaker 4: are like doomers on AI, they think that there's some
• 4:39
  Speaker 4: kind of artificial life form that's about to be created,
• 4:43
  Speaker 4: and this artificial life form will turned into the terminator
• 4:46
  Speaker 4: or some apocalyptise scenario or something like that. Hopefully, what
• 4:50
  Speaker 4: I can do here is bring a little bit of
• 4:52
  Speaker 4: rationality this subject without getting overly technical. But there are
• 4:57
  Speaker 4: a couple of technical concepts that I want to explain, no,
• 4:59
  Speaker 4: I'll make it. And I also want to be able
• 5:04
  Speaker 4: to show what real AI is and how you know
• 5:08
  Speaker 4: what real AI is with versus what is not because
• 5:11
  Speaker 4: there's a lot of junk AI out there. So let's
• 5:14
  Speaker 4: just kind of start with like what is AI? Like fundamentally,
• 5:20
  Speaker 4: at a kindergarten level, what is AI? AI is a
• 5:26
  Speaker 4: mathematical function, and it is primarily based on a reward
• 5:31
  Speaker 4: and cost. There's a function for reward that can be
• 5:34
  Speaker 4: basically anything you want, and then there's a function for
• 5:38
  Speaker 4: cost for that same reward, which can also basically be
• 5:41
  Speaker 4: whatever you want. And the objective in AI is to
• 5:45
  Speaker 4: make these two functions resemble real life as much as possible.
• 5:50
  Speaker 4: So this is really what AI is. It's a type
• 5:53
  Speaker 4: of pattern recognition map. And this pattern recognition map is intense.
• 6:00
  Speaker 4: It's extremely large. Some of these models can have billions
• 6:04
  Speaker 4: of weights, and I'll kind of explain a little bit
• 6:07
  Speaker 4: about that. And because of the massive size of these models,
• 6:12
  Speaker 4: they are also energy intents and the energy you may
• 6:17
  Speaker 4: know in you know, in America, there's this huge push
• 6:20
  Speaker 4: for energy investment. Weightly, we're spending trillions upon trillions of dollars,
• 6:26
  Speaker 4: multiple trillions of dollars into both our energy grid and
• 6:30
  Speaker 4: energy utilities. And then on top of that, AI as
• 6:33
  Speaker 4: well is also independently receiving or trillions of dollars are
• 6:38
  Speaker 4: independently being invested in AI. So there are two major
• 6:44
  Speaker 4: energy intense aspects of AI. There is the training side,
• 6:50
  Speaker 4: which is like what you do to build the model,
• 6:53
  Speaker 4: and then there's the inference side, which is what happens
• 6:56
  Speaker 4: after the model's already done you deployed it. There's people's
• 6:59
  Speaker 4: hands and then they do a querry on something like chat,
• 7:02
  Speaker 4: GPT or whatever, and then that quarry has a small
• 7:06
  Speaker 4: amount of energy that's required to do this thing called inference,
• 7:09
  Speaker 4: which then outputs your answer output to your result. Now,
• 7:13
  Speaker 4: the training side is massive, and the inference side individually
• 7:17
  Speaker 4: is very small, But where the massive energy constraint comes
• 7:21
  Speaker 4: from is the scale. So once these developed models get
• 7:25
  Speaker 4: deployed out into the world, the energy demand from the
• 7:29
  Speaker 4: sum total of them all combined. That's where the energy
• 7:33
  Speaker 4: intensity goes through the roof. So two sides that are
• 7:37
  Speaker 4: heavily energy intense that are required in order to make
• 7:40
  Speaker 4: AI work. So what you can see from this though,
• 7:44
  Speaker 4: and what you should kind of interpret in your head
• 7:46
  Speaker 4: just starting with this, is that if an AI model
• 7:49
  Speaker 4: requires that much energy in order to generate the model
• 7:53
  Speaker 4: and then be able to use it, you're probably in
• 7:56
  Speaker 4: the vacuum stage, a vacuum tube stage, because this is
• 8:01
  Speaker 4: nowhere near the efficiency of the compute that happens in
• 8:06
  Speaker 4: the human brain. And I'm going to.
• 8:09
  Speaker 1: Start relating hold on because I think this is a
• 8:12
  Speaker 1: two way street. It's it's not only just taking everything
• 8:16
  Speaker 1: that you're you know, sort of, it's not only taking
• 8:20
  Speaker 1: your questions. It's taking all of the information together. So
• 8:24
  Speaker 1: this is a massive data gathering tool as well as
• 8:28
  Speaker 1: a data you know, output tool. Yeah, and that's the
• 8:32
  Speaker 1: main sort of correlation that's being drawn here. I think
• 8:36
  Speaker 1: they're sucking up a lot of core information, psychological information
• 8:40
  Speaker 1: about each each one of us, how we interact with it.
• 8:44
  Speaker 4: Yep. And the kind of that kind of description that
• 8:48
  Speaker 4: you just gave that is most relevant to the concept
• 8:52
  Speaker 4: of l ms, which are large language models. So this
• 8:55
  Speaker 4: is like your chat Ebt, Gemini, crock whatever, take your
• 9:00
  Speaker 4: flavor of the month. There's so many these things. This
• 9:03
  Speaker 4: is a good, good thing you mentioned that, because what
• 9:06
  Speaker 4: I want to show here, and one of the things
• 9:08
  Speaker 4: that I'm hoping to promote, is that computer vision is
• 9:12
  Speaker 4: a way better, more accurate version of AI than a
• 9:17
  Speaker 4: large language model. So what I was going to say
• 9:21
  Speaker 4: earlier was that I'm relating AI to the brain. And
• 9:25
  Speaker 4: what I was originally trying to do here and when
• 9:29
  Speaker 4: I was working on AI, was to model artificial intelligence
• 9:33
  Speaker 4: off the human brain. So you're going to hear me
• 9:35
  Speaker 4: relate to the human brain many times in this presentation.
• 9:39
  Speaker 4: I'll get to that because that's a big part of
• 9:41
  Speaker 4: what I'm doing here. So one of the things to
• 9:43
  Speaker 4: know about that is that in a large language model,
• 9:46
  Speaker 4: an LM is a pattern recognition math built around literally
• 9:51
  Speaker 4: language text. And if you think about the human brain,
• 9:56
  Speaker 4: the language center of the brain is a very small
• 9:59
  Speaker 4: part of the brain right here over here. It's a
• 10:02
  Speaker 4: very small part actually, And I'm not saying it's not important.
• 10:05
  Speaker 4: A large language model and that technology is important, but
• 10:09
  Speaker 4: that is not what's going to get you to this
• 10:11
  Speaker 4: artificial life form. And that's controversial to say because there's
• 10:14
  Speaker 4: so many people that think that a large language model
• 10:17
  Speaker 4: will just suddenly become this artificial life form. But language
• 10:20
  Speaker 4: is not a life that's not how language is a
• 10:23
  Speaker 4: form pattern recognition math. And the easy way to understand
• 10:26
  Speaker 4: this without having any understanding of the math behind it
• 10:30
  Speaker 4: is if you walk into a room and there are
• 10:33
  Speaker 4: ten people speaking different languages that you've never heard before,
• 10:37
  Speaker 4: not languages that you don't understand but have heard before. No, no, no,
• 10:40
  Speaker 4: languages that you've never heard before, and you will not
• 10:43
  Speaker 4: be able to distinguish between all that noise one voice
• 10:47
  Speaker 4: from another. However, if one of those ten people start
• 10:51
  Speaker 4: speaking English in that room, all of a sudden, you'll
• 10:55
  Speaker 4: be able to zone in on that wavelength and filter
• 10:58
  Speaker 4: it out, and you'll be able to hear the English
• 11:01
  Speaker 4: from the noise. This is how you know that language
• 11:04
  Speaker 4: is a pattern recognition math and you're not born with language.
• 11:07
  Speaker 4: Another thing to understand about language, two people in this
• 11:11
  Speaker 4: world can hear the same sound and interpret something totally
• 11:17
  Speaker 4: different from that sound. In other words, there can be
• 11:19
  Speaker 4: a word in English and then another word in a
• 11:22
  Speaker 4: different language that sounds exactly the same. That mean two
• 11:24
  Speaker 4: totally different things. Two different people can understand two different
• 11:28
  Speaker 4: things from this same sound. So this is how you
• 11:31
  Speaker 4: know that language is an acquired skill. That is a
• 11:35
  Speaker 4: form of pattern recognition. And this is not anything that
• 11:39
  Speaker 4: you would call a life form. This is a talent,
• 11:42
  Speaker 4: you could say, but it isn't a life form. And
• 11:45
  Speaker 4: one of the other things is and I actually brought
• 11:48
  Speaker 4: this up just so you could see in case anyone
• 11:52
  Speaker 4: forgot from their seventh grade science class, but the definition
• 11:56
  Speaker 4: of life to this day still does not include intelligence.
• 12:02
  Speaker 4: And in my opinion, intelligence is equal to life these definitions.
• 12:09
  Speaker 4: If you read what this is saying here on the screen,
• 12:11
  Speaker 4: the actual definition of life, this sounds like primates trying
• 12:15
  Speaker 4: to describe humans horribly. By the way, this is not
• 12:19
  Speaker 4: the definition of life. If you take away reproduction, for example,
• 12:23
  Speaker 4: is part of life. It is an essential element of
• 12:26
  Speaker 4: what is considered life. If you took away reproduction from
• 12:29
  Speaker 4: everything on Earth tomorrow, does that mean there's no life
• 12:32
  Speaker 4: on Earth? That is how life works. So what I'm
• 12:37
  Speaker 4: trying to argue here is that intelligence is part of
• 12:41
  Speaker 4: the definition of life. And in case you're interested, there's
• 12:47
  Speaker 4: something else that I want to say before I get
• 12:49
  Speaker 4: into the actual formal presentation as a little set up here,
• 12:52
  Speaker 4: So if you're interested in understanding more about intelligence and perception,
• 12:58
  Speaker 4: my home basis perception. By the way, AI my focus
• 13:01
  Speaker 4: area was perception. If you want to understand perception, it
• 13:07
  Speaker 4: turns out, and what I'm going to demonstrate in this presentation,
• 13:11
  Speaker 4: it turns out that computer vision by accident implicitly understands neuroscience.
• 13:19
  Speaker 4: And I'll describe this in detail as I go through
• 13:21
  Speaker 4: the presentation. But if you want to learn more about
• 13:24
  Speaker 4: what I'm talking about on the neuroscience side, which I'm
• 13:27
  Speaker 4: not going to do too much about. I'll talk mostly
• 13:28
  Speaker 4: on the AI side, but if you want to learn
• 13:30
  Speaker 4: more about the neuroscience side, I highly recommend this lecture series,
• 13:34
  Speaker 4: and this one will break it down easy enough for
• 13:36
  Speaker 4: you to understand some basic neuroscience about vision. So computer vision,
• 13:42
  Speaker 4: in my opinion, doesn't get the rep that it deserves
• 13:44
  Speaker 4: in AI. And here's something to note about all of this.
• 13:50
  Speaker 4: So as I was saying, intelligence is not part of
• 13:52
  Speaker 4: the definition of life, my dad was a neurologist and
• 13:55
  Speaker 4: a psychiatrist, my mom is a pharmacist with a bunch
• 13:57
  Speaker 4: of degrees and all this stuff. There's still disappointed that
• 13:59
  Speaker 4: I'm not a doctor. And what what is interesting and
• 14:04
  Speaker 4: what I've learned over the years is a two major things.
• 14:09
  Speaker 4: So number one in evolution, one of the things, one
• 14:14
  Speaker 4: of the first structures that was ever evolved was a
• 14:18
  Speaker 4: photoreceptor attached to a flagella. A photoreceptor just says, so
• 14:24
  Speaker 4: a flagella is a tail, and it was a photoreceptor
• 14:26
  Speaker 4: was attached to it, and a photoreceptor it just says
• 14:29
  Speaker 4: whether light exists or not, yes or no, one or zero.
• 14:32
  Speaker 4: And so you can see that this is the beginning
• 14:34
  Speaker 4: of binary map, whether light exists or does not exist.
• 14:39
  Speaker 4: So another thing that is commonly known but often forgotten
• 14:43
  Speaker 4: is that eyeballs are brain cells. So this is very
• 14:49
  Speaker 4: important to understand. In the human eye, the ratio of
• 14:53
  Speaker 4: cones uh they're there are these two types of uh self.
• 14:58
  Speaker 4: There's these uh raw shaped cylinders which are gray and
• 15:03
  Speaker 4: they determine grayscale. They determine basically opacity light light, the
• 15:07
  Speaker 4: gradient of light, brightness of light, and grayscale. And then
• 15:11
  Speaker 4: they have these cones RGB red, red, green, blue. The
• 15:16
  Speaker 4: ratio on this is roughly twenty six three one. Roughly
• 15:22
  Speaker 4: if you're a female, you'll probably have a little bit
• 15:24
  Speaker 4: more blue, so twenty for twenty. The ratio of cones
• 15:27
  Speaker 4: in the human eye is overweighted gray cylinders. Ratio of
• 15:31
  Speaker 4: twenty on the gray, six on the red, three on
• 15:35
  Speaker 4: the green, and then one or a little bit higher
• 15:37
  Speaker 4: than one if you're female on the blue cones. So
• 15:41
  Speaker 4: that's interesting. And what are the part that I'm going
• 15:46
  Speaker 4: to skip in a neuroscience is about V one through
• 15:49
  Speaker 4: V four cortex and the visual cortex. And this is
• 15:53
  Speaker 4: outside of the you know, the purview of the presentation.
• 15:56
  Speaker 4: But if you want to go deeper down the eyeball route,
• 16:00
  Speaker 4: recommend after watching this presentation on the AI side. For me,
• 16:04
  Speaker 4: if you want to understand what I was saying earlier
• 16:06
  Speaker 4: about how computer vision experts implicitly, whether they know it
• 16:09
  Speaker 4: or not, accidentally already understand the visual cortext just from
• 16:15
  Speaker 4: computer vision, because computer vision is doing exactly what the
• 16:20
  Speaker 4: visual cortex is doing exactly. It is literal, and I'm
• 16:24
  Speaker 4: going to show published papers that demonstrate this. So what
• 16:27
  Speaker 4: I'm saying is that computer vision in the modern day
• 16:31
  Speaker 4: is mimicking the brain, and it is published and known,
• 16:35
  Speaker 4: and computer vision experts, whether they know it or not,
• 16:38
  Speaker 4: accidentally figured out the visual cortext and understand it implicitly.
• 16:42
  Speaker 4: An example of this in computer vision, we understand why
• 16:47
  Speaker 4: you want to use grayscale to detect motion. First, it's
• 16:51
  Speaker 4: energy efficient and that's what happens in V one and
• 16:55
  Speaker 4: that's why you have predominantly gray cylinders and your eyeball.
• 17:02
  Speaker 4: And then there are functional purposes for RGB as well,
• 17:06
  Speaker 4: and this is also discussed in computer vision. Okay, so
• 17:10
  Speaker 4: that was a good like little preamble for the setup,
• 17:13
  Speaker 4: and hopefully you're seeing that this is shit that no
• 17:15
  Speaker 4: one's ever talked about. And this is real.
• 17:17
  Speaker 2: AI, have you spent a lot of time? Go ahead,
• 17:22
  Speaker 2: I was just gonna say, this is right up your all.
• 17:24
  Speaker 1: He's upgraded all of these symbols in occultism directly to
• 17:28
  Speaker 1: the eyeball, and the eyeball is so central to occultism
• 17:32
  Speaker 1: in general. I mean it's in the word itself. And
• 17:36
  Speaker 1: he breaks down all of these symbols and how they
• 17:38
  Speaker 1: relate to stuff, and it's just like, yeah, it's brain tissue,
• 17:41
  Speaker 1: this is what's programming.
• 17:43
  Speaker 4: I also know the connection and spirituality, and I wasn't
• 17:46
  Speaker 4: going to discuss it that much, but here's something to know.
• 17:48
  Speaker 4: In the beginning, there was light and another thing about Yeah,
• 17:56
  Speaker 4: I know a lot about that, but that's a whole
• 17:58
  Speaker 4: wormhole and if I say something about it, we'll have
• 17:59
  Speaker 4: to oh for like an hour. But in other words, yeah,
• 18:02
  Speaker 4: there is it's it's there's a lot of indication that
• 18:08
  Speaker 4: in religion that there was an understanding of evolution, especially
• 18:11
  Speaker 4: in Hinduism, there was an understanding of evolution. And there's
• 18:15
  Speaker 4: a lot of not just written documentation of this, but
• 18:18
  Speaker 4: there's also archaeology that demonstrates this. Uh literal you can
• 18:23
  Speaker 4: see it with your own eyes, thousands and thousands of
• 18:25
  Speaker 4: years old, and they're demonstrating that they understood evolution and
• 18:28
  Speaker 4: more beyond that.
• 18:30
  Speaker 2: You know, it's real quick, you know, no, go ahead,
• 18:34
  Speaker 2: let's go ahead, let's okay.
• 18:36
  Speaker 1: Thees zinc calcium reaction at conception. It's a flash of light.
• 18:41
  Speaker 4: Yeah, there's a Yeah, it's kind of like the Paho
• 18:43
  Speaker 4: electric effect.
• 18:44
  Speaker 2: But real quick, is she? I think it's interesting.
• 18:48
  Speaker 5: You know.
• 18:48
  Speaker 3: There was when a boy I think actually friends with
• 18:51
  Speaker 3: them pravine. Uh, he was a cosmic summit. I went
• 18:54
  Speaker 3: up to him and I said something to him. I
• 18:56
  Speaker 3: was trying to explain to him. I was like, I know,
• 18:58
  Speaker 3: like what you were showing on the screen, you were
• 19:00
  Speaker 3: showing with something else. I said, but have you ever wondered,
• 19:03
  Speaker 3: like if those are depictions of inside your rible?
• 19:07
  Speaker 2: And he like like lost it.
• 19:09
  Speaker 3: Like he like I looked like I literally like freaked
• 19:11
  Speaker 3: him out to where.
• 19:13
  Speaker 2: He like sat down and sat sat down to study
• 19:15
  Speaker 2: eating his food.
• 19:16
  Speaker 3: He's like, oh yeah, I was like, yo, I literally
• 19:18
  Speaker 3: just bugged this guy the fuck out.
• 19:19
  Speaker 2: But like you get it, you get it.
• 19:21
  Speaker 4: Yeah, Pravine is one of my most favorite.
• 19:25
  Speaker 2: Yeah, yeah, maybe you could maybe you could. Maybe you
• 19:26
  Speaker 2: can tell that and he'll take it a little bit
• 19:28
  Speaker 2: more serious, like.
• 19:29
  Speaker 4: Oh, I've I've talked his ear off about this stuff.
• 19:31
  Speaker 4: He's fully aware of everything.
• 19:33
  Speaker 2: Oh, I don't know. He looked like he was bugged
• 19:34
  Speaker 2: out when I said something to him, like he.
• 19:36
  Speaker 4: Looked at that time. Oh okay if I had just
• 19:41
  Speaker 4: met Pravine at that time, so he didn't know that
• 19:43
  Speaker 4: much about me at that time. But we became good
• 19:45
  Speaker 4: friends over time. Oh. Nice, I'm actually gonna go to
• 19:47
  Speaker 4: Cambodia with him and a couple of months or a
• 19:50
  Speaker 4: few months.
• 19:51
  Speaker 2: Nice, that's awesome.
• 19:52
  Speaker 1: Be careful of those planes.
• 19:54
  Speaker 4: A lot of yeah, yeah, yoh.
• 19:58
  Speaker 2: We'll have to get you on with Day to talk
• 19:59
  Speaker 2: about the eyeball. I did like a five or six.
• 20:01
  Speaker 3: I don't have many plots series with the cult symbolism
• 20:03
  Speaker 3: in the eye, But if you ever want to come
• 20:04
  Speaker 3: back on and talk about that ship, we're talking about.
• 20:06
  Speaker 5: The eyeball at all. Day Man.
• 20:09
  Speaker 2: A lot of my lot of all.
• 20:10
  Speaker 3: My art, believe and I see the crossing of the arms.
• 20:13
  Speaker 3: That's that's the crossing of the optic nerve. Like a
• 20:16
  Speaker 3: lot of my art literally is depicting the inside of
• 20:18
  Speaker 3: the eyeball, the one that I have like a chick.
• 20:20
  Speaker 2: It looks like she's on a sea show.
• 20:22
  Speaker 3: That's like the hyloid collac canal and the oriserata and
• 20:25
  Speaker 3: then the ac humor underneath. I totally all my ship's
• 20:28
  Speaker 3: all basically depicted on an eyeball on the brain man.
• 20:31
  Speaker 4: And don't forget about the pineal gland, right, yeah, yeah,
• 20:34
  Speaker 4: so we got all and then the third eye and everything. Yeah,
• 20:36
  Speaker 4: there's quite a lot to do with vision. And uh,
• 20:42
  Speaker 4: one of the how do I say so? One of
• 20:46
  Speaker 4: the there's another thing that I want to discuss about
• 20:48
  Speaker 4: evolution before I get into the formal presentation, and then
• 20:53
  Speaker 4: we'll go from there. But one more thing about evolution.
• 20:55
  Speaker 4: And I'm not sure how this happened, but for some
• 20:59
  Speaker 4: reason in this zeitgeist, it is commonly thought or said
• 21:04
  Speaker 4: that DNA is the cause of evolution. And this is
• 21:11
  Speaker 4: just not true. There is no scientists saying that. The
• 21:15
  Speaker 4: people who discovered DNA didn't say that. Darwin didn't say that.
• 21:18
  Speaker 4: As far as I know, there is no official source
• 21:22
  Speaker 4: anywhere that is saying that DNA causes evolution. DNA is
• 21:29
  Speaker 4: the product of evolution, and that's why they're correlated, but
• 21:35
  Speaker 4: correlation is not Causation is understood at kindergarten level, and
• 21:38
  Speaker 4: as far as I know, no scientist is actually saying that.
• 21:40
  Speaker 4: But for some reason in the psycheiasis is commonly thought
• 21:43
  Speaker 4: that DNA causes evolution. This is not true, and there
• 21:48
  Speaker 4: are it is academically understood and known. It is academically
• 21:52
  Speaker 4: accepted that the brain has known mechanisms for controlling DNA.
• 21:56
  Speaker 4: And even every in your everyday life, your DNA is
• 22:00
  Speaker 4: not static. Things turn different, Switches turn on and off
• 22:03
  Speaker 4: in your DNA all the time, and so and this
• 22:06
  Speaker 4: mechanism is controlled by the brain, and often environmental factors
• 22:09
  Speaker 4: are the reason for why this happens. So again, and
• 22:15
  Speaker 4: another way, just like if you if you're just really
• 22:17
  Speaker 4: thinking about it from a high level, you've got to
• 22:20
  Speaker 4: understand that, like, uh, there's so much infrastructure that's required
• 22:24
  Speaker 4: in order to make DNA work. In the first place,
• 22:27
  Speaker 4: you need to have the hisstones to zip the file,
• 22:29
  Speaker 4: because I look at DNA as being software. You need
• 22:31
  Speaker 4: to have the zip hisstones to zip the file. And
• 22:33
  Speaker 4: then you need to have a way of unraveling it.
• 22:35
  Speaker 4: You need to have a way of splitting it. You
• 22:37
  Speaker 4: need to have the transcription enzymes to transcribe the DNA.
• 22:41
  Speaker 4: You need to have all the different proteins already there
• 22:43
  Speaker 4: in order to even transcribe them in the first place.
• 22:46
  Speaker 4: So there's and and then so much more. In order
• 22:48
  Speaker 4: to make DNA work, it needs all of these other
• 22:51
  Speaker 4: things as well, and so in order of operations. But
• 22:56
  Speaker 4: how is like DNA by itself doesn't do anything. There's
• 22:58
  Speaker 4: no magic that happens with DNA by itself, And so
• 23:01
  Speaker 4: in order of operations, you would need to have all
• 23:04
  Speaker 4: of this other infrastructure available or at least functional to
• 23:08
  Speaker 4: some degree before DNA becomes useful. So in terms of
• 23:11
  Speaker 4: order or operations, DNA can't be at the front of
• 23:14
  Speaker 4: the line. This is the software. Software doesn't come before
• 23:17
  Speaker 4: the hardware. So this is kind of you know, you
• 23:21
  Speaker 4: can kind of understand this stuff at a pretty basic level.
• 23:24
  Speaker 4: This is all commonly understood and all this stuff, but
• 23:26
  Speaker 4: for some reason, there's just a different idea of how
• 23:30
  Speaker 4: it works out there. So let's get into the presentation.
• 23:35
  Speaker 4: So what I was doing at Arizona State University, as
• 23:39
  Speaker 4: I was working on computer vision applied to artificial intelligence
• 23:44
  Speaker 4: in automated vehicles, and so I was trying to apply
• 23:48
  Speaker 4: camera based systems to automated vehicles. And one of the
• 23:57
  Speaker 4: issues that I ran into in automated vehicles, especially at ASU,
• 24:03
  Speaker 4: is that a overwhelming amount of investment and research in
• 24:08
  Speaker 4: R and B effort, especially in academia but also out
• 24:12
  Speaker 4: in the real business world, has been dedicated to the
• 24:15
  Speaker 4: task of making liedar part of the automated vehicle apparatus system.
• 24:23
  Speaker 4: And what I'm sure I'm going to be arguing in
• 24:25
  Speaker 4: this presentation is that LDAR is completely useless. It's actually work.
• 24:31
  Speaker 4: It makes your system worse, not better. And I want
• 24:35
  Speaker 4: to be clear upfront. Lightar is the real technology and
• 24:39
  Speaker 4: it does have real world application. It's used in space,
• 24:42
  Speaker 4: it's used to map layout. There are actual, real functional
• 24:46
  Speaker 4: purposes for lightar. But what I am going to hit
• 24:49
  Speaker 4: hard on in this presentation is it LDAR makes no
• 24:52
  Speaker 4: sense at all in an automated vehicle. Doesn't matter what
• 24:55
  Speaker 4: your philosophy is, there is no reason to have lightar
• 24:58
  Speaker 4: in a vehicle. So automated vehicle so me since maybe
• 25:04
  Speaker 4: not everyone is familiar with how light our works, just
• 25:06
  Speaker 4: to set it up the way that light our works
• 25:08
  Speaker 4: is so the major companies that you may have seen
• 25:11
  Speaker 4: that have light or Weimo is the big one in
• 25:14
  Speaker 4: the US and Waimo is owned by Google. And then
• 25:18
  Speaker 4: in the past there were other companies that have also
• 25:21
  Speaker 4: used them. There was General Motors GM, but they ended
• 25:27
  Speaker 4: up having a problem a couple of years ago in
• 25:29
  Speaker 4: California where they ended up using a light our based
• 25:31
  Speaker 4: system and this car dragged a person underneath their vehicle
• 25:35
  Speaker 4: for like twenty or thirty feet, and so then they
• 25:38
  Speaker 4: had to end basically operations. They were even caught lying
• 25:40
  Speaker 4: like trying to deceive about it too when they were
• 25:42
  Speaker 4: doing it. It was all over the news. People probably
• 25:44
  Speaker 4: remember another company that was doing it was Uber and
• 25:49
  Speaker 4: I live in Phoenix, Arizona. This is a heavy testing
• 25:52
  Speaker 4: ground for automated vehicles. And the reason why Arizona is
• 25:56
  Speaker 4: a heavy testing ground or Phoenix specifically is a heavy
• 25:59
  Speaker 4: testing ground for automate vehicles is because we have three
• 26:01
  Speaker 4: hundred and sixty days of sunshine and we have some
• 26:03
  Speaker 4: of the widest and best roads in the US because
• 26:07
  Speaker 4: we have massive trucking that comes through here, and so
• 26:10
  Speaker 4: this is and we have a very grid like road system,
• 26:14
  Speaker 4: very squared, grid like road system is very clean at organized,
• 26:18
  Speaker 4: so we're pretty much ideal set up for this, and
• 26:22
  Speaker 4: so WEIMO eventually came here, and what they ended up
• 26:27
  Speaker 4: doing is they ended up going to ASU. WEIMO did,
• 26:29
  Speaker 4: and they ended up investing a huge amount of money
• 26:31
  Speaker 4: into their engineering department. And that was very problematic for me,
• 26:36
  Speaker 4: which I didn't know about at the time, because I
• 26:38
  Speaker 4: was going in there applying with this presentation for a
• 26:42
  Speaker 4: PhD in computer vision to use it for automated vehicles.
• 26:46
  Speaker 4: And so you'll see why this gets to be a
• 26:49
  Speaker 4: big problem when you're in the hornet's nest with WEIMO.
• 26:54
  Speaker 4: So before I get to that, let's kind of build
• 26:56
  Speaker 4: it all up and kind of set up what I'm
• 26:58
  Speaker 4: talking about here. My home basis perception. But and by
• 27:03
  Speaker 4: the way, if this math is scaring you, don't worry.
• 27:05
  Speaker 4: I'm not going to get to mathavy. I'm breaking everything down.
• 27:07
  Speaker 4: You don't need a degree to understand any of this stuff.
• 27:11
  Speaker 4: But what is important here is that there are three
• 27:13
  Speaker 4: major elements to UH automated vehicles for doing the doing
• 27:22
  Speaker 4: controlling automated vehicles. So the first element is perception, and
• 27:27
  Speaker 4: then dynamics and then control and then basically what I
• 27:30
  Speaker 4: have on this side is intro to robotics. So this
• 27:33
  Speaker 4: is this is the functional math for how you do robotics.
• 27:38
  Speaker 4: And what I have here on the left side, this
• 27:41
  Speaker 4: big old crazy looking equation. All this, all this is
• 27:43
  Speaker 4: saying is what is the least wrong answer? So based
• 27:46
  Speaker 4: off of these three equations on the right side, the
• 27:50
  Speaker 4: left side is trying to figure out what is the
• 27:52
  Speaker 4: least wrong answer based off of that, And what you'll
• 27:56
  Speaker 4: see is that perception, which is ZT. Perception is used
• 28:02
  Speaker 4: in all three equations, is fundamental to all three equations.
• 28:05
  Speaker 4: So perception is the most important task in robotics. Perception.
• 28:13
  Speaker 4: So here are some of the major brands. So remember
• 28:19
  Speaker 4: that this presentation was done in twenty twenty three, so
• 28:22
  Speaker 4: some of this is a little bit dated, but almost
• 28:24
  Speaker 4: all of it is still relevant today. This is a
• 28:29
  Speaker 4: company called Argo on the top left here. This was
• 28:32
  Speaker 4: owned by Ford. This is a now defunct company. But
• 28:36
  Speaker 4: they were also trying to force fit lighter into their cars.
• 28:40
  Speaker 4: They weren't able to do it, so they went bankruptor
• 28:43
  Speaker 4: they closed on the company. But what I wanted to
• 28:45
  Speaker 4: point out about this is that they have these light
• 28:49
  Speaker 4: our things on here. But notice the cameras that they
• 28:52
  Speaker 4: used are ring cameras, literal ring cameras, and so you
• 28:56
  Speaker 4: can tell that they went way out of their way
• 28:58
  Speaker 4: to do go all in on cameras. Huh. Of course,
• 29:03
  Speaker 4: Weimo's the big one. This company down here in the
• 29:06
  Speaker 4: center is this should be Wave if I recall correctly,
• 29:11
  Speaker 4: it's been a little while since I've done this. And
• 29:12
  Speaker 4: then there's a couple others. It's fine. There's a Chinese
• 29:14
  Speaker 4: company over here called Xpaying, which people should know about
• 29:17
  Speaker 4: if they care about this stuff. So LDAR is so
• 29:26
  Speaker 4: prevalent and overwhelmingly so. Almost every major company has been
• 29:30
  Speaker 4: trying to force fit LDAR into their automated vehicle, and
• 29:34
  Speaker 4: almost every single one of them is failing. The only
• 29:36
  Speaker 4: one that is achieving scale, despite what narratives are, the
• 29:40
  Speaker 4: only one that is achieving scale on the autonomous vehicles
• 29:44
  Speaker 4: in this world is Tesla. And so Tesla is a
• 29:47
  Speaker 4: camera based system that does not use any other perception sensor,
• 29:52
  Speaker 4: just cameras and way moos use lightar and camera. So
• 29:57
  Speaker 4: here's another thing about LDAR. Even if if you do
• 30:00
  Speaker 4: use a light ar system, no matter what, you still
• 30:03
  Speaker 4: need a camera based system. Doesn't matter who you are,
• 30:06
  Speaker 4: what philosophy you believe in it doesn't matter anything. You
• 30:08
  Speaker 4: still need a camera based system. So I'm going to
• 30:11
  Speaker 4: discuss why that's important or why that is. This is
• 30:19
  Speaker 4: like Chinese companies. I'm going to skip some of this
• 30:21
  Speaker 4: stuff because this presentation is actually quite long, and so
• 30:25
  Speaker 4: I'm going to skip it to condense it quite a
• 30:27
  Speaker 4: lot and stick to really the most important points. So
• 30:32
  Speaker 4: one of the issues is that lighter has got this
• 30:34
  Speaker 4: spinny thing that's going around. You've probably seen it if
• 30:37
  Speaker 4: you've seen a Waymo vehicle or something like that. Zeukes
• 30:41
  Speaker 4: I've just seen in Las Vegas has also started to deploy,
• 30:44
  Speaker 4: which I believe it's owned by Amazon. This is a
• 30:46
  Speaker 4: relatively new addition, but again still using widar. And what
• 30:52
  Speaker 4: a light oar system does is this little spinny thing
• 30:55
  Speaker 4: that I'm talking about. It shoots a laser beam and
• 30:58
  Speaker 4: it shoots a ton of these all over the and
• 31:01
  Speaker 4: then there's like you're supposed to get a reflection back.
• 31:03
  Speaker 4: This is an imperceptible beam of light, laser beam of light,
• 31:07
  Speaker 4: and then based on that time and distance, they're able
• 31:09
  Speaker 4: to get a point return of information on depth. So,
• 31:13
  Speaker 4: in other words, this laser beams functional purpose and what
• 31:17
  Speaker 4: it's doing, is it's trying to measure depth of the
• 31:20
  Speaker 4: things around it. The actual literal laser precise depth calculation,
• 31:26
  Speaker 4: and it is laser precise, it is super precise depth calculation.
• 31:30
  Speaker 4: But this is effectively the entire in its entirety, the
• 31:36
  Speaker 4: full and complete explanation of the functional purpose of why
• 31:41
  Speaker 4: these light oar systems exist on these cars is for
• 31:44
  Speaker 4: precise measurement of depth. But lightar is like a dumber
• 31:51
  Speaker 4: version of sonar, that's the way to think of it.
• 31:53
  Speaker 4: And it creates a point cloud return kind of a
• 31:59
  Speaker 4: grip around it in point cloud return. And I'm going
• 32:02
  Speaker 4: to show videos and pictures in all this of that.
• 32:06
  Speaker 4: But what I want to show first before I get
• 32:07
  Speaker 4: into the videos of that, is I want to show
• 32:11
  Speaker 4: why a few different cases of why lid are doesn't
• 32:14
  Speaker 4: make any sense in an automated vehicle. Why am I
• 32:17
  Speaker 4: saying that so strongly? So this what I have on
• 32:21
  Speaker 4: the screen, And by the way, every slide should have
• 32:23
  Speaker 4: a source on it, So like down here, this is
• 32:26
  Speaker 4: the actual paper, academically accepted paper, and i E. The
• 32:33
  Speaker 4: two major conferences I'm going to be talking about are
• 32:35
  Speaker 4: i EE and CVPR. I EE is like basically the
• 32:40
  Speaker 4: biggest engineering conference in the world, and CVPR is the
• 32:43
  Speaker 4: largest computer vision conference in the world. So all of
• 32:46
  Speaker 4: these papers and slides and presentations that I'm going to
• 32:49
  Speaker 4: be talking about are all from the largest possible conferences,
• 32:53
  Speaker 4: the most academically accepted conferences that there are in the world,
• 32:57
  Speaker 4: not just in America, in the world. So this paper
• 33:03
  Speaker 4: that I have on the screen is a little dated.
• 33:04
  Speaker 4: It's from twenty twenty one, but this is considered originally
• 33:07
  Speaker 4: the foundational paper on identifying what is known as VRUS.
• 33:12
  Speaker 4: VRUS stands for Vulnerable road user and this is a
• 33:16
  Speaker 4: regulatory term, not an AI term. This is a regulatory
• 33:19
  Speaker 4: term and it is basically a catch all phrase for pedestrians, dogs, cats,
• 33:25
  Speaker 4: et cetera. And what this paper is showing is that
• 33:29
  Speaker 4: it at that time and it identified thirty two attributes
• 33:33
  Speaker 4: on identifying a pedestrian, which is like one of the
• 33:35
  Speaker 4: most important tasks that a computer, any automated vehicle needs
• 33:40
  Speaker 4: to do. And if you look at what these attributes
• 33:43
  Speaker 4: are that they're talking about, like a handbag, clothes and
• 33:50
  Speaker 4: things like that, the bags that they're carrying male female.
• 33:56
  Speaker 4: The issue with a point cloud return is that it
• 33:59
  Speaker 4: only gives you the shape. It does not give you
• 34:03
  Speaker 4: the color quality that you get from a camera. So
• 34:07
  Speaker 4: liedar is like a one dimensional point. It's just a
• 34:10
  Speaker 4: point that tells you death and that's it. And you
• 34:13
  Speaker 4: get a whole bunch of points, millions of points, and
• 34:15
  Speaker 4: then hopefully you can identify the shape of an object
• 34:19
  Speaker 4: with these points. That's what light our does. Well. It
• 34:23
  Speaker 4: turns out that if this is what you're prioritizing, and
• 34:25
  Speaker 4: this is what you're trying to maximize for, which is
• 34:27
  Speaker 4: what a lot of pretty much all these light our
• 34:29
  Speaker 4: companies are doing well, you miss out on all of
• 34:33
  Speaker 4: those extra fine quality details that are actually most relevant,
• 34:38
  Speaker 4: the most important aspect is this stuff. And this is
• 34:41
  Speaker 4: academically understood. So this is in twenty twenty one, so
• 34:46
  Speaker 4: this is dated. So this has actually been improved significantly
• 34:48
  Speaker 4: since then. But this is just a cited This is
• 34:50
  Speaker 4: known for like a long time in the industry. I'm
• 34:57
  Speaker 4: gonna kind of skip this part and go to some
• 34:59
  Speaker 4: more of the meat. So right here is something called
• 35:04
  Speaker 4: this is Argo verse, which is Argo that the company
• 35:07
  Speaker 4: that was that company that was owned by Ford that
• 35:11
  Speaker 4: was also trying to force fit LDAR into their into
• 35:14
  Speaker 4: their cars. They're now defunct, but in twenty twenty three
• 35:18
  Speaker 4: at CVPR they presented this information that I'm showing on
• 35:22
  Speaker 4: the screen. Now. One of the things that has happened
• 35:25
  Speaker 4: in academia, and I'm pretty sure, at least as far
• 35:27
  Speaker 4: as i'm aware to this day, is still true. One
• 35:30
  Speaker 4: of the things that has happened in academia is that
• 35:33
  Speaker 4: research around light our degradation over range seems to be
• 35:39
  Speaker 4: heavily suppressed. And before ARGO went defuncts they published this
• 35:45
  Speaker 4: and this is, as far as I'm aware, the only
• 35:48
  Speaker 4: major company that has ever published anything like this. What
• 35:54
  Speaker 4: this is showing on the screen here is the accuracy
• 35:58
  Speaker 4: on object and classification identification using light ar based systems.
• 36:04
  Speaker 4: So you can see in their best case scenario they're
• 36:07
  Speaker 4: getting about fifty one percent accuracy at zero to fifty
• 36:11
  Speaker 4: meters of range, but once you go to fifty to
• 36:15
  Speaker 4: one hundred meters of range, it drops by half, and
• 36:18
  Speaker 4: then once you go one hundred to one hundred and
• 36:20
  Speaker 4: fifty meters a range, it drops by half again. So
• 36:23
  Speaker 4: what this is showing is that it's showing a log
• 36:27
  Speaker 4: to degradation in accuracy every fifty meters using a light
• 36:31
  Speaker 4: ur based system. And I just did the math down here.
• 36:35
  Speaker 4: This is some basic maths. This is about as intense
• 36:38
  Speaker 4: as the math is going to get. At sixty miles
• 36:40
  Speaker 4: per hour, that's twenty six point eight two meters per second.
• 36:44
  Speaker 4: That means fifty meters of accurate range. Gives you two
• 36:48
  Speaker 4: seconds of forecasting time, so you're able to see at
• 36:52
  Speaker 4: two seconds of what ahead of you with with this
• 36:55
  Speaker 4: system at sixty miles per.
• 36:56
  Speaker 1: Hour, this completely takes automated trucking off of the board.
• 37:01
  Speaker 1: You cannot have those type of tolerances if you're going
• 37:04
  Speaker 1: to actually have a truck that's now commanded by AI
• 37:09
  Speaker 1: or whatever.
• 37:10
  Speaker 4: No doubt about that. Yeah, that's why there isn't a
• 37:12
  Speaker 4: light hour based truck as far as I'm aware, it's
• 37:14
  Speaker 4: not one that carries weight. But there is the Tesla
• 37:19
  Speaker 4: semi which does exist and is on the roads. It
• 37:23
  Speaker 4: is working, and there are millions of vehicles Tesla vehicles,
• 37:27
  Speaker 4: automated vehicles on the road. In fact, I have one.
• 37:30
  Speaker 4: Oh I should also mention this. I've been beta testing
• 37:33
  Speaker 4: the Tesla automated system since twenty eighteen. I have more
• 37:37
  Speaker 4: than one hundred thousand miles driven on their system since then.
• 37:39
  Speaker 4: Over multiple cars and it's night and day. Comparatively to
• 37:43
  Speaker 4: anything else that you can never see, it's like ridiculous,
• 37:47
  Speaker 4: how much better it is. And again, Tesla has millions
• 37:50
  Speaker 4: and millions of these vehicles on the road and they're
• 37:54
  Speaker 4: all over the country doing this and actually the elsewhere
• 37:57
  Speaker 4: in the world. But Waimo only operates geo fence spaces
• 38:01
  Speaker 4: and a few select cities. They only have two thousand
• 38:04
  Speaker 4: vehicles in their fleet. And I'm going to get into
• 38:08
  Speaker 4: more about all that, but it's that that's worth keeping
• 38:12
  Speaker 4: in your mind for now, and it's gonna come up
• 38:14
  Speaker 4: later again why that's relevant.
• 38:16
  Speaker 3: So I think I did want to ask you though,
• 38:19
  Speaker 3: like what do you think about like light art kind
• 38:21
  Speaker 3: of being used on the ground to service, Like I mean,
• 38:24
  Speaker 3: you aren't even using that as a way of like
• 38:26
  Speaker 3: kind of like a ground truthing for like these satellite
• 38:29
  Speaker 3: images like even on the Pyramids or other stuff.
• 38:33
  Speaker 1: That's fine, all right, yeah, ye think about this. So
• 38:38
  Speaker 1: they're only taking in one wavelength of light, so they
• 38:43
  Speaker 1: don't know what what the fuck anything is. It's just
• 38:46
  Speaker 1: a distance. You've only got one dimension of vision. It's
• 38:50
  Speaker 1: it's worse than gray scale.
• 38:51
  Speaker 4: Yeah, And I'm setting up a few things here on
• 38:54
  Speaker 4: purpose before I show you the video of what the
• 38:57
  Speaker 4: light are actually looks like, because I want you out
• 38:59
  Speaker 4: kind of see how many different problems problem scenarios there are.
• 39:04
  Speaker 4: And then when I show you the video, you'll see
• 39:06
  Speaker 4: you'll have all the context you need for the video
• 39:08
  Speaker 4: to make sense. So you're going to see that, don't worry,
• 39:11
  Speaker 4: it's coming. You're going to see why how much of
• 39:14
  Speaker 4: a difference there is between light R and basically vision.
• 39:18
  Speaker 4: It's like not even close. So what I was saying
• 39:21
  Speaker 4: earlier though, was that light our based systems require geofencing,
• 39:25
  Speaker 4: meaning they have to map to a specific area and
• 39:29
  Speaker 4: they can only operate within this specific area, whatever this
• 39:32
  Speaker 4: area block is, and they have to go through this
• 39:35
  Speaker 4: area over and over and over again, and then they
• 39:37
  Speaker 4: map it. It's effectively memorizing the area. And that's what
• 39:41
  Speaker 4: geo fencing is called, and that's what WEIMO does. They
• 39:44
  Speaker 4: don't operate everywhere in Phoenix. They operate within confined regions
• 39:47
  Speaker 4: and that's for a reason. And another thing about this
• 39:50
  Speaker 4: is that that means that you're only allowed to do
• 39:53
  Speaker 4: in that system. In a light our based system, you're
• 39:55
  Speaker 4: only at this point as far as I'm aware, and
• 39:58
  Speaker 4: there's no technology that can beat this. Are only allowed
• 40:00
  Speaker 4: to do point to point travel, specific point pre determined
• 40:06
  Speaker 4: to specific point predetermined. And this is different than say
• 40:12
  Speaker 4: at Tesla, which can do anywhere to anywhere travel. And
• 40:16
  Speaker 4: so that means you can start from anywhere go to
• 40:19
  Speaker 4: anywhere Whereas a way, MO has to start in pre
• 40:21
  Speaker 4: defined spots only and go to pre defined spots only,
• 40:26
  Speaker 4: and these have to be these are finite. It's not
• 40:28
  Speaker 4: anywhere in that space. So even within the geofence region,
• 40:32
  Speaker 4: they can only go from specific place to specific place,
• 40:35
  Speaker 4: not anywhere anywhere you want in there. And so one
• 40:41
  Speaker 4: of the things that has happened, and one of the
• 40:43
  Speaker 4: distortions that has happened, this is infected the regulatory space
• 40:47
  Speaker 4: at a kindergarten level, this is broken somehow, some way.
• 40:51
  Speaker 4: Waimo and other light our companies have convinced everyone that
• 40:55
  Speaker 4: point to point travel is more difficult than anywhere to
• 41:01
  Speaker 4: anywhere travel, And to this day these systems are still
• 41:06
  Speaker 4: classified that way, where point to point travel, which is
• 41:10
  Speaker 4: an ADS system in regulatory terms, is somehow, some way
• 41:14
  Speaker 4: from a regulatory perspective, it is somehow more difficult than
• 41:19
  Speaker 4: anywhere to anywhere travel, which is an ADAS system. Sort
• 41:24
  Speaker 4: of well, I should be careful what's saying ADS versus eights. Actually,
• 41:31
  Speaker 4: let me come back to ADS versus ADS later. But
• 41:34
  Speaker 4: the point is that somehow they've convinced regulatory people that
• 41:38
  Speaker 4: point to point is harder than anywhere to anywhere, when
• 41:41
  Speaker 4: it's clearly the other way around. By far, it's exponentially
• 41:44
  Speaker 4: more difficult to do anywhere to anywhere travel than point
• 41:47
  Speaker 4: to point, especially when you're not confined by geofencing. One
• 41:50
  Speaker 4: of the other things is that these systems, these light
• 41:54
  Speaker 4: our based systems, are extremely expensive. Now I'm sure the
• 41:58
  Speaker 4: cost has come down over the past couple of years,
• 41:59
  Speaker 4: but in twenty twenty three, the average cost on those
• 42:02
  Speaker 4: weay moths for the lighter based system that apparatus that
• 42:05
  Speaker 4: they put on there, that cost is fifty thousand dollars
• 42:08
  Speaker 4: just for the system alone. Then you have to at
• 42:11
  Speaker 4: that time roughly maybe it's come down a little, but
• 42:14
  Speaker 4: it's still stupidly expensive. And then for whatever reason, they're
• 42:18
  Speaker 4: putting these things on Jaguars, So that's another eighty thousand.
• 42:22
  Speaker 4: So just between these two things alone, you have more
• 42:26
  Speaker 4: than one hundred and twenty thousand dollars being spent on
• 42:29
  Speaker 4: these systems one hundred and twenty one hundred and thirty thousand,
• 42:32
  Speaker 4: and that's not including a safety driver, that's not including
• 42:35
  Speaker 4: R and B, that's not including any kind of testing or anything.
• 42:38
  Speaker 4: That's just the hardware alone with nothing else. No software
• 42:41
  Speaker 4: is even put in it, No AI has even been
• 42:43
  Speaker 4: put in there yet, So already this stuff is automatically
• 42:48
  Speaker 4: absurdly overpriced to begin with, whereas the Tesla vehicles are
• 42:53
  Speaker 4: thirty forty thousand dollars by comparison all in with everything
• 42:57
  Speaker 4: already done, and so the economics here is something that
• 43:07
  Speaker 4: people need to understand is that there is nothing like
• 43:10
  Speaker 4: the scale is not reasonably possible with a light our
• 43:16
  Speaker 4: based system because the expense on light our based systems,
• 43:20
  Speaker 4: even after the costs have come down, is still absurd.
• 43:24
  Speaker 4: By comparison, a camera costs about one hundred dollars. The
• 43:28
  Speaker 4: cameras that are on the Teslas that they've been doing
• 43:30
  Speaker 4: for years and years that do all this stuff anywhere
• 43:33
  Speaker 4: to anywhere, travel just one hundred bucks each, so the
• 43:37
  Speaker 4: cost difference is absurd. One of the other issues with
• 43:40
  Speaker 4: geofencing is that if geo fencing and basically memorizing the
• 43:46
  Speaker 4: area means that you have to expect that the area
• 43:49
  Speaker 4: is static, because as soon as the area changes in
• 43:52
  Speaker 4: the material way, your training data goes out the window.
• 43:56
  Speaker 4: And so what construction becomes a problem because these space
• 44:00
  Speaker 4: says will continuously change. So as soon as construction arise,
• 44:03
  Speaker 4: all your training data for that area is gone, and
• 44:05
  Speaker 4: then you have to retrain with the construction there. And
• 44:07
  Speaker 4: then as soon as the construction is done, the whole
• 44:09
  Speaker 4: area changed again, and so you have to redo the
• 44:11
  Speaker 4: training with new data all over again. So it is
• 44:14
  Speaker 4: not easily adaptable. So one of the big problems with
• 44:19
  Speaker 4: interacting in the real world is that it's an open
• 44:24
  Speaker 4: set environment. What that means in mathematical terms, what that
• 44:28
  Speaker 4: means is that there's infinite possibilities. So if you're trying
• 44:31
  Speaker 4: to memorize an answer, which is what basically every light
• 44:35
  Speaker 4: our system is trying to do, if you memorize an answer,
• 44:38
  Speaker 4: you cannot possibly memorize infinity. And the problem they're trying
• 44:43
  Speaker 4: to solve is an open set problem, which means there's
• 44:46
  Speaker 4: infinite possibilities, So you cannot memorize infinity, so you are
• 44:51
  Speaker 4: destined to fail from the beginning if your system is
• 44:53
  Speaker 4: dependent on geofencing or memorization, which it's basically the same thing.
• 44:58
  Speaker 4: So just to kind of demonstrate other issues that come
• 45:02
  Speaker 4: up with light AR, this is an intersection in China,
• 45:07
  Speaker 4: And if you look at this intersection in China, just
• 45:09
  Speaker 4: look at the heavy complexity that's involved here, the path
• 45:12
  Speaker 4: and complexity, all the lands, all the turning, all the
• 45:14
  Speaker 4: different conditions that would be required, and the amount of
• 45:17
  Speaker 4: traffic that you would have normally going through this. How
• 45:20
  Speaker 4: intense it would be to navigate and manage all that. Well,
• 45:24
  Speaker 4: this is yet another situation where light R doesn't do
• 45:27
  Speaker 4: shit for you, nothing at all. Light R does not
• 45:31
  Speaker 4: help you with this problem in any way, shape or form,
• 45:34
  Speaker 4: exactly zero. It's useless, doesn't do anything. Same thing as
• 45:39
  Speaker 4: true in America. I was just showing this one because
• 45:40
  Speaker 4: one of the issues is that you want to be
• 45:42
  Speaker 4: able to generalize an answer. So if you try to
• 45:45
  Speaker 4: train on just US infrastructure, then you're pigeonholing yourself to
• 45:49
  Speaker 4: US infrastructure and you can't work anywhere else. And one
• 45:51
  Speaker 4: of the things I'm trying to get it is I'm
• 45:52
  Speaker 4: trying to model off the human brain, so we need
• 45:55
  Speaker 4: to find a generalizable answer that works anywhere regardless of
• 46:00
  Speaker 4: where you are a situation. Here's another problem. You have
• 46:05
  Speaker 4: all kinds of different signs, all kinds of different lights,
• 46:08
  Speaker 4: and they're constantly evolving, and there's so many different versions
• 46:13
  Speaker 4: of them, and they constantly change. And this is another
• 46:16
  Speaker 4: situation where light ar doesn't do shit for you. Exactly zero.
• 46:20
  Speaker 4: There's nothing about lightar that helps you with anything here
• 46:22
  Speaker 4: on the screen, zero exactly zero.
• 46:27
  Speaker 1: So they ishud on our streets. I mean, if they
• 46:30
  Speaker 1: can't read a light, what the fuck are we doing
• 46:33
  Speaker 1: letting it drive around?
• 46:36
  Speaker 4: Not a clue? I mean, I will make the argument
• 46:39
  Speaker 4: for I will a little bit later give the fundamental
• 46:42
  Speaker 4: argument about why light R is used, which is about
• 46:44
  Speaker 4: depth perception. This is the fundamental functional purpose that light
• 46:48
  Speaker 4: our serves. But we need to get a little more
• 46:51
  Speaker 4: involved into this presentation before I really hit it. But
• 46:54
  Speaker 4: the argument for lightar is it is accurate death. And
• 46:59
  Speaker 4: I'll get into that a little later. Let me kind
• 47:01
  Speaker 4: of do all the setup for everyone, because these are
• 47:03
  Speaker 4: all going to be new concepts, so we're gonna want
• 47:06
  Speaker 4: We're going to do this right. Let's put it that way. So,
• 47:10
  Speaker 4: as I was saying before the primary issues that we
• 47:12
  Speaker 4: have an open set environment. If we're trying to interact
• 47:15
  Speaker 4: in the universe, in the real world, then there's going
• 47:18
  Speaker 4: to be an infinite number of things that can happen.
• 47:21
  Speaker 4: So there are three major categories of problems. There are
• 47:24
  Speaker 4: the known common problems that we always know about. That's
• 47:27
  Speaker 4: this green bubble. Then there's the long tail known cases,
• 47:31
  Speaker 4: the rare events but we know about them. Those are
• 47:34
  Speaker 4: that's the blue circle. And then there is this much
• 47:37
  Speaker 4: larger orange circle, which is the long tail unknown events,
• 47:42
  Speaker 4: the unusual events that we don't know about, which is
• 47:46
  Speaker 4: the actual larger set of problems, way larger than the
• 47:50
  Speaker 4: other two. So just to give you some examples, at
• 47:54
  Speaker 4: least once a year, in this top left photo, at
• 47:57
  Speaker 4: least once a year, there might be a dinosaur cross
• 48:00
  Speaker 4: the road. And on the top right photo you have
• 48:04
  Speaker 4: a literal it's I know it's kind of hard to see,
• 48:06
  Speaker 4: sorry small, but that's a literal plane on the road.
• 48:10
  Speaker 4: And then you have a few other different issues here,
• 48:13
  Speaker 4: like on the bottom that and we'll hit on a
• 48:15
  Speaker 4: lot more of this stuff, but you can see how
• 48:17
  Speaker 4: there's a lot of unusual things that happen in fact,
• 48:21
  Speaker 4: the number of weird things that can happen is probably
• 48:24
  Speaker 4: way higher than you really are common Like, even if
• 48:27
  Speaker 4: you drive a lot, you'd be surprised how many weird
• 48:30
  Speaker 4: things happen that you've probably personally never seen before. So
• 48:34
  Speaker 4: in particular, what I want to highlight here is the
• 48:38
  Speaker 4: center top video and the center bottom video. So in
• 48:47
  Speaker 4: the center top video you have a bunch of small
• 48:50
  Speaker 4: mammals all over the road. In the center bottom video,
• 48:54
  Speaker 4: you have like flying pages. Now I was mentioning earlier
• 48:59
  Speaker 4: that the way light our works is that lied Our
• 49:03
  Speaker 4: tries to do object detection and classification by shape. So
• 49:10
  Speaker 4: what really is problematic is if you're trying to classify
• 49:16
  Speaker 4: by shape, what ends up happening is if you get
• 49:20
  Speaker 4: this wrong, meaning you reverse these two scenarios. In the
• 49:25
  Speaker 4: top scenario, you may go very cautiously because of these mammals.
• 49:29
  Speaker 4: In the bottom scenario, you don't want to you're on
• 49:31
  Speaker 4: the highway. On this scenario, you don't want to force
• 49:33
  Speaker 4: the break right away. You would actually go really aggressively
• 49:36
  Speaker 4: because you would know that these are pages. However, if
• 49:39
  Speaker 4: you're classifying by shape, you might reverse these two scenarios,
• 49:43
  Speaker 4: and that's when the horrible stuff happens, and so this
• 49:47
  Speaker 4: is another example where shape can easily be fooled and
• 49:50
  Speaker 4: where the quality, the quality of color becomes very relevant
• 49:56
  Speaker 4: and important. And in fact, not long ago I think
• 49:58
  Speaker 4: it was actually when we were at the comp for
• 50:00
  Speaker 4: instant weimo run over a cat and so uh yeah,
• 50:03
  Speaker 4: so I mean that also became really I'm a big
• 50:06
  Speaker 4: cat fan, So that's that really hurt. And of course
• 50:10
  Speaker 4: you have a lot of unusual scenarios.
• 50:13
  Speaker 1: Top right, what's the difference between a cat and a
• 50:17
  Speaker 1: bag blowing in the wind. If a cat's at full sprint,
• 50:20
  Speaker 1: how do you tell the difference in shape from that?
• 50:23
  Speaker 1: And maybe a bag that's now in the shape of
• 50:26
  Speaker 1: a cat.
• 50:27
  Speaker 4: Yeah, you would be surprised at how much better computers
• 50:30
  Speaker 4: are at computer vision than humans when you really want
• 50:33
  Speaker 4: to do And one of the things that I'm not
• 50:35
  Speaker 4: going to hit on it, but maybe it's worth mentioning
• 50:38
  Speaker 4: since you asked, is that once you can do the
• 50:40
  Speaker 4: visual spectrum, why can't you do everything else? There's nothing
• 50:44
  Speaker 4: stopping you. And so that's exactly what the evolutionary path
• 50:48
  Speaker 4: will look like when you get really advanced. Is that
• 50:50
  Speaker 4: once you can do wavelength light there's nothing stopping. That's
• 50:53
  Speaker 4: a very tight that's like the smallest liver of the
• 50:56
  Speaker 4: wavelength available to you. Once you can do that, you
• 50:58
  Speaker 4: can do all the other stuff. So just add infrared
• 51:01
  Speaker 4: for example, and you can automatically see through smoke, you
• 51:04
  Speaker 4: can see through fog, you can see through you can
• 51:06
  Speaker 4: see at night. You'll be able to do all kinds
• 51:08
  Speaker 4: of stuff. So this is all and it's all vision,
• 51:11
  Speaker 4: not anything to do with light, ar or anything like that.
• 51:16
  Speaker 4: And of course you'll have some unusual situations like in
• 51:19
  Speaker 4: the far right scenario that's a random washer rolling down
• 51:24
  Speaker 4: the road, and so that can also be problematic. Sometimes
• 51:28
  Speaker 4: you'll have trees that are in the middle of the road,
• 51:29
  Speaker 4: like a bottom left and then this situation I'm going
• 51:33
  Speaker 4: to bring up now discuss it a little more. But
• 51:35
  Speaker 4: second from the bottom right, this is a trash can
• 51:39
  Speaker 4: that's blowing from the wind through the road. Now, the
• 51:42
  Speaker 4: reason why in AVS for a brief period of time
• 51:47
  Speaker 4: this scenario became complicated was because there's this thing called
• 51:52
  Speaker 4: kinematic bits. These things that kinematic bits is just motion bits,
• 51:56
  Speaker 4: motion information data about motion. And so so people started
• 52:01
  Speaker 4: training on kinematic bits in addition to the wavelength light
• 52:04
  Speaker 4: and stuff like that. And so what they did was
• 52:07
  Speaker 4: they trained the system to be able to recognize every
• 52:09
  Speaker 4: single trash can there is, and it did it. But
• 52:13
  Speaker 4: one of the problems was that the training that they
• 52:16
  Speaker 4: did was on trash cans that didn't move, and so
• 52:19
  Speaker 4: when they deployed it, there was this trash can that
• 52:22
  Speaker 4: started moving and so it had kinematic bits, but every
• 52:26
  Speaker 4: trash can it was trained on had zero, no kinematic bits.
• 52:29
  Speaker 4: So trash can is not supposed to move. And so
• 52:32
  Speaker 4: you can see how a detail like that will mess
• 52:34
  Speaker 4: up the system. And so one of the things that's
• 52:37
  Speaker 4: important is to understand where are all the advantages and
• 52:42
  Speaker 4: the disadvantages, and kinematic bits are actually an advantage, you
• 52:46
  Speaker 4: just need to make sure you train on them. And
• 52:48
  Speaker 4: so this is earlier. This scenario that I'm talking about
• 52:50
  Speaker 4: has already been figured out. Don't worry. This is you know,
• 52:52
  Speaker 4: long ago. So people have figured this dumb shit out.
• 52:55
  Speaker 4: But all we had to go through the evolutionary process.
• 52:57
  Speaker 4: And I think in some of the that I've seen,
• 53:00
  Speaker 4: you know, because there's so many companies that are trying
• 53:04
  Speaker 4: to do light ar at this point and I see
• 53:07
  Speaker 4: all kinds of horrible stuff, and I see a lot
• 53:09
  Speaker 4: of cheats as well, And I'll talk a little bit
• 53:12
  Speaker 4: about one of those cheats right now, which is a
• 53:15
  Speaker 4: bounding box. So this is a in computer vision, this
• 53:19
  Speaker 4: is not a big deal. But in basically any other
• 53:23
  Speaker 4: like what do you call it, any other modality, this
• 53:27
  Speaker 4: bounding box problem I've seen is a big problem. And
• 53:30
  Speaker 4: even in twenty twenty three, I was still seeing papers
• 53:32
  Speaker 4: that were being submitted to CVPR or to IEEE that
• 53:37
  Speaker 4: we're using bounding boxes. Now, that paper that I was
• 53:40
  Speaker 4: showing before about the pedestrians the vr us in twenty
• 53:43
  Speaker 4: twenty one, that's a found I was saying that the
• 53:45
  Speaker 4: foundational paper heavily cited, well understood, academic accepted, you know,
• 53:50
  Speaker 4: big deal. That paper also demonstrated that bounding boxes are
• 53:55
  Speaker 4: a cheat. And what a bounding box is is that
• 53:57
  Speaker 4: not only do you need to be able to identify
• 53:59
  Speaker 4: what an object is, but you also need to be
• 54:02
  Speaker 4: able to position that object on the frame, be able
• 54:06
  Speaker 4: to identify that position. And what a bounding box is
• 54:09
  Speaker 4: basically a box that goes around the object you're trying
• 54:12
  Speaker 4: to identify. And the issue with this is that a
• 54:16
  Speaker 4: bounding box is going to always be larger than the
• 54:19
  Speaker 4: actual object itself, and it's not going to be the
• 54:23
  Speaker 4: shape of the object. The bounding box is just going
• 54:25
  Speaker 4: to like for example, in this circle that you're seeing here,
• 54:28
  Speaker 4: the bounding box in this example is way larger than
• 54:32
  Speaker 4: the actual truck itself, but so you can see how
• 54:36
  Speaker 4: much larger it is. So in the scenario like you
• 54:39
  Speaker 4: have in the middle here with this overturned truck, if
• 54:43
  Speaker 4: you imagine a bounding box with this kind of error
• 54:46
  Speaker 4: rate on it in terms of the size of its shape,
• 54:49
  Speaker 4: and apply it to this truck, the bounding box is
• 54:51
  Speaker 4: going to be bigger than the frame. Now, why that's
• 54:54
  Speaker 4: important is that it's easier to see on this bottom
• 54:58
  Speaker 4: left photo. That's important is because if you're using a
• 55:02
  Speaker 4: bounding box and it's larger than the object, you can
• 55:06
  Speaker 4: see that there's bounding box on the side here. You
• 55:09
  Speaker 4: can see here this side here is covering up the road.
• 55:13
  Speaker 4: So from a from the AI's perspective, there is no
• 55:17
  Speaker 4: road to travel on, and so you'll see the AI
• 55:20
  Speaker 4: systems start to stutter here because it doesn't think there's
• 55:23
  Speaker 4: actually anywhere to go, any room to go, and so
• 55:26
  Speaker 4: it has no way of path projecting through a bounding
• 55:29
  Speaker 4: box because you're not supposed to interact with this object.
• 55:32
  Speaker 4: So this is why bounding boxes are a cheat. And
• 55:34
  Speaker 4: this is still, as far as I know, this is
• 55:36
  Speaker 4: still actually happening. Unfortunately, in computer vision, I think AI
• 55:40
  Speaker 4: specialists have figured this dumb shit out, but in everything else,
• 55:44
  Speaker 4: I think they haven't. And this is a real shame.
• 55:47
  Speaker 4: And I'll show you the solution to this problem, how
• 55:50
  Speaker 4: you solve the bounding box problem. But I just want
• 55:53
  Speaker 4: to bring it up right now that I've seen many
• 55:55
  Speaker 4: light our based systems use this, and this is a cheat.
• 55:59
  Speaker 4: This is a known cheat, and this is academically published
• 56:02
  Speaker 4: as a cheat, and it's unfortunate that this thing is
• 56:04
  Speaker 4: still happening. So here's the WEIMO system on the on
• 56:14
  Speaker 4: the right here, and you can see the little spinny
• 56:16
  Speaker 4: thing that's a lighter. And like I was saying, no
• 56:18
  Speaker 4: matter what your philosophy is, you still need to have
• 56:21
  Speaker 4: cameras anyway. And what this is doing is it's kind
• 56:23
  Speaker 4: of showing the cleaning system that they're using. So this
• 56:27
  Speaker 4: whole apparatus is being put on top of jaguars in
• 56:31
  Speaker 4: Phoenix and in La San France and maybe a couple
• 56:34
  Speaker 4: other cities now Austin, I think also, and so what
• 56:39
  Speaker 4: I want. So there's a couple major issues with this.
• 56:42
  Speaker 4: So number one, there's a reason why we do wind
• 56:45
  Speaker 4: tunnel testing. And so if you're putting all these light
• 56:48
  Speaker 4: oar systems on it, and it's not just this little
• 56:50
  Speaker 4: thing on top, it's also got light oars on the front,
• 56:53
  Speaker 4: on the two corners on the back, and the two
• 56:55
  Speaker 4: corners I think on the center back as well. So
• 56:58
  Speaker 4: you're increasing the amount of mass and you're increasing the
• 57:00
  Speaker 4: amount of surface area on the exterior of the vehicle
• 57:05
  Speaker 4: in order to do this. And there's a reason why
• 57:06
  Speaker 4: we do windsital testing. So if you do all this,
• 57:09
  Speaker 4: you're gonna basically invalidate all that and this is going
• 57:11
  Speaker 4: to affect your range efficiency. So you're gonna have range
• 57:13
  Speaker 4: problems as a result of this. And this is especially
• 57:15
  Speaker 4: going to become a problem at speed. And so the
• 57:18
  Speaker 4: higher speed you go, the more wind resistance you're going
• 57:20
  Speaker 4: to get in, the more problems you're gonna have with range.
• 57:23
  Speaker 4: Another issue with this, and what I was showing here
• 57:26
  Speaker 4: with this video is the cleaning system. And if you
• 57:30
  Speaker 4: just apply one brain cell. Another thing you can do
• 57:34
  Speaker 4: is you can just take these cameras and put them
• 57:37
  Speaker 4: behind the windshield and then you don't ever have to
• 57:40
  Speaker 4: do this dumb shit ever again. And then you can
• 57:43
  Speaker 4: use the cabin features like the windshield wiper and the
• 57:47
  Speaker 4: defrost settings that are already there and you don't have
• 57:51
  Speaker 4: to build anything new and it only will cost you
• 57:55
  Speaker 4: like a couple hundred dollars and that's it. So that's
• 58:00
  Speaker 4: the kind of dumb we're dealing with. So this is
• 58:03
  Speaker 4: why I needed to do this setup for it before
• 58:05
  Speaker 4: I get into the more complicated stuff, because I need
• 58:07
  Speaker 4: a level set on what kind of level of dumb
• 58:09
  Speaker 4: we're dealing with. So I'm gonna skip this. This is
• 58:15
  Speaker 4: called a distribution shift. I don't want to get too technical,
• 58:17
  Speaker 4: but this is basically saying some stuff about time and
• 58:21
  Speaker 4: how it's relevant. I want to point this company out
• 58:25
  Speaker 4: just briefly. I'm not going to go deep into it,
• 58:26
  Speaker 4: but I just want to just so that people don't
• 58:28
  Speaker 4: think I'm just promoting Tesla whatever, and that's it. This
• 58:31
  Speaker 4: is another company out of the UK. This is the
• 58:35
  Speaker 4: company's name is wave Wayve and this is run by
• 58:39
  Speaker 4: a guy named Alex Kendall, and they're trying to do
• 58:43
  Speaker 4: a monocular based system. So a monocular based system is
• 58:47
  Speaker 4: one camera and they're using like other things like HD
• 58:50
  Speaker 4: maps and probably a couple other things. But the point
• 58:53
  Speaker 4: is their system is one camera and that's all they're using,
• 58:57
  Speaker 4: and this is based out of the UK, and they're
• 59:01
  Speaker 4: trying to build it up. I don't think they're fully
• 59:04
  Speaker 4: deployed yet, or at least not that I'm aware of.
• 59:07
  Speaker 4: But what is important to understand here is that even
• 59:12
  Speaker 4: with one camera, automated vehicles can be achieved. And they've
• 59:17
  Speaker 4: done a really good job with this, Like I'm skipping
• 59:19
  Speaker 4: a lot here, but like it's actually impressive what they're
• 59:22
  Speaker 4: able to do with one camera and how money abstract
• 59:26
  Speaker 4: ways that they've been able to improve on the system
• 59:28
  Speaker 4: that are unconventional. And so the important aspect to understand
• 59:34
  Speaker 4: here is that again, the camera's one hundred dollars and
• 59:37
  Speaker 4: this camera based, this singular camera based system is still
• 59:41
  Speaker 4: capable of doing automated vehicles, and Waimo's spending one hundred
• 59:46
  Speaker 4: and twenty thousand dollars one hundred and thirty thousand dollars
• 59:49
  Speaker 4: per vehicle to not really achieve that so it's one
• 59:54
  Speaker 4: thing for me to say that Waymo's messing up and stuff,
• 59:58
  Speaker 4: but it's another thing to act. Showed the hardcore data
• 1:00:02
  Speaker 4: on it. So I just want to bring this up
• 1:00:05
  Speaker 4: real quick. Well where to go there it is. This
• 1:00:12
  Speaker 4: is from the NITSA dot gov website. So this is
• 1:00:18
  Speaker 4: official crash reporting data. This is a standing general Order
• 1:00:22
  Speaker 4: on crash reporting. You're required to report and this is
• 1:00:27
  Speaker 4: NITSA dot gov if you can see at the top.
• 1:00:30
  Speaker 4: And this is national reporting. So any automated vehicle that
• 1:00:35
  Speaker 4: is operating on public roads you're required to report if
• 1:00:39
  Speaker 4: it's involved in the collision. And what this is again
• 1:00:43
  Speaker 4: national reporting. So I want to iterate again. WEAIMO only
• 1:00:47
  Speaker 4: operates in geofen spaces within specific cities in the country,
• 1:00:53
  Speaker 4: and that's it. WEAIMO. This is this data is being
• 1:00:56
  Speaker 4: reported on. The last reported on November seventeen, twenty twenty five.
• 1:01:00
  Speaker 4: This is continuously updated every couple months, every few months.
• 1:01:03
  Speaker 4: I've been following it since twenty twenty three, and WEIMO
• 1:01:07
  Speaker 4: has about two thousand vehicles on the road. And you
• 1:01:10
  Speaker 4: can see here it's kind of tiny. Here, I'll read
• 1:01:12
  Speaker 4: it for you. It's one thousand four hundred and twenty
• 1:01:15
  Speaker 4: six collisions. So in their fleet of two thousand vehicles
• 1:01:21
  Speaker 4: across only a few different cities, they're reporting one thousand,
• 1:01:26
  Speaker 4: four hundred and twenty six collisions. Now, another thing to
• 1:01:31
  Speaker 4: know about this, Weimo recently expanded to a couple new cities,
• 1:01:37
  Speaker 4: so that fleet of two thousand vehicles is recently like
• 1:01:42
  Speaker 4: increased recently. In May of this year, I saw they
• 1:01:47
  Speaker 4: had like twelve hundred vehicles on the road, fourteen hundred vehicles,
• 1:01:51
  Speaker 4: something like this, and they were reporting roughly eighty percent
• 1:01:55
  Speaker 4: of their fleet has already been involved in a collision.
• 1:01:58
  Speaker 4: And I was I've been watching this chart since twenty
• 1:02:02
  Speaker 4: twenty three, and on average it's eighty percent and sometimes
• 1:02:06
  Speaker 4: I've even seen more than one hundred percent. In other words,
• 1:02:09
  Speaker 4: on average more than one collision per vehicle in their
• 1:02:14
  Speaker 4: fleet on Waimo. Now, let's compare this is Tesla's collisions.
• 1:02:24
  Speaker 4: Tesla has I think five million vehicles on the road
• 1:02:31
  Speaker 4: with the eight ass system, millions and millions of vehicles,
• 1:02:35
  Speaker 4: and also around the world, not just in the United States,
• 1:02:39
  Speaker 4: and they're reporting two thousand I think that says eight
• 1:02:44
  Speaker 4: hundred and forty five collisions with millions of vehicles on
• 1:02:50
  Speaker 4: the road, so pretty large, massive magnitude of difference between
• 1:02:57
  Speaker 4: the two. The collision rate with Waimo's is pretty insane.
• 1:03:05
  Speaker 4: I mean, any other vehicle company that you've ever heard
• 1:03:08
  Speaker 4: of having a sixty eighty hundred percent collision rate and
• 1:03:11
  Speaker 4: still operating after all these years, I've never heard of one,
• 1:03:15
  Speaker 4: but Waimo seems to be able to get away with it.
• 1:03:19
  Speaker 4: And so one of the other issues.
• 1:03:20
  Speaker 3: Here, well, one thing I did want to ask you
• 1:03:23
  Speaker 3: it might be a little too conspiratorial. I was wondering
• 1:03:25
  Speaker 3: you think the LIGHTO is maybe the actually map the area,
• 1:03:29
  Speaker 3: Like it's an excuse, that's just one of the excuses
• 1:03:31
  Speaker 3: it's there.
• 1:03:32
  Speaker 4: There is actually that excuse it is made, and that
• 1:03:34
  Speaker 4: is true.
• 1:03:35
  Speaker 3: Like I wonder if it's actually like collecting data and
• 1:03:38
  Speaker 3: you're thinking like, oh, it's on top of the car
• 1:03:39
  Speaker 3: to help get it somewhere.
• 1:03:40
  Speaker 4: It's too Actually, yeah, so this is actually true. So
• 1:03:43
  Speaker 4: you're you're making a good point. There is actually a
• 1:03:46
  Speaker 4: temporary use for light ar in automated vehicles, and this
• 1:03:51
  Speaker 4: is known so one of them and I will get
• 1:03:53
  Speaker 4: into it a little bit later. But one of the
• 1:03:55
  Speaker 4: major issues is calculating depth This is pretty much one
• 1:03:58
  Speaker 4: of the big problems that needed to be solved in
• 1:04:00
  Speaker 4: AVS is depth calculation. And one of the things that
• 1:04:05
  Speaker 4: they learned is that no matter what your system is,
• 1:04:09
  Speaker 4: even if you're using a computer vision only or into
• 1:04:12
  Speaker 4: system like Tesla, one of the things that they learned
• 1:04:14
  Speaker 4: is that you can temporarily put a light our system
• 1:04:17
  Speaker 4: on the vehicle along with the vision and as you're
• 1:04:21
  Speaker 4: training the model, and you can map the area to
• 1:04:24
  Speaker 4: measure depth calculations around you, and you can use that
• 1:04:29
  Speaker 4: to help train the vision system on how to train depth.
• 1:04:33
  Speaker 4: So but then you would still take the light our
• 1:04:35
  Speaker 4: system off after that. The only reason you would have
• 1:04:37
  Speaker 4: it on is temporarily to get depth calculation to get
• 1:04:40
  Speaker 4: the system kickstarted into training, but then you would remove
• 1:04:44
  Speaker 4: it later on and it would be able to calculate
• 1:04:45
  Speaker 4: depth off of the vision system alone. Now even this
• 1:04:47
  Speaker 4: is not required anymore, but there was. This is an
• 1:04:50
  Speaker 4: argument that has made this is sometimes done and this
• 1:04:53
  Speaker 4: is supposed to be temporary, but this is not what
• 1:04:55
  Speaker 4: Wemo's doing. Weymo's making it, and they've multiple times set
• 1:04:58
  Speaker 4: this in public ands including their executives. They're all saying
• 1:05:03
  Speaker 4: that light oar is here to stay for them, meaning
• 1:05:05
  Speaker 4: that they're trying to make light are a fundamental element
• 1:05:08
  Speaker 4: of their solution. The fund, I should say, the fundamental
• 1:05:13
  Speaker 4: element of their solution, which is nonsense, complete nonsense.
• 1:05:19
  Speaker 1: The vital solution for the Silicon Valley nazis.
• 1:05:23
  Speaker 4: He well, well there's stuff there. It's important to understand
• 1:05:31
  Speaker 4: that Weimo is owned by Google and there are numerous
• 1:05:35
  Speaker 4: incentives for doing this. What I had started off by
• 1:05:38
  Speaker 4: saying was that when Weimo came to Phoenix, they went
• 1:05:42
  Speaker 4: to ASU and they started dumping enormous amounts of money
• 1:05:47
  Speaker 4: into the engineering department at ASU. I didn't know that
• 1:05:49
  Speaker 4: at the time, but that's what ended up getting me
• 1:05:51
  Speaker 4: basically in a lot of problems. And so when I
• 1:05:55
  Speaker 4: was applying to my PhD at ASU, I was using
• 1:06:01
  Speaker 4: this presentation. I presented this to all the lab directors
• 1:06:04
  Speaker 4: at ASU and the Perception and Robotics group, which is
• 1:06:08
  Speaker 4: a five floor building filled with PhDs, and basically none
• 1:06:11
  Speaker 4: of them are working on computer vision. They had one
• 1:06:14
  Speaker 4: robot arm with a single camera and that was like
• 1:06:16
  Speaker 4: their computer vision product. Everything else was light our or
• 1:06:19
  Speaker 4: light our related technologies that they were working on. And
• 1:06:22
  Speaker 4: in fact I had to teach my professors all this
• 1:06:25
  Speaker 4: stuff that I'm talking about. They had no clue about
• 1:06:27
  Speaker 4: computer vision and even some of the basic stuff. At
• 1:06:30
  Speaker 4: a certain point it became completely pointless for MEDI even
• 1:06:32
  Speaker 4: b at ASU, because not the professors, every single person
• 1:06:36
  Speaker 4: in the perception of robotics group, not a single person
• 1:06:38
  Speaker 4: at that time had a clue about anything I'm saying here.
• 1:06:42
  Speaker 4: And it's how do I say this? These problems are
• 1:06:51
  Speaker 4: too dumb for somebody that has a PhD and not
• 1:06:55
  Speaker 4: know about in AI and let alone an entire university
• 1:06:59
  Speaker 4: filled with PhD is not knowing a single clue about
• 1:07:01
  Speaker 4: any of this. That's corruption, that's not a mistake. So
• 1:07:08
  Speaker 4: what I was so earlier, I was saying I wanted
• 1:07:10
  Speaker 4: to do this setup so you can understand some of
• 1:07:12
  Speaker 4: the issues that come with this. Now I'll show you
• 1:07:15
  Speaker 4: what Waimo's light our based system actually sees. So here's
• 1:07:18
  Speaker 4: a visual demonstration of what I'm talking about. So this
• 1:07:22
  Speaker 4: is a point cloud representation of what the Weimo vehicle sees.
• 1:07:26
  Speaker 4: You have the literal cameras on top, and then below
• 1:07:30
  Speaker 4: it is what the light our system is seeing, creating
• 1:07:33
  Speaker 4: a point cloud return representation. This is what I'm showing
• 1:07:37
  Speaker 4: on the bottom is what it means to have a
• 1:07:38
  Speaker 4: point cloud return representation. And like you were saying earlier,
• 1:07:42
  Speaker 4: it's just one dimensional point of light. It's not even
• 1:07:45
  Speaker 4: really the wavelength. It's just that one point and that's it.
• 1:07:48
  Speaker 4: Let me play it again. So this is like a
• 1:07:51
  Speaker 4: person walking across with a box. You can see the
• 1:07:54
  Speaker 4: point cloud representation here again, these people move in this stuff.
• 1:07:59
  Speaker 4: You can see it kind of misses the second person,
• 1:08:01
  Speaker 4: it looks like, but it's very low quality, right, Like
• 1:08:06
  Speaker 4: you would never trade the top for the bottom, right though,
• 1:08:11
  Speaker 4: But somehow they've convinced you that this is exactly what
• 1:08:13
  Speaker 4: you should do. One more time, just to iterate the point.
• 1:08:18
  Speaker 4: This is what a point cloud representation means. Now remember
• 1:08:22
  Speaker 4: this video because I'm going to show you what you
• 1:08:25
  Speaker 4: can do with computer vision, and anybody that has understands
• 1:08:29
  Speaker 4: video games and how video games work and all that stuff,
• 1:08:32
  Speaker 4: you're about to have your mind blown because it's exactly
• 1:08:34
  Speaker 4: the same thing. So whatever you can do in video games,
• 1:08:38
  Speaker 4: that's what you can do with computer vision, and you
• 1:08:40
  Speaker 4: cannot do that dumb shit with light art. Okay, this
• 1:08:47
  Speaker 4: is in my opinion, the most important slide, and this
• 1:08:51
  Speaker 4: is the slide that got me in trouble. I know
• 1:08:54
  Speaker 4: there's a lot on here. I'm going to break it down.
• 1:08:56
  Speaker 4: It's not that hard, but this is really worth knowing.
• 1:09:01
  Speaker 4: And in my opinion, like I sh, I want to
• 1:09:08
  Speaker 4: mention that there's a lot of things that I'm not saying.
• 1:09:11
  Speaker 4: Some of it is to condense, but a lot of
• 1:09:14
  Speaker 4: it is also to avoid creating even more problems for myself.
• 1:09:19
  Speaker 4: I'm happy to talk about things if you want to,
• 1:09:21
  Speaker 4: but I'm gonna skip some of the really bad stuff
• 1:09:25
  Speaker 4: that could get me in a lot of trouble. But
• 1:09:27
  Speaker 4: this is this is good. This got me in enough
• 1:09:28
  Speaker 4: trouble what I have here on the screen, This got
• 1:09:30
  Speaker 4: me in enough trouble. But I already did this, so
• 1:09:33
  Speaker 4: I'll go ahead and talk about it. So this is
• 1:09:36
  Speaker 4: the state. So if you ever see the phrase soda
• 1:09:41
  Speaker 4: Sota state of the art, soda state of the art,
• 1:09:46
  Speaker 4: So this is the state of the art. Way MOO
• 1:09:49
  Speaker 4: leaderboard results. On June twenty ninth, twenty twenty three WHIPS.
• 1:09:54
  Speaker 4: This was presented at CVPR twenty twenty three, the Major
• 1:09:57
  Speaker 4: Computer Vision Conference by Chen Wu who's for WAMEL. I
• 1:10:01
  Speaker 4: have the link here and this is the paper that
• 1:10:03
  Speaker 4: is associated with that presentation on the left side. So
• 1:10:08
  Speaker 4: this is the reference paper that I'm talking about on
• 1:10:10
  Speaker 4: the left side. Here on the left side you have
• 1:10:13
  Speaker 4: this table. This table represents what is industry standard for
• 1:10:20
  Speaker 4: object classification around the world. Everyone in the world that
• 1:10:27
  Speaker 4: is doing object identification and classification in AI, they all
• 1:10:34
  Speaker 4: use MIOU mean intersection over union in this task. This
• 1:10:41
  Speaker 4: is industry standard, not just in America, everywhere in the world.
• 1:10:46
  Speaker 4: This is well understood, well known. This is standard. What
• 1:10:51
  Speaker 4: MIOU is that there is a minimum. It's at the
• 1:10:54
  Speaker 4: time it was a minimum sixteen classes, but actually the
• 1:10:59
  Speaker 4: numbers in significantly since then. I think in this case
• 1:11:02
  Speaker 4: WEIMO was doing twenty two which at the time in
• 1:11:04
  Speaker 4: twenty twenty three was acceptable to do twenty two classes.
• 1:11:08
  Speaker 4: Since then this number has increased to even more classes.
• 1:11:10
  Speaker 4: But just know at the time this is actually industry
• 1:11:14
  Speaker 4: standard and accepted. You're going to have two sets of
• 1:11:18
  Speaker 4: data here, validation set and the test set. The validation
• 1:11:22
  Speaker 4: set is the performance of the model on the training data,
• 1:11:26
  Speaker 4: and then the test set, which is the real measure
• 1:11:30
  Speaker 4: of performance. The test set is on data after the
• 1:11:34
  Speaker 4: model has been trained and is being tested on scenarios
• 1:11:38
  Speaker 4: that it hasn't ever seen before. So on average, you
• 1:11:41
  Speaker 4: would expect that the validation set would be or I mean,
• 1:11:45
  Speaker 4: the test set would be lower performance than the validation set,
• 1:11:48
  Speaker 4: So that's supposed to be it. So it's all these
• 1:11:50
  Speaker 4: different classes. They measure the performance on each one and
• 1:11:53
  Speaker 4: then they do an intersection over union on them, and
• 1:11:56
  Speaker 4: then they average it, and that's what this is table
• 1:11:58
  Speaker 4: is saying. So what you're getting here is forty six
• 1:12:01
  Speaker 4: point eighty two percent accuracy on MIOU from WEIMO at
• 1:12:07
  Speaker 4: this one and they presented us this table was on
• 1:12:11
  Speaker 4: the last page of their paper here and it was
• 1:12:16
  Speaker 4: not referenced in the paper, and this table was hidden
• 1:12:20
  Speaker 4: at the very end, not talked about. Everything else on
• 1:12:25
  Speaker 4: this slide is what weimo's actually doing. This table is
• 1:12:30
  Speaker 4: industry standard. Everything else is not. This is what weimo's doing,
• 1:12:34
  Speaker 4: So here we go. What you need to know is this.
• 1:12:39
  Speaker 4: Look at this line that says swformer dot threef hours,
• 1:12:43
  Speaker 4: referring to WEIMO, and then also down here s w
• 1:12:47
  Speaker 4: former threef hours. These two lines are the lines you
• 1:12:50
  Speaker 4: need to look at. What you'll see is they're showing
• 1:12:53
  Speaker 4: the performance using this metric apaph on pedestrians, for example,
• 1:13:00
  Speaker 4: Whatever the hell this apaph metric is not miou and
• 1:13:06
  Speaker 4: I have never seen this metric before, so I had
• 1:13:08
  Speaker 4: to go find it. What the hell is this apaaph
• 1:13:11
  Speaker 4: thing they're showing on this whatever this metric is apaph,
• 1:13:15
  Speaker 4: whatever the hell that means? This thing is showing an
• 1:13:18
  Speaker 4: eighty two point nine percent accuracy rate up here or
• 1:13:23
  Speaker 4: maybe down here, like eighty two point one three on
• 1:13:25
  Speaker 4: the pedestrian L one whatever ap me and aph. So
• 1:13:30
  Speaker 4: I went and looked for where the hell did this
• 1:13:32
  Speaker 4: come from? And that's what's on the right. And it
• 1:13:35
  Speaker 4: turns out this metric apaph it was also made by
• 1:13:41
  Speaker 4: Waimo in a paper from twenty nineteen, which I'm showing
• 1:13:44
  Speaker 4: here on the right side. And what this effectively does
• 1:13:49
  Speaker 4: this is the formula for it on the right side. Whoops.
• 1:13:52
  Speaker 4: What this does is it takes this data on the left,
• 1:13:57
  Speaker 4: runs it through this magic formula on the right, and
• 1:14:00
  Speaker 4: outputs these results. That's literally what it does. I'm not exaggerating.
• 1:14:05
  Speaker 4: That's what it does. It takes this data, runs it
• 1:14:08
  Speaker 4: through that formula, manufactures this data and as you can see,
• 1:14:13
  Speaker 4: the performance that they're showing off of this metric, which
• 1:14:16
  Speaker 4: is what they're using for their safety reports and what
• 1:14:19
  Speaker 4: they published in their safety data in twenty twenty three
• 1:14:22
  Speaker 4: and twenty twenty four. This data here is almost double
• 1:14:27
  Speaker 4: the performance of the industry standard, and they totally ignored
• 1:14:30
  Speaker 4: the industry standard forty six percent versus eighty two point
• 1:14:34
  Speaker 4: nine percent or whatever on a metric that they invented.
• 1:14:38
  Speaker 4: So let me make it clear here. WEIMO invented the
• 1:14:41
  Speaker 4: metric that nobody in the world uses, and then they
• 1:14:45
  Speaker 4: use that metric for their safety data, and that metric
• 1:14:50
  Speaker 4: that they invented is showing a performance that is nearly
• 1:14:53
  Speaker 4: double the actual performance that is known as industry standard
• 1:14:57
  Speaker 4: around the world. So this is important to know. Now,
• 1:15:03
  Speaker 4: one other thing that I was saying earlier and the
• 1:15:05
  Speaker 4: previous side was the degradation of lid or over range.
• 1:15:09
  Speaker 4: So what you're getting these point cloud returns. One of
• 1:15:13
  Speaker 4: the things that happens is that the number of points
• 1:15:17
  Speaker 4: that you get from the light our system the farther
• 1:15:21
  Speaker 4: you go out is lower. That's why the quality goes down.
• 1:15:25
  Speaker 4: So when I was saying fifty meters one hundred meters,
• 1:15:28
  Speaker 4: one hundred and fifty meters, the argoverse results were showing
• 1:15:31
  Speaker 4: degradation at a log two rate half every fifty meters.
• 1:15:35
  Speaker 4: This this thing that I just described, this is called sparsity.
• 1:15:39
  Speaker 4: In AI terms, this is referred to as sparsity. How
• 1:15:43
  Speaker 4: sparse are the points? You want to have higher resolution data?
• 1:15:48
  Speaker 4: More points? Well, these results were showing degradation at range,
• 1:15:54
  Speaker 4: and I was also stating that it seems like research
• 1:15:58
  Speaker 4: on this specific topic about degradation over range seems to
• 1:16:01
  Speaker 4: be heavily suppressed because I just can't find anything anyone
• 1:16:06
  Speaker 4: really doing any serious research on it. And it's not,
• 1:16:10
  Speaker 4: in my opinion, it's impossible that nobody thought about that.
• 1:16:13
  Speaker 4: For how prevalent and common, and how many businesses, major
• 1:16:17
  Speaker 4: multi billion dollar businesses have been using lightar and failed,
• 1:16:21
  Speaker 4: the graveyard of businesses that have tried to make lightar
• 1:16:25
  Speaker 4: work and failed, there is no possible way that nobody
• 1:16:30
  Speaker 4: thought about degradation of lightar over range. I promise you
• 1:16:35
  Speaker 4: this is like from an engineering perspective, you have to
• 1:16:38
  Speaker 4: be completely idiot. You have to be a complete idiot
• 1:16:40
  Speaker 4: to not think about that. This is like even in cameras,
• 1:16:44
  Speaker 4: you think about like video games, you know about like
• 1:16:46
  Speaker 4: how you have lower quality pixels. Further down you go
• 1:16:49
  Speaker 4: and in order to say, remember, you have to understand
• 1:16:51
  Speaker 4: this stuff. It's like at an elementary level. There's no
• 1:16:55
  Speaker 4: possible way people didn't think about that. So why is
• 1:16:58
  Speaker 4: that important? I want to read this thing that came
• 1:17:00
  Speaker 4: from the paper. Some of the latest commercial light ours
• 1:17:03
  Speaker 4: can sense up to two hundred and fifty and three
• 1:17:06
  Speaker 4: hundred meters in all directions around the vehicle, leading to
• 1:17:10
  Speaker 4: a large range of point clouds. Okay, so they're fully
• 1:17:13
  Speaker 4: aware of point how point clouds work, and they're talking
• 1:17:16
  Speaker 4: about how long range lightar can go up to this range.
• 1:17:19
  Speaker 4: But notice they don't tell you how sparse it is
• 1:17:21
  Speaker 4: at that point. So let's go to this bottom right here,
• 1:17:25
  Speaker 4: I'm going to read this. Our experiments are primarily based
• 1:17:30
  Speaker 4: on the challenging WEAIMO open data set. Okay, One thing
• 1:17:33
  Speaker 4: to know about this WEIMO open data set. At the
• 1:17:36
  Speaker 4: time that I published this, there were a total which
• 1:17:39
  Speaker 4: is November twenty twenty three, at the time I was
• 1:17:43
  Speaker 4: publishing this, which is four or five months after CVPR.
• 1:17:47
  Speaker 4: This CVPR, they had a total of six submissions to
• 1:17:52
  Speaker 4: the WEIMO open data set. By comparison, in Nvidia at
• 1:17:57
  Speaker 4: on their data set had more than four hundred submissions
• 1:18:04
  Speaker 4: and I'm bringing that up because the next line they
• 1:18:08
  Speaker 4: say is which has been adopted in many recent state
• 1:18:12
  Speaker 4: of the art three D detection methods. As far as
• 1:18:18
  Speaker 4: I'm aware, basically nobody in the world uses the Waymo
• 1:18:21
  Speaker 4: open data set other than weymo, and even the submissions,
• 1:18:25
  Speaker 4: the six submissions that I saw in November twenty twenty three,
• 1:18:28
  Speaker 4: even those submissions were from nobody's I've never heard of
• 1:18:31
  Speaker 4: any of them. But on the in Vidia data set,
• 1:18:35
  Speaker 4: with more than four hundred, every major computer vision every
• 1:18:40
  Speaker 4: major university that competes in computer vision has competed there,
• 1:18:43
  Speaker 4: including in Vidia and all other major computer vision participants,
• 1:18:50
  Speaker 4: it's like the standard. Effectively, the data set contains one
• 1:18:55
  Speaker 4: hundred and fifty seven split into seven hundred and ninety
• 1:18:58
  Speaker 4: eight training, two hundred two validation one hundred fIF Each
• 1:19:00
  Speaker 4: scene has about two hundred frames, where each frame captures
• 1:19:05
  Speaker 4: the full three hundred and sixty degrees around the EGO vehicle.
• 1:19:08
  Speaker 4: The data set has one long range lighter with the
• 1:19:12
  Speaker 4: range capped at seventy five meters, four near range lightar,
• 1:19:18
  Speaker 4: and five cameras, so it doesn't matter that the long
• 1:19:24
  Speaker 4: range lighter can since up to two hundred and fifty
• 1:19:29
  Speaker 4: to three hundred meters. They're capping it at seventy five.
• 1:19:32
  Speaker 4: And this helps support what I was saying earlier and
• 1:19:35
  Speaker 4: what Argoverse results were showing at the same timeframe as this,
• 1:19:39
  Speaker 4: that lighter degrades over range. And notice the seventy five
• 1:19:43
  Speaker 4: meter cap just a little beyond the fifty meter degradation.
• 1:19:46
  Speaker 1: How much you would have bet that that seventy five
• 1:19:49
  Speaker 1: meter cap that they're giving it is probably a way
• 1:19:52
  Speaker 1: over estimation as well.
• 1:19:55
  Speaker 4: Yeah, I'm pretty sure. Yeah, you're going to see some stuff.
• 1:20:02
  Speaker 4: I'm just being a little I'm not trying to be.
• 1:20:08
  Speaker 1: Yeah, they're overestimated. Even in that seventy five meters. They
• 1:20:11
  Speaker 1: would have gone higher if they could have.
• 1:20:15
  Speaker 4: Yeah, they would have gone high there. I mean, the
• 1:20:17
  Speaker 4: seventy five meters is sixty miles an hour, gives you
• 1:20:19
  Speaker 4: less than three seconds of forecasting time, you know, So
• 1:20:22
  Speaker 4: that's still not much better, you know. So let me
• 1:20:29
  Speaker 4: talk a little bit. I mean, I'll skip all this
• 1:20:31
  Speaker 4: technical stuff, but if anyone cares and wants to know
• 1:20:34
  Speaker 4: about the architecture and how it really works for the
• 1:20:36
  Speaker 4: camera based system, this is actually a really useful slide.
• 1:20:39
  Speaker 4: It pretty much lays out the actual architecture if anyone
• 1:20:41
  Speaker 4: actually cares. But I'm going to try and skip all
• 1:20:44
  Speaker 4: the really technical stuff. I do want to just mention
• 1:20:49
  Speaker 4: something from this paper, which was a big deal. This
• 1:20:53
  Speaker 4: is called one Former. This was a big deal in
• 1:20:55
  Speaker 4: computer vision and all this stuff. The element here that
• 1:20:59
  Speaker 4: I'm just going to briefly is that, as you would expect,
• 1:21:03
  Speaker 4: it turns out that vision also improves large language models semantics,
• 1:21:10
  Speaker 4: and it's actually a two way road. They kind of
• 1:21:13
  Speaker 4: go hand in hand. But the point is that vision
• 1:21:17
  Speaker 4: is an augment to an LLM. So another way of
• 1:21:21
  Speaker 4: saying that is being able to see a red wagon
• 1:21:25
  Speaker 4: allows you to be able to describe a red wagon
• 1:21:29
  Speaker 4: in language better. And this one form is kind of
• 1:21:32
  Speaker 4: showing all that. Let me skip some of this stuff.
• 1:21:38
  Speaker 4: This is technical stuff. It's kind of cool, shows some
• 1:21:42
  Speaker 4: really interesting things, but I kind of want to get
• 1:21:45
  Speaker 4: away from it. Segment anything for anyone in computer vision
• 1:21:48
  Speaker 4: is totally worth knowing. This is a This was made
• 1:21:52
  Speaker 4: by Meta and when it came out, it was a
• 1:21:56
  Speaker 4: big deal because it did it did object classification pretty
• 1:22:02
  Speaker 4: robustly across a wide variety of topics, not just avs,
• 1:22:06
  Speaker 4: like not just like useful in the av industry, but
• 1:22:11
  Speaker 4: useful across a variety of tasks, and it was a
• 1:22:15
  Speaker 4: big deal. So this is where we'll get into the
• 1:22:20
  Speaker 4: argument for depth and why light oars were adopted. So again,
• 1:22:25
  Speaker 4: the primary and significant function that lightar provides avs is
• 1:22:32
  Speaker 4: a depth calculation. And the reason why this is completely
• 1:22:37
  Speaker 4: useless because number one, there's no reason to have super
• 1:22:40
  Speaker 4: precise death depth. There's no reason to have laser precision
• 1:22:43
  Speaker 4: on depth. The argument went like this, the major problem
• 1:22:49
  Speaker 4: in avs is collision, and because lightar does accurate depth perception,
• 1:22:57
  Speaker 4: highly accurate, this is how you have collision. This was
• 1:23:01
  Speaker 4: the crux and fundamental purpose of the argument for lidar.
• 1:23:06
  Speaker 4: But if you think about your human eyes and how
• 1:23:08
  Speaker 4: you do it in real life, you're not trying to
• 1:23:11
  Speaker 4: calculate the distance between you and an object in front
• 1:23:14
  Speaker 4: of you. The only thing you care about when you
• 1:23:18
  Speaker 4: in your everyday real life is you care about this
• 1:23:21
  Speaker 4: object is in front of that object, but behind that object.
• 1:23:26
  Speaker 4: This is called relative distance. So this is how the
• 1:23:30
  Speaker 4: brain does it, and this is actually the better way
• 1:23:35
  Speaker 4: of doing it. It's way more efficient. So one of
• 1:23:38
  Speaker 4: the reasons why we have binocular vision two eyes is
• 1:23:42
  Speaker 4: because to having two eyes aka two cameras is how
• 1:23:47
  Speaker 4: our brain does depth perception on a relative basis. When
• 1:23:51
  Speaker 4: you have two fixed points that you're using as a
• 1:23:54
  Speaker 4: camera with known dimensions, you can create triangles off of this,
• 1:24:00
  Speaker 4: and then everything becomes geometry and that's it. You don't
• 1:24:04
  Speaker 4: you can even make up then like the actual real
• 1:24:06
  Speaker 4: precise numbers don't even matter. You can be as long
• 1:24:09
  Speaker 4: as they're relatively correct. It doesn't actually matter what the
• 1:24:13
  Speaker 4: numbers are. You can even make up numbers as you go.
• 1:24:15
  Speaker 4: The only thing that matters is relative to death.
• 1:24:18
  Speaker 1: Now, insects, look at the eyes on insects. It's just
• 1:24:22
  Speaker 1: massive eyes, just over and over again, just so that
• 1:24:25
  Speaker 1: it has perfect you know, it's it's got depth perception
• 1:24:28
  Speaker 1: and everything else. Yeah, all all you know, wired together
• 1:24:32
  Speaker 1: with those eyes.
• 1:24:33
  Speaker 4: And you'll even notice that the eyes on say a fly,
• 1:24:37
  Speaker 4: for example, are like they look complicated, but they're actually
• 1:24:41
  Speaker 4: just repeating the same thing over and over again, which
• 1:24:44
  Speaker 4: one fundamental concept and what they care about. A fly
• 1:24:48
  Speaker 4: cares about the most is motion detection. And that's why
• 1:24:52
  Speaker 4: the eyes bulge out and they can see behind them
• 1:24:54
  Speaker 4: and all around. That's why the globe shape and they're
• 1:24:56
  Speaker 4: doing object detection sorry, motion motion de and that's the
• 1:25:02
  Speaker 4: fundamental reason why it is. And you can even get
• 1:25:05
  Speaker 4: hints of evolutionary purpose evolutionary progress. Sorry, you can get
• 1:25:09
  Speaker 4: hints of evolutionary progress just by looking at the eye
• 1:25:12
  Speaker 4: throughout all the species. The more evolved the eye, probably
• 1:25:15
  Speaker 4: the more intelligent the species. If all the eye that
• 1:25:18
  Speaker 4: they could this is some animal or or whatever it
• 1:25:21
  Speaker 4: can do, is look do objects or motion detection, then
• 1:25:25
  Speaker 4: it's probably a primitive eye that probably has gray cylinders
• 1:25:27
  Speaker 4: or something equivalent, right, gray scale. But the more evolved
• 1:25:31
  Speaker 4: the eye is, with more color perception and more quality,
• 1:25:35
  Speaker 4: then this is probably a more intelligent species. And one
• 1:25:38
  Speaker 4: of the things that I kind of I'm not going
• 1:25:40
  Speaker 4: to hit too hard on this, but one of the
• 1:25:42
  Speaker 4: things I want to demonstrate is that consciousness is a gradient.
• 1:25:45
  Speaker 4: One of the things is that there's not a real
• 1:25:47
  Speaker 4: definition of consciousness. And why is it difficult to measure?
• 1:25:51
  Speaker 4: It's because consciousness is a gradient. It's not one. It's
• 1:25:54
  Speaker 4: not like you just suddenly have a certain level that
• 1:25:57
  Speaker 4: you hit and then all of a sudden you're conscious.
• 1:25:59
  Speaker 4: Nonsciousness is a gradient, and it goes from simple to complex,
• 1:26:04
  Speaker 4: and it can get even more complex than we even are.
• 1:26:08
  Speaker 4: So I want to get to the real fun stuff.
• 1:26:15
  Speaker 4: So I was just talking about depth perception. So and
• 1:26:21
  Speaker 4: earlier I had mentioned that bounding boxes are a cheat. Well,
• 1:26:26
  Speaker 4: this is the solution to that. To those problems, depth
• 1:26:31
  Speaker 4: perception and bounding boxes are both solved by this. So
• 1:26:36
  Speaker 4: in the AV industry it has been completely adopted, including Waimo,
• 1:26:41
  Speaker 4: every single major company or philosophy. It doesn't matter what
• 1:26:46
  Speaker 4: your belief system is. Every participant in the AV industry
• 1:26:51
  Speaker 4: has adopted this thing called an occupancy network. And this
• 1:26:57
  Speaker 4: is one of two AI cons that I'm going to
• 1:27:00
  Speaker 4: be demonstrating. This is when earlier I was saying I
• 1:27:04
  Speaker 4: was hoping to teach people what real AI is and
• 1:27:07
  Speaker 4: distinguish that from fake AI. This is the one of
• 1:27:10
  Speaker 4: the two things you need to know in order to
• 1:27:13
  Speaker 4: understand what is real AI and what is not. An
• 1:27:17
  Speaker 4: occupancy network is one, and the next thing I'll talk
• 1:27:20
  Speaker 4: about is something called nerve. These two concepts you must
• 1:27:23
  Speaker 4: know if you want to understand anything about AI going forward.
• 1:27:26
  Speaker 4: And this is what distinguishes real AI from all the
• 1:27:30
  Speaker 4: other junk. So an occupancy network, what it is is
• 1:27:35
  Speaker 4: it's a grid pattern that is made around the vehicle itself,
• 1:27:40
  Speaker 4: and this grid pattern is filled with these Minecraft looking
• 1:27:43
  Speaker 4: boxes all around it. And these Minecraft looking boxes, these
• 1:27:48
  Speaker 4: boxes are called voxels, and these voxels are made are
• 1:27:53
  Speaker 4: heat mapped all around the vehicle. So it's basically a grid.
• 1:27:57
  Speaker 4: And then you put objects on this grid and it
• 1:27:59
  Speaker 4: doesn't matter the exact precise distance as long as you
• 1:28:02
  Speaker 4: get it pixel correct. Whatever your pixel resolution is will
• 1:28:06
  Speaker 4: be the resolution of your occupancy network and these voxels.
• 1:28:10
  Speaker 4: What is really simple and powerful about an occupancy network
• 1:28:18
  Speaker 4: and why it was adopted by the industry is because
• 1:28:21
  Speaker 4: it gives you a simple mechanism, a zero and one
• 1:28:25
  Speaker 4: binary mechanism for drive and not drive. So you can
• 1:28:30
  Speaker 4: see here in blue, this blue area is where you
• 1:28:34
  Speaker 4: can drive, and then everything else is no drive, don't drive.
• 1:28:39
  Speaker 1: If you're gonna if you're gonna relate this to a sense,
• 1:28:42
  Speaker 1: it would have to be touch, right, because each and
• 1:28:45
  Speaker 1: every one of these things have different bumps and different layers,
• 1:28:49
  Speaker 1: so that as you're reaching out, you're touching these boxes,
• 1:28:52
  Speaker 1: which is a generalized form, and you're getting the feel
• 1:28:55
  Speaker 1: for it how far or close it is. So it's
• 1:28:58
  Speaker 1: a very tactile sais with this form right here.
• 1:29:02
  Speaker 4: Good, So you're absolutely right, and so eventually I'm going
• 1:29:06
  Speaker 4: to show the evolution of this. So this is way.
• 1:29:08
  Speaker 4: This is an old version. I know that says twenty
• 1:29:10
  Speaker 4: twenty three, but this is actually an old version of
• 1:29:12
  Speaker 4: Tesla's occupancy network. I think it is. Actually this is
• 1:29:15
  Speaker 4: the twenty twenty version, if I recall correctly. So this
• 1:29:18
  Speaker 4: is an old version, and I'm just showing this to
• 1:29:21
  Speaker 4: show you the evolution of what happened in the industry
• 1:29:25
  Speaker 4: the occupancy because this is fundamentally important an occupancy network,
• 1:29:29
  Speaker 4: and you're right. You are going to get the texture
• 1:29:32
  Speaker 4: quality as a result of this, and there's a few
• 1:29:35
  Speaker 4: there's actually a number of other benefits. But first and foremost,
• 1:29:40
  Speaker 4: the thing to understand here is that the reason why
• 1:29:43
  Speaker 4: this was adopted into the av industry. No matter your philosophy,
• 1:29:47
  Speaker 4: whether you're using lidar or not, it doesn't matter, you're
• 1:29:50
  Speaker 4: still going to be doing this is because it provided
• 1:29:53
  Speaker 4: an automatic implicit collision avoidance system. This through software alone,
• 1:30:02
  Speaker 4: this occupancy network automatically created an implicit collision avoidance system,
• 1:30:09
  Speaker 4: and that alone was what was a lot enabled it
• 1:30:13
  Speaker 4: which made it attractive to put into avs. Now, there
• 1:30:18
  Speaker 4: are many other benefits that I'm going to talk about
• 1:30:20
  Speaker 4: that ended up happening, and this evolved even further. But
• 1:30:24
  Speaker 4: important to understand here that basically, think of Minecraft. You
• 1:30:27
  Speaker 4: have a zero one mechanism for where you can drive
• 1:30:30
  Speaker 4: and where not to drive. And in the you know,
• 1:30:34
  Speaker 4: in the zero part, the part you don't drive these
• 1:30:37
  Speaker 4: little boxes. What they've done is they've added in a
• 1:30:40
  Speaker 4: bunch of little data points, so they add further data
• 1:30:44
  Speaker 4: into the not drive part. And then the drive part
• 1:30:47
  Speaker 4: is basically the road. That's it, just the road, and
• 1:30:50
  Speaker 4: that's simple, binary and super powerful. And it turns out
• 1:30:56
  Speaker 4: and this is this is the solution to the bounding problem.
• 1:31:01
  Speaker 4: And I'm going to get into that. So the bounding
• 1:31:03
  Speaker 4: box problem I was saying was a was a cheat.
• 1:31:07
  Speaker 4: Oh shit, I'm sorry, I need to talk about nerves first.
• 1:31:12
  Speaker 4: Pause on the bounding box problem. I need to talk
• 1:31:14
  Speaker 4: about this first. I got a mixed up. So the
• 1:31:16
  Speaker 4: I was saying, there are two things that we need
• 1:31:18
  Speaker 4: to know in real AI. The occupancy network is one,
• 1:31:24
  Speaker 4: and then the second thing you need to know. And
• 1:31:25
  Speaker 4: this is as technical as I'm going to get there
• 1:31:28
  Speaker 4: is nerves now NERF, n ERF, NERF. This thing was invented.
• 1:31:34
  Speaker 4: This is a technology that was invented in twenty nineteen,
• 1:31:38
  Speaker 4: I think or twenty eighteen, twenty nineteen, and this stands
• 1:31:42
  Speaker 4: for neural radiance field. And now, if you play video games,
• 1:31:47
  Speaker 4: you're going to have an advantage in understanding all this because
• 1:31:50
  Speaker 4: a neural radiance field is very similar in concept to
• 1:31:54
  Speaker 4: raytracing in a video game. And so what a radiance
• 1:31:58
  Speaker 4: field is is that it's the ray from a pixel
• 1:32:05
  Speaker 4: on a frame. It's the ray vector that is pretty
• 1:32:10
  Speaker 4: much identical to what ray tracing is in video games,
• 1:32:14
  Speaker 4: and it's each pixel doing that ray tracing. And that's
• 1:32:19
  Speaker 4: what a NERF is. That's originally what a NERF was.
• 1:32:22
  Speaker 4: So NERF started out in twenty nineteen as a new technology,
• 1:32:27
  Speaker 4: and what happened is this thing that's called a radiance field,
• 1:32:31
  Speaker 4: this NERF thing, it evolved into a whole branch of technologies. Now,
• 1:32:36
  Speaker 4: this whole branch of technologies in AI is just colloquially
• 1:32:40
  Speaker 4: collectively referred to as NERF. But they haven't since then.
• 1:32:45
  Speaker 4: Since then, they have made like thousands and thousands of
• 1:32:48
  Speaker 4: different evolutions of this thing. Called nerve. Now, earlier I
• 1:32:53
  Speaker 4: was showing the occupancy network is these like Minecraft looking
• 1:32:57
  Speaker 4: boxes fossils. What happened oh time is that those voxels
• 1:33:03
  Speaker 4: reduced in size over time and eventually became single points.
• 1:33:09
  Speaker 4: And what you were able to do with that is
• 1:33:12
  Speaker 4: you were able to make an occupancy network and the
• 1:33:15
  Speaker 4: NERF effectively the same thing. So occupancy networks and nurse
• 1:33:21
  Speaker 4: initially started out as two independent, completely different things, and
• 1:33:26
  Speaker 4: over time they came together and became basically the same thing.
• 1:33:33
  Speaker 4: So another way of understanding this is that ray tracing
• 1:33:36
  Speaker 4: in video games and occupancy networks in aves basically the
• 1:33:41
  Speaker 4: same thing. And so this is like if you play
• 1:33:45
  Speaker 4: video games, that should be nuts because that means that
• 1:33:47
  Speaker 4: everything that is happening in vehicles in robotics is a
• 1:33:51
  Speaker 4: video game. It is like literally it is that. It
• 1:33:54
  Speaker 4: is not like that, it is that equal to that.
• 1:34:00
  Speaker 4: So these two things became the same thing. This is
• 1:34:03
  Speaker 4: why I'm saying you need to know these two things,
• 1:34:05
  Speaker 4: not that you have to understand everything about it. You
• 1:34:08
  Speaker 4: just need to know what this is because this is
• 1:34:11
  Speaker 4: real AI and you're gonna see what happens with this.
• 1:34:14
  Speaker 4: So I'll even see that word. So NERF is a
• 1:34:18
  Speaker 4: new concept, I'm sure to people. So I'm just gonna
• 1:34:21
  Speaker 4: show like a few examples so that people get an
• 1:34:23
  Speaker 4: understanding of what it is. So initially NERF. So initially
• 1:34:29
  Speaker 4: NERF was slow, you know, like all new things. But
• 1:34:33
  Speaker 4: then eventually there was this paper that was called plinoxiles
• 1:34:37
  Speaker 4: and they figured out how to do it way faster,
• 1:34:39
  Speaker 4: especially with GPUs. And you'll see in this video here
• 1:34:44
  Speaker 4: the difference. So the left one is like what it
• 1:34:46
  Speaker 4: was originally trying to do. This is what NERF is
• 1:34:48
  Speaker 4: originally trying to do. It's trying to render the object.
• 1:34:52
  Speaker 4: But you can see that it was like slow and
• 1:34:54
  Speaker 4: has some problems and has some difficulty. But then of
• 1:34:57
  Speaker 4: course they basically sped this thing up like really fast.
• 1:35:01
  Speaker 4: So what I'm going to eventually build up to is
• 1:35:03
  Speaker 4: can you do this in real time? Yes you can,
• 1:35:04
  Speaker 4: And so I'm going to build up to, Yes, you
• 1:35:07
  Speaker 4: can render NERVE aka ocupency network in real time. If
• 1:35:11
  Speaker 4: you're an AI person, this matters to you because if
• 1:35:13
  Speaker 4: you're an av navigating through the world, it's one thing
• 1:35:17
  Speaker 4: to be able to build of the environment around you,
• 1:35:20
  Speaker 4: but can you do it in real time? Which is
• 1:35:22
  Speaker 4: an even harder right, you have to do it fast.
• 1:35:25
  Speaker 4: So can you do radiance fields aka ocupency networks in
• 1:35:28
  Speaker 4: real time? The answer is going to be yes, and
• 1:35:30
  Speaker 4: I'm going to demonstrate that these are the results if
• 1:35:34
  Speaker 4: you need to care about this. So let me show
• 1:35:37
  Speaker 4: you one step of evolution. This is not the modern
• 1:35:40
  Speaker 4: version of the occupens the network in the tessel system.
• 1:35:42
  Speaker 4: This is like what twenty twenty two, So you can
• 1:35:44
  Speaker 4: see just in a couple of years how the evolution
• 1:35:47
  Speaker 4: is lower resolution I mean sorry, higher resolution voxels, smaller vauxels,
• 1:35:53
  Speaker 4: and what you're getting here. You can see that this
• 1:35:56
  Speaker 4: is still a zero one mechanism, just like I was
• 1:35:58
  Speaker 4: saying before for drive, no drive. But look at this.
• 1:36:00
  Speaker 4: So these are the cameras on top, the literal cameras
• 1:36:03
  Speaker 4: on top, and then this is what the car sees.
• 1:36:06
  Speaker 4: So this is the evolution of the occupancy network over
• 1:36:09
  Speaker 4: probably a couple of years. So a little bit higher
• 1:36:12
  Speaker 4: resolution you're seeing, and you can see the car shapes
• 1:36:16
  Speaker 4: are starting to become more defined. And remember how I
• 1:36:19
  Speaker 4: was saying, uh, bounding boxes are a cheat. These little
• 1:36:24
  Speaker 4: shapes right here, this is how you get around the
• 1:36:27
  Speaker 4: bounding box problem. Once you apply them to a grid
• 1:36:30
  Speaker 4: and you make the voxels as small as possible to
• 1:36:33
  Speaker 4: individual points. Then the task becomes getting the shapes pixel perfect,
• 1:36:39
  Speaker 4: and you have a pixel perfect shape through an occupancy
• 1:36:42
  Speaker 4: network for free. And still so many people are using
• 1:36:46
  Speaker 4: bounding boxes, which is a total cheat.
• 1:36:53
  Speaker 1: This reminds me of Blender.
• 1:36:56
  Speaker 4: Blender, Oh Blender, yeah, right.
• 1:36:59
  Speaker 1: The three D software, Like you could just plug just
• 1:37:03
  Speaker 1: straight into there and have a total model of the
• 1:37:05
  Speaker 1: entire place, and then I guess share it between different vehicles.
• 1:37:09
  Speaker 1: Isn't that what's also happening as well, Like there's a
• 1:37:12
  Speaker 1: sharing between different vehicles with all these.
• 1:37:16
  Speaker 4: I think that, I mean, this is that's more theoretical.
• 1:37:19
  Speaker 4: How do I say it's it's not commercially available like
• 1:37:22
  Speaker 4: Tesla's don't do it yet, but this is part of
• 1:37:25
  Speaker 4: the plan that they're working on. Yeah, and yeah, that
• 1:37:30
  Speaker 4: that you can assume that that'll be obvious. It's yeah.
• 1:37:34
  Speaker 4: I would say that in terms of priorities, like what
• 1:37:37
  Speaker 4: I'm talking about here, making this perfect, getting it exactly
• 1:37:41
  Speaker 4: right becomes is a way higher priority for everyone in
• 1:37:43
  Speaker 4: the world, And that's what they're This is where we're at.
• 1:37:46
  Speaker 4: We're still in the vacuum tube stage of AI. Every
• 1:37:48
  Speaker 4: once acond we're like, no, no, no, we're not. And I'm
• 1:37:52
  Speaker 4: trying to show some of the evolution in the progress
• 1:37:54
  Speaker 4: because this is the state of the art stuff. And
• 1:37:56
  Speaker 4: if you understand video games, you understand that we're still right,
• 1:38:00
  Speaker 4: like this is like not even nineteen nineties graphics yet, right,
• 1:38:04
  Speaker 4: So we're getting there. So, as I was saying before,
• 1:38:09
  Speaker 4: you get object shapes for free, and this is the
• 1:38:12
  Speaker 4: solutions to the bounding box problem. And this was a
• 1:38:14
  Speaker 4: major problem in the industry and still probably is for
• 1:38:17
  Speaker 4: some dumb people. But this is how you solve it.
• 1:38:19
  Speaker 4: And this is a really really important thing, getting object
• 1:38:24
  Speaker 4: shapes for free, Like it's up there with collision avoidance
• 1:38:28
  Speaker 4: in terms of importance one of the and this is
• 1:38:33
  Speaker 4: like kind of important too, especially if you're an AI
• 1:38:36
  Speaker 4: you understand how significant this is. Probably most people won't
• 1:38:39
  Speaker 4: understand how big of a deal this is. But this
• 1:38:41
  Speaker 4: is a one of the other things you get out
• 1:38:43
  Speaker 4: of an ocupency network is that it reasons well within
• 1:38:46
  Speaker 4: about occluded objects, So objects that are traveling behind other
• 1:38:50
  Speaker 4: objects and partially occluded. This system inherently and automatically just
• 1:38:55
  Speaker 4: understands that because it's just a grid network around you.
• 1:38:58
  Speaker 4: Very simple heat map. But again, as I was saying,
• 1:39:02
  Speaker 4: the most important thing about the occupancy network for avs
• 1:39:05
  Speaker 4: is a collision avoidance system. It implicitly has this. Now
• 1:39:09
  Speaker 4: that that is not to say that it will automatically
• 1:39:12
  Speaker 4: go on the correct path. No, it just means it
• 1:39:16
  Speaker 4: will consistently choose a collision avoidant path. So, as you
• 1:39:22
  Speaker 4: can see here, this car went out of control. It
• 1:39:24
  Speaker 4: almost collided in the right corner, but the collision avoid
• 1:39:26
  Speaker 4: the occupancy network avoided it moved over to the left.
• 1:39:29
  Speaker 4: This is clearly the wrong side of the road, but
• 1:39:31
  Speaker 4: it avoided another collision. It stabilized and that's the value originally,
• 1:39:36
  Speaker 4: initially that was the major value, and it was sufficient
• 1:39:39
  Speaker 4: to sell it into the av industry just for this alone,
• 1:39:42
  Speaker 4: because there's a software alone doing this.
• 1:39:45
  Speaker 5: Close your words, looking to the darkness far the blazing
• 1:39:53
  Speaker 5: start focus on it be called the don't feel let
• 1:40:05
  Speaker 5: the should be done.
• 1:40:09
  Speaker 4: And does
• 1:40:12
  Speaker 1: You the shoes