Latest / Elon Musk Podcast / Why Tesla Must Retrofit Four Million Cars

Transcript

Why Tesla Must Retrofit Four Million Cars · Elon Musk Podcast

0:00
Elon Musk recently admitted that approximately 4 million Tesla
0:04
vehicles currently on the road lack the hardware to achieve
0:07
unsupervised full self driving. Yeah, and that physical
0:10
limitation, I mean, it forces this enormous retrofit effort
0:14
for older cars. It really reveals a hidden
0:17
ceiling in the tech world. We're looking at how this
0:20
insatiable demand for processing power is, you know, physically
0:23
altering in car computers. It's forcing companies to scrap
0:26
their Rd. Maps and creating a hard divide
0:29
and how different automakers approach the future of driving.
0:32
Right. So how does a company pivot when
0:34
its core autonomous driving software suddenly outgrows the
0:38
silicon powering its fleet? To answer that, you really have
0:41
to look closely at the physical limitations of the computers
0:43
already welded inside these vehicles.
0:45
Like if you look at Tesla's older hardware, which they call
0:48
Hardware 3, there is a severe physical constraint happening
0:52
right on the board itself. OK, what kind of constraint?
0:54
Well, it's memory bandwidth is only one eighth that of its
0:57
successor hardware 4. Wait, 1/8.
1:00
So memory bandwidth. We're talking about the actual
1:03
speed at which data travels back and forth between the memory
1:06
chips and the main processor, right?
1:07
Precisely, And the reason memory bandwidth is causing such an
1:12
emergency right now comes down to how autonomous driving
1:15
software is currently being built.
1:17
The entire industry is abandoning old methods and
1:20
moving toward what are called end to end neural networks.
1:23
Right. If you look at how selfdriving
1:25
software worked in the past, it was highly modular.
1:28
You had one specific piece of software tasked with looking at
1:31
camera pixels and, say, identifying a stop sign.
1:35
Right. And then a totally separate
1:36
piece of software applied. A hard coded rule written by a
1:39
human that told the car you know to apply the brakes of a stop
1:42
sign was detected. Yeah, exactly.
1:44
But end to end, neural networks throw out those hard coded rules
1:47
entirely. They take raw camera pixels and
1:49
map them directly to driving actions like steering and
1:52
braking. So the software just learns by
1:54
watching millions of hours of human driving video rather than
1:58
following a rigid list of instructions typed out by a
2:00
programmer in an office. That is the fundamental shift,
2:04
and to execute that kind of learning, these systems rely on
2:07
large transformer models, and these models require massive
2:12
amounts of extremely high speed memory to store what engineers
2:16
call activation maps. Activation Maps.
2:18
Yeah, you could visualize an activation map as the car's
2:21
short term working memory. When you approach a busy,
2:24
chaotic intersection, you naturally track moving objects,
2:28
right? Yeah, of course the computer
2:29
needs to do the exact same thing.
2:31
It needs to like remember that a pedestrian stepped off the curb
2:34
3 seconds ago, even if that pedestrian is currently hidden
2:37
behind a delivery truck that just pulled forward.
2:39
Oh I see, so it is less about how fast the processor
2:42
calculates the math and more about how much data can
2:45
physically fit through the pipe at any given microsecond.
2:48
Yes, it sounds like trying to drink from a fire hose with a
2:51
coffee stir. That analogy she perfectly
2:53
captures the bottleneck. Hardware 3 relies on older LPDDR
2:57
4 memory. It simply cannot move the data
3:00
fast enough to feed the demands of these large transformer
3:02
models, and the processor itself might have the capability to do
3:05
the math. But it's just sitting there
3:07
waiting. Exactly.
3:08
It sits there, starved for data, waiting for the memory to catch
3:11
up and deliver the next frame of the environment.
3:13
And this physical traffic jam on the circuit board dictates the
3:17
overall size of the neural networks the car is allowed to
3:20
run. Because if the pipe is too
3:22
small, you cannot run the bigger, smarter network.
3:25
Right, which totally prevents true unsupervised autonomy.
3:28
The car literally cannot hold enough of the surrounding
3:32
environment in its active memory to safely navigate without a
3:35
human sitting there ready to grab the wheel.
3:37
Wait, back up a second. Sure, if those 4 million
3:40
hardware 3 cars currently sitting in driveways cannot do
3:44
unsupervised driving, what happens to them?
3:46
I mean, people paid thousands of dollars for software they were
3:49
told would eventually drive them to work while they slept.
3:52
Yeah, well, Tesla plans to establish specialized micro
3:55
factories in major urban areas to physically swap out the
3:59
computers and the cameras in those Hardware 3 vehicles.
4:02
Micro factories. Yeah.
4:04
And while they try to figure out the logistics of that, they are
4:06
offering a software compromise called FSD version 14 Lite.
4:10
It's a It's an interim solution designed to run on the older
4:13
hardware until the cars can be physically rebuilt.
4:16
Setting up entirely new micro factories just to swap out
4:19
computers? That sounds like an astronomical
4:21
expense. You have to secure real estate
4:23
in expensive cities, hire specialized technicians, build
4:26
isolated supply chains strictly for retrofitting older models.
4:29
I mean, why not just send people to the regular service centers?
4:32
Because standard service centers are just too inefficient for a
4:35
complex swap like this. You have to remember you are not
4:38
just unbolting a metal box and plugging in a new one.
4:42
You have to carefully remove the old cameras, install new high
4:45
resolution 5 megapixel cameras, integrate the new AI4 compute
4:49
module and then this is the hard part, precisely align every
4:53
single sensor so the new software receives perfect
4:57
undistorted visual data. Yeah, service centers are
5:00
designed for standard maintenance.
5:01
They fix brakes, align tires, replace cracked windshields.
5:05
Exactly. They're not built to perform
5:08
major surgical overhauls on the sensory nervous system of a
5:12
complex machine. Relying on the existing service
5:15
center network would clog up normal operations for years, and
5:18
it would just create an impossible financial burden.
5:21
But even with specialized micro factories, the cost of ripping
5:24
apart millions of cars just to fulfill a past promise seems
5:27
financially ruinous anyway. Every hour a technician spends
5:31
doing surgery on a four year old car is an hour they aren't
5:34
building a new one. That financial reality is
5:36
exactly why offering heavily discounted trade insurance to
5:39
new vehicles equipped with hardware for might end up being
5:42
their primary strategy. Oh, to avoid the physical
5:45
retrofits entirely. Right.
5:48
Think about the psychology of the consumer.
5:50
If you own a hardware 3 vehicle and you are offered a compelling
5:53
enough discount on a brand new car that already has the
5:56
advanced hardware, you are highly likely to take the
5:59
upgrade. Win win.
6:00
Sort of. The customer gets a new car and
6:03
the company completely avoids the agonizingly slow, expensive
6:06
process of tearing down and rebuilding a used vehicle in a
6:09
temporary micro factory. But meanwhile they are
6:13
continuing to change the hardware on the production line.
6:15
Right now Tesla is already shipping a modified 3 chip
6:20
computer which is labeled AP45 in some recent model wise and
6:24
they just announced a future upgrade called AI4 plus that
6:27
double S the system RAM to 64 gigabytes.
6:30
Yeah. So the transition from A2 chip
6:32
architecture to a three chip architecture introduces a really
6:36
vital concept called triple modular redundancy.
6:39
OK, what does that mean? Well if you look at their
6:41
previous computers, they used 2 chips for redundancy.
6:44
If chip A failed completely, chip B took over.
6:48
Simple, but a three chip layout allows the system to actively
6:52
vote on reality. Vote on reality like an election
6:55
for what the car should do next. In a way, yes.
6:58
Imagine a scenario where, I don't know, a cosmic ray strikes
7:02
the silicon and physically flips a bit from A0 to A1, causing a
7:05
momentary glitch. OK.
7:06
Or perhaps the software on one chip briefly misinterprets a
7:10
weird shadow on the road as a concrete barrier.
7:13
If you only have two chips and they disagree, the system panics
7:16
enhance control back to the human.
7:18
Right, because it doesn't know which one is right.
7:20
Exactly, but with three chips they can compare the results If
7:24
chip A hallucinates a wall but chips B&C both agree the road is
7:28
completely clear, The system boasts to ignore chip A.
7:32
The outlier is outvoted. Wow.
7:34
This ensures continuous operation and prevents the car
7:37
from phantom breaking or, you know, violently disengaging on
7:40
the highway. Three silicon brains debating
7:43
the safest action in real time. That is wild.
7:46
It is, and a three chip setup serves another critical function
7:49
too. It allows one of the chips to
7:51
run highly experimental software in the background.
7:54
In the industry, this is known as shadow mode.
7:56
Oh, so the car physically drives using the proven stable software
8:00
running on 2 chips? Yep.
8:01
Well, the third chip silently tests next generation code
8:04
against the real world conditions the driver is
8:07
experiencing. Exactly.
8:08
It learns and processes the environment without ever risking
8:11
passenger safety because its outputs are physically
8:14
disconnected from the steering wheel.
8:15
And what about the AI4 Plus upgrade?
8:17
Because doubling the RAM to 64 gigabytes seems entirely focused
8:21
on that memory pipe issue we discussed earlier it.
8:24
Totally addresses the core problem of neural network
8:26
weights. The upcoming AI4 plus hardware
8:30
increases the raw compute speed and the memory bandwidth by
8:34
roughly 10%, which is, you know, a nice bump.
8:37
But the critical change is that massive increase in RAM.
8:41
End to end. Neural networks are not static.
8:43
They're constantly growing the weights, which are the billions
8:48
of mathematical parameters the artificial intelligence learns
8:51
and refines during its training phase in the data center.
8:54
They take up physical space in the car's memory.
8:57
As the models get smarter and more capable of handling complex
9:00
edge cases, those weights get heavier.
9:03
Doubling the RAM prevents the current fleet from hitting a
9:05
computational wall as the software inevitably becomes more
9:07
complex over the next couple of years.
9:09
Hold on, if they are upgrading the hardware again right now,
9:12
our current hardware for owner is going to face the exact same
9:15
obsolescence issues as hardware 3 owners in a few years.
9:18
I mean, if the neural networks just keep getting fatter, a 64
9:21
gigabyte limit will eventually reach 2.
9:23
You are identifying the central tension in the entire autonomous
9:27
driving space. Right now, a company can claim
9:30
their current hardware is capable of unsupervised driving,
9:35
but when they simultaneously release incremental upgrades
9:38
with double the memory and extra processors, well, it reveals a
9:43
lack of certainty. Yeah, no kidding.
9:45
It really highlights the immense difficulty of predicting exact
9:49
hardware requirements for software that hasn't actually
9:52
been fully solved yet. You're trying to build the
9:54
perfect box for an intelligence that hasn't finished growing.
9:57
And the hardware road map extends even further right.
9:59
Tesla's next generation AI5 chip has officially completed its
10:03
design phase, but according to the sources, it will actually be
10:06
prioritized for their optimist humanoid robot and their data
10:09
centers rather than being immediately deployed into cars.
10:12
Yeah, the compute requirements for robotics are just on an
10:14
entirely different level than driving.
10:16
Well, a humanoid robot requires unstructured 3D spatial
10:19
reasoning and constant tactile feedback from its environment.
10:23
Look, navigating A2 dimensional road is certainly difficult, but
10:27
roads have clear rules, ainted lines and a general flow of
10:31
traffic. The environment is somewhat
10:32
constrained. Exactly.
10:34
Now picture a robot walking through a cluttered human
10:37
kitchen. It has to recognize a fragile
10:40
glass sitting precariously on a counter, calculate the exact
10:44
pressure needed to pick it up without shattering it, navigate
10:47
around a dog sleeping on the floor, and place the glass into
10:50
a dishwasher. That is a fundamentally
10:52
different engineering challenge. Uses orders of magnitude more
10:55
compute intensive than staying between 2 white lines on a
10:57
highway. Because a humanoid robot is
11:00
dealing with infinite variations of human environments rather
11:03
than a standardized public Rd. system.
11:05
Exactly. So.
11:06
To handle the immense processing load of physical AI, the AI5
11:10
chip offers 5 five times the memory bandwidth of the current
11:14
AI 4 architecture. Five times.
11:16
Wow. Yeah, and the physical
11:17
manufacturing of these chips is becoming a bottleneck in itself.
11:21
To secure the massive volume of silicon required for both
11:24
millions of cars and millions of robots, Tesla is partnering with
11:29
SpaceX and Intel to construct a massive semiconductor
11:32
fabrication facility in Texas. They're calling it Terafab.
11:35
Terafab. This facility basically opens up
11:38
the ability for them to manufacture their own logic
11:40
chips and memory completely under one roof.
11:42
It sounds like building a silicon fortress.
11:45
Instead of relying entirely on external foundries spread across
11:49
the globe, they are attempting to control the physical
11:51
manufacturing of their brains from the ground up.
11:54
Right, because the standard practice across the tech
11:56
industry is fables design. You design the chip architecture
11:59
on a computer in California, and you pay another company in Asia
12:02
to actually bake the silicon and manufacture it.
12:04
Right, But relying on external supply chains creates delays.
12:09
Building a research grade wafer fab in Texas that integrates
12:12
mask, fabrication, logic, chips, memory, and packaging in the
12:16
exact same physical building. It creates the fastest possible
12:20
iteration cycle. Oh, I see.
12:22
You can design a new chip architecture on Tuesday, test
12:25
it, and scale it into production without ever waiting in line
12:27
behind other tech giants at an external foundry.
12:30
Precisely. Well, we should look at the rest
12:33
of the industry because the approaches are really splitting
12:35
here. While Tesla develops this highly
12:37
specialized vision focused silicon relying on older ARM
12:41
Cortex A72 CPU's and GDR6 memory, competitors like BYD,
12:46
Lucid and Xiaomi are adopting Nvidia's Drive Thor platform.
12:51
Yeah, and NVIDIA store platform represents a completely
12:54
different engineering philosophy.
12:55
How so? Thor utilizes cutting edge
12:57
manufacturing processes and server grade CPU's to deliver
13:00
2000 teraflops of raw compute power.
13:04
That's a lot of math. It is, and it does not just
13:06
focus on driving. It is designed to run the entire
13:09
vehicle. Thor handles the complex
13:11
autonomous driving neural networks, but it also has the
13:14
power to run the infotainment screen, process the digital
13:17
dashboard, render 3D graphics for passengers, and manage in
13:20
car voice assistance simultaneously.
13:22
So it is an all in one supercomputer for the car, just
13:25
brute force doing everything at once.
13:26
Yes, and that brute force requires immense power.
13:30
Tesla's custom architecture, on the other hand, deliberately
13:33
sacrifices general purpose CPU power to maximize one specific
13:38
thing, memory bandwidth. Back to the firehose.
13:41
Exactly. They use GDDR 6 memory, which is
13:44
the type of memory typically found in high end graphics cards
13:47
for video games. They use it to heavily
13:49
prioritize the rapid continuous transfer of high resolution
13:53
video streams. You have to remember they do not
13:55
use Lidar, lasers or radar sensors anymore.
13:58
Right, they rely entirely on flat cameras.
14:00
Right, so to understand depth, to know exactly how many feet
14:03
away a vehicle or a pedestrian is, the computer has to
14:06
constantly analyze flat 2 dimensional pixels and calculate
14:10
3D geometry in real time. They call that pseudo Lidar
14:13
right? Deriving 3D depth from 2D
14:14
images. Exactly, and running pseudo
14:16
lidar on a high resolution camera streams simultaneously
14:20
requires moving a tremendous amount of video data without a
14:22
single millisecond of lag. That is why they willingly
14:25
accept older, less powerful CPU cores in their custom chips.
14:30
They trade general processing capability in exchange for
14:32
maximum memory bandwidth, ensuring the video data never
14:36
ever stops flowing into the neural network.
14:38
So does a custom hyper specialized design focus purely
14:42
on video data give an automaker an edge?
14:44
Or does Nvidia's massive all in one compute power offer a safer
14:49
bet for companies trying to build the cars of the future?
14:52
You know, the race for full autonomy is no longer just a
14:54
software problem solved by clever code.
14:57
It is a brutal battle defined by physical silicon limits, the
15:01
realities of hardware manufacturing, and just how fast
15:03
you can push data through a wire.
15:05
Right, it leaves you wondering, will the constant escalating
15:08
need for more processing power turn modern cars into temporary
15:12
tech gadgets that inevitably age out and require replacement
15:15
every few years, rather than long term assets that sit in our
15:17
garages for decades? If you're not subscribed yet,
15:20
take a second and hit follow on whatever app you're using.
15:23
It helps us keep making this. We appreciate you being here.
15:26
Also, check out our YouTube channel for more business and
15:28
tech updates. There's a link in the
15:30
description.