Latest / Elon Musk Podcast / Musk's $25 Billion Custom AI Chip Factory

Transcript

Musk's $25 Billion Custom AI Chip Factory · Elon Musk Podcast

0:00
Elon Musk's Tesla and SpaceX are they're spending 20 to $25
0:04
billion to build a custom semiconductor fabrication plant
0:08
down in Texas. Yeah.
0:09
And this is specifically to manufacture to a nanometer
0:13
artificial intelligence chips. Yeah, the massive scale of that
0:17
financial commitment requires, you know, serious pause.
0:20
I mean, the Terafab project aims to produce 100 to 200 billion
0:25
custom chips annually, and that level of volume goes way beyond
0:31
just like outfitting a fleet of vehicles, so.
0:33
We have this fierce battle forming for control over
0:35
physical hardware right in front of us.
0:38
On one side you've got established semiconductor giants
0:40
pushing these massive new computing platforms, and on the
0:44
other, a vehicle manufacturer attempting to build the
0:46
factories themselves. Our conversation focuses on the
0:49
tension between those two approaches, looking strictly at
0:52
the physical limitations of computing based on the
0:54
materials. Which really alters the basic
0:56
mechanics of how artificial intelligence gets physically
0:58
built and distributed globally. So what happens to the global
1:01
tech economy when the biggest buyers of computer chips decide
1:06
to become the manufacturers? Well, the entire computing
1:09
ecosystem has to reevaluate how hardware is designed.
1:12
I mean, the demands of the software are just stretching the
1:15
physical limits of silicon to the breaking point.
1:18
Right. And when you look at the
1:19
established semiconductor giants, NVIDIA is moving in a
1:22
direction that feels entirely focused on brute force.
1:26
Oh, they absolutely are. Their Vera Rubin platform
1:29
utilizes 6 different specialized chips that function as a single
1:33
supercomputer. 6 chips. Yeah.
1:35
And doing this drops inference token cost by a factor of 10 and
1:39
it delivers five times the inference compute of the
1:42
previous Blackwell generation. Wait, back up a second.
1:44
Clarify the exact difference between inference and training
1:47
before we move on, just so we're clear.
1:48
Sure. So training is the education
1:50
phase. You feed the artificial
1:52
intelligence billions of pieces of data over several months, so
1:55
it learns the patterns. Inference is when the artificial
1:58
intelligence actually takes a test.
2:00
It takes a prompt from you. The user applies the rules it
2:04
learned and generates a response.
2:06
Got it? Historically, training required
2:08
massive supercomputers, while inference could just run on your
2:11
phone or, you know, lighter hardware.
2:14
But the requirements for inference are exploding right
2:16
now. Because the models themselves
2:18
are fundamentally changing, we are moving toward HNTK AI, where
2:22
systems act and reason autonomously.
2:25
They don't just generate a paragraph of text.
2:27
No. They write code, test it,
2:29
realize it failed, rewrite it. Exactly, and communicate with
2:32
other AI agents to solve multi step problems for you.
2:35
Let me give an example to make sure we have the mechanics
2:37
right. Go for it.
2:39
Instead of just asking a program to write a recipe, you are
2:41
asking it to check your fridge, order the missing groceries,
2:45
track the delivery and Preheat your oven.
2:48
The system has to constantly pull new data, evaluate its own
2:52
logic, and make physical or digital actions on your behalf.
2:55
Exactly. And the raw capability required
2:58
for that level of continuous reasoning comes with immense
3:01
power requirements. The new Reuben chip uses 2200
3:05
watts of power per die. That is exactly double the power
3:10
draw of the previous. Generation and unbelievable.
3:12
To put 2200 watts in perspective, that is the
3:17
equivalent of running a heavy duty industrial microwave
3:20
continuously on a piece of silicone the size of a drink
3:22
coaster. I have to push back on the
3:24
practicality of this. It sounds like dropping a
3:27
massive freight locomotive engine into a standard sedan
3:30
chassis, right? How can standard data centers
3:33
possibly cool or provide enough electricity for racks filled
3:37
with these 2200 Watt tips? If you have hundreds of
3:40
thousands of these running simultaneously, the math on the
3:43
power grid just stops working. Well, standard data centers
3:46
actually cannot handle it. That extreme power density means
3:49
if you put a standard air cooling fan on that chip, the
3:52
heat generation is so intense the silicon would literally melt
3:55
itself into slag. Wow, I'm built.
3:57
It yeah, the thermal limits of the materials are being pushed
4:00
to their absolute breaking point.
4:01
The power draw requires purpose built infrastructure.
4:04
Which means the consequence of that shift severely limits who
4:07
can participate. The massive power requirement of
4:10
the Reuven platform restricts this highly capable tier of
4:13
computing to only the wealthiest corporations.
4:16
We are seeing a transition from regular data centers to
4:19
specialized AI factories. These are GW scale facilities
4:23
specifically built to churn out AI tokens.
4:26
So it pushes everyone else out. Exactly.
4:29
This concentrates the most powerful artificial intelligence
4:31
capabilities into the hands of a few massive tech conglomerates
4:35
who can actually afford the electricity and the concrete
4:38
infrastructure structure. So you have these massive power
4:40
hungry processors, but processing power requires
4:43
matching memory speed. What happens when you pair a
4:46
blazing fast processor with slow memory?
4:49
The processor ends up sitting completely idle.
4:52
The industry is currently transitioning to HBM 4 memory
4:55
stacks to solve this. Companies like Micron are
4:57
manufacturing units that offer 36 to 48 gigabytes of capacity
5:01
and two terabytes per second of bandwidth per stack.
5:04
Let me make sure I understand the mechanics of this
5:06
bottleneck. Having a supercomputer brain
5:09
paired with slow memory feels like having a genius locked in a
5:14
room, but you are forcing them to learn entirely through a tiny
5:18
plastic straw. That's a good way to picture.
5:20
It you are sliding tiny pieces of information under the door
5:23
one at a time, the genius can solve anything instantly, but
5:26
they spend 99% of their time just waiting for you to slide
5:30
the next piece of paper. The data cannot get in or out
5:33
fast enough to matter, wasting all that electricity.
5:36
That bottleneck is known across the industry as the memory wall.
5:40
When an agentic AI is reasoning through a complex problem, it
5:44
has to remember the entire history of its actions.
5:47
The context window. Yes, the context window.
5:50
If the memory bandwidth is too slow, the AI cannot retain its
5:53
train of thought. HBM 4 solves this by changing
5:56
the physical geometry of the hardware.
5:57
How so? Instead of laying memory chips
5:59
flat next to the processor on a green motherboard, data travels
6:03
across microscopic highways. It stacks memory chips
6:06
vertically like a skyscraper. And how does the data get up and
6:09
down the skyscraper? By punching microscopic holes
6:12
straight through the silicon to physically connect the floors,
6:14
they are called through silicon vias.
6:17
This allows vast amounts of data to flow simultaneously up and
6:21
down the stack, rather than waiting in a single file line.
6:24
But pushing that much data through microscopic holes has to
6:27
generate an unbelievable amount of friction and heat.
6:30
It does, and the memory requirements do not stop there.
6:34
Future projections extend all the way to HBM 8, predicting
6:38
staggering 15,000 Watt GPU. 15,000 watts.
6:42
This extreme bandwidth is strictly required to hold the
6:45
context window for massive neural networks.
6:47
Which goes back to the cooling problem.
6:49
The consequence of this massive heat generation forces the
6:52
entire industry to abandoned traditional air cooling and
6:56
adopt direct to chip liquid cooling and full immersion
6:59
systems completely. This alters physical data center
7:02
construction from the ground up. You are looking at facilities
7:04
where the server racks are literally dunked into vats of
7:07
non conductive engineered fluids, or where chilled liquid
7:11
is piped directly over the bare silicon dye.
7:14
So you can no longer retrofit an old warehouse to be a data
7:17
center. No, you have to design the
7:18
complex fluid plumbing before you even pour the concrete.
7:22
So if these massive liquid cooled data centers are hitting
7:26
their physical limits just to run artificial intelligence, how
7:30
do you put that same kind of brain into a car driving down
7:33
the highway? You cannot put a water cooling
7:35
tower on the roof of a passenger sedan.
7:38
That is exactly where the battle shrinks down to fit inside
7:41
vehicles. Nvidia's Drive Thor delivers
7:44
2000 teraflops using a custom 4 nanometer process.
7:49
OK, this single chip is powerful enough to run the entire vehicle
7:52
stack, including all the autonomous driving functions and
7:55
the dashboard. Infotainment brands like
7:57
Mercedes and BYD are already adopting it.
8:00
Hold on. Explain what a nanometer process
8:03
actually means in this context, and what a teraflop does for the
8:06
driver sitting behind the wheel. Sure.
8:08
Nanometers measure the size of the microscopic transistors
8:11
carved into the chip. Smaller the number, the more
8:13
switches you can pack onto the silicon.
8:16
Think of it like shrinking the width of the roads in a city so
8:18
you can build more highways in the same amount of space.
8:21
Exactly. Electrons travel faster and less
8:24
power is wasted as heat. 4 nanometer process is incredibly
8:28
dense and a teraflop is a trillion calculations per
8:32
second. So 2000 teraflops means the car
8:35
can calculate 2000 trillion mathematical operations every
8:39
single second to understand its surroundings, right?
8:42
And how does Drive Thor actually manage all those calculations
8:45
without melting? It achieves this by taking
8:47
server grade ARM cores, the basic thinking units or managers
8:51
of the chip, and combining them with an integrated transformer
8:54
engine that is the exact same hardware logic used in large
8:57
language models packed right into the car.
9:00
Meanwhile, you look at Tesla's current hardware AI 4, and it
9:03
relies on older 7 nanometer Samsung nodes, legacy ARM cores
9:08
and a memory bandwidth of just 384 gigabytes per second.
9:11
The contrast in raw specifications is massive when
9:14
you put them side by side. Which makes me think Tesla is
9:17
bringing an outdated smartphone processor to a supercomputer
9:21
fight. I mean they are at a severe
9:23
hardware disadvantage trying to run massive vision only neural
9:28
networks on silicon that is generations behind what NVIDIA
9:31
is offering to the rest of the automotive industry.
9:33
Well that assumes everyone is trying to run the exact same
9:36
software Fairpoint. Tesla utilizes strict software
9:40
hardware Co design because they control the software entirely.
9:43
They strip out generic tasks. Tesla's upcoming AI 5 chip is
9:48
projected to operate on just 150 watts, while rivaling the
9:51
performance of a 700 Watt server GPU.
9:54
Wait, hold on, how does a 150 Watt chip match the performance
9:57
of a massive server GPU? You cannot just magically ignore
10:01
the physics of computing power. It is the difference between a
10:04
generalist and a specialist drive.
10:06
Thor has to be compatible with whatever operating system
10:09
Mercedes, BYD or any other manufacturer wants to use.
10:13
It carries transistors specifically dedicated to
10:15
rendering high resolution graphics for the dashboard,
10:18
running third party voice assistance, and handling
10:20
generalized safety redundancies. And AI5 doesn't do that.
10:23
No, AI5IS stripped of all that generalized compatibility.
10:27
Every single circuit is physically carved into the
10:30
silicon specifically to process Tesla's proprietary neural
10:34
network matrix math. Matrix math being the specific
10:37
type of grid based calculation that neural networks use to
10:41
understand images from the car's cameras.
10:43
Yes. Imagine a spreadsheet with a
10:45
million cells representing pixels from a camera.
10:48
Neural networks multiply every cell by every other cell to find
10:51
edges, shapes and objects like pedestrians or stop signs.
10:54
In general, processors do this clumsily, moving data back and.
10:57
Forth exactly, AI 5 is physically wired to do only the
11:00
specific math problem perfectly. That physical specialization
11:04
saves hundreds of watts of power.
11:06
The consequence of that extreme vertical integration naturally
11:09
limits Tesla's speed, though the delays in the AI5 chip force
11:13
their upcoming cyber cab to launch on older hardware.
11:16
Does. But it opens up the extreme
11:18
energy efficiency required for edge computing in cars and
11:21
battery powered humanoid robots. And edge computing just means
11:25
the processing happens locally on the physical device itself,
11:29
rather than sending data back and forth to a cloud server.
11:32
Right. If you are building millions of
11:33
Optimus humanoid robots, you cannot strap a 700 Watt liquid
11:38
cooled server to their backs. The battery would drain in
11:41
minutes. Exactly.
11:42
The robot needs to process visual data and balance itself
11:45
instantly, which requires A specialized low power inference
11:49
engine right there in its chest. Tesla's response to future
11:52
supply constraints regarding those specialized engines is the
11:55
Terrafab project. The facility in Texas is a joint
11:58
venture with SpaceX. The addition of SpaceX
12:01
fundamentally shifts the compute allocation. 80% of the compute
12:04
output is actually earmarked for space applications like
12:08
satellite logic and rocket navigation.
12:10
Leaving only 20% for vehicles and robots, right?
12:13
I'm highly skeptical of this execution.
12:15
Mastering atomic level physics and securing extreme ultraviolet
12:19
lithography equipment takes established foundries decades of
12:22
painful iteration. Let me explain extreme
12:25
ultraviolet lithography for a second, because it illustrates
12:27
your point perfectly. To carve A2 nanometer transistor
12:32
you have to blast droplets of molten tin with a high power
12:36
laser 50,000 times a second. Wow.
12:38
And that generates light with a wavelength so tiny it can etch
12:42
patterns almost at the scale of single atoms.
12:46
The mirrors required to direct that light are some of the
12:48
flattest surfaces ever created in the universe.
12:51
That's insane. If you scaled one of those
12:53
mirrors to the size of a country, the tallest mountain
12:56
would be less than a millimeter high.
12:58
Exactly. You were talking about a car
13:00
company attempting to run A2 nanometer fab.
13:03
It is highly improbable they can spin up a pristine clean room
13:07
environment and start churning out perfectly etched silicon
13:10
wafers without massive yield issues.
13:13
They would just be printing expensive pieces of sand.
13:15
That skepticism is entirely warranted if they were doing it
13:18
completely alone, but Terrafab relies on a hybrid model.
13:21
How does that work? Tesla is supplying massive
13:23
capital and extreme volume demand while partnering with
13:26
established foundries to utilize their existing process
13:29
technology. So they're not inventing the
13:31
manufacturing process from scratch.
13:33
No, not. They are providing the billions
13:35
of dollars in the guaranteed demand to accelerate a partner's
13:39
existing technology road map on a dedicated line.
13:42
Exactly when you are projecting a need for hundreds of billions
13:46
of chips for autonomous vehicles, robots and satellites,
13:49
you simply cannot wait in line behind Apple and NVIDIA for
13:52
manufacturing capacity. You have to fund your own line.
13:56
The consequence here alters the core business model of vehicle
14:00
manufacturing. By funding their own silicon
14:02
production, they insulate themselves from global supply
14:05
chain shocks and create a massive dedicated compute
14:09
foundry for the space economy. Space applications require
14:12
highly specific radiation hardened hardware.
14:15
When a computer is operating outside the Earth's atmosphere,
14:18
stray cosmic rays can actually strike the silicon and flip a
14:22
bit from A0 to A1. Just a single ray.
14:25
Yes, and that single microscopic event can cause a rocket
14:28
navigation system to fail instantly.
14:31
Specialized physical shielding and redundant logic gates carved
14:35
right into the chip. When you combine the compute
14:38
needs of Starlink satellites communicating in orbit, Starship
14:41
calculating autonomous atmospheric reentry, and Tesla
14:45
vehicles navigating city streets, the sheer volume
14:49
justifies building a dedicated fabrication plant.
14:52
They are treating silicon the same way they treated battery
14:55
cells, bringing it in house to guarantee supply.
14:58
So mastering artificial intelligence requires total
15:01
control over the physical silicon through massive capital
15:05
investment and intense energy infrastructure.
15:07
It leaves you wondering what happens to the traditional tech
15:10
sector when physical industry takes over the silicon.
15:13
If a vehicle and rocket manufacturer suddenly controls a
15:16
massive share of the world's most advanced chip production,
15:19
are they even a car company anymore, or are we just riding
15:22
inside computers? If you're not subscribed yet,
15:24
take a second and hit follow on whatever app you're using.
15:27
It helps us keep making this. We appreciate you being here.