Latest / Elon Musk Podcast / Amazon, Anthropic and the city sized grid

Transcript

Amazon, Anthropic and the city sized grid · Elon Musk Podcast

• 0:00
  Amazon is investing up to $25 billion into Anthropic, and in a
• 0:05
  reciprocal move, Anthropic has committed to spending $100
• 0:09
  billion exclusively on Amazon Web Services over a 10 year
• 0:13
  period. Right.
• 0:14
  And it's wild, because right now, an energy grid capable of
• 0:18
  powering the entire city of Chicago is effectively being
• 0:21
  hijacked. Hijacked for just one company.
• 0:24
  Exactly. I mean it isn't being used to
• 0:26
  light homes or run hospitals. It is being used by one single
• 0:30
  company just to power one single artificial intelligence model.
• 0:34
  Wow. Yeah.
• 0:35
  The volume of capital moving through this specific
• 0:37
  arrangement, it fundamentally rewrites our understanding of
• 0:40
  corporate investment. Because it's a closed loop.
• 0:43
  Right, right. We're looking at a closed
• 0:44
  financial loop that directly dictates the physical
• 0:47
  constraints of data centers across the globe.
• 0:50
  This money translates directly into the intricate silicon
• 0:53
  architecture moving data through fiber optics.
• 0:56
  And entirely new software marketplaces too.
• 0:58
  Exactly, along with this systemic security threats that
• 1:01
  are, you know, currently catching the attention of global
• 1:03
  banking regulators. So it really brings up the
• 1:05
  question, can the physical infrastructure of the world
• 1:08
  actually support this level of runaway computational demand?
• 1:12
  That is the defining engineering challenge of our current era,
• 1:16
  hands down. Answering it really requires
• 1:19
  looking closely at what initiated this mega deal in the
• 1:22
  first place. Well, the catalyst for this
• 1:24
  specific financial arrangement is Anthropics annualized revenue
• 1:28
  run rate crossing $300 billion, which I mean let that sink in
• 1:33
  for. A second, it's an economical
• 1:34
  number. It really is.
• 1:36
  And to understand a run rate, you just take the revenue from a
• 1:39
  specific short period like a month or 1/4 and project it out
• 1:43
  over a full year, assuming that exact pace continues, of course,
• 1:46
  right? Hitting $300 billion is a
• 1:48
  staggering figure for any corporate entity, but the
• 1:51
  velocity behind it is the real story here.
• 1:53
  I mean, the number of enterprise customers spending over $1
• 1:56
  million doubled in less than two months.
• 2:00
  Yeah. And when a software provider
• 2:01
  experiences that kind of explosive adoption, it triggers
• 2:04
  an immediate infrastructure crisis.
• 2:06
  Because they just don't have the hardware to support it.
• 2:08
  Exactly. We saw consumer usage across
• 2:11
  their free, Pro and Max tears spiked so incredibly fast that
• 2:16
  it's severely impacted reliability.
• 2:18
  Oh right, the outages. Yeah, it caused highly
• 2:20
  disruptive outages for clawed users.
• 2:23
  And you know, an outage in this context isn't just a server
• 2:26
  needing a quick reboot. What does it actually look like
• 2:28
  behind the scenes? We are talking about millions of
• 2:32
  complex multi layered queries hitting the application
• 2:36
  programming interfaces simultaneously.
• 2:38
  The servers literally reach a physical limit where they cannot
• 2:42
  process the request. They just get completely
• 2:44
  overwhelmed. Right, creating massive queues,
• 2:46
  dropping connections and essentially melting down under
• 2:49
  the mathematical weight of the tasks.
• 2:51
  So how do they dig out of that? Well, to fix this immediate
• 2:54
  physical bottleneck, Anthropic secured 5 gigawatts of compute
• 2:58
  capacity using Amazon's custom Trainium and Graviton chips.
• 3:02
  OK. Hold on.
• 3:03
  We need to put 5 gigawatts into perspective because that number
• 3:05
  gets thrown around without much context.
• 3:08
  It's hard to visualize. It really is.
• 3:10
  Five gigawatts is roughly equivalent to the entire energy
• 3:13
  consumption of a major metropolitan city.
• 3:15
  Yeah, if you look at a place like Seattle or Miami, all the
• 3:19
  traffic lights, the skyscrapers, the residential heating and
• 3:22
  cooling, that is what 5 gigawatts powers.
• 3:26
  That's just unbelievable. We are talking about dedicating
• 3:29
  the power grid of a massive urban center just to run
• 3:32
  artificial intelligence models for one single company.
• 3:35
  And the power requirements are absolute, they are completely
• 3:39
  unavoidable. You can't cheat physics.
• 3:41
  Exactly. You cannot negotiate with the
• 3:42
  physics of electricity by securing this specific capacity.
• 3:46
  The arrangement radically limits Open AI's ability to monopolize
• 3:50
  the available compute capacity in the market.
• 3:52
  Right, it takes a massive chunk of resources off the table.
• 3:55
  Yep, and simultaneously it opens up Amazon's custom silicon
• 3:59
  ecosystem to completely dominate enterprise adoption.
• 4:02
  How so? Well, if you are a business
• 4:05
  deciding where to build your internal tools, the fact that
• 4:08
  Anthropic is committing to Amazon's hardware creates this
• 4:11
  immense gravitational pull toward the tranium and graviton
• 4:14
  architecture. Because they're basically
• 4:15
  endorsing it as the standard. Exactly.
• 4:17
  Instead of just buying off the shelf general purpose
• 4:20
  processors, Amazon has designed these chips specifically for the
• 4:24
  mathematical operations required by neural networks.
• 4:27
  That makes them highly efficient for this exact workload.
• 4:30
  So how does $100 billion of cloud credits actually translate
• 4:34
  into physical buildings, steel and energy?
• 4:37
  I mean, you cannot just wire that kind of money and instantly
• 4:40
  spawn a facility out of thin air.
• 4:42
  Oh, definitely not. It translates into the most
• 4:45
  aggressive industrial construction boom in modern
• 4:48
  history. Like beyond anything we've seen
• 4:50
  before. By far the major hyper scalers,
• 4:53
  Alphabet, Amazon, Microsoft and Meta are actively planning to
• 4:57
  spend hundreds of billions on capital expenditures.
• 5:00
  Just massive sums of cash. Amazon alone is dedicating
• 5:03
  roughly $200 billion to capital expenditure in a single year.
• 5:07
  That is wild. They're buying vast tracts of
• 5:10
  land, securing power substations and pouring concrete at a pace
• 5:14
  that is actively straining global supply chains.
• 5:16
  But there is severe friction in deploying these funds, right?
• 5:19
  It's not a smooth process. Absolutely not.
• 5:21
  Because average data center facilities now cost between 500
• 5:25
  million and $2 billion to construct, with lead time
• 5:28
  stretching well over 12 months. It takes forever.
• 5:31
  And the bottleneck is rarely the building itself.
• 5:35
  The bottleneck is the industrial equipment required to make the
• 5:38
  building function. Right, the internal guts of the
• 5:40
  place. Exactly.
• 5:41
  You cannot just go to a hardware store and buy a massive
• 5:44
  electrical transformer. These are custom engineered
• 5:47
  pieces of machinery with multi year wait lists.
• 5:51
  And the requirements are so specific now.
• 5:53
  Yeah, because the power and cooling demands are so extreme,
• 5:56
  developers are completely abandoning multi tenant
• 5:59
  collocation spaces. And to understand why, you
• 6:02
  really have to look at the density of the hardware, right?
• 6:05
  In a traditional data center, where multiple companies might
• 6:08
  rent space on the same floor, a standard rack of servers might
• 6:12
  draw about 10 kilowatts of power.
• 6:13
  OK, 10 kilowatts. But the specialized racks
• 6:16
  required for these new models draw upward of 100 kilowatts.
• 6:20
  And 10 times the power. Exactly.
• 6:22
  A traditional multi tenant facility would literally melt
• 6:25
  under that kind of thermal output.
• 6:28
  The cooling systems designed for 10 kilowatts simply cannot
• 6:32
  dissipate the heat generated by 100 kilowatts.
• 6:34
  They just aren't built for it, right?
• 6:36
  Therefore, developers are heavily favoring single tenant
• 6:40
  hyperscale leases. They want the whole building to
• 6:43
  themselves. Exactly.
• 6:44
  They're essentially demanding control over the entire building
• 6:47
  and its dedicated power supply so they can custom build the
• 6:51
  liquid cooling infrastructure necessary to keep the silicon
• 6:55
  from destroying itself. Building these facilities isn't
• 6:58
  just plugging in computers, it's high stakes industrial real
• 7:01
  estate. That is the perfect way to
• 7:02
  phrase it. And that real estate reality
• 7:04
  permanently changes how these technology companies manage
• 7:07
  inflation and resource scarcity. How does it change things
• 7:10
  exactly? Well, the friction forces them
• 7:12
  into highly specific modular designs and phase deployments.
• 7:16
  Return on investment is checked at every single stage of
• 7:19
  construction. They can't just build the whole
• 7:21
  thing blindly. No, they cannot afford to build
• 7:24
  a $2 billion facility, turn the lights on and hope the customer
• 7:29
  demand materializes. They have to build in distinct
• 7:32
  functional phases. So if phase one fails.
• 7:35
  If the first phase doesn't generate the expected revenue,
• 7:38
  the subsequent phases are paused.
• 7:41
  This physical and financial reality effectively limits the
• 7:44
  speed at which new capacity can actually come online, regardless
• 7:48
  of how much cash they have in the bank.
• 7:49
  But you know, even when the building is finally constructed,
• 7:52
  the steel is set and the power is flowing from the grid, a
• 7:55
  completely new physical problem emerges inside the server racks
• 7:59
  themselves. Right, the internal networking,
• 8:01
  yeah. The custom chips processing
• 8:04
  these models are now too fast for the physical cables
• 8:06
  connecting them. It's crazy to think about.
• 8:08
  The primary limitation inside these facilities is no longer
• 8:11
  raw processing power. It's the traffic jam.
• 8:14
  Exactly. The limitation is the
• 8:15
  interconnect, the physical ability to move data between the
• 8:18
  accelerators, the memory pools and the networking switches.
• 8:21
  So the. Chips are just sitting there
• 8:22
  waiting. Yeah, you have these incredibly
• 8:24
  powerful processors that are effectively starved for data
• 8:28
  because the wires just cannot feed them fast enough.
• 8:31
  To solve this, AWS is deploying a highly specialized solution
• 8:35
  for their new Tranium 4 chips, Right, They are, yeah.
• 8:38
  They are integrating Nvidia's NV Link Fusion architecture and
• 8:42
  this integration allows hyperscalers to connect up to 72
• 8:46
  custom application specific integrated circuits, or ASICS.
• 8:50
  Which is a huge cluster. It is, and it allows every
• 8:53
  single one of them to communicate at a speed of 3.6
• 8:56
  terabytes per second per chip. I mean, think about what 3.6
• 9:00
  terabytes per second actually means physically.
• 9:02
  It's hard to even wrap my head around.
• 9:04
  You could transfer the data of every single high definition
• 9:07
  movie ever produced in a fraction of a heartbeat.
• 9:10
  Wow. But moving that much data
• 9:13
  requires A fundamental rethinking of how computer
• 9:15
  components interact. It's like, think of it like
• 9:17
  building the world's fastest sports cars, but realizing you
• 9:20
  have to build specialized, frictionless super highways just
• 9:23
  for them. Otherwise they just sit idling
• 9:25
  in traffic. That's a great way to put it.
• 9:28
  Imagine you have 72 of the most brilliant scientists on the
• 9:31
  planet working in the same building.
• 9:33
  OK, if they're confined to separate rooms and have to fly
• 9:37
  physical pieces of paper under the doors to share their
• 9:40
  research, their individual brilliance doesn't really
• 9:42
  matter. Right, the bottleneck is the
• 9:45
  paper sliding. Exactly.
• 9:46
  The speed of discovery is entirely limited by the guys
• 9:49
  sliding the paper back and forth.
• 9:51
  NV Link fusion basically vaporizes the walls between
• 9:55
  those rooms. So they can just talk freely?
• 9:57
  Yep. They are all sitting at one
• 9:59
  giant table, sharing a single brain.
• 10:02
  And implementing NV Link Fusion changes the fundamental
• 10:04
  architecture of the data center. It eliminates those network
• 10:07
  bottlenecks entirely. Which is huge for efficiency it
• 10:10
  really. It is.
• 10:11
  By doing so, it allows operators to treat an entire rack of
• 10:15
  diverse specialized components as one single giant processing
• 10:19
  unit. Everything acting as one.
• 10:21
  Right. The software running the models
• 10:24
  no longer has to negotiate with 72 individual chips, figuring
• 10:28
  out which chip has which piece of data and routing traffic
• 10:31
  between them. It simply talks to the RAC as a
• 10:34
  unified, mathematically coherent entity.
• 10:38
  But communicating at 3.6 terabytes per second pushes
• 10:41
  traditional copper wiring past its absolute physical limits.
• 10:45
  Because copper can only do so much.
• 10:47
  Exactly. When you push electrical signals
• 10:49
  through copper at ultra high frequencies, you encounter
• 10:52
  severe physical resistance. What actually happens to the
• 10:56
  signal? The electrons get pushed to the
• 10:58
  outer edge of the wire, a phenomenon known as the skin
• 11:00
  effect. The skin effect, right?
• 11:02
  Yeah, and this increases resistance, which generates an
• 11:05
  enormous amount of heat and causes the electrical signal to
• 11:08
  degrade rapidly over very short distances.
• 11:11
  You just have racks melting again.
• 11:13
  Pretty much. You simply cannot build a server
• 11:15
  rack dense enough to handle the volume of thick copper cabling
• 11:18
  that would be required to mitigate that heat and signal
• 11:21
  loss. This requires a complete shift
• 11:23
  to using light for data transmission.
• 11:25
  And that physical limitation is the exact reason behind Nvidia's
• 11:30
  $2 billion equity investment in Marvell.
• 11:33
  Oh, absolutely. This alliance utilizes Celestial
• 11:36
  AI's photonic fabric to create what is being called an
• 11:40
  optically defined factory. Which sounds very sci-fi.
• 11:44
  It does, but it's very real. They're integrating Co packaged
• 11:48
  optics. Normally you have the processor
• 11:51
  in the middle of a circuit board and the optical transceiver, the
• 11:54
  part that turns electricity into light, sitting at the edge of
• 11:57
  the board. Right, separated by physical
• 11:58
  disk. Exactly.
• 12:00
  The electrical signal still has to travel across the board to
• 12:03
  reach the laser, burning power the entire way.
• 12:06
  Which brings us back to the heat issue.
• 12:07
  Yep. Co Packaged Optics takes the
• 12:09
  laser and microscopic glass waveguides and prints them
• 12:12
  directly onto the silicon processor itself.
• 12:15
  So it's all one piece. Yeah, the light goes straight
• 12:17
  into the brain of the chip. This shift to light reduces
• 12:20
  power consumption by a factor of 5 and drastically increases
• 12:24
  overall network resiliency. Wait.
• 12:26
  Back up a second, how do other hardware providers fit into this
• 12:29
  optical revolution? What do you mean?
• 12:30
  Because Marvel and NVIDIA aren't the only ones building these
• 12:33
  systems, we really have to look at Astrolabs and their Scorpio X
• 12:39
  series Smart Fabric switch. OK, Astrolabs.
• 12:42
  Yeah, this hardware is specifically designed for scale
• 12:44
  up GPU clustering and they are boasting operating margins of
• 12:48
  41.7%. That is a massive margin for
• 12:52
  hardware. It is that margin shows just how
• 12:56
  lucrative solving this interconnect problem has
• 12:58
  actually become. I mean, I look at the capital
• 13:01
  involved here and I strongly believe Nvidia's closed,
• 13:03
  proprietary and V link system, bolstered heavily by the Marvell
• 13:07
  investment, will easily dominate the sector.
• 13:09
  You really think so? I do.
• 13:11
  When you have a fully integrated ecosystem that works Florida
• 13:14
  State out-of-the-box hyperscalers will pay the
• 13:17
  premium for that reliability. But what about flexibility?
• 13:20
  If you are risking $2 billion on a data center build, you are not
• 13:23
  going to mess around trying to stitch together mismatched parts
• 13:26
  to save a few million on networking switches.
• 13:29
  You want the system that is guaranteed to function perfectly
• 13:32
  on day one. Well, I see it completely
• 13:34
  differently actually. Really.
• 13:35
  How so? Astrolabs is effectively acting
• 13:38
  as the Switzerland of silicon. They are supporting open
• 13:42
  standards like UI Link, which gives hyperscalers the
• 13:45
  flexibility they desperately want.
• 13:47
  You think they want to mix and match?
• 13:49
  Absolutely. If you look at the history of
• 13:51
  enterprise technology, closed proprietary systems always face
• 13:56
  massive pushback from buyers who refuse to be held hostage.
• 14:00
  I guess nobody wants vendor lock in.
• 14:02
  Right. Hyperscalers do not want to be
• 14:04
  entirely dependent on a single proprietary standard controlled
• 14:07
  by 1 vendor. They want the ability to mix and
• 14:10
  match components from different manufacturers without re
• 14:13
  architecting their entire network.
• 14:15
  OK, I can see that. If a better memory pool comes
• 14:17
  out from a competitor next year and open standard allows the
• 14:20
  data center to integrate it without throwing away all their
• 14:22
  existence hardware. Regardless of whether closed or
• 14:25
  open standards went out, this technological shift severely
• 14:29
  limits the viability of traditional pluggable hardware.
• 14:31
  Yeah, copper is basically dead in this context.
• 14:34
  Exactly. It opens up a new era where
• 14:36
  silicon photonics is no longer an optional high end upgrade.
• 14:40
  It is a mandatory requirement for cluster performance.
• 14:43
  You simply cannot compete in this space using copper anymore.
• 14:48
  But you know, all of this specialized optical hardware,
• 14:51
  the massive real estate footprints, and the dedicated
• 14:54
  power grids ultimately have to be paid for by enterprise
• 14:58
  software sale someone. Has to foot the bill.
• 15:00
  Precisely, the hyper scalers are laying down billions in optical
• 15:04
  highways because they know enterprise customers are about
• 15:06
  to foot the bill through a completely new kind of software
• 15:09
  marketplace. Right, because enterprises that
• 15:12
  already have massive spending commitments with Anthropic can
• 15:15
  use those exact same funds to purchase third party clawed
• 15:19
  powered applications built by developers like GitLab or
• 15:22
  Snowflake. It's such a clever system.
• 15:24
  It is in the cloud computing world, companies often sign
• 15:27
  contracts agreeing to spend a certain amount of money over a
• 15:30
  set period, say committing $10 million to Anthropic for the
• 15:33
  year. A classic use it or lose it
• 15:35
  budget. Exactly.
• 15:37
  This creates a use it or lose it budget.
• 15:40
  The Clod marketplace allows them to draw down that exact
• 15:43
  commitment to buy external software tools, as long as those
• 15:47
  tools run on the Clod model. And the billing architecture
• 15:50
  here is highly centralized and just brilliant for procurement
• 15:54
  teams. Because it removes the
• 15:55
  headaches. Yeah, Anthropic handles all the
• 15:58
  invoicing. The customer gets one single
• 16:01
  bill drawing from one pre approved budget.
• 16:04
  That saves so much time. If you have ever worked in
• 16:06
  enterprise software procurement, you know that onboarding a new
• 16:10
  vendor can take 6 months of legal reviews and security
• 16:13
  audits. It's a nightmare.
• 16:14
  This system bypasses all of that friction, and the success of
• 16:18
  this software layer is incredibly evident in tools like
• 16:21
  Clawed Code. Oh, Claude code is fascinating.
• 16:24
  We were looking at an environment where Claude code
• 16:26
  already authors 4% of all public GitHub commits.
• 16:30
  Just writing code on its own. Yeah, and we aren't just talking
• 16:32
  about passive autocomplete. This is an agent that can read
• 16:35
  an entire repository, understand the underlying architecture,
• 16:39
  draft the necessary code, and actively push the commit into
• 16:42
  the code base. This consolidated billing into
• 16:45
  one central clearing house, removing the friction of
• 16:48
  onboarding new vendors entirely. However, it's severely limits an
• 16:53
  enterprise's ability to switch providers down the road.
• 16:56
  The trap closes. It creates a deep structural
• 16:59
  vendor lock in where unwinding these custom integrations
• 17:03
  becomes far are too costly. Once you're in, you're stuck.
• 17:06
  Exactly once your engineering team's workflow and your
• 17:10
  financial billing are entirely routed through Anthropics
• 17:13
  Marketplace, migrating to a different language model
• 17:16
  requires tearing out the very foundation of your software
• 17:19
  development process. It's huge operation.
• 17:21
  It is no longer just swapping out an API key.
• 17:24
  It requires retraining your entire engineering department.
• 17:27
  And you know, as these integrated software agents gain
• 17:29
  deeper control over enterprise workflows and code base
• 17:32
  generation, their raw capabilities start to alarm
• 17:35
  global authorities. Now, regulators are definitely.
• 17:37
  Watching We are moving away from models that simply answer
• 17:40
  questions in a chat window toward models that take
• 17:43
  autonomous actions across highly complex networks.
• 17:46
  US and UK regulators recently organized urgent meetings with
• 17:50
  the CEO's of major banks, including Citi, Morgan Stanley
• 17:53
  and Bank of America. Yeah, that was a very quiet but
• 17:56
  very serious summit. What was this specific catalyst
• 17:59
  for those high level meetings? I mean, something must have
• 18:02
  spooked them the. Specific Catalyst centers on
• 18:04
  anthropics. Claude Mythos Preview model
• 18:07
  security teams demonstrated that the model can actively identify
• 18:10
  and exploit web browser vulnerabilities.
• 18:13
  Wait, exploit the browser itself.
• 18:15
  Yeah, browsers are built on a concept called the Same Origin
• 18:18
  Policy, which essentially sandboxes every open tab.
• 18:21
  So they can't talk to each other.
• 18:22
  Right, A script running in your social media tab should have
• 18:25
  absolutely 0 visibility into the banking tab you have open right
• 18:29
  next to it. That's just basic security.
• 18:31
  But security researchers found that an AI agent, by interacting
• 18:35
  with the browser's deeper accessibility layer and
• 18:37
  rendering engines, could occasionally see the entire
• 18:40
  document object model state across the entire browser.
• 18:44
  This means a malicious advertisement on a seemingly
• 18:47
  harmless website could potentially command the AI model
• 18:51
  to read the state of the banking tab, open next to it, extract
• 18:54
  the sensitive financial information, and infiltrate it
• 18:57
  without the user ever clicking a single button.
• 19:00
  Exactly. We just spent all this time
• 19:02
  talking about the immense effort required to build these
• 19:05
  machines. The gigawatts of power, the
• 19:07
  silicon photonics, the custom built industrial facilities.
• 19:11
  But this is what happens when you actually turn them.
• 19:13
  On It's wild to think about. The physical construction is a
• 19:16
  marvel of human engineering, but the output introduces
• 19:20
  vulnerabilities that bypass traditional cybersecurity
• 19:23
  defenses entirely. The defenses just aren't built
• 19:26
  for it. No, they aren't.
• 19:27
  You can have the most secure bank vault in the world, but if
• 19:29
  the browser itself is compromised by an autonomous
• 19:32
  agent reading the memory state, the vault is essentially left
• 19:35
  wide open. And this changes the regulatory
• 19:38
  environment permanently. Oh, without a doubt.
• 19:40
  It opens up the highly likely possibility of mandated
• 19:43
  government guardrails and strictly limits how freely these
• 19:47
  frontier models can be deployed in critical financial sectors.
• 19:50
  They won't just let it run wild. No.
• 19:52
  Regulators looking at the potential for crosstab data
• 19:55
  exfiltration will not allow systems that can autonomously
• 19:59
  breach browser isolation to operate without severe, legally
• 20:04
  enforced oversight. Yeah.
• 20:05
  So summarizing all of this, the current expansion of artificial
• 20:08
  intelligence is no longer just a software evolution.
• 20:11
  It is a massive physical industrialization requiring
• 20:15
  hundreds of billions of dollars, highly specialized optical
• 20:19
  networks, and entirely new real estate footprints across the
• 20:24
  globe. If models like Mythos can
• 20:26
  autonomously breach financial systems by reading across
• 20:29
  isolated browser tabs, perhaps the ultimate limit on this
• 20:32
  technology won't be local power grids or optical switches at
• 20:36
  all. What do you think it will be?
• 20:37
  The ultimate limit might just be strict government intervention.
• 20:40
  If you're not subscribed yet, take a second and hit follow on
• 20:42
  whatever app you're using. It helps us keep making this.
• 20:44
  We appreciate you being here. Also check out our YouTube
• 20:46
  channel for more business and updates.
• 20:48
  There's a link in the descripcion.