Latest / Elon Musk Podcast / AI Tokens worth $250,000

Transcript

AI Tokens worth $250,000 · Elon Musk Podcast

• 0:00
  NVIDIA CEO Jensen Huang stated that a software engineer earning
• 0:03
  a $500,000 salary must consume at least $250,000 worth of AI
• 0:09
  tokens annually or he would be deeply alarmed.
• 0:12
  Yeah. That is a wild ratio to think
• 0:15
  about. And just to clarify, a token is
• 0:18
  basically the absolute basic unit that artificial
• 0:20
  intelligence uses to process and generate text.
• 0:23
  So if you take a common word like unbelievable, the AI
• 0:27
  doesn't actually read it as one whole piece.
• 0:29
  It breaks it into these smaller chunks, unbelieve and able and
• 0:33
  those individual chunks, those are tokens.
• 0:36
  What we're seeing in the industry right now is this
• 0:38
  massive shift. Computing power is moving from
• 0:40
  just being this, you know, background infrastructure cost
• 0:43
  into a direct, measurable unit of human productivity.
• 0:47
  Brings us to our mission for this analysis.
• 0:49
  Today we are looking through a stack of research papers,
• 0:51
  industry surveys, and technical interviews to answer this exact
• 0:55
  question. Does spending hundreds of
• 0:56
  thousands of dollars on compute actually make an engineer 10
• 1:00
  times more productive? Or are companies just funding
• 1:03
  the illusion of speed? That is the core issue really,
• 1:06
  because right now token budgets are essentially becoming a
• 1:10
  direct recruiting tool. Wow, really?
• 1:12
  Oh, yeah, They are sitting right alongside your salary, your
• 1:15
  annual bonuses, your equity packages.
• 1:17
  I mean, we are looking at what is basically the fourth pillar
• 1:20
  of compensation. So people are actually asking
• 1:23
  about this in interviews. Exactly.
• 1:24
  Candidates interviewing at Open AI are actively asking about
• 1:27
  their compute budget before they even consider accepting a job
• 1:31
  offer. That is insane.
• 1:33
  Right, and there are engineers burning through 10 billion
• 1:36
  tokens in a single week. 10 billion.
• 1:39
  Yeah, there was this one developer at Purcell who
• 1:42
  actually racked up a $10,000 bill in one single day just
• 1:45
  deploying agents to build a a new service.
• 1:48
  I mean, if you're listening to this and wondering how someone
• 1:50
  even spends $10,000 in a day, they are basically writing
• 1:55
  scripts that automatically trigger hundreds of language
• 1:59
  models to work simultaneously. Precisely.
• 2:01
  And if those models get stuck in a feedback loop, you know,
• 2:04
  continuously talking to each other, requesting new data, the
• 2:07
  billing meter just spins completely out of control.
• 2:10
  It just runs the credit card right up.
• 2:12
  Right. It's like think about hiring a
• 2:14
  top tier biotech scientist. You pay that scientist a massive
• 2:18
  salary, but you do not just put them in an empty room with a
• 2:21
  notepad. No, of course not.
• 2:22
  They require a massive laboratory materials budget.
• 2:25
  You know, centrifuges, chemical regions, sterile environments
• 2:28
  just to function. Huang himself actually compared
• 2:32
  engineers who refused to use tokens to chip designers,
• 2:36
  insisting on using paper and pencil instead of computer aided
• 2:39
  design tools. Which is a great analogy,
• 2:41
  honestly. And this completely alters how
• 2:43
  organizations have to approach their hiring and their corporate
• 2:46
  budgets. How so?
• 2:47
  Well, a candidate's inherent value to a company is now
• 2:50
  fundamentally tied to their ability to efficiently spend the
• 2:53
  company's compute resources. Oh, I see.
• 2:56
  You aren't just hiring a coder anymore, you are hiring a
• 2:59
  manager of digital resources sources.
• 3:01
  The resource just happens to be tokens, and the budget required
• 3:04
  to sustain them is just enormous.
• 3:06
  Right, so if individual compensation is changing that
• 3:08
  drastically, the structure of the entire workforce must warp
• 3:12
  to accommodate it. Right, Absolutely, it has to.
• 3:15
  Because engineers are transitioning away from writing
• 3:18
  individual lines of code, they're spending their days
• 3:20
  writing ideas, defining architectures, and kind of
• 3:24
  orchestrating massive teams of digital workers.
• 3:26
  Yeah, the orchestration part is key.
• 3:28
  NVIDIA actually projects a future where there's 75,000
• 3:31
  human employees will work alongside 7.5 million AI agents.
• 3:37
  Which is just a staggering number.
• 3:39
  That is a 100 to one ratio of machines to humans.
• 3:43
  And that completely redefines what a software engineer
• 3:45
  actually does during their 9 to 5 schedule.
• 3:48
  Wait, back up 100 digital workers for every human?
• 3:53
  How does a single person even manage that volume of output?
• 3:56
  I mean logistically. Well, they absolutely do not do
• 3:59
  it manually. Software tools, code, compilers,
• 4:01
  relational databases, they are all going to see massive spikes
• 4:04
  in overall usage because these agents are basically the new
• 4:06
  power users right through. Especially systems like Open
• 4:09
  Claw, which is essentially this framework that allows an AI to
• 4:12
  browse the web, click buttons and type in terminal windows
• 4:15
  just like you would. These agents are operating
• 4:18
  around the clock. So they're just running 24/7.
• 4:20
  Exactly. The human is not reviewing every
• 4:23
  single action. The human is directing the
• 4:25
  agents with high level goals and then those agents are
• 4:28
  interacting directly with the enterprise software.
• 4:32
  You know, querying the databases and running automated tests
• 4:35
  continuously. Which creates an environment of
• 4:37
  extreme parallel workflows. Yes, humans are no longer
• 4:41
  bottlenecked by their own typing speed, which opens up concurrent
• 4:44
  project development on just a massive scale.
• 4:47
  Oh, completely. Like if you are an engineer, you
• 4:50
  can have 50 agents compiling your code while fifty other
• 4:53
  agents run complex security tests all happening
• 4:55
  simultaneously. Your output is completely
• 4:58
  detached from how fast your fingers move across a physical
• 5:01
  keyboard. But, and this is a big but, with
• 5:03
  those agents running continuously in the background,
• 5:06
  the sheer output seems massive. Which leads directly to the
• 5:09
  question of actual measurable productivity, because generating
• 5:13
  text quickly is fundamentally different from producing a
• 5:17
  finished working application that customers can actually use.
• 5:20
  See I agree, but there is this severe disconnect between how
• 5:25
  fast developers feel they are working and how fast they are
• 5:28
  actually shipping finished products.
• 5:30
  Oh, for sure. If you are a software engineer
• 5:32
  listening right now, you have probably felt that intense
• 5:36
  dopamine rush of generating a massive scaffolding of code in a
• 5:40
  few seconds. It.
• 5:41
  Feels great. It does.
• 5:43
  You type a prompt into your window, and an entire functional
• 5:47
  module just appears on your screen.
• 5:50
  It feels incredibly efficient. You feel like you're doing the
• 5:52
  work of 10 people. I completely see why it feels so
• 5:55
  fast, but I have to push back a bit because developers are
• 5:57
  hitting what we call the 70% problem A. 70% problem.
• 6:01
  Yeah, the artificial intelligence handles the easy
• 6:03
  70%, instantly laying down the basic syntax and structure.
• 6:07
  But that final 30% requires painstaking human logic, deep
• 6:11
  systemic understanding, and intense review to actually work
• 6:15
  in a production environment. So the last mile is the hardest.
• 6:18
  Exactly because the machine lacks actual intent, it
• 6:22
  generates mathematical patterns that look correct to the human
• 6:25
  eye. But the human has to ensure
• 6:27
  those patterns actually solve the specific, nuanced business
• 6:31
  problem the company is facing. Well, backing that up, a
• 6:34
  randomized trial conducted by MBTR actually showed that
• 6:38
  experienced developers using these tools were 19% slower at
• 6:42
  completing their tasks, even though they self reported
• 6:45
  feeling 20% faster. Which is a wild psychological
• 6:48
  trick being played on the human brain it.
• 6:50
  Really is. They felt faster, but they were
• 6:53
  empirically slower. And there's a completely
• 6:55
  separate DX study tracking enterprise companies that found
• 6:58
  a 65% increase in AI tool usage. You did only a 10% gain in
• 7:03
  actual pull request throughput. Only 10%.
• 7:06
  Yeah, and a pull request is just the formal process where a
• 7:09
  developer submits their finished code to be merged into the main
• 7:11
  company project. So two usage skyrockets, but
• 7:15
  finished work barely moves at all.
• 7:16
  Furthermore, a Stack Overflow survey revealed that only 16.3%
• 7:22
  of developers feel these tools make them highly productive.
• 7:25
  The empirical numbers simply do not support that emotional
• 7:28
  feeling of speed. So the focus of engineering work
• 7:31
  is forced to shift from creative problem solving to heavy
• 7:35
  validation and editing exactly, which heavily limits the
• 7:38
  perceived return on investment for senior talent.
• 7:41
  A senior engineer is spending more time untangling the
• 7:44
  machines overly complex mess than they would have spent just
• 7:47
  writing the simple, elegant code correctly from scratch.
• 7:51
  And because engineers are severely bogged down validating
• 7:54
  this endless stream of machine output, cognitive fatigue sets
• 7:58
  in. And that creates massive
• 8:00
  vulnerabilities when that validation process inevitably
• 8:03
  fails. Right, because you get tired.
• 8:04
  You are no longer writing software, you are proofreading a
• 8:07
  machine that never sleeps and never stops typing.
• 8:10
  And high speed generation introduces severe technical debt
• 8:13
  and major security risks into corporate systems.
• 8:16
  And a PIRO study actually found that machine generated code
• 8:19
  introduced 322% more privilege escalation paths.
• 8:23
  Which is terrifying. It is, and privilege escalation
• 8:26
  is basically when a standard user is accidentally given the
• 8:29
  keys to the Kingdom, allowing them to access administrative
• 8:33
  controls they should never have. The study also found 153% more
• 8:39
  fundamental design flaws. And the worst part is these
• 8:42
  broken commits were merged into the main projects four times
• 8:45
  faster than human written code. Four times faster, so it
• 8:49
  completely bypassed normal review scrutiny.
• 8:51
  Exactly. People trust the output far too
• 8:54
  much because it just looks so clean on the surface.
• 8:56
  There was also a 40% increase in exposed secrets.
• 8:59
  Exposed secrets like passwords. Yeah, because these models need
• 9:03
  massive amounts of context to write good code.
• 9:05
  Developers are just copying and pasting their entire working
• 9:08
  environments into the prompt window.
• 9:10
  Oh, so they are accidentally including their company's
• 9:13
  private passwords and live API keys sending highly classified
• 9:17
  data directly to external cloud servers?
• 9:19
  Hold on, so the AI operates like a highly confident intern who
• 9:23
  writes incredibly fast but routinely leaves the front door
• 9:26
  completely the unlocked? That is a highly accurate way to
• 9:28
  put it. Yeah.
• 9:30
  In fact, pull requests containing machine generated
• 9:33
  code requires 60% more security comments from human reviewers
• 9:38
  just to catch those basic mistakes.
• 9:41
  The digital intern produces an incredible volume of text, but
• 9:44
  the senior staff pays a heavy price in continuous oversight
• 9:47
  and correction. Unbridled agent deployment is
• 9:50
  just too dangerous on its own. Then it's forcing companies to
• 9:53
  adopt strict control planes and policy based guardrails just to
• 9:57
  monitor what the agents are doing on the network.
• 10:00
  You have. To You cannot just let 7 million
• 10:02
  digital workers loose on your enterprise systems without a
• 10:05
  massive, dedicated security apparatus tracking their every
• 10:08
  single move. But despite these glaring
• 10:10
  security flaws and the obvious workflow bottlenecks we just
• 10:13
  talked about, the sheer demand for compute continues to
• 10:16
  skyrocket, and it's fundamentally altering global
• 10:19
  macroeconomics. Yeah, the physical
• 10:20
  infrastructure required to power this transition is staggering.
• 10:24
  It is the Jebens paradox basically takes effect here.
• 10:27
  As we make scaffolding code cheaper and more efficient to
• 10:30
  roduce, we consume vastly more of it, which requires
• 10:33
  unprecedented physical infrastructure.
• 10:35
  Right, like with steam power. Exactly.
• 10:38
  When steam edges became more efficient in the 19th century,
• 10:41
  we did not use less coal. We found entirely new ways to
• 10:44
  use steam power, driving coal consumption through the roof.
• 10:48
  As tokens get cheaper, we deploy millions more agents to do
• 10:52
  thousands of new tasks. And while the Bureau of Labor
• 10:55
  Statistics shows only a 1.3% aggregate productivity
• 10:59
  improvement across the entire workforce. 1.3%.
• 11:02
  Just 1.3. Meanwhile, NVIDIA currently has
• 11:05
  order visibility of $1 trillion for its next generation
• 11:09
  Blackwell and Vera Rubin architectures just to meet the
• 11:12
  global demand for tokens I. Have to ask though, why drop $50
• 11:16
  billion to build a single inference factory?
• 11:19
  That massive capital expenditure seems entirely out of scale,
• 11:23
  with a 1.3% productivity gain. It does seem crazy on paper.
• 11:27
  It sounds like burning money to chase a trend that is barely
• 11:31
  moving the economic needle. Well, Huang's logic is that
• 11:34
  highly dense multibillion dollar factories are required to
• 11:37
  produce at the absolute lowest cost per Watt.
• 11:40
  Cost per Watt. Right, you need astronomical
• 11:43
  scale to make the energy consumption viable for the
• 11:45
  business. In a $50 billion budget, 20
• 11:48
  billion goes strictly to land acquisition, electricity,
• 11:52
  substations, and industrial cooling facilities.
• 11:55
  Wow, those are fixed costs regardless of the specific chips
• 11:59
  used inside the building. By maximizing the sheer
• 12:02
  throughput of the entire data center, the actual cost to
• 12:05
  produce one single token drops drastically.
• 12:08
  So the entire economic value of the IT sector becomes tethered
• 12:11
  directly to energy efficiency. Exactly.
• 12:13
  Tokens per Watt becomes the defining metric of corporate
• 12:16
  profitability. We have moved away from software
• 12:18
  algorithms and we are now dealing with a brutal physics
• 12:21
  problem of Power Distribution and heat management.
• 12:24
  And because cheap tokens require these massive physical $50
• 12:28
  billion factories and intense power grids, compute becomes a
• 12:31
  tangible physical choke point. Which means computing power is
• 12:35
  the ultimate lever for regulating and governing
• 12:38
  artificial intelligence globally.
• 12:40
  Yes, because algorithms and training data are intangible.
• 12:44
  They are easily shared across the Internet in a matter of
• 12:46
  seconds. But compute relies on a highly
• 12:49
  inelastic supply chain heavily dependent on EUV, lithography,
• 12:54
  and dedicated localized power infrastructure.
• 12:57
  Right? You can copy a data set
• 12:59
  instantly. You cannot copy a $50 billion
• 13:02
  power dense factory. Wait, EUV lithography?
• 13:05
  Just so everyone listening is on the same page, we are talking
• 13:08
  about the extreme ultraviolet lasers used to print the
• 13:11
  microscopic patterns onto the silicon chips, right?
• 13:14
  Exactly. These are some of the most
• 13:16
  complex, physical and rare their manufacturing machines on the
• 13:19
  planet. Compute is detectable, it's
• 13:21
  excludable, and it's completely quantifiable.
• 13:24
  You can literally see it, right? Governments can track large data
• 13:27
  centers from space using satellite imagery.
• 13:29
  They can subsidized specific research facilities or
• 13:32
  physically restrict international access to advanced
• 13:34
  chips. They know exactly where the
• 13:36
  power is flowing on the National Grid.
• 13:38
  So this physical reality opens up the potential for entirely
• 13:42
  new international control mechanisms.
• 13:44
  Like a Strategic Compute Reserve or even a shared international
• 13:48
  CERN for artificial intelligence.
• 13:51
  That's a fascinating concept. Yeah, nations can pool their
• 13:54
  financial resources to build this capital intensive
• 13:57
  infrastructure, ensuring they maintain strict control over the
• 14:00
  very physical substrate that the technology requires to function.
• 14:04
  We are entering an era where an engineer's value is directly
• 14:07
  measured by the volume of compute they can safely command.
• 14:11
  However, managing an autonomous workforce is far messier and
• 14:14
  slower than it feels, introducing severe security
• 14:17
  risks that require constant human oversight.
• 14:20
  If human engineers are no longer hired to do manual coding, how
• 14:23
  does someone ever get an entry level job to gain the experience
• 14:27
  needed to manage a fleet of 100 digital workers?
• 14:30
  If you're not subscribed yet, take a second and hit follow in
• 14:33
  whatever app you're using. It helps us keep making this.
• 14:35
  We appreciate you being here.