Latest / In The Loop / The Real Cost Of AGI—According To OpenAI

Transcript

The Real Cost Of AGI—According To OpenAI · In The Loop

• 0:00
  This week, Open AI's leaked projections show that they're
• 0:03
  going to spend $115 billion between now and 2029.
• 0:08
  That's $80 billion more than they'd forecast to spend just
• 0:12
  earlier this year. In today's episode, we're going
• 0:15
  to follow this money trail and try to understand exactly what
• 0:19
  the real cost of Open AI reaching AGI is going to be and
• 0:24
  where all of this money is going to go.
• 0:26
  This is in the loop with Jack Horton.
• 0:28
  I hope you enjoy the show. So let's start with the crazy
• 0:43
  spending figures because opening and now projecting and expecting
• 0:47
  to burn $115 billion between now and 2029.
• 0:50
  So to put that into context, that's about $10 billion more
• 0:53
  than they initially projected. And broadly, they're expecting
• 0:57
  to burn over $8 billion this year and then $17 billion next
• 1:00
  year. And losses are expected to peak
• 1:02
  at about 45 billion in 2028 before apparently turning
• 1:07
  profitable in 20-30, which is initially a year later than they
• 1:11
  actually hope for. Now you may be wondering what's
• 1:13
  actually the big driver of all this cost that?
• 1:15
  Where does the money actually go?
• 1:17
  And well, as I'll go through today, it's compute and training
• 1:20
  costs because Open AI are pouring money into data centres
• 1:24
  and chips. And as I'll come on to
• 1:27
  potentially training data to fund their business, one of the
• 1:31
  big spending sprees is going to come through building data
• 1:33
  centres as part of Project Stargate.
• 1:36
  For example, their partnership with Oracle.
• 1:38
  Now Project Stargate is a huge initiative to build our
• 1:41
  essentially a massive network of mega AI data centres.
• 1:46
  And in July, Open AI and Oracle announced a deal to add 4.5
• 1:50
  gigawatts of new cloud capacity across the US.
• 1:53
  To put this into context, that's the equivalent of 2 Hoover Dams,
• 1:57
  which would power 4 million homes.
• 2:00
  This 4.55 GW expansion is part of Stargate's big objective to
• 2:05
  invest about $500 billion into 10 gigawatts of AI
• 2:09
  infrastructure by 2029. Again, these data centres are
• 2:13
  the key to making AI work. The plan is to run about 2
• 2:17
  million new AI chips across the Stargate sites.
• 2:20
  That's going to enable and unlock that 10 GW capacity,
• 2:25
  which is a lot of money. Roughly I think about 30 to
• 2:28
  $40,000 per high end chip. You're looking at about 60 to
• 2:32
  $80 billion in hardware purchases to fuel the 2 million
• 2:37
  AI chips needed for Stargate. And that's before considering
• 2:40
  replacements and upgrades and expansion.
• 2:43
  To put that into context, again, no cloud project has really ever
• 2:48
  tried to assemble this much compute under a single umbrella.
• 2:51
  And as you can see here, it's why the likes of Nvidia's data
• 2:54
  centre revenue is just exploding from virtually nothing to many
• 2:58
  10s of billions per year. And this is where I think the
• 3:00
  economics of this business model is really interesting to those
• 3:03
  listening because if you actually start to break down
• 3:07
  maybe who captures what in that $1.00 transaction for some sort
• 3:11
  of AI service roughly. So maybe a big slice, maybe 25
• 3:16
  to $0.40 of that $1.00 goes towards paying for GPU hardware,
• 3:21
  which basically goes straight to NVIDIA because they're the ones
• 3:25
  selling GPUs. Another slice, let's say 10 to
• 3:28
  $0.20 might go to that cloud provider who owns the data
• 3:31
  centre to pay for power and cooling and rent and obviously
• 3:35
  profit. And then maybe the remaining 30
• 3:38
  cents goes to the AI firms, costs, research data and stuff
• 3:43
  and then hopefully trying to turn that into profit.
• 3:46
  So it's very, very expensive. And now it's probably no
• 3:49
  surprise to anybody that Open AI are trying to now invest in
• 3:53
  their own data centres and chips.
• 3:55
  So Open AI and Broadcom are going to aim to produce their
• 3:58
  first in house chip and processor by 2026.
• 4:02
  And if Open AI shifts to custom chips with Broadcom hosted in
• 4:08
  maybe self run or partner run data centres, it obviously will
• 4:12
  reduce dramatically the amount of money NVIDIA makes and also
• 4:17
  the likes of Microsoft and Amazon and the data centre
• 4:19
  providers. And the end result is that more
• 4:23
  of that $1.00 will just go to open AIS margin.
• 4:25
  And well, that's the theory anyway.
• 4:27
  And to me, this article breaking down the real finances of this
• 4:31
  update really shows that Open AI clearly trying to grab the lead
• 4:35
  in infrastructure and to capture all that future revenue and
• 4:38
  potential. So by outspending others and
• 4:41
  building out the infrastructure, which reduces their own cost
• 4:44
  long term, they're trying to secure the future.
• 4:46
  So they're actually looking at a major jump in revenue to keep up
• 4:49
  with all these costs. And so they're expecting a total
• 4:52
  revenue about $13 billion this year, which is $300 million
• 4:56
  higher than their earlier forecast this year.
• 4:59
  And by 20-30, they're expecting $200 billion in revenue, which
• 5:02
  is 15% higher than they previously estimated.
• 5:05
  And most of this they're expecting to come through
• 5:07
  ChatGPT again. So if they can get their
• 5:10
  business model off the reliance of the expensive NVIDIA chips
• 5:14
  and not relying so heavily on very expensive data centers,
• 5:18
  their profits go up and their margins go up.
• 5:20
  Now, what I would say is that this plan to escape that
• 5:23
  dependency can very easily backfire.
• 5:25
  There's a reason that not many companies have managed to
• 5:28
  squeeze their way into the data center industry or the chip
• 5:31
  industry. They're apparently spending
• 5:32
  about $10 billion with Broadcom to develop these custom chips.
• 5:36
  But it's very risky and it's very expensive because if their
• 5:39
  custom chips do not work or they underperform or they get
• 5:43
  delayed, they're stuck buying the NVIDIA chips anyway,
• 5:46
  potentially much worse off after spending billions on a
• 5:49
  completely fair project. And plus, most of the big
• 5:52
  companies like Google and Microsoft are also producing
• 5:55
  their own chips. So by the time the chip arrives,
• 5:58
  maybe all the advantages are very short lived, who knows?
• 6:01
  But you might also argue that therefore they have absolutely
• 6:04
  no choice because their competitors are going to spend
• 6:07
  huge amounts of money. And if they're producing their
• 6:10
  own chips opening, I must also compete on this front.
• 6:14
  Otherwise it's game over. Now, these things that they're
• 6:27
  trying to do are really not easy.
• 6:28
  It's worth noting, I guess, this huge need for power and money to
• 6:33
  fuel this business model, It's insane.
• 6:35
  For example, in 2023, data centres were taking up about
• 6:39
  4.4% of all electricity in the US, and it's expected to triple
• 6:44
  by 2028. And also, they just take a huge
• 6:47
  amount of time to get going. For example, the time to connect
• 6:50
  a new project to the grid is about five to seven years.
• 6:55
  Again, you can't just build a data centre and plug it into the
• 6:57
  wall. You have to actually work with
• 6:59
  the National Grid to get it integrated.
• 7:02
  And there's massive cues to obviously get that going.
• 7:05
  And we're literally seeing companies like Microsoft and
• 7:07
  Meta partner with nuclear plants and natural gas providers to try
• 7:12
  and fuel these data centres. Again, there's something to note
• 7:15
  here from an environmental impact perspective, it's worth
• 7:18
  doing the reading and analysis. It's going to have a massive
• 7:21
  impact on the environment. Of course.
• 7:23
  I hope and I really hope that we continue along the line of a
• 7:27
  massive amount of efficiency gains continually every year so
• 7:31
  it becomes more efficient. We waste less water because
• 7:36
  obviously this amount of data centre and this amount of power
• 7:39
  is going to continually make a very negative impact long term
• 7:42
  for the environment. Although I do think we'll
• 7:45
  increase efficiencies and the net gain over all of more
• 7:50
  intelligence for every single person and society as a whole
• 7:53
  will pay off anyway. This is why data centers make so
• 7:56
  much money. Only a handful of companies in
• 7:58
  the world typically are able to do this, and they're called
• 8:00
  hyperscalers, the likes of Google and Microsoft and Amazon.
• 8:04
  And I think a great example of this is Oracle.
• 8:07
  They've spent about $20 billion this year just on data centres.
• 8:10
  And much of it's just a service, open AIS, initial investment.
• 8:14
  And I guess the beauty of this business model really is that
• 8:17
  cloud providers have very limited risk.
• 8:19
  They're essentially renting servers to other AI companies.
• 8:23
  Now they have to put a huge amount of cash and time upfront
• 8:26
  to even be able to rent that out.
• 8:29
  But if open AI goes bankrupt, they'll just try and rent the
• 8:33
  servers out to someone else. Or at least that's the theory.
• 8:36
  And this is really one of the big problems right now with the
• 8:38
  business model of AI, because data centres plus chips cost a
• 8:43
  huge amount of money. And this is where a lot of the
• 8:45
  talk of bubbles has come from. Because really, if a huge amount
• 8:49
  of the revenue being captured is being captured from just a
• 8:52
  handful of small companies, then it obviously stirs up
• 8:56
  conversations of a bubble because all it takes is a
• 8:58
  handful of companies to go wrong and collapse and it's seemingly
• 9:02
  game over. Now, I've done in a previous
• 9:04
  episode just a couple of weeks ago, why I do not think we're in
• 9:07
  a full bubble, but I would expect some form of correction
• 9:10
  in the coming period. But I'll get on to that again a
• 9:13
  little bit further into the episode.
• 9:25
  It's worth going also to onto something that's just happened
• 9:28
  in the last week as well. That's going to cause a huge
• 9:30
  amount of extra or unexpected investment by the likes of Open
• 9:35
  AI just to train their models. Until now, most companies in the
• 9:38
  AI space that are training models treated data as free.
• 9:41
  They would scrape text from books, from websites, and they
• 9:45
  would rarely ever pay. And that era is coming to an end
• 9:48
  because this week, Anthropic have finally agreed to pay a
• 9:50
  $1.5 billion settlement that covers half a million books,
• 9:55
  which puts the cost of a book roughly at $3000 each.
• 10:00
  Now, the lawyers have called it the largest copyright recovery
• 10:03
  in history. On top of that, Anthropic are
• 10:06
  having to delete all of the data from pirated books that they'd
• 10:09
  previously stored. And the settlement doesn't even
• 10:11
  grant a blanket right to generate text based on those
• 10:15
  books. So Anthropic could still face
• 10:18
  claims if Claude's outputs infringe copyright.
• 10:21
  So you can just imagine the potential ramifications of
• 10:25
  something like this happening in the AI industry to an open AI,
• 10:27
  to other big providers in this race to AGI.
• 10:41
  And if you add a historical comparison to this huge amount
• 10:45
  of infrastructure build out and fear but also excitement in
• 10:48
  society right now, there are great historical comparisons.
• 10:52
  For example, during Britain's massive railway, what's
• 10:55
  described as railway mania in the 1840s, railway investments
• 10:59
  peaked at about 7% of GDP per year.
• 11:02
  And then duringthe.com bubble peak around the early 2000's,
• 11:06
  the tech and telecom industries, how we're spending about 1.5 to
• 11:10
  2% of USGDP on infrastructure. Now to put that into context, AI
• 11:15
  related data centre spending in 2025 is expected to reach about
• 11:19
  2% of USGDP. So already matchingthat.com
• 11:23
  bubble peak. Now, just like in the railway
• 11:25
  mania andthe.com bubble, lots of these companies collapsed and
• 11:29
  you know, we were left with huge amounts of redundant train line.
• 11:32
  So the key question really is, are we going to end up with more
• 11:35
  GPU capacity than we actually need or use or is demand truly
• 11:40
  that big? Now I could be wrong and I do
• 11:43
  expect some form of correction in the incoming period, but I do
• 11:45
  not expect it to be like the.com era.
• 11:48
  So in conclusion, we're seeing one of the biggest
• 11:50
  infrastructure spends in history, companies with $115
• 11:53
  billion burn rates. History suggests that we may
• 11:57
  then see a correction and this spend might not be something
• 12:01
  that can be maintained for that long.
• 12:03
  To me, it just shows that this path to AGI, wherever you define
• 12:07
  that as, is going to cost a seriously large amount of money
• 12:11
  and the biggest companies in the world are betting everything on
• 12:15
  them being first to reach whatever that definition might
• 12:17
  be. I, for one, I'm just excited to
• 12:20
  watch this play out. Thank you for listening.
• 12:22
  This is in the loop and I'll see you next week.