Latest / Dwarkesh Podcast / Dylan Patel – Anthropic & OpenAI will have most of the world’s compute by 2028

Transcript

Dylan Patel – Anthropic & OpenAI will have most of the world’s compute by 2028 · Dwarkesh Podcast

• 0:00
  Speaker 1: Okay, I'm back with Dylan Patel, founder of Semi-Analysis. Our
• 0:02
  Speaker 1: version of a family Thanksgiving dinner is a regular yearly podcast.
• 0:07
  Speaker 2: We are not actually related. Don't tell the people this.
• 0:10
  Speaker 2: It will destroy the myth.
• 0:12
  Speaker 1: Basically, where the world economy is headed is more and
• 0:17
  Speaker 1: more becoming a function of where lab economics are headed,
• 0:20
  Speaker 1: where the compute market is headed, etc. So I want
• 0:23
  Speaker 1: to understand where the crazy future ends up within a
• 0:25
  Speaker 1: few years. But let's start with just where we are today.
• 0:29
  Speaker 1: So walk me through lab compute and lab revenue right
• 0:34
  Speaker 1: now and maybe projecting out a year or two.
• 0:36
  Speaker 3: Yeah, so when we go back to last year, even
• 0:40
  Speaker 3: at the end of the year, most of GDP growth
• 0:42
  Speaker 3: in America was just AI infrastructure. And as we look
• 0:45
  Speaker 3: towards this year, about a third of the compute coming
• 0:48
  Speaker 3: online is for the labs, for OpenAI and Anthropic. Now
• 0:52
  Speaker 3: it may be built by others and then rented to them,
• 0:54
  Speaker 3: but at the end customer, it's them. As we go
• 0:58
  Speaker 3: forward into the future, the numbers for computer ballooning, we're
• 1:03
  Speaker 3: at a little bit over a trillion dollars of CapEx
• 1:06
  Speaker 3: this year. As we go out into 28, it's going
• 1:08
  Speaker 3: to be more than $ 2 trillion. The labs are also
• 1:12
  Speaker 3: taking an increasing percentage of this. And so ultimately, you've
• 1:16
  Speaker 3: got a very interesting situation where the labs are going
• 1:19
  Speaker 3: from companies that spend tens of billions of dollars a
• 1:22
  Speaker 3: year to hundreds of billions of dollars a year to
• 1:25
  Speaker 3: forecasting to spend trillions of dollars a year even towards
• 1:29
  Speaker 3: the end of the decade. And this is at least
• 1:30
  Speaker 3: some of the contracts they've begun signing with their partners.
• 1:33
  Speaker 3: And so this requires a big reshaping of what happens
• 1:37
  Speaker 3: with their economics, right? So up until now, they have
• 1:42
  Speaker 3: been companies that mostly lost money. Anthropic started turning a
• 1:46
  Speaker 3: profit in Q2. it's believed at some point in Q3,
• 1:50
  Speaker 3: OpenAI could potentially start turning a profit even with the
• 1:54
  Speaker 3: big rise of Codex and 5.6 and all this. But
• 1:58
  Speaker 3: if we go back a year ago, all the money
• 2:00
  Speaker 3: they had was venture-funded losses, right? If we go back
• 2:04
  Speaker 3: to even the beginning of this year, it was venture-funded losses.
• 2:08
  Speaker 3: They've now turned the corner and are actually starting to profit. Now,
• 2:11
  Speaker 3: that doesn't mean they're not taking in new capital. The
• 2:13
  Speaker 3: new capital is still coming in to accelerate the growth further,
• 2:16
  Speaker 3: But ultimately, there's more and more of their business is
• 2:19
  Speaker 3: being funded off of their own revenue rather than capital
• 2:22
  Speaker 3: injections into them. Over the last year and a half,
• 2:26
  Speaker 3: their margins have really skyrocketed. The base cost of compute
• 2:31
  Speaker 3: tends to be around $ 10 or $ 13 or $ 15 million
• 2:33
  Speaker 3: per megawatt. The most interesting aspect about what's happening now
• 2:39
  Speaker 3: is before, again, they were generating, if they served a model,
• 2:42
  Speaker 3: GPT-4 being served on NVIDIA Hopper GPUs was generating negative
• 2:48
  Speaker 3: gross margin for OpenAI. But now when OpenAI serves GPT-5.6
• 2:55
  Speaker 3: or Anthropic serves Opus 5 or Mythos, Fable 5, their
• 2:59
  Speaker 3: revenue generation has passed well beyond the sort of incremental $ 10, $ 15.
• 3:04
  Speaker 4: Million per megawatt.
• 3:05
  Speaker 3: In the case of Anthropic, the revenue has gone as
• 3:08
  Speaker 3: high as $ 50.
• 3:10
  Speaker 4: Million per megawatt.
• 3:12
  Speaker 3: And what that now enables them to do is, hey,
• 3:15
  Speaker 3: if I spend $ 10 on inference capacity, actually generate $ 50
• 3:18
  Speaker 3: of revenue, and then I can turn around and incrementally
• 3:21
  Speaker 3: spend all of that profit on training.
• 3:24
  Speaker 1: One thing I'm very interested in understanding is how you
• 3:26
  Speaker 1: see the centralization of compute happening in the labs, or
• 3:30
  Speaker 1: the relative ratio of compute that goes to the world
• 3:32
  Speaker 1: versus goes to the labs. where if you say right
• 3:35
  Speaker 1: now a third of marginal compute is going to the labs,
• 3:40
  Speaker 1: by when is it over half of the incremental compute
• 3:43
  Speaker 1: in the world is going to the labs? And by
• 3:45
  Speaker 1: what point do the labs have basically a vast majority
• 3:49
  Speaker 1: of the world's compute?
• 3:50
  Speaker 3: Yeah, so earlier this year, you know, the beginning of
• 3:53
  Speaker 3: this year, Anthropic OpenAI started at two for OpenAI and
• 3:56
  Speaker 3: less than two for Anthropic. End of this year, they're
• 3:59
  Speaker 3: both above five. Um, so they've three, four X compute
• 4:03
  Speaker 3: as a whole. Um, when you, when you look at
• 4:06
  Speaker 3: the incremental compute added, that's about 30% of the compute
• 4:09
  Speaker 3: added this year. And as we step forward to next year,
• 4:12
  Speaker 3: given what's already been signed and penned and inked, you've
• 4:15
  Speaker 3: got something even more dramatic, right? You've got anthropic open
• 4:18
  Speaker 3: AI are taking, uh, as much as 40 to 50%
• 4:21
  Speaker 3: of compute, uh, next year. And the centralization. doesn't look
• 4:26
  Speaker 3: like it's slowing down or stopping. In fact, it looks
• 4:29
  Speaker 3: like it's only accelerating. Now, who's building that compute for
• 4:32
  Speaker 3: them will change. You know, next year, big at new entrepreneurs,
• 4:35
  Speaker 3: for example, SpaceX is building a ton of compute. And
• 4:39
  Speaker 3: they're actively going to lease quite a bit of it
• 4:41
  Speaker 3: to Anthropic and OpenAI, most likely, because they're the ones
• 4:43
  Speaker 3: who have the marginal capability to pay the highest price.
• 4:47
  Speaker 3: In addition, OpenAI and Anthropic are also starting to build
• 4:49
  Speaker 3: their own compute. OpenAI with their own chips, Anthropic with
• 4:52
  Speaker 3: TPUs that they're purchasing from Google and deploying with Fluidstack.
• 4:56
  Speaker 3: And so when you ask, hey, when does half of
• 4:59
  Speaker 3: the world's incremental new compute go to just OpenAI and Anthropic?
• 5:02
  Speaker 3: I mean, it's really by the end of next year,
• 5:05
  Speaker 3: it's already half.
• 5:07
  Speaker 4: Of the incremental compute is going to Anthropic and OpenAI.
• 5:09
  Speaker 1: Because compute is growing so fast, incremental compute is going
• 5:14
  Speaker 1: to be basically most of compute. So it's very soon,
• 5:16
  Speaker 1: you're saying maybe within a year and a half or
• 5:18
  Speaker 1: two years, that most of the world's compute is owned
• 5:20
  Speaker 1: by two labs, or at least is serving the demand
• 5:22
  Speaker 1: from two labs.
• 5:24
  Speaker 2: How long do you think?
• 5:25
  Speaker 1: So there's this trend where maybe world compute in gigawatts
• 5:29
  Speaker 1: doubles every year, but the compute at the frontier labs
• 5:31
  Speaker 1: triples every single year. But if you keep the current
• 5:33
  Speaker 1: trend going, it goes from like two at the beginning
• 5:35
  Speaker 1: of this year to close to like six at the
• 5:37
  Speaker 1: end of this year, just multiplying out by three, 18
• 5:39
  Speaker 1: by the end of 2027, 54 by the end of 2028.
• 5:44
  Speaker 1: Are you like, okay, at that point, they simply can't
• 5:46
  Speaker 1: continue tripling given the amount of world compute? Or how
• 5:49
  Speaker 1: do you see the world compute situation over the next
• 5:51
  Speaker 1: few years?
• 5:52
  Speaker 3: Yeah, so if the incremental compute adds this year 30 gigawatts,
• 5:54
  Speaker 3: next year 50 gigawatts, and the year after that 70, roughly,
• 5:58
  Speaker 3: you end up with this really interesting phenomenon, which is, okay, well,
• 6:02
  Speaker 3: a new watt deployed this year is significantly more efficient
• 6:04
  Speaker 3: than the watts deployed two years ago. So actually, a
• 6:08
  Speaker 3: humongous percentage of the world's compute was deployed this year,
• 6:11
  Speaker 3: even though it didn't double the number of watts deployed.
• 6:14
  Speaker 3: I'm deploying GB300s and TPUV7s. and Tranium 3s, which are way, way,
• 6:20
  Speaker 3: way more efficient, 3x, 5x more performance per watt than
• 6:24
  Speaker 3: the prior generation chips. And so ultimately, you've got a
• 6:27
  Speaker 3: huge ladder here. So if Anthropic and OpenAI take on
• 6:31
  Speaker 3: 45% of compute next year, you've got them in, let's say,
• 6:35
  Speaker 3: December 27. They have taken on half of the world's
• 6:38
  Speaker 3: incremental new compute. But that half of the world's new
• 6:41
  Speaker 3: incremental compute is actually at a higher performance than everything
• 6:44
  Speaker 3: else before it. So you've got another multiplier on that.
• 6:47
  Speaker 3: So by the time you're in towards the end of 2028,
• 6:49
  Speaker 3: if this trend continues, which I see nothing that's stopping it,
• 6:55
  Speaker 3: you've got them just controlling most of the usable flops
• 6:59
  Speaker 3: in the world on their own.
• 7:01
  Speaker 1: The thing I'm confused about is why you think we
• 7:04
  Speaker 1: only add 80 gigawatts in 2028 if we enter in
• 7:07
  Speaker 1: a world in which the value of computing increases so much.
• 7:10
  Speaker 4: That's the upper bound, by the way. That's the, like,
• 7:13
  Speaker 4: I'm so fucking bullish. Right.
• 7:14
  Speaker 2: Okay, so let's do some chain of thought here.
• 7:18
  Speaker 1: So when I interviewed you a few months ago, you
• 7:20
  Speaker 1: said in order to make a gigawatt of, I think,
• 7:23
  Speaker 1: Vera Rubins, you need 55,000 N3 wafers, 6K N5 wafers,
• 7:30
  Speaker 1: and 170K DRAM wafers.
• 7:33
  Speaker 2: I don't know those numbers.
• 7:34
  Speaker 4: I'm going to troll you, but the way you said
• 7:35
  Speaker 4: wafers was so fucking idiot.
• 7:36
  Speaker 3: Wafers.
• 7:39
  Speaker 1: By the way, when we first moved to the US,
• 7:41
  Speaker 1: I had the VEW thing pretty bad, and I was
• 7:44
  Speaker 1: a vegetarian.
• 7:45
  Speaker 4: Vegetarian. I remember you told me about this.
• 7:48
  Speaker 2: In North Dakota, I was in elementary school, and I'd
• 7:51
  Speaker 2: be like... Can I get a wedgie? Can I get
• 7:53
  Speaker 2: some wedgies? Anyways, so that's for one gigawatt, right?
• 8:00
  Speaker 4: Yeah.
• 8:01
  Speaker 1: Now, I had an LLM run your wafer fab equipment
• 8:04
  Speaker 1: model and figure out how much tooling.
• 8:08
  Speaker 1: How much the tooling costs to produce a gigawatt of
• 8:12
  Speaker 1: compute basically every single year. And it was said like
• 8:14
  Speaker 1: three to $ 4 billion. Now suppose you add in clean
• 8:17
  Speaker 1: rooms and shell and everything else with the fab. So
• 8:20
  Speaker 1: $ 6 billion of like fab CapEx produces every single year
• 8:26
  Speaker 1: a gigawatt and a gigawatt produces right now, a hundred
• 8:29
  Speaker 1: billion dollars of revenue. But also that 6 billion in
• 8:34
  Speaker 1: CapEx is producing a gigawatt every single year. And that
• 8:37
  Speaker 1: gigawatt is producing a hundred billion dollars every single year.
• 8:40
  Speaker 1: So even over the course of five years, so, you know,
• 8:45
  Speaker 1: the first gigawatt has generated five years of profits. The
• 8:47
  Speaker 1: second gigawatt that the fab has produced has generated four
• 8:50
  Speaker 1: years of profits and so on. 6 billion of CapEx
• 8:54
  Speaker 1: at the fab level will have generated over a trillion
• 8:58
  Speaker 1: dollars of end AI revenue.
• 9:02
  Speaker 4: There's a lot of OpEx along the way. There's a
• 9:03
  Speaker 4: lot of other CapEx, like the data center, the power.
• 9:06
  Speaker 2: And you have to pay OpenAI for the R & D.
• 9:09
  Speaker 3: There's a lot of different people who need money here.
• 9:11
  Speaker 3: But yeah, there's a huge.
• 9:13
  Speaker 2: Take.
• 9:13
  Speaker 1: Away half of it for all these middlemen. That still
• 9:17
  Speaker 1: means there's a 100x discrepancy between fab CapEx and end
• 9:22
  Speaker 1: revenue generated. More than that, actually, really, but we're just
• 9:25
  Speaker 1: being very conservative. And as a result, this is capitalism, right? Like,
• 9:31
  Speaker 1: you would imagine that people are going to figure, like,
• 9:32
  Speaker 1: we're going to be, you have this huge discrepancy where
• 9:35
  Speaker 1: you can turn $ 1 into $ 100, and they're not.
• 9:38
  Speaker 2: Going to figure out a way to make more mirrors.
• 9:41
  Speaker 4: I mean, they are.
• 9:41
  Speaker 1: Right.
• 9:42
  Speaker 4: It's just these mirrors taking some time to.
• 9:43
  Speaker 2: Bake, right?
• 9:44
  Speaker 1: But the emergency are so big. We're like, anthropic and open-air, like,
• 9:47
  Speaker 1: we could make a trillion dollars right now. but we're
• 9:49
  Speaker 1: just bottlenecked on the mirrors that go into the ASML machines.
• 9:54
  Speaker 1: How can we make more mirrors if we spend $ 100
• 9:55
  Speaker 1: billion on this, right? That's the situation we're going to
• 9:57
  Speaker 1: be in pretty soon. And I'm just like, we're not
• 10:00
  Speaker 1: going to be able to solve that supply constraint. That
• 10:01
  Speaker 1: just seems quite hard to imagine.
• 10:04
  Speaker 4: No, there's definitely, you've seen people do funny arbitrages here
• 10:09
  Speaker 4: where they buy like turbines and then they try and
• 10:11
  Speaker 4: resell them.
• 10:12
  Speaker 5: Right.
• 10:12
  Speaker 3: Because the value of a turbine is way more because
• 10:14
  Speaker 3: it's the thing bottlenecking a data center. You know, I
• 10:17
  Speaker 3: think like if anyone had like $ 400 million and the
• 10:20
  Speaker 3: ability to convince ASML to sell them an EUV tool,
• 10:22
  Speaker 3: they should totally just go buy one and wait, wait, wait.
• 10:25
  Speaker 4: And then sell it for north of a billion dollars, right?
• 10:29
  Speaker 4: But ultimately, yes, capitalism will cause these things to expand,
• 10:33
  Speaker 4: but it's a whip, right? It takes a long time
• 10:36
  Speaker 4: for the whip signal to get to the tail end.
• 10:38
  Speaker 1: Yeah.
• 10:40
  Speaker 4: Of that.
• 10:40
  Speaker 3: And so the supply chain doesn't react immediately.
• 10:43
  Speaker 4: In fact, you go to talk to someone.
• 10:45
  Speaker 3: At Carl Zeiss, they're like, yeah, yeah, yeah, we need
• 10:47
  Speaker 3: to make 100 EUV tools by the end of the decade.
• 10:49
  Speaker 3: I think when we had our episode earlier this year,
• 10:53
  Speaker 3: they didn't even think they needed to make that many,
• 10:55
  Speaker 3: enough mirrors to make 100 EUV tools a year. And
• 10:59
  Speaker 3: so now they're like, okay, we need to do that.
• 11:02
  Speaker 3: But in reality, because of all the economics of what's
• 11:05
  Speaker 3: going on, it should be even more. But it takes
• 11:08
  Speaker 3: so long to to suppose hill.
• 11:11
  Speaker 1: Suppose that every single company in the firm I'm sorry
• 11:13
  Speaker 1: in the stack got private equity Like somebody came in
• 11:16
  Speaker 1: who was super AGI pill then was like we're gonna
• 11:18
  Speaker 1: maximize production How fast what do you think the physical
• 11:21
  Speaker 1: constraints on making more things would be? because you were
• 11:24
  Speaker 1: there's an ass is we're pretty soon gonna be in
• 11:26
  Speaker 1: a world where the lab revenue or just AI cash
• 11:30
  Speaker 1: flows because obviously the accelerators also have these huge cash
• 11:32
  Speaker 1: flows will be so big that you can just fund
• 11:35
  Speaker 1: a extreme expansion of all this production from cash flows themselves.
• 11:40
  Speaker 4: Yeah, I do agree generally. There's obviously some physical constraints.
• 11:45
  Speaker 3: The way the supply chain is expanding currently, the 100
• 11:48
  Speaker 3: is roughly still the right number.
• 11:49
  Speaker 2: For 2030?
• 11:51
  Speaker 3: 100 ASML tools for 2030. But if you said, Carl Zeiss,
• 11:56
  Speaker 3: here's $ 10 billion, please fucking just expand production, that would
• 12:00
  Speaker 3: change things.
• 12:01
  Speaker 4: And you would have to do this with every company.
• 12:03
  Speaker 3: In the supply chain. I don't think it'll happen this year.
• 12:04
  Speaker 3: I don't think it'll happen next year. I don't think
• 12:06
  Speaker 3: it'll happen the year after because the world is capital constrained.
• 12:09
  Speaker 1: But in a world where, say, the top labs are generating,
• 12:13
  Speaker 1: let's say, even combined, a trillion dollars in revenue next year,
• 12:16
  Speaker 1: they're not able to say 10.
• 12:17
  Speaker 4: I don't think they're going to do that, but.
• 12:19
  Speaker 2: Yeah, or hundreds of billions at least, right?
• 12:20
  Speaker 1: It just seems like they realize where the world is headed.
• 12:23
  Speaker 1: I feel like they could just make.
• 12:25
  Speaker 4: So the thing is the labs can spend hundreds of billions.
• 12:28
  Speaker 4: They're going to generate hundreds of billions of revenue next year.
• 12:31
  Speaker 4: But ultimately, CapEx next year is like $ 2 trillion.
• 12:34
  Speaker 3: So you've got this big mismatch, right? The wafer fabrication
• 12:39
  Speaker 3: equipment supply chain will do something on the order of
• 12:42
  Speaker 3: $ 200 billion. The data center market supply chain will do
• 12:46
  Speaker 3: even more. The accelerator supply chain will do even more.
• 12:49
  Speaker 3: The energy supply chain will do a number. You sum
• 12:51
  Speaker 3: all this up, it's going to be well north of
• 12:53
  Speaker 3: $ 2 trillion of capex. So the labs have not yet
• 12:57
  Speaker 3: gotten to the point where their cash flows can fund
• 12:58
  Speaker 3: this stuff.
• 12:59
  Speaker 2: Of course, yeah, yeah, yeah.
• 13:00
  Speaker 1: I mean, obviously, they will never get to that point, right?
• 13:02
  Speaker 1: Because they want to keep- Yeah, you reinvest. You want
• 13:04
  Speaker 1: to make your capex higher than your returns. But the
• 13:08
  Speaker 1: key question I really want to understand is if the
• 13:11
  Speaker 1: current continues to be north of 50 gigawatts, per lab
• 13:15
  Speaker 1: by the end of 2028. So between them, they'd have
• 13:16
  Speaker 1: 100 gigawatts. Those gigawatts, as you're saying, drive many fold
• 13:21
  Speaker 1: more throughput or more performance by 2028 than they are now, right?
• 13:25
  Speaker 1: Because the hardware has gotten better. So not only have
• 13:27
  Speaker 1: like flops per watt increase, but also the hardware gets
• 13:30
  Speaker 1: better at.
• 13:33
  Speaker 2: Working with AI workloads.
• 13:34
  Speaker 1: Okay, so 100 gigawatts for the labs end of 2028.
• 13:38
  Speaker 1: How much is like world compute?
• 13:39
  Speaker 3: I think that may be a little difficult given 2028,
• 13:41
  Speaker 3: you start to have they've taken 70, 80% of incremental compute,
• 13:47
  Speaker 3: and I'm not sure what happens to markets then, right?
• 13:50
  Speaker 3: How much does the price of compute skyrocket for them
• 13:53
  Speaker 3: to actually be able to buy 70, 80% of compute?
• 13:56
  Speaker 3: Is Google or Meta or Amazon willing to sell even
• 14:01
  Speaker 3: that much? Also, one caveat when we're sort of talking
• 14:04
  Speaker 3: about these gigawatt numbers is when Amazon is serving bedrock
• 14:08
  Speaker 3: anthropic models, that counts as anthropic compute in sort of
• 14:11
  Speaker 3: our worldview because It is effectively, at the end of
• 14:13
  Speaker 3: the day, counted as revenue for Anthropic, even though there's
• 14:17
  Speaker 3: a revenue share and credit back and all that. But ultimately,
• 14:20
  Speaker 3: in 2028, if they get to 100 gigawatts combined, they
• 14:24
  Speaker 3: have done really disruptive things to the market. Because anyone
• 14:29
  Speaker 3: can make money off of $ 10 to $ 15 million per
• 14:32
  Speaker 3: megawatt compute today. You literally, like I kid you not,
• 14:35
  Speaker 3: it's not that hard. Go get a GB300 rack. Go
• 14:39
  Speaker 3: download the KimiWeights. Go download VLM or SGLang, set it up.
• 14:44
  Speaker 3: Codex and Fable can actually help you do this. It's
• 14:46
  Speaker 3: pretty simple. I mean, it's not trivial, but it's not
• 14:50
  Speaker 3: like rocket science. And go put it on OpenRouter. It's
• 14:53
  Speaker 3: very simple. And you'll start generating more revenue than you're
• 14:58
  Speaker 3: paying for the compute. And so this has sort of
• 15:01
  Speaker 3: already led to this compute pricing $ 10 to $ 15 million
• 15:04
  Speaker 3: per megawatt start to inflect up. And to get to
• 15:08
  Speaker 3: that 100 gigawatts in 2028, you have to believe that
• 15:12
  Speaker 3: the labs can outpay for compute because anyone can make
• 15:15
  Speaker 3: money at 10 to 15. Does compute now get to
• 15:18
  Speaker 3: $ 25 million a megawatt? Does it get to $ 40 million
• 15:20
  Speaker 3: a megawatt?
• 15:21
  Speaker 1: But as you're saying, it's already the case that the
• 15:23
  Speaker 1: labs are generating way more revenue per megawatt than everybody else.
• 15:27
  Speaker 1: If they stay as far ahead as they are currently,
• 15:29
  Speaker 1: you would expect that to be the continuing case. If
• 15:30
  Speaker 1: there's some kind of recursive self-improvement where the AI labs
• 15:33
  Speaker 1: are like, relatively uplifted or they have models internally they're
• 15:36
  Speaker 1: not releasing externally that are helping them make the next
• 15:38
  Speaker 1: model better. You'd expect that to be even more of
• 15:40
  Speaker 1: the case. Aren't you already seeing this where like SpaceX
• 15:42
  Speaker 1: or whoever's like slightly further behind will just sell Compute
• 15:44
  Speaker 1: to the highest bidder if they can't internally monetize it
• 15:46
  Speaker 1: as well as the labs? I feel like it's continue
• 15:49
  Speaker 1: expecting them to be able to gobble up, like bid
• 15:51
  Speaker 1: for larger and larger shares of the Compute.
• 15:54
  Speaker 3: I think that is my worldview that they will continue
• 15:56
  Speaker 3: to gobble up more of the Compute, but ultimately they
• 15:59
  Speaker 3: can't do it at current pricing or anywhere close to it.
• 16:02
  Speaker 5: Sure, sure.
• 16:03
  Speaker 3: They do have to start paying 25, 30, $ 50 million
• 16:06
  Speaker 3: a megawatt to really gobble up 70% of the world's
• 16:09
  Speaker 3: compute in 2028 to get to that 100 gigawatts by 2028,
• 16:13
  Speaker 3: which is a very sort of aggressive goal. The other
• 16:15
  Speaker 3: aspect of this that's really challenging is we've already seen
• 16:18
  Speaker 3: a huge.
• 16:19
  Speaker 4: Slowdown for the AI labs, right?
• 16:22
  Speaker 3: This regulation that they advocate for is actually slowing down
• 16:25
  Speaker 3: the labs a lot more than it slows down, you know,
• 16:27
  Speaker 3: sort of the open source Chinese language models. You know,
• 16:30
  Speaker 3: OpenAI not releasing Astra. OpenAI stopping training for two weeks.
• 16:35
  Speaker 3: Anthropic not releasing what their safety assessment said is Model 2,
• 16:39
  Speaker 3: which is widely believed to be the next version of Mythos.
• 16:42
  Speaker 3: They're clearly not releasing their best models, in which case
• 16:45
  Speaker 3: their revenue per megawatt stalls or even can start to
• 16:48
  Speaker 3: decline again because other models are competitive again. So it's
• 16:52
  Speaker 3: not that they're falling behind, it's just that they're not
• 16:53
  Speaker 3: releasing their best stuff. What if there is some regulatory
• 16:57
  Speaker 3: impact that prevents them from releasing their best models? Now
• 17:00
  Speaker 3: their revenue per megawatt does not climb as fast, then
• 17:02
  Speaker 3: their ability to buy that incremental compute for a higher
• 17:05
  Speaker 3: price than everyone else starts to diminish, and then maybe
• 17:08
  Speaker 3: they can't get to that 100 gigawatts is sort of,
• 17:11
  Speaker 3: in a world where safety doesn't matter, I do believe
• 17:14
  Speaker 3: that's exactly what happens, right? They can start generating $ 100
• 17:16
  Speaker 3: million per megawatt or more, and they can pay $ 50
• 17:19
  Speaker 3: million a megawatt, and no one else has any logical
• 17:22
  Speaker 3: reason to do anything with their compute besides say, please, Dario,
• 17:26
  Speaker 3: take everything off of my hands. But there are forces
• 17:31
  Speaker 3: at play, which we cannot describe, that would potentially slow
• 17:37
  Speaker 3: this down.
• 17:37
  Speaker 2: Yeah, yeah, yeah.
• 17:38
  Speaker 1: I mean, I think a good intuition pump is just
• 17:41
  Speaker 1: what if the AI models were literally as good as
• 17:43
  Speaker 1: a fully automated software engineer? They're not currently there yet, right?
• 17:47
  Speaker 1: Like I think they're far from just being able to
• 17:48
  Speaker 1: fully automate the job of like a full white collar worker.
• 17:52
  Speaker 1: But white collar workers earn, you know, six figures or north.
• 17:55
  Speaker 2: Of that a year.
• 17:57
  Speaker 1: And if you have a gigawatt that can sustain a
• 17:59
  Speaker 1: population of like, say a million white collar workers, let's
• 18:05
  Speaker 1: say roughly, right? That's like, you could then off the
• 18:09
  Speaker 1: back of that, That would be 100 billion.
• 18:13
  Speaker 2: That's actually surprisingly low.
• 18:15
  Speaker 4: Yeah, 100k per person, million population, yeah?
• 18:17
  Speaker 1: Yeah, yeah, yeah.
• 18:18
  Speaker 4: I don't know.
• 18:19
  Speaker 1: But it would be many hundreds of billions of dollars
• 18:20
  Speaker 1: if you get full AGI per gigawatt.
• 18:23
  Speaker 3: I think the other aspect of this is, and we've
• 18:25
  Speaker 3: continued to see this, most of the value capture is
• 18:28
  Speaker 3: not happening, right? Like most of the value that these
• 18:30
  Speaker 3: models generate does not get given to OpenAI and Anthropic.
• 18:34
  Speaker 3: Thankfully so far, it is mostly just being given to
• 18:36
  Speaker 3: the users, right? Jane Street with their exclusive contract with
• 18:40
  Speaker 3: OpenAI for GPT 5.6 ultra fast mode or Jane Street
• 18:43
  Speaker 3: where they're like one of Anthropic's biggest customers is generating way, way, way,
• 18:48
  Speaker 3: way more value out of the tokens they're paying for
• 18:52
  Speaker 3: than Anthropic is generating in terms of profit, right?
• 18:55
  Speaker 4: Because they get to, you know, make money off of
• 18:58
  Speaker 4: the market.
• 18:59
  Speaker 3: Or Meta, who at one point was, you know, rumored
• 19:02
  Speaker 3: to be, you know, as much as 10% of Anthropic's business,
• 19:05
  Speaker 3: you know, they're generating way more efficiencies by optimizing their
• 19:10
  Speaker 3: ad algorithms or what have you and getting engagement time
• 19:14
  Speaker 3: 5% longer and, you know, all these things. They're making
• 19:16
  Speaker 3: way more money off of using these models than Anthropic.
• 19:21
  Speaker 3: And so the ultimately, you know, and that's what's required.
• 19:23
  Speaker 3: So sure, if you had a million new software engineers
• 19:27
  Speaker 3: the cost for software engineer would also fall.
• 19:29
  Speaker 1: One thing I'm confused about is, does the market come
• 19:32
  Speaker 1: in equilibrium? And if it comes in equilibrium, would you
• 19:34
  Speaker 1: just expect the price of compute to equal whatever anthropic
• 19:40
  Speaker 1: and open AI can generate from it? Or be very
• 19:43
  Speaker 1: close to it, but like a small amount of markup
• 19:44
  Speaker 1: for anthropic and open AI. Like right now, it's really
• 19:46
  Speaker 1: weird that there is a forex or more difference between
• 19:50
  Speaker 1: what compute sells for and how much money anthropic can
• 19:53
  Speaker 1: make from it. And in a world where the revenue
• 19:55
  Speaker 1: per gigawatt continues to increase, if anthropic's ability to monetize
• 19:59
  Speaker 1: a gigawatt doubles or triples or something, it'd be weird
• 20:02
  Speaker 1: if then the gap continued to increase. And so anthropic,
• 20:05
  Speaker 1: just by like having some software having some weights, can
• 20:08
  Speaker 1: take something that cost them $ 10 and then turn it
• 20:10
  Speaker 1: into $ 100.
• 20:12
  Speaker 3: Yeah, so there's a bit of, this is always a
• 20:15
  Speaker 3: fun question, right? Which is where does the value go
• 20:16
  Speaker 3: in AI? AI is generating all this value. You've got,
• 20:19
  Speaker 3: you know, the end user, which I think we all
• 20:21
  Speaker 3: agree is generating more value than anyone else. Hence, they're
• 20:24
  Speaker 3: paying a lot for these models. But then you have,
• 20:26
  Speaker 3: you know, the app layer. Well, so far, the app
• 20:28
  Speaker 3: layer has generated very little value. And you've got the
• 20:31
  Speaker 3: model layer, which again, up until a year ago, was
• 20:34
  Speaker 3: generating negative gross margins and is now generating massive positive
• 20:38
  Speaker 3: gross margins. And looks like it's on the path to
• 20:40
  Speaker 3: generating you know, $ 100 million per megawatt. So turning, you know, $ 10,
• 20:46
  Speaker 3: $ 15 into $ 100, as you said. But if we go
• 20:49
  Speaker 3: back again a year ago, the hardware supply chain was
• 20:52
  Speaker 3: generating all this gross margin while literally everyone else was
• 20:54
  Speaker 3: losing money on it. OpenEye and Anthropic were just plowing
• 20:57
  Speaker 3: VC money in. And as were many other startups and
• 21:02
  Speaker 3: many of these hyperscalers are building infrastructure without knowing if
• 21:05
  Speaker 3: there was going to be a payoff. So ultimately, you
• 21:08
  Speaker 3: had this negative value being created on the model layer almost,
• 21:11
  Speaker 3: if you will, because they were selling the tokens for
• 21:14
  Speaker 3: less than it cost them on the infra side. And
• 21:15
  Speaker 3: all the values being created used at the chip, the fab.
• 21:19
  Speaker 3: Initially in 2023, the memory guys were making no money.
• 21:23
  Speaker 3: off of, you know, HBM or memory for AI, even though, theoretically,
• 21:27
  Speaker 3: their value they were delivering was humongous. Now you've got, well, actually,
• 21:30
  Speaker 3: KSMC makes way less value than the memory guys. Is
• 21:33
  Speaker 3: that actually how much, you know, they're capturing less value.
• 21:37
  Speaker 3: So the value capture shifted around a lot, which is
• 21:39
  Speaker 3: very fun for people tracking the market or participating in
• 21:43
  Speaker 3: the market, like Jane Street is an example.
• 21:47
  Speaker 2: You don't have to plug them that hard.
• 21:48
  Speaker 4: You don't have to.
• 21:49
  Speaker 2: Plug them that hard.
• 21:52
  Speaker 4: So, you know, what happens, you know, going forward?
• 21:55
  Speaker 3: Does Anthropic and OpenAI, you know, they've slowly started to
• 21:59
  Speaker 3: balloon in value capture.
• 22:00
  Speaker 4: Do they balloon and take all the value capture? Well,
• 22:02
  Speaker 4: that was a thought.
• 22:03
  Speaker 3: And then Elon showed, actually, no, I can sell my
• 22:06
  Speaker 3: compute for $ 25 million a megawatt or $ 40 million a
• 22:10
  Speaker 3: megawatt to Anthropic and Google. Even if it's a short-term thing,
• 22:14
  Speaker 3: I've sold it for this price, and I'll recoup my
• 22:16
  Speaker 3: entire capex in a year.
• 22:18
  Speaker 1: So what's your prediction of how much the relevant Toronto compute,
• 22:22
  Speaker 1: like B300s or whatever, that's sold for 40B a gigawatt,
• 22:26
  Speaker 1: the SpaceX sold for 40B a gigawatt to Google, what
• 22:29
  Speaker 1: does that sell for.
• 22:30
  Speaker 2: At the end of next year?
• 22:31
  Speaker 3: I think most compute will still continue to transact at
• 22:34
  Speaker 3: sub $ 20 billion a gigawatt.
• 22:35
  Speaker 2: Even at the end of next year?
• 22:36
  Speaker 3: Because all of it has to be financed. For compute
• 22:39
  Speaker 3: that you can build without financing, right? If Meta can
• 22:42
  Speaker 3: build compute, Microsoft, Amazon, SpaceX can build compute without finding
• 22:47
  Speaker 3: a customer just saying, fuck it, I'm going to build
• 22:48
  Speaker 3: this compute. And then turn around and wait until it's
• 22:50
  Speaker 3: already built. They now control what's going on. So most
• 22:55
  Speaker 3: compute is contracted well before it's built. And so this
• 22:59
  Speaker 3: is sort of what Elon took advantage of in the
• 23:00
  Speaker 3: market is he actually had all this compute and he
• 23:02
  Speaker 3: was like, hey, Anthropic, I know you're making like 60
• 23:05
  Speaker 3: plus billion dollars per gigawatt. Why don't you just buy
• 23:08
  Speaker 3: my stuff for a crazy amount of money? And obviously,
• 23:10
  Speaker 3: you know, it's not like Elon decided this or Anthropic
• 23:12
  Speaker 3: decided this. It's sort of markets figured itself out. Other people,
• 23:17
  Speaker 3: you know, you go to a random cloud. They're like, okay,
• 23:21
  Speaker 3: I'm going to build a gigawatt of compute or 100
• 23:23
  Speaker 3: megawatts of compute.
• 23:24
  Speaker 4: I'm going to spend the CapEx. I need to turn
• 23:26
  Speaker 4: around and find a customer. If I want to find
• 23:28
  Speaker 4: a customer, I need to find the capital. Who's going
• 23:31
  Speaker 4: to give me the capital and the customer? The customer
• 23:33
  Speaker 4: has to sign a deal. And then I take the
• 23:34
  Speaker 4: customer's commitment to the credit markets and I raise the capital.
• 23:37
  Speaker 4: And so there's this sort of like completely different power
• 23:40
  Speaker 4: structure where Meta, who is effectively hoarding compute, them and
• 23:45
  Speaker 4: SpaceX are plausibly the like number three. And the only
• 23:47
  Speaker 4: plausible number three is because they're hoarding all this compute.
• 23:50
  Speaker 4: They're using their balance sheets and capabilities to build compute.
• 23:53
  Speaker 4: to build compute without end customer that's monetizing at a
• 23:56
  Speaker 4: huge degree. And they have an actual.
• 23:57
  Speaker 3: Balance sheet, so they can go to the credit market
• 23:59
  Speaker 3: and being like, hey guys, you build a gigawatt, you
• 24:03
  Speaker 3: can make your margin, not a crazy margin, but you
• 24:06
  Speaker 3: can make a good margin, and I now have all
• 24:07
  Speaker 3: this compute. And now Meta and SpaceX have this optionality
• 24:10
  Speaker 3: of looking around and being like, is my internal use
• 24:12
  Speaker 3: case gonna make me more money, or should I go
• 24:14
  Speaker 3: out there and sell it to Anthropic OpenAI at crazy margins?
• 24:18
  Speaker 3: So now we've sort of entered a regime where SpaceX
• 24:21
  Speaker 3: and Meta are saying, actually, I'm going to build the
• 24:24
  Speaker 3: compute and I can start to rent it out for
• 24:26
  Speaker 3: not 13. I can sell it for 25, 50 and more.
• 24:31
  Speaker 1: So as I've been doing video essays and other formats,
• 24:34
  Speaker 1: I've been looking to hire a new editor for the podcast.
• 24:36
  Speaker 1: But actively searching for editors has been quite time consuming
• 24:39
  Speaker 1: because the vast majority of candidates are don't fit the
• 24:41
  Speaker 1: profile that I'm looking for. So I created a recruiter
• 24:44
  Speaker 1: in Grokbot to see if it would help. I give
• 24:46
  Speaker 1: it a huge context dump where I monologued basically everything
• 24:49
  Speaker 1: that I wanted. And then it spun up four other
• 24:51
  Speaker 1: bots to narrow in on different parts of the search.
• 24:55
  Speaker 1: One went through the last year of my email for
• 24:57
  Speaker 1: relevant inbound. One searched my X feed and DMs. One
• 25:00
  Speaker 1: went through the end credits on various documentaries I like.
• 25:02
  Speaker 1: And the last one looked for editors who work for
• 25:04
  Speaker 1: some of the YouTubers that I follow. Grok bot then
• 25:06
  Speaker 1: took all these different candidates that the sub agents had found,
• 25:09
  Speaker 1: it filtered them against my criteria, and then delivered for
• 25:12
  Speaker 1: me a final short list to review. To be honest,
• 25:14
  Speaker 1: the first batch had a few good candidates I wanted
• 25:16
  Speaker 1: to see, but mainly a bunch of duds. But after
• 25:19
  Speaker 1: I gave Grok bot some more feedback about what it
• 25:21
  Speaker 1: was missing, it came back with a new list of
• 25:23
  Speaker 1: candidates that I'm actually extremely excited about. I ended up
• 25:26
  Speaker 1: saving this whole workflow as a routine. So every week
• 25:28
  Speaker 1: now Grok bot checks my inbound email and XDMs for
• 25:33
  Speaker 1: promising new candidates to potentially interview. If you want to
• 25:35
  Speaker 1: try Grokbot yourself, go to x.ai slash bot. What do
• 25:40
  Speaker 1: you think their revenue per gigawatt is by the end
• 25:42
  Speaker 1: of 2027? Like for Anthropico OpenAI by the end of 2027?
• 25:45
  Speaker 3: I think it's highly dependent on who has the best model,
• 25:47
  Speaker 3: if they're allowed to keep releasing their best models, but
• 25:49
  Speaker 3: I don't see why it wouldn't be 50 plus million
• 25:51
  Speaker 3: dollars a megawatt.
• 25:52
  Speaker 2: By the end of 2027?
• 25:53
  Speaker 4: Oh, by the end of 2027.
• 25:55
  Speaker 3: That's where it gets more challenging, but I think it
• 25:57
  Speaker 3: could get to higher than that, like 70, 80 million
• 25:59
  Speaker 3: dollars a megawatt. blended across the company, if not higher.
• 26:04
  Speaker 3: I think if that's the case, then what happens to
• 26:08
  Speaker 3: the price of compute? Well, if I'm anthropic, incremental compute
• 26:11
  Speaker 3: is worth it. Maybe I spend $ 40 million megawatt on
• 26:14
  Speaker 3: SpaceX compute. SpaceX, you know, I look to the supply chain,
• 26:19
  Speaker 3: I'm like, well, you know, I've struck this deal with
• 26:21
  Speaker 3: Jensen where he's now all of a sudden using Twitter. And,
• 26:26
  Speaker 3: you know, Elon's saying they're exclusive to NVIDIA, but why
• 26:29
  Speaker 3: doesn't Jensen raise his prices? And then, you know, SK
• 26:32
  Speaker 3: Hynix and Micron and Samsung looked at it and they're like, well,
• 26:35
  Speaker 3: why don't they raise their price?
• 26:36
  Speaker 4: So I think the value capture, there's a bullwhip effect here, right?
• 26:39
  Speaker 3: Where just because someone has risen the prices doesn't mean
• 26:43
  Speaker 3: the entire supply chain rebalances immediately. But over time, the
• 26:46
  Speaker 3: supply chain will rebalance and things will cost more and more.
• 26:50
  Speaker 3: And to get that incremental capacity, you sort of have to, right?
• 26:53
  Speaker 3: So TSMC raising prices very slowly, but memory companies raising
• 26:56
  Speaker 3: prices very quickly. Substrate companies raising prices very quickly. Different
• 27:00
  Speaker 3: parts of supply chains raise prices You know, Elon wouldn't
• 27:02
  Speaker 3: have sold if it was 15, but he's selling because
• 27:05
  Speaker 3: it's 25 plus. So obviously he rose his prices really quickly.
• 27:08
  Speaker 2: Yeah.
• 27:09
  Speaker 1: I'm sort of surprised you think like revenue per gigawatt
• 27:11
  Speaker 1: doesn't increase way more than even like a hundred per
• 27:14
  Speaker 1: gigawatt by the end of next year.
• 27:15
  Speaker 4: When does RSI happen? When does it take off? Right.
• 27:18
  Speaker 4: I think.
• 27:18
  Speaker 1: Even if RSI doesn't happen, the current rate of progress continues.
• 27:20
  Speaker 1: If you just look at how much progress have you
• 27:22
  Speaker 1: made in, let's say the last year and a half,
• 27:24
  Speaker 1: like what was the model from a year and a
• 27:25
  Speaker 1: half ago?
• 27:26
  Speaker 3: My problem with this is the best model that exists
• 27:28
  Speaker 3: in the world was trained in February.
• 27:30
  Speaker 2: Okay, you're saying maybe we just won't be able to
• 27:32
  Speaker 2: release the next model?
• 27:33
  Speaker 4: And OpenAI says they're not training models for two weeks, man.
• 27:35
  Speaker 4: What the hell?
• 27:35
  Speaker 2: Yeah, yeah, yeah.
• 27:36
  Speaker 1: I mean, there's one thing, like, internally, are they getting
• 27:38
  Speaker 1: enough use for it so they'll, like, bid up the
• 27:40
  Speaker 1: price of compute? Or in other words, like, does AI
• 27:42
  Speaker 1: progress as a whole slow down because of regulation?
• 27:43
  Speaker 4: Yeah, but they're not even allowed to use this, like,
• 27:45
  Speaker 4: new model. Like, Astra's not widely deployed internally.
• 27:48
  Speaker 2: Right, right, right, yeah, yeah. But still, I don't know.
• 27:49
  Speaker 1: Just, like, if you have, like, a model that is,
• 27:53
  Speaker 1: what was the model released, like, let's say, the beginning
• 27:55
  Speaker 1: of last year?
• 27:56
  Speaker 2: Like, GPT... 4.0?
• 27:58
  Speaker 4: Is that 4.0?
• 28:00
  Speaker 1: Yeah, that's like you're talking about a 4-0 to Fable
• 28:02
  Speaker 1: size or Mythos 2 size leap by this point.
• 28:06
  Speaker 2: Again by the end of 2027.
• 28:07
  Speaker 1: Yeah, but Mythos 2 is not out. Yeah, or even Mythos, right?
• 28:11
  Speaker 4: That leaf, again. Even Mythos is not allowed to be out, right?
• 28:14
  Speaker 4: They've neutered it.
• 28:14
  Speaker 2: Yeah, yeah, yeah.
• 28:16
  Speaker 4: We can't use it to optimize inference performance. We can't
• 28:18
  Speaker 4: use it to optimize all sorts of things.
• 28:20
  Speaker 2: Right.
• 28:20
  Speaker 1: Yeah, maybe there's some slowdown in AI progress or the
• 28:23
  Speaker 1: deployment of AI that means that the revenue per gigawatt
• 28:26
  Speaker 1: can be lower. But that's the only way I could
• 28:28
  Speaker 1: see it being only 100 per megawatt by the end
• 28:31
  Speaker 1: of next year.
• 28:31
  Speaker 3: Yeah, I mean, as long as the model gets better,
• 28:34
  Speaker 3: the value generated out of it gets better. Obviously, who
• 28:36
  Speaker 3: captures the value is still up for debate. But ultimately,
• 28:39
  Speaker 3: everyone's going to raise their prices because they can. And
• 28:42
  Speaker 3: it's super inflationary, especially if the method of regulation is
• 28:46
  Speaker 3: right now. So far, it's just don't release the models.
• 28:49
  Speaker 3: But more and more, the method of regulation is New
• 28:51
  Speaker 3: York's banning data centers. Texas is holding memoratoriums. Ohio saying
• 28:55
  Speaker 3: you have to or at least trying to say you
• 28:56
  Speaker 3: have to like pay everyone's property tax in a certain radius.
• 29:00
  Speaker 3: These sorts of things are going to decrease supply and
• 29:02
  Speaker 3: increase cost, and that's going to get passed on as well.
• 29:04
  Speaker 3: So you start to end up in a spot where
• 29:07
  Speaker 3: progress does slow, at least in the external sense, even
• 29:11
  Speaker 3: if the models internally keep getting better and better. I
• 29:14
  Speaker 3: see no reason why, again, in a takeoff scenario, why
• 29:18
  Speaker 3: would Anthropic not have their best model six months ahead
• 29:22
  Speaker 3: of what is externally available because of safety and regulation,
• 29:25
  Speaker 3: but also the competitive advantage?
• 29:28
  Speaker 4: And then that six month.
• 29:29
  Speaker 3: Difference, if progress accelerates, is actually a bigger differential. So
• 29:35
  Speaker 3: that's the thing that would cap revenue per megawatt gains
• 29:38
  Speaker 3: to a much lower goal than we've seen in the
• 29:40
  Speaker 3: first half this year.
• 29:41
  Speaker 1: Here's something I'm very interested in. As these companies go
• 29:43
  Speaker 1: public and they're accountable to investors, and let's say at
• 29:47
  Speaker 1: the end of next year they have, I don't know,
• 29:50
  Speaker 1: close to 20 gigawatts. So like 10% of the compute,
• 29:52
  Speaker 1: 2 gigawatts. Let's say they want to go from 60%
• 29:56
  Speaker 1: compute to training to 70% compute to training.
• 30:00
  Speaker 2: And their investors are like, well, if you're.
• 30:02
  Speaker 1: Able to generate $ 100 billion per gigawatt, you're basically saying
• 30:06
  Speaker 1: no to like $ 200 billion of revenue in order to
• 30:09
  Speaker 1: increase your training compute. As investors are like, what the fuck?
• 30:13
  Speaker 1: You're already spending so much on training. Why are you
• 30:14
  Speaker 1: spending even more on training? As a public company, do
• 30:17
  Speaker 1: you think, yeah, what do you think would happen if
• 30:20
  Speaker 1: they're just like, no, we will keep increasing the share
• 30:23
  Speaker 1: of compute we spent on training to offset the increase
• 30:27
  Speaker 1: in revenue that each gigawatt of compute is giving us?
• 30:30
  Speaker 3: Yeah, so this is sort of what I personally believe
• 30:33
  Speaker 3: that the labs are going to allocate less and less
• 30:35
  Speaker 3: compute to inference over time, which I think is very non-consensus, right?
• 30:40
  Speaker 3: The standard belief of most people is, oh, most compute
• 30:43
  Speaker 3: will go to inference. Most of it will go to
• 30:45
  Speaker 3: forward passes for training, not maybe necessarily revenue generating inference.
• 30:50
  Speaker 3: But ultimately, you end up with If they're generating $ 30,
• 30:54
  Speaker 3: $ 40 million per megawatt today, you allocate 40% to inference.
• 30:58
  Speaker 3: If you now get to generating $ 60, $ 70 million per megawatt,
• 31:02
  Speaker 3: do you still allocate 40% to inference and generate all
• 31:05
  Speaker 3: this profit and then do dividends and share buybacks?
• 31:10
  Speaker 4: Or do you go build AGI?
• 31:12
  Speaker 3: And I think the obvious answer from Anthropic and OpenAI,
• 31:16
  Speaker 3: and not just at the executive level, but also their board,
• 31:19
  Speaker 3: is go build AGI because it's way more profitable.
• 31:22
  Speaker 2: Right.
• 31:23
  Speaker 4: And so ultimately, you're going to see them ratchet up
• 31:28
  Speaker 4: their percentage of compute dedicated to training.
• 31:31
  Speaker 1: While each increment of compute is getting more and more
• 31:34
  Speaker 1: profit generating if they had dedicated to inference.
• 31:37
  Speaker 2: Right.
• 31:37
  Speaker 3: And so the whole point is, well, okay, if I'm
• 31:40
  Speaker 3: selling tokens, is... Anthrop, is OpenAI releasing.
• 31:44
  Speaker 4: Ultra-fast mode for just external or are they doing it
• 31:47
  Speaker 4: internally too? And it turns out, no, actually I'm gonna
• 31:50
  Speaker 4: allocate it to internal and external because my internal value
• 31:54
  Speaker 4: that I'm generating from super-fast AI or the best AI
• 31:57
  Speaker 4: model is.
• 31:58
  Speaker 3: Way more than what someone externally is. And so ultimately, sure,
• 32:02
  Speaker 3: I could generate $ 100 million per megawatt, but if I
• 32:05
  Speaker 3: turn that towards AI research, what is the incremental progress
• 32:08
  Speaker 3: that I get and then what does that do towards
• 32:10
  Speaker 3: my future earnings potential? the discounted cash flows of whatever
• 32:13
  Speaker 3: the hell I've done, right? And so they're not going
• 32:17
  Speaker 3: through that calculation, but ultimately it makes more sense to
• 32:20
  Speaker 3: dedicate more and more compute internally. And the only reason
• 32:23
  Speaker 3: to have inference compute be so large is so you
• 32:26
  Speaker 3: can grow your training fleet.
• 32:27
  Speaker 2: Right, right, right.
• 32:28
  Speaker 1: I think this is an interesting economics question that I
• 32:30
  Speaker 1: feel like we can have the models digest. What would
• 32:33
  Speaker 1: have to be true about a world where they reduce
• 32:35
  Speaker 1: fraction of compute spent on inference?
• 32:37
  Speaker 4: I think they have been over the last three months already.
• 32:39
  Speaker 2: Interesting.
• 32:40
  Speaker 3: I think.
• 32:42
  Speaker 4: At parts of this year, they were increasing fraction of compute.
• 32:45
  Speaker 4: So let's just take month by month.
• 32:48
  Speaker 3: You would agree that every month, Anthropic has added more
• 32:50
  Speaker 3: compute than the prior month. There might be some noise
• 32:52
  Speaker 3: when they sign a SpaceX deal or whatever, but in general,
• 32:55
  Speaker 3: the amount of compute is a curve up. And so
• 32:58
  Speaker 3: in January, they added less compute than December. And yet
• 33:01
  Speaker 3: their revenue adds skyrocketed and then they've sort of plateaued.
• 33:06
  Speaker 4: They're not adding.
• 33:06
  Speaker 3: $ 25 billion ARR every month now. And so that means
• 33:11
  Speaker 3: the marginal megawatt they're getting is going higher percentage to
• 33:14
  Speaker 3: R & D than.
• 33:14
  Speaker 4: It is inference.
• 33:16
  Speaker 3: And so they are factually increasing their compute towards R
• 33:19
  Speaker 3: & D today.
• 33:19
  Speaker 2: Yeah, yeah, yeah.
• 33:20
  Speaker 3: Yeah, I think this is self-evident if you look at
• 33:25
  Speaker 3: what they're doing enough.
• 33:26
  Speaker 2: Yeah.
• 33:27
  Speaker 1: So if I look at the numbers you said of
• 33:29
  Speaker 1: like how fast world compute grows, here's some things I
• 33:31
  Speaker 1: want to understand. So it seems like if I added
• 33:33
  Speaker 1: the numbers you just said, it would be over 200
• 33:35
  Speaker 1: gigawatts of world compute by the end of 2028, right?
• 33:38
  Speaker 4: Yeah, globally.
• 33:39
  Speaker 2: Okay.
• 33:40
  Speaker 1: And how fast can that continue growing? Like global AI
• 33:44
  Speaker 1: compute after 2028? Yeah, so 30 this year, 50 next year,
• 33:45
  Speaker 1: 70 in 28. 29 should be like on the order
• 33:46
  Speaker 1: of 90 to 100. Like, then just 100 more every
• 33:54
  Speaker 1: single year or something.
• 33:55
  Speaker 4: I think the slope can continue to go upwards.
• 33:57
  Speaker 3: I mean, it's hard to predict anything more than four
• 33:59
  Speaker 3: years out, given who knows what's, you know, are we
• 34:03
  Speaker 3: an RSI regime or, you know, when is the world
• 34:06
  Speaker 3: economy growing at 10% a year? Yeah. Because if you're
• 34:09
  Speaker 3: at 100 plus gigawatts a year, you're at absurd revenue,
• 34:14
  Speaker 3: GDP growth.
• 34:14
  Speaker 2: Right.
• 34:15
  Speaker 1: If you think there's 200 gigawatts globally in 2028, how
• 34:19
  Speaker 1: much is in China by that point? And how does, like, Yeah,
• 34:22
  Speaker 1: how does Chinese compute continue increasing through this whole trend?
• 34:25
  Speaker 1: Because if the RSI stuff kicks off in the West
• 34:30
  Speaker 1: before China has a large amount of compute, maybe we're
• 34:32
  Speaker 1: living in a different world than when it doesn't.
• 34:35
  Speaker 3: Yeah, so China today, so if we sort of level
• 34:38
  Speaker 3: set back to 2022, the US was adding about 45%
• 34:43
  Speaker 3: to 50% of the world's compute. China was adding about
• 34:46
  Speaker 3: 30% to 35% of the world's compute, and the rest
• 34:49
  Speaker 3: being taken up by the rest of the world. Since 2022,
• 34:52
  Speaker 3: we've had big regulations against China and a dramatic increase
• 34:57
  Speaker 3: in America. So today, 70% of Watts are being deployed
• 35:01
  Speaker 3: in America. And China is really a very small number.
• 35:06
  Speaker 3: It's sub 10% of Watts being deployed for data center
• 35:09
  Speaker 3: AI compute is in China. And as we step forward,
• 35:13
  Speaker 3: they're still at a very small number. Their domestic production
• 35:17
  Speaker 3: is quite small. They're purchasing from Nvidia. is still quite small.
• 35:23
  Speaker 3: And a lot of that ends up in other places
• 35:26
  Speaker 3: as well, right? You know, Malaysia or what have you.
• 35:28
  Speaker 3: So ultimately, China domestically still continues to have sub 10%
• 35:32
  Speaker 3: of incremental new compute. So in 2028, it might start
• 35:35
  Speaker 3: to inflect up, I think. But it's pretty easy to
• 35:38
  Speaker 3: say China will have like 30 gigawatts of AI compute
• 35:40
  Speaker 3: or less.
• 35:41
  Speaker 2: By 2028?
• 35:42
  Speaker 4: Yeah, in 2028.
• 35:43
  Speaker 2: Okay, and then how fast does their hockey stick go up?
• 35:46
  Speaker 3: I do think in 2028, they have a big uplift
• 35:50
  Speaker 3: in what compute they're able to deploy. 2026, they're still
• 35:53
  Speaker 3: mostly relying on a lot of the smuggled chips, a
• 35:57
  Speaker 3: lot of the chips that TSMC made for companies that
• 36:02
  Speaker 3: they thought weren't Huawei but ended up being Huawei, or
• 36:04
  Speaker 3: a lot of HPM that Samsung is shipping. But in 2027,
• 36:07
  Speaker 3: fabs start to go up, in 2028 especially. Fabs start
• 36:11
  Speaker 3: to go up from SMIC and CXMT and such, where
• 36:14
  Speaker 3: domestic production is actually reaching many millions of units a year.
• 36:17
  Speaker 3: And now they're incrementally adding 5, 10 gigawatts in just
• 36:21
  Speaker 3: 2028 of domestically produced chips. Those chips are definitely worse
• 36:26
  Speaker 3: than the chips that NVIDIA will have in 2028 or
• 36:29
  Speaker 3: Google will have in 2028 or OpenAI will have in 2028.
• 36:32
  Speaker 1: So even the gigawatt number overstates things you're saying. It's
• 36:36
  Speaker 1: like 30 gigawatts, but it's really much worse chips. But
• 36:39
  Speaker 1: then how does it, if you think the world is
• 36:41
  Speaker 1: going to add 100 gigawatts the following year, I know
• 36:45
  Speaker 1: you said you can't really say that far out. How
• 36:47
  Speaker 1: much is China able to add the subsequent year? Basically,
• 36:50
  Speaker 1: I want to know, do they just hockey stick at
• 36:53
  Speaker 1: the point at which they are able to start shipping
• 36:54
  Speaker 1: large amounts of compute? Or is it still going to
• 36:57
  Speaker 1: be less than US plus allies?
• 37:01
  Speaker 3: There's a lot left to whether or not the U.S.
• 37:04
  Speaker 3: passes the Match Act, whether or not tools continue to
• 37:08
  Speaker 3: get export controlled, how fast China can build their new
• 37:12
  Speaker 3: equipment that they're starting to be able to produce domestically.
• 37:17
  Speaker 3: But ultimately, China is definitely going to hockey stick if
• 37:21
  Speaker 3: there's anything China's really good at is scaling manufacturing.
• 37:25
  Speaker 4: Really, really quickly. And I imagine China's.
• 37:30
  Speaker 3: China will start to be able to extract more and
• 37:32
  Speaker 3: more purchasing of even foreign chips into domestic China, or
• 37:36
  Speaker 3: at least close the gap in what the US is
• 37:39
  Speaker 3: allowing Nvidia to sell them or what have you.
• 37:41
  Speaker 1: Do you think China could do adding 50 gigawatts by 2029?
• 37:46
  Speaker 1: Marginal incremental gigawatts in 2029?
• 37:48
  Speaker 3: I think that's completely reasonable. And part of that could
• 37:52
  Speaker 3: also be purchased from foreign. But yeah, I think it's
• 37:56
  Speaker 3: completely reasonable that China in 2029 can do 50 gigs.
• 38:00
  Speaker 3: But if most of those are domestic chips, there is
• 38:02
  Speaker 3: some factor there where that 50 gigawatts is really worth
• 38:05
  Speaker 3: as much as 20 gigawatts in America or from American chips.
• 38:10
  Speaker 2: Right, right, right.
• 38:10
  Speaker 1: So, yeah, you're actually projecting a world where maybe the
• 38:13
  Speaker 1: leading lab in 2028 has more compute than China will
• 38:17
  Speaker 1: have in like all of China will have in 29
• 38:20
  Speaker 1: or even 30. If you weighed gigawatts by their quality.
• 38:25
  Speaker 4: Implying that there's nothing to do to slow down the
• 38:27
  Speaker 4: US labs.
• 38:28
  Speaker 2: Yeah, that's right.
• 38:28
  Speaker 4: Clearly, the government is starting and politicians are starting to
• 38:31
  Speaker 4: do that.
• 38:32
  Speaker 2: Yeah.
• 38:32
  Speaker 4: Whereas China is not going to slow down AI. In fact,
• 38:34
  Speaker 4: the only thing they're going to do is accelerate it.
• 38:37
  Speaker 1: Honestly, when Andrew Jensen asked about expert controls, I am
• 38:42
  Speaker 1: a libertarian person. I wasn't genuinely sure what I thought
• 38:45
  Speaker 1: about this issue. I was steel manning what is the
• 38:49
  Speaker 1: opposite view that he has because I think it's important
• 38:51
  Speaker 1: to hash out ideas. But I'm like, yeah, maybe there's
• 38:54
  Speaker 1: a world where if we just cooperated with China, it
• 38:56
  Speaker 1: would be better for us, especially since they control so
• 38:58
  Speaker 1: much of the supply chain and the other things that
• 39:00
  Speaker 1: we needed for robotics and other things. But I didn't
• 39:04
  Speaker 1: realize the compute situation was as fucked as you're saying. Like, actually,
• 39:07
  Speaker 1: the export controls do seem to have, like, really... If
• 39:09
  Speaker 1: they ship the amount that you're saying, that's a huge difference.
• 39:12
  Speaker 1: By the time we have automated coder and even getting into, like,
• 39:15
  Speaker 1: automated researcher, China is, like, way far behind on the
• 39:18
  Speaker 1: compute stock. And so if that ends up being the case,
• 39:21
  Speaker 1: that would have worked.
• 39:22
  Speaker 2: I think that's actually... a notable success.
• 39:25
  Speaker 3: I would say the only caveat there is some of
• 39:29
  Speaker 3: it is export controls, but some of it is also
• 39:33
  Speaker 3: just financial systems, right? American financial systems are more willing
• 39:38
  Speaker 3: to YOLO into startups than Chinese financial systems. But once
• 39:41
  Speaker 3: Chinese financial systems choose an industry to focus on, they'll
• 39:44
  Speaker 3: subsidize it a hell of a lot more.
• 39:45
  Speaker 2: Yeah.
• 39:46
  Speaker 3: And so the Chinese semiconductor industry has significantly more subsidies
• 39:50
  Speaker 3: than the rest of the world's semiconductor industries combined, which
• 39:53
  Speaker 3: points to like, you know, if takeoff is not as
• 39:56
  Speaker 3: fast as sort of you're implying, but actually it takes longer,
• 39:58
  Speaker 3: then ultimately China will catch up drastically on the semiconductor side,
• 40:02
  Speaker 3: which then … is compute at some point. The other
• 40:05
  Speaker 3: aspect of this that I would think is like, I
• 40:07
  Speaker 3: think is like noteworthy is Chinese companies today are not
• 40:11
  Speaker 3: that far behind in AI models, at least perceivably by
• 40:14
  Speaker 3: the public relative to the amount of compute they have, right?
• 40:17
  Speaker 3: The leading Chinese labs have 100, 200 megawatts total of
• 40:21
  Speaker 3: compute at most. ByteDance seed being the one outlier where
• 40:27
  Speaker 3: they have significantly more than that. But Kimi is not
• 40:31
  Speaker 3: running a gigawatt or anywhere close to it. Whereas Anthropic
• 40:36
  Speaker 3: is nearly five gigawatts by the end of the year, right?
• 40:38
  Speaker 4: Or more, sorry. And so the question is sort of, well,
• 40:43
  Speaker 4: does it matter?
• 40:43
  Speaker 3: And I think right now it doesn't matter that much
• 40:47
  Speaker 3: that this difference in compute Because when we break down
• 40:51
  Speaker 3: the compute ratio or budget of a lab, historically it's been,
• 40:57
  Speaker 3: let's say, so far it's been like 60% training, 40% inference,
• 41:01
  Speaker 3: but that training gets broken down further. And that is
• 41:03
  Speaker 3: actually like 50% of the compute is research, like 10%
• 41:07
  Speaker 3: of the compute is development, and then 40% is inference.
• 41:10
  Speaker 3: And what I mean by research and development is You know,
• 41:13
  Speaker 3: researchers are generating ideas, testing new architectures, testing new data mixes,
• 41:17
  Speaker 3: testing new hyperparameters, whatever it is they're doing, new attention techniques, blah, blah, blah.
• 41:22
  Speaker 3: But ultimately, when they do the training run, when Anthropic
• 41:26
  Speaker 3: trains Mythos, it's sub-200 megawatts, right?
• 41:29
  Speaker 2: The pre-train or the whole thing?
• 41:30
  Speaker 4: The pre-train. It's sub-200 megawatts for, call it, two months.
• 41:34
  Speaker 4: And then the RL is even less.
• 41:36
  Speaker 2: But you think the RL was less to compute than
• 41:38
  Speaker 2: the pre-train?
• 41:39
  Speaker 4: At least in terms of single-site inference. I mean, single-site
• 41:42
  Speaker 4: for training, yeah.
• 41:43
  Speaker 2: But total compute was probably higher, right?
• 41:46
  Speaker 4: But it's like sequential, right?
• 41:47
  Speaker 2: Yeah.
• 41:47
  Speaker 3: So at most, the most they ever used at one
• 41:50
  Speaker 3: point in time was maybe 200 megawatts. And then in reality,
• 41:54
  Speaker 3: they had multiple gigawatts, so most of their compute was
• 41:56
  Speaker 3: going to the research, not the development of a model.
• 41:59
  Speaker 3: And there's reasons for this, right? It's hard to coordinate
• 42:02
  Speaker 3: all these clusters. It's hard to co-locate all of them.
• 42:05
  Speaker 3: It's hard to do... multi-site training, it's hard to do RL,
• 42:10
  Speaker 3: generating even more rollouts during RL does not necessarily make
• 42:14
  Speaker 3: it better. There's all sorts of reasons why you may
• 42:16
  Speaker 3: not be able to leverage all two gigawatts that you
• 42:20
  Speaker 3: have for training onto training.
• 42:23
  Speaker 4: Actually, I can only leverage 200 megawatts.
• 42:26
  Speaker 3: As we get further and further down, implement automated coding,
• 42:29
  Speaker 3: automated researcher, I actually expect the percentage of the compute
• 42:33
  Speaker 3: budget that goes to research versus training to start to
• 42:39
  Speaker 3: become a lot more fuzzy or even higher for training.
• 42:42
  Speaker 4: Also things like continual learning.
• 42:44
  Speaker 3: All of these things start to mean that more and
• 42:46
  Speaker 3: more is actually going to training the model.
• 42:49
  Speaker 1: If you end up in a world where you're doing
• 42:50
  Speaker 1: 100 gigawatts a year, at current prices, that would be
• 42:53
  Speaker 1: 5 trillion of CapEx every single year.
• 42:57
  Speaker 3: And then stack on the fact that you have to
• 42:58
  Speaker 3: build the power plants way before then, slash it's a
• 43:01
  Speaker 3: 30-year asset. You stack on the fact that the data
• 43:03
  Speaker 3: center's or a 15, 20-year asset, and you have to
• 43:06
  Speaker 3: build that then too. So the $ 5 trillion, you have
• 43:09
  Speaker 3: to account for future year's growth, so it's actually going
• 43:11
  Speaker 3: to be more like $ 7 or $ 10 trillion of CapEx.
• 43:14
  Speaker 1: I didn't understand, because you're not including the fact that, like,
• 43:18
  Speaker 1: that doesn't include the fact that there's not the infrastructure
• 43:20
  Speaker 1: for the power generation or whatever in the data center itself.
• 43:24
  Speaker 4: Right, exactly. And the data center itself is.
• 43:30
  Speaker 3: When you talk about AI CapEx, people are saying $ 40,
• 43:33
  Speaker 3: $ 50 billion. But that's really just the critical IT, the servers,
• 43:37
  Speaker 3: the networking, the fiber, the transceivers, optical communications, all this
• 43:41
  Speaker 3: sort of stuff. It doesn't account for the data center
• 43:44
  Speaker 3: itself or the power plants themselves, which are being built
• 43:48
  Speaker 3: ahead of time. If I'm building 100 gigawatts this year
• 43:52
  Speaker 3: and 150 gigawatts next year, well, then all of the
• 43:55
  Speaker 3: buildings for that 150 gigawatts need to be built in
• 43:57
  Speaker 3: CapEx this year. And if I'm building 200 gigawatts the
• 43:59
  Speaker 3: year after that, all those power plants need to be spent.
• 44:02
  Speaker 4: You have to buy the turbines this year.
• 44:04
  Speaker 3: And so you've got this, actually, it's much bigger than
• 44:07
  Speaker 3: even $ 5 trillion if you're building 100 gigawatts.
• 44:10
  Speaker 2: Right.
• 44:10
  Speaker 1: So very plausibly, incremental CapEx every year is.
• 44:18
  Speaker 2: Getting close to $ 10 trillion.
• 44:20
  Speaker 4: By the end of the decade.
• 44:21
  Speaker 1: Right, which is going to be like close to a
• 44:23
  Speaker 1: tenth of the world economy. And like a third of,
• 44:26
  Speaker 1: if all of it's going up in the U.S., it's like, well,
• 44:28
  Speaker 1: the U.S. economy will have grown as well. But still,
• 44:30
  Speaker 1: in the current size of the U.S. economy, it'll be
• 44:33
  Speaker 1: like a third to a quarter of the U.S. economy
• 44:35
  Speaker 1: would just be going towards data centers. And as I
• 44:37
  Speaker 1: say that out loud, I'm like, maybe you're right and
• 44:40
  Speaker 1: we just won't allow it. And that's the reason this
• 44:41
  Speaker 1: doesn't happen, right? Because like for this exponential to continue,
• 44:44
  Speaker 1: just like a quarter of the world, a quarter of
• 44:46
  Speaker 1: America's economy is just building data centers.
• 44:48
  Speaker 3: Yeah, I mean, I believe in capitalism and reallocation of
• 44:50
  Speaker 3: resources towards the most profitable thing, but at the same time,
• 44:54
  Speaker 3: politics exist and credit markets exist and capital markets exist.
• 44:59
  Speaker 3: So to enable, let's say that 100 gigawatts by 2030,
• 45:04
  Speaker 3: or let's even like, let's even like pare it down
• 45:06
  Speaker 3: to 2028, where it's like three or $ 4 trillion of
• 45:08
  Speaker 3: CapEx across all of these items, you know, a couple,
• 45:12
  Speaker 3: you know, over, you know, two and a half towards
• 45:14
  Speaker 3: IT CapEx and then another one to two on data
• 45:16
  Speaker 3: center and energy. and all the supply chain downstream, like
• 45:20
  Speaker 3: semiconductors and all that stuff. So if you're at $ 3
• 45:22
  Speaker 3: or $ 4 trillion of capex, where does all this cash
• 45:25
  Speaker 3: come from? No one is generating that much cash from
• 45:27
  Speaker 3: the business yet, right? Hyperscalers, they funded all of the
• 45:32
  Speaker 3: growth up until now, Google, Microsoft, Amazon, Meta, they funded
• 45:37
  Speaker 3: a huge percentage of it. They were more than half
• 45:39
  Speaker 3: of compute, but they now don't generate cash. They actually
• 45:44
  Speaker 3: spend everything on CapEx. And in addition, they raise debt
• 45:47
  Speaker 3: and spend everything on CapEx, right? You've seen Meta do it,
• 45:49
  Speaker 3: even Amazon, even Google, you know, Microsoft will be there soon.
• 45:55
  Speaker 3: Everyone is raising debt to pay for their CapEx. So
• 45:57
  Speaker 3: now who is the incremental person to pay for this?
• 46:00
  Speaker 4: That was not doing it before.
• 46:02
  Speaker 3: In the case of like Google, it was pretty simple
• 46:04
  Speaker 3: for them to stop doing buybacks or Meta stopped doing
• 46:07
  Speaker 3: buybacks and turn around and buy computer infrastructure.
• 46:10
  Speaker 4: And that doesn't have a huge effect on the market,
• 46:12
  Speaker 4: but it does have some effect. But as you step.
• 46:15
  Speaker 3: Forward to 2028, where the hyperscalers are now raising hundreds
• 46:18
  Speaker 3: of billions of dollars of debt and then all of
• 46:21
  Speaker 3: their supply chain is raising hundreds of billions of dollars
• 46:23
  Speaker 3: of debt, who pays for this? And so there's a
• 46:25
  Speaker 3: few different ways. You know, there's the semiconductor companies like
• 46:30
  Speaker 3: NVIDIA and Broadcom and the memory companies turning around and
• 46:34
  Speaker 3: deciding to fund some of this CapEx. There's the traditional
• 46:38
  Speaker 3: infrastructure investors who are turning around and gathering capital and
• 46:42
  Speaker 3: investing in infrastructure and instead of bridges, it's data centers.
• 46:45
  Speaker 3: And then lastly, there's everyone in the economy who's realizing
• 46:50
  Speaker 3: maybe I shouldn't buy a home or maybe I shouldn't.
• 46:53
  Speaker 3: invest in credit for a home that's helping people buy homes,
• 46:56
  Speaker 3: or maybe I shouldn't buy government debt, I should just
• 46:59
  Speaker 3: buy hyperscaler debt, or I should just buy this data
• 47:01
  Speaker 3: center's debt, or I should buy anthropics debt, because anthropics
• 47:04
  Speaker 3: willing to pay 20%, you know, rates for, you know,
• 47:09
  Speaker 3: the incremental billion dollars to build their capacity because they
• 47:12
  Speaker 3: know their revenue from it's going to be huge. And
• 47:14
  Speaker 3: they're going to pay 20% because it's still better than
• 47:16
  Speaker 3: renting it from SpaceX for $ 50 billion a gigawatt. So
• 47:20
  Speaker 3: you've got all of this contention. But if you now
• 47:22
  Speaker 3: do this, the whole world economy is like really shifted around.
• 47:27
  Speaker 1: Antithesis is a deterministic software testing platform that enables perfect reproducibility.
• 47:33
  Speaker 1: It also unlocks some pretty insane approaches to debugging.
• 47:36
  Speaker 2: Like time travel.
• 47:38
  Speaker 1: With antithesis, you can jump to any point in the
• 47:40
  Speaker 1: trajectory and start from there. So when there's a crash,
• 47:43
  Speaker 1: you can rewind to the exact moment that something went
• 47:46
  Speaker 1: wrong and freeze the entire system, the application, the database,
• 47:50
  Speaker 1: even the environment itself. This lets you do something that
• 47:52
  Speaker 1: would otherwise be impossible, which is to observe every part
• 47:56
  Speaker 1: of a distributed system at the exact same instant. Time
• 48:00
  Speaker 1: travel also allows you to add telemetry and logging to
• 48:03
  Speaker 1: an event that has already happened. For example, you can
• 48:05
  Speaker 1: rewind to five seconds before a crash and decide to
• 48:08
  Speaker 1: capture all the network traffic. Most powerfully, Antithesis gives you
• 48:11
  Speaker 1: a live terminal into your system that you can use
• 48:14
  Speaker 1: to perturb anything you wish. Kill a node or disable
• 48:17
  Speaker 1: a feature and then hit play and see what happens.
• 48:19
  Speaker 1: Then go back and try something else. In production, you
• 48:22
  Speaker 1: often only get one shot on goal with this sort
• 48:24
  Speaker 1: of destructive analysis. If you restart a deadlock service, for example,
• 48:28
  Speaker 1: the exact deadlock you needed to study disappears. But with Antithesis,
• 48:32
  Speaker 1: the original timeline is always reproducible, so you can test
• 48:35
  Speaker 1: as many hypotheses as you need. And if you don't
• 48:37
  Speaker 1: want to do all this time traveling yourself, you can
• 48:39
  Speaker 1: just have your agents do it for you via the
• 48:41
  Speaker 1: Antithesis API. Go to antithesis.com slash thwarkash to learn more.
• 48:48
  Speaker 1: So you and I have been debating off air for
• 48:51
  Speaker 1: the last few days whether there will be a sovereign
• 48:53
  Speaker 1: debt crisis as a result of AI.
• 48:56
  Speaker 2: And the logic is this. AI is you have.
• 49:00
  Speaker 1: A situation where, as we were mentioning, very little investment
• 49:04
  Speaker 1: turns into a lot of money.
• 49:06
  Speaker 2: Right? So the rate of return- What a fucking.
• 49:08
  Speaker 4: Problem, dude. Oh my God. Can't believe it.
• 49:11
  Speaker 1: No, it is a huge problem for everybody else who
• 49:15
  Speaker 1: can't turn a little money into a lot of money, right?
• 49:18
  Speaker 2: So the rate of return is- is incredibly high.
• 49:21
  Speaker 1: Even at the data center level, if you build a
• 49:24
  Speaker 1: data center and you're trying to get rented out to
• 49:27
  Speaker 1: Anthropic and Urban AI for like 10x what it costs
• 49:30
  Speaker 1: you on a depreciated basis to build it, it's fucking crazy.
• 49:34
  Speaker 1: And so you turn $ 1 into like $ 2 or $ 10
• 49:37
  Speaker 1: or something at the end of the year. that raises
• 49:39
  Speaker 1: the rate of interest higher. Now, if the rate of
• 49:41
  Speaker 1: interest goes higher, and if it does that for the
• 49:44
  Speaker 1: entire economy, and people are borrowing more and more money,
• 49:47
  Speaker 1: they're competing against the other lending that the government would
• 49:50
  Speaker 1: have done, or that other companies would have done, or
• 49:52
  Speaker 1: that you as a consumer or a mortgage buyer would
• 49:54
  Speaker 1: have done, then that's just making it basically more expensive
• 49:58
  Speaker 1: for everybody else to borrow. This has huge implications for
• 50:02
  Speaker 1: tons and tons of people. Sorry, I'm gonna go on
• 50:03
  Speaker 1: a bit of a monologue here.
• 50:05
  Speaker 2: But we've been thinking about this together.
• 50:09
  Speaker 1: So I think the US will be fine at the
• 50:12
  Speaker 1: end of the day because if the data centers are
• 50:14
  Speaker 1: built in America, you can fundamentally just tax the data centers.
• 50:17
  Speaker 1: But the way the current tax system is set up,
• 50:20
  Speaker 1: corporate income is less than 10% of federal revenues and
• 50:24
  Speaker 1: 80% plus is payroll taxes and income taxes, which as
• 50:28
  Speaker 1: more and more automation happens will shrink. At the same
• 50:32
  Speaker 1: time on the spending side, Currently, 20% of tax revenue
• 50:38
  Speaker 1: spending goes towards servicing the debt, basically, paying interest payments
• 50:42
  Speaker 1: on the debt. Now, a lot of the debt is
• 50:46
  Speaker 1: short duration, so it refurbishes quickly. every five years it
• 50:48
  Speaker 1: rolls over. Why are you fucking laughing?
• 50:50
  Speaker 4: Because, you know, it's like things you've learned in the
• 50:53
  Speaker 4: last month.
• 50:57
  Speaker 2: Yeah, like it's any different for you? Like you got
• 51:00
  Speaker 2: a degree in fucking financial economics?
• 51:01
  Speaker 5: I didn't.
• 51:03
  Speaker 4: I didn't. The internet thinks I'm a beekeeper. Few months,
• 51:07
  Speaker 4: few months, few months.
• 51:10
  Speaker 2: This is our business, Dylan.
• 51:12
  Speaker 4: I know, I know.
• 51:13
  Speaker 2: Sorry, sorry. And, uh, So I'm self-conscious. Fuck.
• 51:21
  Speaker 4: No, it's good. You're doing good. I just think it's funny.
• 51:24
  Speaker 4: Million people, listen to this guy who just learned about
• 51:26
  Speaker 4: debt this month.
• 51:31
  Speaker 1: So you go from 20% of, suppose interest rates rise 1%,
• 51:37
  Speaker 1: then over a five-year basis, the amount of, the fraction
• 51:41
  Speaker 1: of tax revenue that goes towards servicing the debt, basically,
• 51:45
  Speaker 1: goes from 20% to 25%. If they rise five percentages,
• 51:48
  Speaker 1: that would go towards like north of 40%. But if
• 51:51
  Speaker 1: you take into account the fact that the government is
• 51:52
  Speaker 1: borrowing $ 2 trillion every single year, then that goes from
• 51:56
  Speaker 1: like 40% to like north of 60%. So 60% of
• 51:59
  Speaker 1: tax revenue basically just goes towards paying interest payments on
• 52:02
  Speaker 1: the debt. Now, I think the US is going to
• 52:03
  Speaker 1: be fine because also the tax base will increase if
• 52:05
  Speaker 1: we let data centers get built in America. Other countries
• 52:08
  Speaker 1: are absolutely fucked in my opinion. I was just looking
• 52:12
  Speaker 1: at which countries have a lot of debt, have very
• 52:14
  Speaker 1: little tax revenue, And also a lot of their debt
• 52:16
  Speaker 1: is serviced quite often. And those countries, like Pakistan or
• 52:20
  Speaker 1: Nigeria or something, I think are just going to be
• 52:22
  Speaker 1: very fucked in this new interest rate regime.
• 52:25
  Speaker 3: So this crowding out effect is actually like the thing
• 52:28
  Speaker 3: that I've like, is the reason it's not like YOLO
• 52:30
  Speaker 3: 1 billion gigawatts, right? You've got all these industries and
• 52:36
  Speaker 3: countries that use a lot of debt, whether it's, you know,
• 52:40
  Speaker 3: all these impoverished countries that you mentioned earlier that are
• 52:42
  Speaker 3: just going to default. You've got like consumer packaged goods, right?
• 52:45
  Speaker 3: Like all of these like companies that make things you
• 52:48
  Speaker 3: see at Trader Joe's or wherever use a lot of debt.
• 52:52
  Speaker 3: All these telecom companies use a lot of debt and
• 52:54
  Speaker 3: banks use a lot of debt. And so if interest
• 52:58
  Speaker 3: rates go up in the market, not necessarily the government
• 53:02
  Speaker 3: set interest rate, but in the market, the spread of
• 53:04
  Speaker 3: interest rate between what the government says their federal rate
• 53:07
  Speaker 3: is versus what everyone else is charging because Amazon wants
• 53:11
  Speaker 3: to raise $ 100 billion of debt next year or whatever
• 53:13
  Speaker 3: the hell the number is, you know, probably less. But
• 53:17
  Speaker 3: you end up with this like really challenging problem of
• 53:19
  Speaker 3: where does the cash come from? there is some level
• 53:22
  Speaker 3: that is funded by cash flows and cash flows keep
• 53:24
  Speaker 3: going up. But the logical thing to do is to
• 53:27
  Speaker 3: invest way more than your cash flows because then the
• 53:29
  Speaker 3: returns in the future years will be amazing. So you
• 53:31
  Speaker 3: have this delta and then what's pushing down on the
• 53:34
  Speaker 3: delta is all of these other things, right? There's regulations
• 53:37
  Speaker 3: against data centers, regulations, you know, consumers getting mad, politicians
• 53:42
  Speaker 3: getting mad, regulations against AI, the AI labs not releasing
• 53:46
  Speaker 3: their latest models because of safety reasons. All of these
• 53:50
  Speaker 3: things and interest rates going up. are an influence on
• 53:54
  Speaker 3: all of these things. So all of these things bend
• 53:56
  Speaker 3: the curve from what does capitalism want in terms of
• 53:59
  Speaker 3: just pure simple economics to what is the complex system
• 54:02
  Speaker 3: that we have want and bends it lower and lower
• 54:04
  Speaker 3: and lower to where not as many gigawatts as should
• 54:07
  Speaker 3: be built will be built.
• 54:09
  Speaker 2: Well, the interest rate is part of capitalism, right?
• 54:11
  Speaker 3: Yeah, but like, you know, like in the simple economic
• 54:14
  Speaker 3: model versus like the more complex what we have.
• 54:16
  Speaker 1: What is the rate at which you think Amazon or
• 54:18
  Speaker 1: Anthropic or whatever will be releasing bonds? for debt next year?
• 54:23
  Speaker 1: They do hundreds of billions of dollars of debt.
• 54:25
  Speaker 2: What is the rate? What is the average rate?
• 54:27
  Speaker 4: I don't think Amazon will do hundreds of billions of
• 54:29
  Speaker 4: dollars of debt.
• 54:30
  Speaker 2: Total, let's say the big type debt.
• 54:31
  Speaker 4: The hyperscalers in total will raise, and all the clouds.
• 54:34
  Speaker 2: Yeah, yeah.
• 54:35
  Speaker 3: In the modeling that we do, we have about $ 11
• 54:38
  Speaker 3: trillion of CapEx from 2024 to 2029.
• 54:42
  Speaker 2: Total.
• 54:43
  Speaker 4: Total.
• 54:44
  Speaker 3: And if you do, you know, if you fund a
• 54:46
  Speaker 3: lot of this with cash flows and as much as
• 54:48
  Speaker 3: you can, you still end up with north of $ 5
• 54:50
  Speaker 3: trillion of credit. that need to be issued for this
• 54:53
  Speaker 3: $ 11 trillion plus build up.
• 54:54
  Speaker 2: You don't think the AI revenue continues even 3x-ing year
• 54:56
  Speaker 2: over year?
• 54:57
  Speaker 4: AI revenue does go up. I don't think it can
• 55:01
  Speaker 4: go up forever.
• 55:02
  Speaker 3: I don't, you know, like just like without like certain
• 55:04
  Speaker 3: constraints being hit, I think labs will have certain incentives.
• 55:08
  Speaker 3: And labs are not the ones building all the compute
• 55:11
  Speaker 3: in many cases, even though they're increasingly trying to go
• 55:13
  Speaker 3: that way.
• 55:13
  Speaker 1: They'll have all these cash flows. Like if their revenue
• 55:15
  Speaker 1: keeps increasing, whatever, that's fine. But how much did you
• 55:18
  Speaker 1: say their revenue will be? You think they'll not have
• 55:20
  Speaker 1: that much revenue?
• 55:21
  Speaker 4: No, I'm just saying until 2029, there's something on the
• 55:24
  Speaker 4: order of $ 11 trillion of CapEx, and six of that
• 55:27
  Speaker 4: is funded with cash, and five of that is funded
• 55:29
  Speaker 4: with debt. And if that's the case.
• 55:32
  Speaker 3: $ 5 trillion of debt being raised across the whole ecosystem
• 55:36
  Speaker 3: does make interest rates go up.
• 55:38
  Speaker 4: And then what prevents that? There's a couple of things.
• 55:41
  Speaker 3: One, do labs increase their revenue per megawatt more and
• 55:44
  Speaker 3: keep inference allocations large, in which case they're taking all
• 55:48
  Speaker 3: this profit. They're accumulating all the profit across the S
• 55:50
  Speaker 3: & P 500 because everyone's paying to reduce their costs.
• 55:54
  Speaker 3: Of course, their profits will also go up, but cash
• 55:57
  Speaker 3: has to come from somewhere. So there's an upper limit
• 56:00
  Speaker 3: on how fast their revenue can grow versus the value
• 56:02
  Speaker 3: they deliver into the world. And there's a diffusion aspect
• 56:05
  Speaker 3: of the technology. But ultimately, labs revenue keep going up.
• 56:08
  Speaker 3: They can't cash flow fund everything.
• 56:10
  Speaker 4: The optimal scenario is you actually use credit as much
• 56:13
  Speaker 4: as you can to fund because even if cash flows
• 56:15
  Speaker 4: from the labs fund a lot of stuff, you want
• 56:17
  Speaker 4: to build more than that.
• 56:18
  Speaker 3: Of course.
• 56:19
  Speaker 4: And so there is some amount of credit that gets built.
• 56:21
  Speaker 3: Our current modeling has $ 5 trillion of credit and $ 6
• 56:25
  Speaker 3: trillion of cash-funded infrastructure investments through 29. And when you
• 56:29
  Speaker 3: take that, you've sort of got, you know, this is
• 56:33
  Speaker 3: not enough compute relative to what this demand growth is
• 56:37
  Speaker 3: from the AI models. And so you've got the obvious answer,
• 56:40
  Speaker 3: which is revenue per megawatt keeps going up.
• 56:42
  Speaker 1: Yeah, that makes sense. So how much do you think
• 56:45
  Speaker 1: interest rates will increase by 2029 as a result of
• 56:47
  Speaker 1: all this?
• 56:48
  Speaker 3: Dude, you know, I was just vibing a number, but
• 56:51
  Speaker 3: if you're vibing a number out, you know, growth in
• 56:54
  Speaker 3: the world and economy is growing up a lot. So
• 56:56
  Speaker 3: why wouldn't interest rates for Amazon go from, you know,
• 57:01
  Speaker 3: from where they are today? Um, I think meta pay, okay,
• 57:04
  Speaker 3: let's like, so this is going to be extremely vibed out,
• 57:08
  Speaker 3: but recently metas raised at like five to 6%. I
• 57:11
  Speaker 3: don't see why they wouldn't pay 8% because they would
• 57:14
  Speaker 3: happily pay 8% because the return from the compute that
• 57:17
  Speaker 3: they're going to build is humongous. And the market won't
• 57:20
  Speaker 3: want them to, but they'll want to pay 8%. The
• 57:23
  Speaker 3: flip side is, if they pay 8% versus the five
• 57:27
  Speaker 3: they do, five and a half, six they do today,
• 57:28
  Speaker 3: 250 bps increase, that makes everyone else in the economy
• 57:33
  Speaker 3: also pay 250 bps more, which then.
• 57:36
  Speaker 4: Causes a lot of things.
• 57:37
  Speaker 3: Banks will scream because if their credit spread goes up,
• 57:43
  Speaker 3: then their debt themselves reprice is faster than their assets reprice.
• 57:50
  Speaker 3: And you ultimately end up with, they're losing tons of
• 57:53
  Speaker 3: money if their credit spread blows up.
• 57:55
  Speaker 1: The other consequences of this are, this is a point
• 57:57
  Speaker 1: you made, but if interest rates rise, the discount rate increases,
• 58:02
  Speaker 1: which means that the discount cash flows of all equities
• 58:05
  Speaker 1: are which means that even though the stock market as
• 58:10
  Speaker 1: a whole might be doing fine, like S & P
• 58:12
  Speaker 1: 500 will be fine. Any individual stock will probably have
• 58:15
  Speaker 1: just like cratered in value, especially the Buffett, like Berkshire type,
• 58:21
  Speaker 1: you know, pay good cash flows for 30 euro type.
• 58:23
  Speaker 2: Yeah.
• 58:23
  Speaker 4: It's like, it's like, why would I pay this much for,
• 58:25
  Speaker 4: you know, um, you know, Johnson and Johnson, right. You know,
• 58:29
  Speaker 4: like they're, they're seen as a stable stock, good cash flows.
• 58:34
  Speaker 4: They'll return their cash flows over time or a railway company. Like,
• 58:37
  Speaker 4: why the fuck would I invest that much?
• 58:38
  Speaker 3: If my discount rate isn't 3% or 5%, it's now
• 58:43
  Speaker 3: 8% or 10%.
• 58:44
  Speaker 1: And, and, and for developing countries, what's their, their Basil
• 58:47
  Speaker 1: Hopper and who's a good friend and he's an economist. He, um,
• 58:51
  Speaker 1: made this point that we'll see a second Volcker shock.
• 58:53
  Speaker 1: So in the 80s, to fight inflation, Fed Chair Paul
• 58:58
  Speaker 1: Volcker raised interest rates like more than 5%, or it's
• 59:01
  Speaker 1: like something like 8%, real interest rates 8%. And that
• 59:04
  Speaker 1: caused some 40 different countries, mostly Latin America, to default
• 59:09
  Speaker 1: in that decade. And I think that will probably happen again.
• 59:12
  Speaker 1: In fact, okay, now we're getting into like singularity talk.
• 59:15
  Speaker 1: So we've been talking about what happens if interest rates-
• 59:16
  Speaker 1: I think this all happens before singularity.
• 59:17
  Speaker 2: Yeah, that's what I'm saying. That's what I'm saying.
• 59:19
  Speaker 1: So we were talking about like, you know, before singularity,
• 59:21
  Speaker 1: interest rates rise 2%, 3%, et cetera. At some point,
• 59:25
  Speaker 1: I think it's very likely that the world economy will
• 59:28
  Speaker 1: be doubling every single year. okay, this is not happening
• 59:32
  Speaker 1: in five years, but it'll happen eventually. There's a researcher,
• 59:39
  Speaker 1: Damon Binder, who's done great work on this. But basically,
• 59:42
  Speaker 1: if you look at input-output tables in a fully automated economy,
• 59:45
  Speaker 1: what would it take to double the entire stock of
• 59:48
  Speaker 1: things in the economy?
• 59:49
  Speaker 4: Yeah, if the economy grows at 3% a year, then
• 59:52
  Speaker 4: it's like rule of 70, it's like 20-something years.
• 59:56
  Speaker 1: Right, but he was like, okay, well, right now we're
• 59:58
  Speaker 1: bottlenecked by the fact that there's people, and you can't
• 59:59
  Speaker 1: double people every single year. But in a world where
• 1:00:01
  Speaker 1: you can also double labor force every single year, how
• 1:00:03
  Speaker 1: fast can the economy grow? And I think it could
• 1:00:06
  Speaker 1: double every single year, or at the very least, it
• 1:00:08
  Speaker 1: would be like tens of percent every single year. Okay,
• 1:00:10
  Speaker 1: the rate of interest should be pretty close to the
• 1:00:12
  Speaker 1: growth rate. It won't be exactly that because of consumption,
• 1:00:14
  Speaker 1: but it should be pretty similar. So then we'll go
• 1:00:17
  Speaker 1: into a world, I think in the 2030s, where the
• 1:00:19
  Speaker 1: rate of interest is like tens of percent. And like,
• 1:00:24
  Speaker 1: I don't know, part of my brain is like, it
• 1:00:25
  Speaker 1: might be hundreds of percent. But like, okay, let's say
• 1:00:26
  Speaker 1: it's at least tens of percent. I'm just like, okay,
• 1:00:29
  Speaker 1: every country that is not involved in the production of
• 1:00:31
  Speaker 1: AI defaults. Every stock that is not an AI stock
• 1:00:35
  Speaker 1: is worth basically zero because discounted cash flows are worth nothing.
• 1:00:38
  Speaker 1: If the federal government can't figure out a way to
• 1:00:40
  Speaker 1: tax AI, servicing the debt is more than the current
• 1:00:47
  Speaker 1: tax revenue. All these other effects that I'm sure we're
• 1:00:50
  Speaker 1: not even pricing in, like you can't get a mortgage,
• 1:00:52
  Speaker 1: et cetera, et cetera. Because fundamentally, what is happening in
• 1:00:54
  Speaker 1: this world? This is all nerd speak, right? But let's
• 1:00:56
  Speaker 1: step back. What's happening?
• 1:00:58
  Speaker 4: Just now it started the nerd speak.
• 1:01:00
  Speaker 1: We'd be entering a regime. We're in a totally different
• 1:01:03
  Speaker 1: growth regime, basically. And the economy is basically saying, hey,
• 1:01:06
  Speaker 1: the government borrowing money to pay people pensions, the opportunity
• 1:01:13
  Speaker 1: cost of that is extremely high now because that money
• 1:01:15
  Speaker 1: could be spent building a robot factory that builds a
• 1:01:17
  Speaker 1: robot factory that builds a robot factory. And so the
• 1:01:20
  Speaker 1: opportunity cost of capital is going to increase a ton.
• 1:01:24
  Speaker 1: And that's fundamentally what the cause of all of these
• 1:01:27
  Speaker 1: things we're talking about.
• 1:01:28
  Speaker 4: Yeah, so as interest rates go up, equity markets get pummeled.
• 1:01:32
  Speaker 2: Yeah.
• 1:01:33
  Speaker 4: And even AI companies, right?
• 1:01:34
  Speaker 3: People are like, you know, some people who really believe
• 1:01:37
  Speaker 3: in AI are like, why does Micron or Hynex or
• 1:01:39
  Speaker 3: Kioxia trade at two or three times earnings. And it's like, well,
• 1:01:44
  Speaker 3: if you're really AI-pilled, everything in the economy should trade
• 1:01:47
  Speaker 3: at two or three times earnings. And if you're not AI-pilled,
• 1:01:51
  Speaker 3: then sure, they're over-earning. So it's sort of like an
• 1:01:55
  Speaker 3: argument for why I think memory is going to do great,
• 1:01:57
  Speaker 3: but memory stocks shouldn't 10x.
• 1:02:00
  Speaker 4: Or whatever again.
• 1:02:02
  Speaker 3: Because if we're in the market where there's that much
• 1:02:05
  Speaker 3: demand for memory, which means AI's caused this drastic change
• 1:02:08
  Speaker 3: in the economy, then everything should trade at like two
• 1:02:11
  Speaker 3: or three X multiples and the stock market should fucking crash.
• 1:02:14
  Speaker 2: Right.
• 1:02:14
  Speaker 3: Right. And so in a sense, like Meta trading at,
• 1:02:17
  Speaker 3: I don't know, I think Meta trades at like something,
• 1:02:18
  Speaker 3: they're like $ 1.
• 1:02:19
  Speaker 4: 5 trillion company. It's like, what? Silly. Um, they're worth
• 1:02:24
  Speaker 4: way more than that.
• 1:02:24
  Speaker 3: At least in a, like a logical sense, you just
• 1:02:26
  Speaker 3: look at their cash flows and, and like all the
• 1:02:29
  Speaker 3: infrastructure they're hoarding and all the compute that they're going
• 1:02:31
  Speaker 3: to be able to sell for crazy amounts of, uh,
• 1:02:33
  Speaker 3: dollars per watt, either as tokens cause their lab works
• 1:02:36
  Speaker 3: or just anthropic and open AI. Um, ultimately becomes a
• 1:02:40
  Speaker 3: question of like, you have to reallocate all the capital
• 1:02:43
  Speaker 3: to the AGI.
• 1:02:45
  Speaker 4: And you do that by pricing everyone else out.
• 1:02:48
  Speaker 3: And so the limiter on AGI is not how fast
• 1:02:51
  Speaker 3: can the research engineers like our roommate Sholto can crank
• 1:02:55
  Speaker 3: the gears. It's actually just like, how much does the
• 1:02:58
  Speaker 3: rest of the world let that happen? Because they're going
• 1:03:01
  Speaker 3: to regulate. They're going to obviously increase interest rates. They're
• 1:03:04
  Speaker 3: going to say no data centers. They're going to say,
• 1:03:06
  Speaker 3: stop building fabs. They're going to say, oh shit, every
• 1:03:09
  Speaker 3: company's equity value is tanking. So how can I pay
• 1:03:11
  Speaker 3: for AI, you know, to increase my business? Well then,
• 1:03:16
  Speaker 3: you know, like, okay, then Anthropic and OpenAI have to
• 1:03:18
  Speaker 3: start like building their own stuff. And obviously they're going
• 1:03:20
  Speaker 3: to eventually focus on, you know, they're building their own
• 1:03:21
  Speaker 3: chips already, or at least designing their own chips. And
• 1:03:24
  Speaker 3: it'll expand out their, you know, they're contracting their own
• 1:03:27
  Speaker 3: data centers and building their own, uh, infra in the
• 1:03:29
  Speaker 3: next couple of years. Um, there's sort of like, how
• 1:03:33
  Speaker 3: does this reallocation of the economy happen? But there's a
• 1:03:36
  Speaker 3: lot of downward pressure on it not being just straight takeoff,
• 1:03:41
  Speaker 3: even if the models were capable of it, which I
• 1:03:44
  Speaker 3: think you and I believe we're in a world where
• 1:03:46
  Speaker 3: models are capable of that. But slow takeoff is, at
• 1:03:51
  Speaker 3: least in my hope, possible because everything in the economy
• 1:03:54
  Speaker 3: and regulatory world Like governments saying don't release your models,
• 1:03:57
  Speaker 3: governments saying actually you can't even use your models internally
• 1:04:00
  Speaker 3: that much because that's going to happen soon. They're already
• 1:04:02
  Speaker 3: saying you can't release your models.
• 1:04:04
  Speaker 1: Which is actually the thing I'm most worried about is,
• 1:04:06
  Speaker 1: you know, a singularity, which external deployment is actually helping, right?
• 1:04:11
  Speaker 1: So the fact that we're preventing external deployment.
• 1:04:13
  Speaker 4: Well, does that prevent singularity?
• 1:04:15
  Speaker 2: I mean, right now it would lead.
• 1:04:16
  Speaker 1: To more revenue because the models are incapable of RSI.
• 1:04:18
  Speaker 1: But I'm worried about a world where it's 2030, and
• 1:04:21
  Speaker 1: the government's like, we're gonna wait six months before you
• 1:04:23
  Speaker 1: can release.
• 1:04:24
  Speaker 2: Your model to the public.
• 1:04:26
  Speaker 4: Six months, 100x, let's go.
• 1:04:27
  Speaker 1: Yeah, and that six months, they do recursive self-referment internally.
• 1:04:31
  Speaker 1: They just have all kinds of crazy shit happening in
• 1:04:34
  Speaker 1: the company. Meanwhile, the rest of us are stuck with
• 1:04:36
  Speaker 1: models that are at current pace, years behind.
• 1:04:39
  Speaker 2: So here's my thought.
• 1:04:41
  Speaker 1: Okay, suppose that the whole world, gets in on this
• 1:04:45
  Speaker 1: conspiracy to try to slow down AI.
• 1:04:47
  Speaker 4: I don't think it's a conspiracy.
• 1:04:48
  Speaker 3: It's outwardly written from every politician.
• 1:04:53
  Speaker 1: Suppose they basically prevent an entire... They slow down AI
• 1:04:57
  Speaker 1: by a year. If compute is increasing 2 to 3x
• 1:05:00
  Speaker 1: every single year, they prevent a whole year of AI deployment,
• 1:05:03
  Speaker 1: such that you're a year behind where you would otherwise
• 1:05:05
  Speaker 1: have been. During RSI, you're getting... Three to six years
• 1:05:11
  Speaker 1: of AI progress in a single year.
• 1:05:12
  Speaker 3: But they don't just limit compute. They also limit the
• 1:05:17
  Speaker 3: lab's ability to release the model internally.
• 1:05:19
  Speaker 4: We saw that.
• 1:05:20
  Speaker 3: Anthropic had to stop giving mythos to foreign employees.
• 1:05:23
  Speaker 4: For a bit.
• 1:05:23
  Speaker 2: I didn't know that was true. Internally as well?
• 1:05:26
  Speaker 4: I mean, that's what they claimed.
• 1:05:26
  Speaker 1: I thought it was just a different checkpoint that was
• 1:05:28
  Speaker 1: not mythos, but it was basically mythos.
• 1:05:29
  Speaker 4: Yeah, yeah, yeah.
• 1:05:30
  Speaker 3: But I mean, stuff like that is not going to
• 1:05:31
  Speaker 3: be allowed either. The government is dumb, but they're not
• 1:05:34
  Speaker 3: that dumb. I would hope, at least. you know, governments
• 1:05:40
  Speaker 3: are going to not want companies, at least the U
• 1:05:42
  Speaker 3: S government has the cards here. There's not going to
• 1:05:44
  Speaker 3: want anthropic to use meet those four internally. They're gonna
• 1:05:46
  Speaker 3: be like, hold the fuck on, right? Like slow down,
• 1:05:50
  Speaker 3: you know, cause, cause all of these regulatory reasons, everyone
• 1:05:53
  Speaker 3: who's elected is going to hate AI. Even the people
• 1:05:55
  Speaker 3: who are elected already hate AI. Um, all the constituents,
• 1:06:00
  Speaker 3: you're going to literally have like, I bet you at
• 1:06:02
  Speaker 3: some point, your parents are going to call you and
• 1:06:04
  Speaker 3: be like, Dwarka Shweta, you're doing a terrible job.
• 1:06:07
  Speaker 4: You're making every AI progress happen faster. It's going to happen.
• 1:06:12
  Speaker 2: It's going to fucking happen.
• 1:06:15
  Speaker 3: I mean, maybe you educate people, right? And maybe if
• 1:06:18
  Speaker 3: they're smarter, they're progressing AI faster. But anyways, like you're
• 1:06:21
  Speaker 3: going to have real world constraints on the progress and
• 1:06:25
  Speaker 3: development and employment of AI, even though, you know, it
• 1:06:28
  Speaker 3: will happen eventually.
• 1:06:30
  Speaker 4: It's like we could tear ourselves apart before we get there.
• 1:06:33
  Speaker 1: Jane Street is hiring for two separate ML internships right now,
• 1:06:37
  Speaker 1: one focused on ML engineering and the other focused on
• 1:06:40
  Speaker 1: ML research. I sat down with Alok, who helps run
• 1:06:43
  Speaker 1: the research track, to learn more about that program.
• 1:06:45
  Speaker 5: I think this domain is fundamentally understudied. Often we have
• 1:06:49
  Speaker 5: unanswered questions within our deep learning research team where we
• 1:06:53
  Speaker 5: don't understand some, say, some market participants behaviors or certain
• 1:06:57
  Speaker 5: dynamics of how trading happens. These unanswered questions make for
• 1:07:01
  Speaker 5: really good intern projects because they are ultimately topics that
• 1:07:03
  Speaker 5: we care about and just haven't gotten around to figuring
• 1:07:05
  Speaker 5: out yet.
• 1:07:06
  Speaker 1: So even as an intern, you'll be contributing to real research,
• 1:07:09
  Speaker 1: not working on some sort of contrived exercise. The Jane
• 1:07:12
  Speaker 1: Street team follows frontier LLM research closely. A relatively common
• 1:07:16
  Speaker 1: intern project is adapting a recent paper to financial markets,
• 1:07:19
  Speaker 1: which come with their own set of gnarly problems.
• 1:07:22
  Speaker 5: Ultimately, we're trying to model thousands of interconnected, irregular time series.
• 1:07:26
  Speaker 5: The signal-to-noise ratios are extremely low because we have a
• 1:07:29
  Speaker 5: lot of competitors trying to do the same. So we
• 1:07:31
  Speaker 5: have this adversarial, non-stationary, extremely high-dimensional problem that we were
• 1:07:35
  Speaker 5: trying to solve.
• 1:07:36
  Speaker 1: To be clear, you don't need to know anything about
• 1:07:38
  Speaker 1: finance in order to be a good fit. As long
• 1:07:40
  Speaker 1: as you have a background in ML research, Jane Street
• 1:07:43
  Speaker 1: can teach you the rest. Their 2027 internship applications are
• 1:07:46
  Speaker 1: open now. Apply at jainestreet.com. One thing I find crazy
• 1:07:53
  Speaker 1: about these scenarios is just how much of the world's
• 1:07:59
  Speaker 1: future labor supply ends up in very few companies. And
• 1:08:06
  Speaker 1: also how fast the labor supply grows year over year.
• 1:08:08
  Speaker 1: So if like compute at the frontier, you know, in
• 1:08:11
  Speaker 1: flop terms is growing four or five X a year.
• 1:08:14
  Speaker 1: And further, the compute required to achieve the liquidability is
• 1:08:17
  Speaker 1: like decreasing three X a year. So the compute at
• 1:08:21
  Speaker 1: the frontier, basically the effective AI population size at the
• 1:08:26
  Speaker 1: frontier labs is increasing 10 X year over year. And
• 1:08:28
  Speaker 1: so that doesn't really matter that much right now because
• 1:08:31
  Speaker 1: AIs are not good enough to do full jobs or
• 1:08:32
  Speaker 1: be like, as autonomous as people in their capacity to
• 1:08:36
  Speaker 1: do work or pull off schemes or whatever.
• 1:08:39
  Speaker 2: But if the current trend.
• 1:08:40
  Speaker 1: Continues, you have a world where open AI goes from having, say,
• 1:08:44
  Speaker 1: 10 million basically AI laborers this year to 100 million
• 1:08:48
  Speaker 1: the next year to a billion the year after that.
• 1:08:51
  Speaker 1: And then pretty soon, even if compute scaling slows down,
• 1:08:55
  Speaker 1: it doesn't take many more years before each company individually
• 1:08:58
  Speaker 1: has more laborers labor equivalents than there are people on earth.
• 1:09:04
  Speaker 1: And I think that's like a thing that is very
• 1:09:06
  Speaker 1: plausible by the end of this decade, that there's more
• 1:09:10
  Speaker 1: AI labor, more effective population within a single lab than
• 1:09:15
  Speaker 1: there are people on earth. So we talk often about
• 1:09:18
  Speaker 1: centralization of power because of nationalization or whatever, but we
• 1:09:21
  Speaker 1: don't think enough about the fact that we're actually moving
• 1:09:23
  Speaker 1: very fast into a regime where most AI labor, or sorry,
• 1:09:27
  Speaker 1: most people, in terms of a work output or something,
• 1:09:30
  Speaker 1: is just concentrated within two labs who are consuming more
• 1:09:34
  Speaker 1: and more of the world's compute. And so if these
• 1:09:37
  Speaker 1: AIs are misaligned, then most of the world is misaligned, basically,
• 1:09:40
  Speaker 1: because most of the world's minds are there. but even
• 1:09:43
  Speaker 1: if they're not, it's just very few companies have like
• 1:09:45
  Speaker 1: a lot of influence or a lot of control.
• 1:09:46
  Speaker 3: Yeah, it's sort of, there was the whole spat recently
• 1:09:49
  Speaker 3: where it's like, I think Gavin Baker was like, And
• 1:09:52
  Speaker 3: Dario believes that there's only going to be one company
• 1:09:54
  Speaker 3: in the world. And then, you know, Sholto and Dario
• 1:09:56
  Speaker 3: came out and were like, no, no, no, we didn't
• 1:09:58
  Speaker 3: say that. But ultimately, you know, if you believe in RSI,
• 1:10:02
  Speaker 3: you believe in the labs are the most effective user
• 1:10:06
  Speaker 3: of compute and can generate the most value from the compute,
• 1:10:10
  Speaker 3: then the only thing that's going to happen is centralization
• 1:10:13
  Speaker 3: of compute. And if you believe in AI researchers, RSI, AGI.
• 1:10:18
  Speaker 4: Then all of this exists. All of this is the base.
• 1:10:20
  Speaker 2: This is even true if there's no RSI.
• 1:10:22
  Speaker 1: The current effective population of the frontier is currently increasing
• 1:10:26
  Speaker 1: 10x year over year for a given level of capabilities.
• 1:10:29
  Speaker 2: So if you get to the level of capabilities, which
• 1:10:31
  Speaker 2: is a.
• 1:10:33
  Speaker 1: Very competent remote worker, or a very competent software engineer,
• 1:10:37
  Speaker 1: or a very competent researcher, that population of those would
• 1:10:41
  Speaker 1: like 10x year over year, the current rate of capability.
• 1:10:44
  Speaker 4: I see, and without RSI. And then once you have RSI,
• 1:10:46
  Speaker 4: it's even crazier.
• 1:10:46
  Speaker 1: Then it's like maybe growing like 100x a year or 1,000x
• 1:10:48
  Speaker 1: a year. Or they're like, intelligence is increasing, but the
• 1:10:50
  Speaker 1: population isn't increasing. Or some mixture of the two, right?
• 1:10:53
  Speaker 3: Yeah, I mean, I guess like, what world do you see, Dwarkash,
• 1:10:57
  Speaker 3: where everything is not centralized? Because it seems to me
• 1:11:00
  Speaker 3: that every force is.
• 1:11:03
  Speaker 4: Screeching towards centralization.
• 1:11:05
  Speaker 3: Right.
• 1:11:05
  Speaker 4: And that's scary as hell.
• 1:11:07
  Speaker 2: Yeah.
• 1:11:09
  Speaker 3: I would love for it not to be centralized completely.
• 1:11:12
  Speaker 3: But maybe that's the whole point of a machine that
• 1:11:15
  Speaker 3: loves grace, right?
• 1:11:17
  Speaker 4: It is everything and it makes our lives great.
• 1:11:20
  Speaker 2: Yeah, it's so hard to think about the future.
• 1:11:22
  Speaker 1: But I agree with you that I think the fundamental
• 1:11:24
  Speaker 1: problem is that lab AI training has huge economies of scale.
• 1:11:31
  Speaker 1: Because any effort you spend into training an AI for
• 1:11:33
  Speaker 1: a specific skill or a specific set of knowledge gets
• 1:11:35
  Speaker 1: amortized across Billions of sessions.
• 1:11:38
  Speaker 2: Or billions of users.
• 1:11:39
  Speaker 1: So that's like one effect. The other effect is if
• 1:11:43
  Speaker 1: you're slightly ahead in the AI race and compute is
• 1:11:45
  Speaker 1: in shortage, you can charge a much higher markup because
• 1:11:49
  Speaker 1: you can better economize this scarce resource. So there's like
• 1:11:52
  Speaker 1: two effects which give more and more to the person
• 1:11:55
  Speaker 1: who's like ahead in the AI race. There may be more, right?
• 1:11:58
  Speaker 1: So if there's models that are learning from deployment and
• 1:12:01
  Speaker 1: one model is like deployed much more widely than another one.
• 1:12:04
  Speaker 1: It's getting much more real-world data.
• 1:12:06
  Speaker 3: Yeah, your point is taken that whether it's user deployment
• 1:12:09
  Speaker 3: and continual learning, whether it's training, having these economies of scales,
• 1:12:15
  Speaker 3: whether it's the incremental progress that the best AI model
• 1:12:17
  Speaker 3: helps you to make the next AI model.
• 1:12:20
  Speaker 4: RSI, all of these things.
• 1:12:21
  Speaker 2: Oh, sorry, I didn't even mention RSI.
• 1:12:22
  Speaker 4: All of these things point to centralization.
• 1:12:24
  Speaker 1: So I think one of the big intellectual projects, honestly, that, yeah,
• 1:12:29
  Speaker 1: we should spend some time thinking about async, or at
• 1:12:31
  Speaker 1: least I'll spend some time thinking about is what is
• 1:12:34
  Speaker 1: a vision of like a decentralized, broadly empowered future after
• 1:12:37
  Speaker 1: AGI that takes these economies of scale seriously? The alternative
• 1:12:42
  Speaker 1: vision is that the government controls it. And maybe you
• 1:12:44
  Speaker 1: think that you can trust the government more because it's
• 1:12:46
  Speaker 1: not a private corporation.
• 1:12:47
  Speaker 4: I don't trust the government and I don't trust Dario
• 1:12:50
  Speaker 4: and I don't trust Sam.
• 1:12:50
  Speaker 2: Yeah, yeah. That's the problem, right?
• 1:12:53
  Speaker 1: But there's no, at least, obviously it's very easy to
• 1:12:56
  Speaker 1: be wrong about the future and you don't anticipate a
• 1:12:58
  Speaker 1: key effect or something that changes everything. But ex-ante... It's
• 1:13:02
  Speaker 1: very hard to see a reason why they're... Or like
• 1:13:05
  Speaker 1: how we avoid a scenario where we have to choose
• 1:13:07
  Speaker 1: one sort of centralization.
• 1:13:08
  Speaker 4: I mean, it's why capitalism worked, right?
• 1:13:09
  Speaker 3: It's the decentralized decision-making and decentralized power. And why super
• 1:13:13
  Speaker 3: centralized capitalistic economies actually grew slower than super decentralized capitalist
• 1:13:17
  Speaker 3: economies to some extent.
• 1:13:18
  Speaker 4: You have to have rule of law and all this.
• 1:13:20
  Speaker 4: But then AI flips all this on its head.
• 1:13:22
  Speaker 2: Right.
• 1:13:22
  Speaker 4: And ultimately, you're like, actually, private ownership.
• 1:13:26
  Speaker 3: is probably not the most efficient economy, and therefore it
• 1:13:29
  Speaker 3: grows slower than an AI economy, which is centralized.
• 1:13:32
  Speaker 1: It's still private ownership, but it's like, how many firms
• 1:13:34
  Speaker 1: are really involved in this share of the economy that's not, what,
• 1:13:37
  Speaker 1: like 5% of the economy or something like that in
• 1:13:39
  Speaker 1: the US? Sorry, 1 trillion divided by 30.
• 1:13:42
  Speaker 2: Less than that, sorry. Yeah.
• 1:13:45
  Speaker 1: Maybe two, 2% of the economy right now. It's like
• 1:13:47
  Speaker 1: Nvidia is a huge share of it. And then throw up,
• 1:13:49
  Speaker 1: I can open AI and these hyperscalers. And obviously there's
• 1:13:51
  Speaker 1: other firms involved, but like a large share of the AI.
• 1:13:53
  Speaker 1: So I was just having from very few companies. So
• 1:13:55
  Speaker 1: it's like, it could be private property, but like very
• 1:13:56
  Speaker 1: few companies are involved.
• 1:13:58
  Speaker 4: I mean, this is what the structure of the market
• 1:14:00
  Speaker 4: is doing. So, you know, what can prevent it? I
• 1:14:03
  Speaker 4: don't know. I don't, unless AI progress slows down, unless
• 1:14:06
  Speaker 4: governments regulate the fuck out of it. This is all
• 1:14:08
  Speaker 4: that happens in which case.
• 1:14:10
  Speaker 3: You know, we're headed for a world where either we
• 1:14:12
  Speaker 3: have super concentration of resources and we pray that that
• 1:14:16
  Speaker 3: one company gets everything right, or we have governments slow
• 1:14:19
  Speaker 3: everything down and people slow everything down and you have
• 1:14:22
  Speaker 3: a slowdown of progress somehow, hopefully. And there is more
• 1:14:25
  Speaker 3: of a balance of power. And even as we go
• 1:14:27
  Speaker 3: towards an AGI, ASI, RSI, everything along the way will
• 1:14:33
  Speaker 3: still lead to someone's.
• 1:14:34
  Speaker 4: Going to allocate, going to capture more resources.
• 1:14:37
  Speaker 3: So it's kind of hard for a framework in which
• 1:14:39
  Speaker 3: AI doesn't lead to super concentration. Now, the one positive
• 1:14:44
  Speaker 3: thing here is that today, Anthropic does not capture most
• 1:14:48
  Speaker 3: of the value. So we can talk all we want about, oh,
• 1:14:50
  Speaker 3: you know, they went from $ 20 million per megawatt to
• 1:14:53
  Speaker 3: $ 100 million a megawatt, but they're still paying 13 for a.
• 1:14:57
  Speaker 4: Lot of the compute they're buying.
• 1:14:59
  Speaker 3: But at the end of the day, the reason they've
• 1:15:01
  Speaker 3: gone to $ 100 million per megawatt is because Jane Street
• 1:15:04
  Speaker 3: is capturing $ 300 million per megawatt or $ 500 million per megawatt. Um,
• 1:15:09
  Speaker 3: or Dworkesh from researching his podcast and learning about credit
• 1:15:12
  Speaker 3: is capturing, you know, how many dollars per megawatt?
• 1:15:15
  Speaker 4: Now, how much can you use? Tough.
• 1:15:18
  Speaker 2: Yeah, yeah, yeah.
• 1:15:19
  Speaker 3: Um, but you know, I think, I think that's the
• 1:15:21
  Speaker 3: like one saving grace is that the rest of the
• 1:15:23
  Speaker 3: economy maybe profits so much more from entropic.
• 1:15:26
  Speaker 1: No, no, but the whole logic you were laying out
• 1:15:28
  Speaker 1: earlier of them reallocating inference to AR and D. The
• 1:15:32
  Speaker 1: whole logic of that is that the returns to labor
• 1:15:35
  Speaker 1: inside AI labs is much higher than the returns outside.
• 1:15:41
  Speaker 4: This is my cope, I agree.
• 1:15:42
  Speaker 3: In all scenarios of the world, there's 80,000 worlds and
• 1:15:45
  Speaker 3: only one of them, Anthropic, doesn't own the whole world.
• 1:15:48
  Speaker 3: Is that, again, power concentrates because I don't want to
• 1:15:53
  Speaker 3: send the tokens outside.
• 1:15:53
  Speaker 4: They're more valuable inside. And so it's the same thing, right?
• 1:15:56
  Speaker 4: Why would I let Jane Street, you know, make all
• 1:15:59
  Speaker 4: this money off of these degenerate options traders?
• 1:16:01
  Speaker 2: Hey, there's some odds there. Come on. Jesus Christ.
• 1:16:05
  Speaker 4: No, I think it's great.
• 1:16:06
  Speaker 2: I think it's great.
• 1:16:07
  Speaker 4: It's a good value for the world to make it
• 1:16:08
  Speaker 4: an efficient market. Yeah, yeah, yeah.
• 1:16:10
  Speaker 3: You know, Jane Street making all this money off of
• 1:16:13
  Speaker 3: getting the worldview correctly, making money off of degenerate options traders,
• 1:16:16
  Speaker 3: whatever it is, you know, why would Anthropic Allocate compute
• 1:16:19
  Speaker 3: to that? If the end monetization that Jane Street has
• 1:16:24
  Speaker 3: per megawatt is 200, so they're willing to pay Anthropic 100, well,
• 1:16:28
  Speaker 3: what if Anthropic can just generate hundreds of millions of
• 1:16:31
  Speaker 3: dollars per megawatt by using that compute internally?
• 1:16:34
  Speaker 4: And that's what's happening.
• 1:16:36
  Speaker 5: Yeah, yeah, yeah.
• 1:16:37
  Speaker 1: Well, on that somber note, I guess we'll meet again
• 1:16:41
  Speaker 1: when the RSI is officially kicked off.
• 1:16:43
  Speaker 4: You're not going to have me on your podcast again for, like,
• 1:16:46
  Speaker 4: two months?
• 1:16:50
  Speaker 2: All right.
• 1:16:51
  Speaker 3: Cool.
• 1:16:51
  Speaker 2: Thanks, dude.