Latest / In the Loop AI / Elon's Recipe for Winning the AI Race: Grok5 and Colossus

Transcript

Elon's Recipe for Winning the AI Race: Grok5 and Colossus · In the Loop AI

0:03
Josh: It's been a big couple of weeks for elon we had a few pretty hit
0:06
Josh: episodes last week talking about starlink talking about the ai5 chip
0:09
Josh: and this week it's just another big breakthrough ejaz this week we're coming
0:13
Josh: out with a lot of new grok and xai news which is pretty exciting i mean one
0:17
Josh: of the leading headlines he said i now think xai has a chance of reaching agi
0:21
Josh: with grok 5 never thought that before and now there's two things that kind of
0:25
Josh: spawn this one which we'll get into a little bit later, which is the Grok Fast model.
0:29
Josh: It is remarkable. It is a full order of magnitude better than anything else for its size.
0:33
Josh: And it is really, really impressive. But the thing we're going to start with,
0:36
Josh: Ejaz, is this chart that we're showing on screen right here,
0:39
Josh: which is the single thing that convinced Elon, wait a second,
0:41
Josh: maybe, maybe just maybe, Grok 5 could actually lead to AGI.
0:45
Josh: And it's because we're seeing this crazy anomaly on the chart where Grok 4 was
0:49
Josh: kind of ahead, but somehow without any new major release, Grok 4 is now way ahead.
0:54
Josh: So Ejaz, can you explain to us like what's going on in this chart?
0:58
Josh: How did they get so good so fast without a major new model release?
1:01
Josh: I mean, this didn't even come from XAI, did it?
1:03
Ejaaz: It's a good question. And no, it didn't come directly from XAI.
1:07
Ejaaz: It actually came from two random AI researchers, one called Jeremy Berman and
1:12
Ejaaz: the other one called Eric Pang, who tweaked Grok 4's model,
1:18
Ejaaz: also known as fine tuning, to basically make it a hell of a lot smarter.
1:22
Ejaaz: And so they put it to the ultimate test, Josh.
1:25
Ejaaz: It's this thing called the Arc AGI benchmark.
1:30
Ejaaz: And for those of you who have not been spending all your time researching benchmarks,
1:36
Ejaaz: the Arc AGI benchmark tests how good your AI model is at being successful.
1:41
Ejaaz: Intelligently human. What I mean by that is it presents the AI model with puzzles
1:47
Ejaaz: that it's never seen before, that it can't possibly have been trained to solve
1:52
Ejaaz: and sees how good it does. Now, Josh, let me ask you this question.
1:55
Ejaaz: Before Grok 4 itself was released, what do you think the highest score was on this benchmark?
2:02
Josh: Lower, but I'm not sure how much lower. I don't know this particular numbers,
2:05
Josh: but maybe I'll guess five to 10% lower than what the best is now,
2:10
Josh: kind of like an incremental improvement.
2:11
Ejaaz: Nope nope nope it was way way
2:14
Ejaaz: lower in fact it only scored between five to eight
2:17
Ejaaz: percent from the top models from open ai google and all those kinds of things
2:20
Ejaaz: and then grok 4 came yeah grok 4 came along and it broke that frontier and scored
2:27
Ejaaz: 22 percent guess how much these two models that these two random ai researchers um scored on wait
2:35
Josh: So you're I'm looking at the screen. I'm seeing 79.6%. Is that right?
2:39
Josh: Is this a 4X multiple on base Grokfor?
2:42
Ejaaz: 80%. And this had nothing to do with the XAI team at all. I want you to focus
2:48
Ejaaz: on this chart that I'm showing you right now.
2:50
Ejaaz: And look at my cursor circling around these two orange dots that are off into the distance.
2:56
Ejaaz: You see Grokfor thinking over here, which was basically the heaviest,
3:02
Ejaaz: most expensive model that Elon and the XAI team released.
3:05
Ejaaz: Um when they launched croc 4 and they were just completely beaten by these two
3:10
Ejaaz: models but i'm sure you're probably thinking josh how the hell did these two
3:15
Ejaaz: researchers do that and um you know why aren't they being hired by elon immediately they
3:20
Josh: Don't have the resources of a giant lab.
3:22
Ejaaz: Like they're
3:23
Josh: Competing against i mean if you remember these people are getting billion dollar
3:26
Josh: offers to come work for a single employer and there's a collection of these
3:29
Josh: so how is it that one individual it's being a collection of these people.
3:32
Ejaaz: So these two researchers introduced two novel
3:36
Ejaaz: ways of training their models one is called
3:39
Ejaaz: open source program synthesis and the
3:42
Ejaaz: other is called test time adaptations before
3:45
Ejaaz: i get into an explanation as to how these work i
3:49
Ejaaz: want to remind the audience that what really makes a model really intelligent
3:54
Ejaaz: um is largely part in due to the data that it's trained on people spend so much
4:01
Ejaaz: money i'm talking hundreds of millions to billions of to acquire the best data to train their models.
4:06
Ejaaz: And the reason why this is so important is the model, when it's trying to answer
4:10
Ejaaz: a question, draws on the data that it's been trained on, right? So it's hoping that
4:17
Ejaaz: it can look back on the data that it's been trained on and find the right answer
4:21
Ejaaz: somewhere in all of these tokens and characters, right, Josh?
4:25
Ejaaz: These researchers decided to flip that completely on its head.
4:29
Ejaaz: It's this thing called open source program synthesis where the model designs
4:33
Ejaaz: its own bespoke solutions in real time.
4:36
Ejaaz: So it doesn't even look at the data that it was trained on.
4:39
Ejaaz: It just looks at the puzzle that it's presented with and it tries to break it
4:43
Ejaaz: down into smaller components. So let's say the puzzle has 10 different steps
4:48
Ejaaz: to reach to the end goal, the correct answer.
4:50
Ejaaz: It'll break it down into 10 different little steps, whereas normally a model
4:55
Ejaaz: would just look at the complete set of 10 steps and think, hmm,
4:58
Ejaaz: how do I get from step one to step 10?
5:00
Ejaaz: It just solves each step one at a time.
5:03
Ejaaz: And that was like the massive breakthrough that they made.
5:06
Ejaaz: And if this sounds familiar, you're probably thinking of this technique known
5:11
Ejaaz: as reinforcement learning,
5:13
Ejaaz: which basically has like the model like repeatedly go at a problem over and
5:17
Ejaaz: over again this is pretty similar but it's the next step up in that field
5:22
Josh: Okay got it yeah this this news kind of really annoyed me because of how seemingly
5:26
Josh: simple it was i mean jeremy burman in the case of this i got some examples of
5:30
Josh: specifically how he did it and he he was originally writing in python code but
5:35
Josh: then he switched to just writing instructions in plain english and i think this
5:38
Josh: is such an important thing that a lot of people forget.
5:40
Josh: I mean, myself included, I'm speaking for myself here, that a lot of this really
5:43
Josh: challenging, difficult work with engaging with LLMs is really just done in plain English.
5:48
Josh: You're just writing sentences to a model in hopes that it produces a better
5:52
Josh: output for you. It's not this crazy complex code base, although that exists deep down.
5:56
Josh: But the way that they achieve this is actually just by writing plain English.
5:59
Josh: So I did a little bit of digging. I have a few notes on how it works.
6:02
Josh: And his system, it basically starts by having Grok 4. He chose Grok
6:05
Josh: for is this model of choice it produces 30 english descriptions
6:08
Josh: of rules to transform inputs into outputs so it
6:12
Josh: takes that and then it tests these descriptions on training examples by
6:15
Josh: pretending each is a test and scoring how well
6:18
Josh: they match the correct outputs and then the top five descriptions get
6:21
Josh: revised individually with feedback on mistakes like highlighting the
6:24
Josh: wrong cells and stuff like that and then it combines the elements into
6:26
Josh: the top one to create these pool descriptions so it kind of has this iterative loop
6:30
Josh: where it tests itself it creates more examples it
6:33
Josh: gets better data it confirms that it's the right output and that's
6:36
Josh: generally why you see the actual outputs of this model are
6:38
Josh: a little more expensive but the quality of it is amazing because it
6:41
Josh: just continues to do this like self-iterative loop on itself and
6:44
Josh: get better and better and better again all in plain english so if you are listening
6:48
Josh: to this podcast in english you are fully capable of doing this because you speak
6:53
Josh: the language and this isn't anything crazy it's just very refined um prompts
6:57
Josh: that you're feeding to a model that result in these unbelievable outputs that
7:00
Josh: are now best in the world.
7:02
Josh: That's the coolest part to me, EJS. I don't know about EJS.
7:05
Ejaaz: No, no, I agree. And it reminds me of Andrew Carpathy's hit tweet three months
7:11
Ejaaz: ago where he goes, the new number one programming language
7:16
Ejaaz: Turned out to be English. It's English. Right?
7:19
Ejaaz: And kind of like to emphasize, again, how important this is.
7:25
Ejaaz: This isn't just another frontier breakthrough of another benchmark.
7:29
Ejaaz: I'm talking about the hardest benchmark that has just been 3x'd by two random researchers.
7:37
Ejaaz: Right? This is, again, puzzles that are problem sets that an AI model has never
7:44
Ejaaz: seen before. Typically, when you put an AI model up against a benchmark,
7:48
Ejaaz: it has some kind of context.
7:49
Ejaaz: Kind of think of yourself taking an exam at school or at university.
7:53
Ejaaz: You can look at past papers. You can look at books. You kind of know what topics
7:57
Ejaaz: they're going to talk about.
7:58
Ejaaz: This is completely foreign to an AI model. And therefore, it is the hardest test.
8:02
Ejaaz: So to have something achieve this almost feels like, and Josh,
8:06
Ejaaz: I hate to say it, but I have to say it, like AGI.
8:10
Ejaaz: And I think the fact that none other than Elon himself was taken completely aback by this.
8:16
Ejaaz: I mean, again, to reiterate the tweet, I now think XAI has a chance of achieving
8:20
Ejaaz: AGI with Grok5, never thought that before.
8:23
Ejaaz: And the fact that he is now saying, hey, by the way, Grok5 starts to train in
8:29
Ejaaz: a few weeks and you know what?
8:31
Ejaaz: I think it's going to be out by the end of this year.
8:35
Ejaaz: I think just speaks to the importance of this development.
8:38
Josh: Yeah, I think one of the things that was really startling for me was the realization
8:44
Josh: of how little resources it takes to get so good. And then I was wondering, well, why?
8:50
Josh: Clearly, this isn't anything super novel, although they did do some unique training frameworks.
8:56
Josh: And I think the reason that I, the conclusion that I came to was just scale.
8:59
Josh: I mean, the cost per query, the cost per token of these new super high end models
9:04
Josh: that just came out is very high.
9:06
Josh: And you can't really scale that to a lot of people because the companies are
9:10
Josh: just resource constrained.
9:11
Josh: So it leads me to believe and leads me to think, well, what happens when a company
9:15
Josh: with a lot of resources dedicates all of their brainpower to this specific type
9:19
Josh: of reinforcement learning, like we're going to see with Grok5,
9:21
Josh: and they do so in a way that's compressed enough, that's efficient enough to
9:25
Josh: actually run it at scale on the servers without melting everything down without
9:29
Josh: charging $1,000 a month per membership.
9:31
Josh: And I think that's probably what we see with Grok5 is this new juiced up reinforcement
9:35
Josh: learning, but efficient and actually built for scale.
9:40
Josh: I mean, even if it just launches at the specs of these two individual researchers,
9:43
Josh: that's a huge win because that's an incredible model.
9:46
Ejaaz: Yeah. And it's open source. It's open source and available for everyone.
9:50
Josh: It's pretty remarkable. Yeah. So I think very interesting things coming.
9:53
Josh: If I was a betting man, I would be betting big on Grok5. I think they very much
9:57
Josh: see a solution that people really want.
9:59
Ejaaz: I was just thinking about why
10:01
Ejaaz: both of us are finding this development both amazing, but really annoying.
10:05
Ejaaz: And I think it's because to some degree
10:08
Ejaaz: we both believe that in order for AI models today
10:11
Ejaaz: to get to AGI we would need to completely re-architect
10:15
Ejaaz: how they're designed you know transformers was
10:18
Ejaaz: the big breakthrough that's why models that we know and use today
10:21
Ejaaz: are so smart but it's not as
10:24
Ejaaz: smart as we expected it and there was this kind of like lag of improvement and
10:27
Ejaaz: now we suddenly see a 3x improvement where this model is kind of breaking this
10:32
Ejaaz: leading benchmark and so now I I think I'm starting to believe that maybe if
10:38
Ejaaz: we invest hundreds of billions of dollars in the post-training part,
10:42
Ejaaz: where typically we've been investing in the pre-training, in the compute,
10:45
Ejaaz: but if we invest it in the post-training, we may clearly reach AGI before redesigning
10:51
Ejaaz: the entire thing up front.
10:53
Ejaaz: Does that resonate with you, Joshua? Or am I just, do I sound crazy?
10:56
Josh: No, it does. It does. It's funny because we frequently record the show and you
10:58
Josh: expect to be surprised and then something happens.
11:00
Josh: You're like, oh my, I wasn't expected to be surprised in that way. and this
11:04
Josh: is one of those things where i mean i wasn't expecting to see a new leader in
11:08
Josh: between major model releases from an independent researcher so the fact that
11:11
Josh: this is even possible really just blows the doors off of a lot of expectations
11:15
Josh: i had and this isn't even the only interesting news this week from the xai team
11:19
Josh: because they released new model alert grok for,
11:23
Josh: fast let me tell you each as when i saw how this model worked i was like whoa
11:29
Josh: this is um again blown away super.
11:32
Ejaaz: Impressed can you run us through some of the highlights please
11:34
Josh: This spec sheet yeah so first of all the leading headline
11:38
Josh: two million token context window is outrageous i think the current leader is
11:42
Josh: google with the gemini model they have 2.5 pro and flash both of them i believe
11:46
Josh: have a million tokens uh this is two million tokens of context for those that
11:51
Josh: aren't aware context is the basically active memory of a language model it's the more context you can,
11:56
Josh: collect, the more clarity it has into the actual data that it's talking about
12:00
Josh: and conversing with, you want that number to be bigger.
12:03
Josh: This is the biggest by far, by a doubling.
12:05
Josh: So that's a really important headliner. The second one, probably even more outrageous,
12:10
Josh: 47 times cheaper than ROC4.
12:12
Josh: Which is crazy because when you look at it on
12:15
Josh: the scale below if you can scroll down just a little
12:18
Josh: bit grok 4 is right in line with every other great
12:21
Josh: model it is a grok 4 fast is just beneath
12:24
Josh: o3 it's above deep seek it's above cloud 4 sonnet it's a above cloud 4 opus
12:30
Josh: it's just like this remarkable model that is better than a lot of the leading
12:34
Josh: models but 47 cheaper than the base model and i think that's going to be a pretty
12:38
Josh: interesting thing when we get into like scaling these models and using them for code.
12:42
Josh: EJ, as we talked last week about how good the Grok model is for coding because
12:46
Josh: it was so cheap and so effective.
12:48
Josh: This is another case of that. And the way they did that, I was so interested
12:51
Josh: in how they were able to come up with like the secret sauce to do it.
12:54
Josh: And basically what they did is they taught to model to spend its brainpower
12:57
Josh: only on tools when it helps.
13:00
Josh: So they use this like large scale reinforcement learning to train Grok for fast
13:03
Josh: to choose when to think in depth and when to answer questions quickly.
13:07
Josh: So what that resulted in was about 40% what we're seeing here on the screen.
13:11
Josh: 40% fewer thinking tokens on average than we've gotten from the previous model,
13:16
Josh: which is a significant difference. Oh, and by the way, it's number one on Elmarina.
13:19
Josh: So this was crazy. EJs, what were your reactions when you saw the team drop this?
13:23
Ejaaz: I already thought these tokens were cheap. I thought these models were cheap enough.
13:27
Ejaaz: Do you remember when OpenAir released GPT-5? They kept flexing GPT-5 mini saying,
13:33
Ejaaz: hey, you now have the power of our previous best model, but actually it's more intelligent.
13:37
Ejaaz: And it's like, I think it was something like five times cheaper.
13:40
Ejaaz: And I was like, holy shit, holy crap. I was like, that is like crazy magnitude.
13:46
Ejaaz: And now it's like, now we've got 47X cheaper than Grok 4.
13:50
Ejaaz: Grok 4, by the way, was already cheap compared to some of the Frontier bottles.
13:54
Ejaaz: So I don't know how far this can go, but kind of zooming out,
13:58
Ejaaz: I have never been more confident that,
14:01
Ejaaz: than now that cutting edge super intelligence will be available for anyone and everyone.
14:08
Ejaaz: This isn't going to be some kind of closeted technology where only the rich
14:13
Ejaaz: can buy devices and run it.
14:15
Ejaaz: I think anyone and everyone will have fair game access to this.
14:18
Ejaaz: And think about the dynamics that that changes up, Josh.
14:22
Ejaaz: Like you can have someone in the complete middle of nowhere with a cell phone
14:26
Ejaaz: attached to Elon Musk's new 5G Starlink satellite that's beaming down to him.
14:31
Ejaaz: And he could kind of produce something that the world ends up using because
14:35
Ejaaz: he has access to this cheap model that is actually super intelligent and can
14:39
Ejaaz: be used to create whatever crazy invention that he has or she has that dreams
14:43
Ejaaz: up. I just think this is insane.
14:46
Josh: Yeah, the efficiency improvements are the thing that's always most exciting
14:48
Josh: to me because, I mean, as we get more cheaper tokens and as the tokens become
14:52
Josh: more portable and lightweight, I mean, you could have the world of knowledge
14:55
Josh: on your phone even without necessarily an internet connection because these
14:59
Josh: models are getting so lightweight, so condensed.
15:00
Josh: Um so effective it's like it's really it's
15:03
Josh: unbelievably impressive and what i was really interested in
15:06
Josh: is comparing this to the other models because i know google was
15:09
Josh: kind of doing a similar thing they were leading along the frontier and oh yeah
15:13
Josh: here's this post from gavin baker that i love because it shows how google has
15:17
Josh: kind of dominated this thing called the pareto frontier and on the chart you
15:19
Josh: can very clearly see how there's this kind of arc that hugs the outer bounds
15:23
Josh: of all of the models and it shows that like gemini pro has been really good on a few
15:28
Josh: So I briefly want to just talk about the Pareto Frontier concept because it's
15:32
Josh: really interesting and it will explain to you exactly why Grok4Fast is way out there.
15:37
Josh: It totally shattered what it is. So, I mean, basically, it's funny.
15:40
Josh: I was doing a little bit of research on this and the Pareto Frontier is done
15:44
Josh: by an Italian economist named Velfredo Pareto.
15:47
Josh: So I just thought that was a fun fact because great name. Basically,
15:50
Josh: it comes from the economist and decision theory.
15:52
Josh: And it's a way to decide optimal trade-offs when you have multiple objectives
15:55
Josh: you're trying to achieve all at the same time. So imagine you're trying to optimize
15:58
Josh: two things that might conflict a little bit, like you want to make a product
16:01
Josh: as powerful as possible, but also inexpensive as possible, like these models.
16:05
Josh: So in this scenario, there's a set of best possible solutions where you can't
16:09
Josh: improve one aspect, like the power, without making the other aspect, like the cost worse.
16:13
Josh: And what we're seeing in this chart here is Google has made a series of those
16:17
Josh: decisions, those tradeoffs that
16:18
Josh: have led to the absolute Pareto optimal outcome along this outer band.
16:22
Josh: What grok has done is they actually made a new trade-off
16:25
Josh: that isn't necessarily a trade-off it's more of an innovation that allows
16:29
Josh: them to unlock this perceived frontier this limiting factor
16:32
Josh: that was on the outer band and just shatter it and create a new pareto
16:35
Josh: optimal trade-off using these best things and they did that by doing a lot of
16:39
Josh: magic but basically what they have now is they have a really smart model that
16:43
Josh: actually sits above gemini 2.5 flash and not too far below the pro model but
16:47
Josh: it is a order of magnitude cheaper and i think that's where that outlier that
16:52
Josh: cost effectiveness is really unbelievable when it comes to,
16:55
Josh: um distributing these tokens widely so now if you're writing code if you're
16:59
Josh: creating an application if you're just if you're paying for tokens this is very
17:02
Josh: clearly the model you want to use.
17:03
Ejaaz: What you just described is elon and
17:06
Ejaaz: xai literally charting a new path which
17:09
Ejaaz: is kind of like um very behavioral of elon in general um and another thing that
17:16
Ejaaz: i thought was really cool about this is the reinforcement learning infrastructure
17:19
Ejaaz: team was kind of key behind getting this model as fast and as cheap and as efficient
17:24
Ejaaz: as we're describing it, right, Josh?
17:27
Ejaaz: They used this kind of like agent framework, which was extremely compatible
17:31
Ejaaz: with the infrastructure that they used to train and iterate on this model in the first place.
17:36
Ejaaz: And what I wanted to point out here is there's a theme between the two topics
17:40
Ejaaz: that we've discussed so far on this episode, Josh.
17:42
Ejaaz: Number one, when we described the two models that the researchers created that
17:47
Ejaaz: broke the ArcGIS benchmark, they specifically used a technique which used reinforcement
17:54
Ejaaz: learning, a new reinforcement learning technique.
17:57
Ejaaz: And the reason, if you remember, why Jeremy Berman picked Grok4 specifically
18:01
Ejaaz: was he said it was the best reasoning model because in the way that had been
18:07
Ejaaz: trained via reinforcement learning.
18:08
Ejaaz: And now we're seeing yet again, this GrokFast model achieving what it can because
18:14
Ejaaz: of reinforcement learning.
18:15
Ejaaz: So I'm seeing a theme or noticing a theme here where XAI and Elon are basically
18:19
Ejaaz: the leaders in reinforcement learning,
18:22
Ejaaz: which i think is going to probably play in their favor maybe it's a hint that
18:26
Ejaaz: the models that are going to be closest to agi that are the quickest that are
18:29
Ejaaz: the cheapest are embedded in reinforcement learning techniques that are just
18:33
Ejaaz: completely breakthrough
18:34
Josh: Yeah it seems like they the team really reasons
18:37
Josh: i mean this is a core elon uh notion but
18:40
Josh: like they really do reason from first principles and what's important and what matters and
18:44
Josh: you're seeing that throughout the entire product as they advance and
18:46
Josh: i think what's really exciting what i'm most stoked about um
18:50
Josh: for this show in general is is to compare this
18:53
Josh: next round of models like will gemini 3
18:56
Josh: and grok 5 like how are they
18:58
Josh: going to compete with each other because those are both going to be remarkable
19:02
Josh: models and it seems to me like those are like those are currently the top dogs
19:06
Josh: i mean as far as gpt5 was kind of a little bit of a miss anthropic's been a
19:10
Josh: little bit quiet uh gemini and xai are on fire but this also there was there
19:16
Josh: was one last thing of news before we sign off today well i.
19:18
Ejaaz: Was I was gonna say, like I'm highlighting this sentence here for those who are just listening.
19:23
Ejaaz: And it says, you know, we built this reinforcement learning infrastructure team
19:28
Ejaaz: with a new agent framework to help train Grok4 fast.
19:31
Ejaaz: But specifically so that we can harness the power of Colossus 2.
19:35
Ejaaz: And if I remember correctly, Josh, there was some breaking news around Colossus
19:39
Ejaaz: 2. Elon was getting into some fights. Can you walk us through it?
19:43
Josh: Yeah, it's funny. There was this whole report from SemiAnalysis,
19:45
Josh: which does a really great job. I would highly recommend checking them out.
19:48
Josh: And they released this report on the XAI data center build out.
19:52
Josh: And it was so funny to see, because a lot of times you just see satellite pictures
19:55
Josh: or you read headlines and you're not really sure what's going on.
19:57
Josh: The sole purpose of SemiAnalysis is to actually have boots on the ground,
20:01
Josh: check the satellite images, and look at it with a scientific engineering point
20:05
Josh: of view where they actually understand what is going on.
20:07
Josh: And they shared their findings in one of these articles. And I found one of
20:10
Josh: these stories was so funny because it's such a testament to how the XAI team
20:13
Josh: works, where they were having problems with their energy generation in Memphis,
20:18
Josh: Tennessee, because people were complaining and they were having a tough time getting permits.
20:21
Josh: And the core crux of every large AI data center is energy.
20:26
Josh: So they were like, this is unacceptable. well, we need energy immediately.
20:29
Josh: So what do they do? Well, they jumped over the state lines.
20:32
Josh: They went over to Mississippi a couple of miles down the road and they built
20:37
Josh: these new generators right down the road across the state line.
20:40
Josh: They got the permits they needed.
20:42
Josh: They said, we don't, you don't want us, Tennessee. We'll just go right over to Memphis.
20:45
Josh: You could see here, they took the power lines, they ran them back into Tennessee
20:49
Josh: and now they're powering the data center.
20:51
Josh: And part of the article was, was this funny story, but part of the article also is,
20:55
Josh: is colossus 2 um being built
20:57
Josh: in the sheer scale that colossus 2 is going to be and it's going
21:00
Josh: to be over a gigawatt of energy um which
21:03
Josh: is i don't know how many hundreds of thousands of homes is
21:06
Josh: going to power but this is like a remarkable amount of power and a tremendous
21:10
Josh: amount of gpus and they're planning to make these all coherent and they're using
21:13
Josh: them exclusively i believe to train this new grok 5 model so as this new training
21:17
Josh: center comes online they will be using this new cutting-edge world's largest
21:21
Josh: supercomputer to train the world's perceivedly best model.
21:25
Josh: But I found this funny because the day that this article came out,
21:28
Josh: there was another post from another CEO of a very prominent company saying,
21:32
Josh: hey, wait a second, we have something a little bit bigger.
21:36
Josh: Than Colossus 1 currently. And that was from Microsoft's CEO, Satya Nadella.
21:42
Josh: And he had this post where he said they just added over two gigawatts of new energy capacity.
21:49
Josh: So Ejaz, this is just a really crazy brawl between these people who are building
21:54
Josh: larger and larger AI data centers.
21:55
Josh: And it eventually leads to the big news that just dropped a little earlier today.
21:58
Josh: But do you have any commentary before we get to the huge number?
22:01
Ejaaz: Yeah, there's actually one thing I wanted to point out, which is when Elon first
22:05
Ejaaz: announced that he was building out this Colossus 2 data center,
22:09
Ejaaz: it made headlines that it cost $20 billion.
22:13
Ejaaz: And everyone thought it was crazy. Everyone was yelling, this is an AI CapEx bubble.
22:18
Ejaaz: There is no products that prove that all this investment makes sense.
22:23
Ejaaz: And now you have Satya Nadella, CEO of Microsoft, announcing that he's probably
22:28
Ejaaz: going to be investing twice as much of that to build two gigawatts of new capacity.
22:34
Ejaaz: Again, validating that there is a need for energy and compute to train these new models.
22:41
Ejaaz: Don't forget that Microsoft last week acquired a random European data center for,
22:47
Ejaaz: I think it was about, what, $10 billion, which caused its stock price to 3x
22:51
Ejaaz: because it itself wasn't worth that much at the time of the reporting happening.
22:57
Ejaaz: And then it leads us to the even bigger announcement, which released this morning,
23:02
Ejaaz: which is NVIDIA will be investing not one,
23:06
Ejaaz: not 10, not 20, but $100 billion in OpenAI over the next couple of years.
23:14
Ejaaz: And you might be asking why?
23:16
Ejaaz: Well, it's because OpenAI is going to be investing in so many data centers that
23:22
Ejaaz: is going to produce so much power. I don't know how many gigawatts.
23:25
Ejaaz: I think it's actually 10 gigawatts which is 10x colossus 2 5x um uh fair water
23:32
Ejaaz: which is satya nadella's thing for all my mathematician mathematician fans out
23:37
Ejaaz: there it is just crazy josh are we in a bubble or is there a need for all of this
23:42
Josh: So here's the thing as i keep going back
23:45
Josh: and forth about the bubble conversation because a hundred billion
23:48
Josh: dollars is such an outrageous amount of money to spend on making
23:52
Josh: what is already a remarkable language model even more
23:55
Josh: remarkable um like the product is great and
23:58
Josh: at least me personally as a user of these
24:01
Josh: products i'm definitely getting closer to a wall of things that i use them for
24:05
Josh: where if a model is marginally smarter my experience doesn't get that much better
24:10
Josh: um but i was so that's like one school of thought and then the other is thinking
24:15
Josh: well this is probably the only thing we'll ever need to spend money on going forward ever.
24:19
Josh: So it makes sense to throw all of it at it now.
24:22
Josh: Because in the case that you do solve AGI, you get hyperintelligence,
24:26
Josh: it solves all of your problems. And it gives you the better questions to ask
24:30
Josh: in order to solve better problems. So it really...
24:33
Josh: It would appear, assuming that we continue on this trajectory of improvement,
24:38
Josh: that it makes sense to take every disposable dollar you can to get better and better compute.
24:44
Josh: And this will probably just extend forever. As we are able to harness more energy
24:47
Josh: from the sun, from nuclear energy, a lot of that new energy and compute will
24:51
Josh: just go to making better AI, which will then serve better downstream effects for how society works.
24:56
Josh: So is it a bubble on the long term? I think absolutely not on the short term.
25:00
Josh: I don't know. Where do you get the revenue from?
25:03
Josh: I don't know. I mean, it's a ton of money, but you know what?
25:05
Ejaaz: I think the reason why you and I feel this disassociation between the amount,
25:11
Ejaaz: how large these numbers are in investing in infrastructure versus what we're
25:15
Ejaaz: actually seeing is we're not going to be seeing AGI before some other fields
25:20
Ejaaz: or some other professions see it first, right?
25:23
Ejaaz: The clear example is coding. Coding has just
25:26
Ejaaz: been on an absolutely exponential improvement rate
25:29
Ejaaz: that has beaten out any other ai feature ever you now
25:32
Ejaaz: have ai models that can code as well
25:35
Ejaaz: as a senior staff engineer which is getting paid like
25:38
Ejaaz: 300 to 500 a year right um so my guess is this investment is worth it um my
25:45
Ejaaz: guess is the investment is going to come to fruition in professions in use cases
25:50
Ejaaz: in jobs that we won't see but we'll maybe talk about or see the kind of like effects.
25:55
Ejaaz: Maybe it's in science where we create a new drug that cures cancer or whatever that might be, right?
26:00
Ejaaz: I think different types of professionals will see AGI and reap the rewards of
26:04
Ejaaz: this investment before average consumers see it.
26:09
Ejaaz: And then I think the other thing that I want to mention, Josh,
26:11
Ejaaz: is this isn't specific to US or Western spending.
26:16
Ejaaz: In fact, our foes over the seas in China or in Asia have been working on this
26:23
Ejaaz: for like the last five years.
26:25
Ejaaz: They've been building out massive data centers, which I think has like build
26:30
Ejaaz: up an aggregate of like 300 gigawatts over the next five years, at least.
26:35
Ejaaz: And they've been investing in this so heavily. So it's not just a Western thing.
26:40
Ejaaz: It's an Asian thing as well. China is investing so heavily in
26:43
Ejaaz: this if this is a bubble if we are completely
26:46
Ejaaz: wrong this will be the uh biggest most highest profile l that the world has
26:52
Ejaaz: taken it's not just going to be a us thing it's not just going to be a map something
26:56
Ejaaz: it's not just going to be a sam altman thing it's going to be an everyone's
26:59
Ejaaz: involved type of thing kind of like world ending event yeah
27:03
Josh: Too big to fail so i i do love
27:06
Josh: this incentive structure where everyone is incentivized to make it work because
27:09
Josh: everyone's equally at risk in terms of their exposure to the technology so that
27:14
Josh: i think i could be happy to sleep at night where at least u.s and china are
27:17
Josh: aligned in one thing in which they want to achieve agi they want the smartest
27:21
Josh: models they're going to make their money pay off the best they can so,
27:25
Josh: All the power to them. But I think, is that a wrap for us today,
27:29
Josh: EJ? We got anything else?
27:30
Ejaaz: That is a wrap,
27:30
Josh: Josh. That's it. That's a wrap on our little like XAI mini episode.
27:34
Josh: There was one fact that I wanted to just do a little like fun fact check, which is a gigawatt.
27:39
Josh: And according to Grok, it powers approximately 750,000 to 850,000 average US homes per one gigawatt.
27:46
Josh: So the scale we're talking is like a tremendous amount of gigawatts.
27:50
Josh: I mean, this NVIDIA project is 10 of those, which means that's about,
27:53
Josh: I mean, on the high end, eight and a half million U.S.
27:56
Josh: Homes can be powered by a singular data center. So we're going to hope this works out.
28:00
Josh: I think right now it seems like, I mean, Grok is cooking. The XAI team is on
28:04
Josh: fire and they are in between models.
28:06
Josh: I cannot wait until they get this new Colossus training cluster up or even Microsoft's.
28:11
Josh: I mean, Microsoft's got a huge cluster.
28:13
Josh: What are you doing with it, dog? Like, let's see. Let's see your stats.
28:16
Josh: Let's see your numbers. Put a number up on the ArcGIS leaderboard. um
28:19
Josh: but yeah i think that's that's a wrap on all the fun exciting new
28:22
Josh: things about xai the comment section is by energy stocks yeah
28:25
Josh: by energy stocks um i we read
28:28
Josh: all the comments i read every single comment i try to reply to them too so i
28:31
Josh: would love for you to share either what you think about the show or or who
28:35
Josh: you think is winning this ai race currently do you like are
28:38
Josh: we just kind of like do we have elon derangement syndrome are
28:41
Josh: we just kind of like obsessed with everything he builds or is this it feels
28:44
Josh: like it's pretty grounded i feel like we have some good examples about how well
28:47
Josh: they're doing so i'd love to hear if you agree or disagree that
28:50
Josh: would be a fun little thing for the comments but anyway that's a wrap on today's episode
28:53
Josh: we have a couple more exciting ones coming this week so buckle up uh
28:56
Josh: the next one the next one coming i think ejaz myself
28:59
Josh: and we might even have a guest for that episode we'll be probably in an all-out
29:02
Josh: brawl it's good that we're recording remotely because we might like the blood
29:05
Josh: could possibly be drawing so yeah buckle up for for that one there's a lot to
29:10
Josh: look forward to this week but that's it for this week this episode so thank
29:13
Josh: you so much for watching as always please don't forget to subscribe like comment,
29:16
Josh: all the fun things, share it with your friend.
29:18
Music: Music