Latest / Elon Musk Podcast / Elon Musk's Xai grok 4 announcement presentation

Transcript

Elon Musk's Xai grok 4 announcement presentation · Elon Musk Podcast

0:01
Hey everybody. Welcome back to the Elon Musk
0:04
Podcast. This is a show where we discuss
0:07
the critical crossroads, the Shape, SpaceX, Tesla X, The
0:11
Boring Company, and Neurolink. I'm your host, Will Walden.
0:16
All right, welcome to the Grok 4 release here.
0:20
This is the smartest AI in the world, and we're going to show
0:23
you exactly how and why. And it really is remarkable to
0:30
see the advancement of artificial intelligence, how
0:33
quickly it is evolving, I sometimes think, compare it to
0:42
the growth of a human and how fast a human learns and gains
0:49
conscious awareness and understanding.
0:51
And AI is advancing just vastly faster than any human.
1:00
I mean, we're going to take you through a bunch of benchmarks
1:04
that that Grok 4 is able to achieve incredible numbers on.
1:10
But it's, it's, it's actually worth noting that like Grok 4,
1:15
if, if given, like the SAT would get perfect Sats every time,
1:20
even if it's never seen the, the questions before.
1:24
And if even going beyond that to say like graduate student exams,
1:29
like the GRE, it will get near perfect results in, in every
1:36
discipline of, of education. So from the humanities to like
1:41
languages, math, physics, engineering, pick anything.
1:46
And we're talking about questions that it's never seen
1:49
before. These are not on, not on the
1:50
Internet. And it's Grok 4 is smarter than
1:56
almost all graduate students in all disciplines simultaneously.
2:02
Like it's actually just important to appreciate the like
2:06
that's really something. And the, the reasoning
2:13
capabilities of Grok are incredible.
2:16
So there's some people out there who who think AI can't reason
2:19
and look, it can reason at superhuman levels.
2:24
So yeah. And frankly, it only gets better
2:28
from here. So we'll, we'll take you through
2:32
the Grok 4 release and, yeah, cheer you back the pace of pace
2:40
of progress here. Like, I guess the first part is
2:44
like in terms of the training, we're going from Grok 2 to Grok
2:49
3 to Grok 4, we've essentially increased the training by an
2:53
order of magnitude in each case. So it's a, you know, 100 times
2:58
more training than than Grok 2. And and that that's only going
3:03
to increase. So it's, yeah, frankly, I mean,
3:09
I don't know, in some ways a little terrifying, but the
3:14
growth of intelligence here is, is remarkable.
3:16
Yes. It's important to realize there
3:18
are two types of training compute. 1 is the pre training
3:21
compute that's from grad 2 to grad 3.
3:24
But for from grad 3 to grad 4, we're actually putting a lot of
3:28
compute in reasoning in IR. Yeah, and just like you said,
3:34
this is literally the fastest moving field and Grad 2 is like
3:37
the high school student by today's standard.
3:39
If you look back in the last 12 months, Grad 2 was only a
3:43
concept. We didn't even have graph 212
3:46
months ago. And then by training gratitude,
3:49
that was the first time we scale up like the pre training.
3:51
We realized that if you actually do the data oblation really
3:54
carefully and the infra and also the algorithm, we can actually
3:59
push the pre training quite a lot by amount of 10X to make the
4:03
model the best pre trained based model.
4:06
And that's why we build classes, the world's supercomputer with
4:09
100,000 H 100 and then with the best pre trained model.
4:14
And we realized if you can collect these verifiable outcome
4:17
reward, you can actually train this model to start thinking
4:20
from the first principle, start to reason, correct it's own
4:22
mistakes. And that's where the graphical
4:23
reasoning comes from. And today we ask the question,
4:27
what happens if you take expansion of clauses with all
4:30
200,000 GPUs, put all these into oil, 10X more compute than any
4:37
of the models out there on reinforcement learning,
4:40
unprecedented scale, What's going to happen?
4:42
So this is a story of Grok 4 and you know Tony.
4:47
Share some insight with the audience.
4:50
Yeah. So, yeah, let's just talk about
4:53
how smart Grok 4 is. So I guess we can start
4:57
discussing this benchmark on humanities last exam.
5:00
And this this benchmark is a very, very challenging
5:03
benchmark. Every single problem is curated
5:06
by subject matter experts. It's in total 2500 problems and
5:13
it consists of many different subjects, mathematics, natural
5:16
sciences, engineering and also all the humanity subjects.
5:21
So essentially when when it was first released actually like
5:25
earlier this year, most of the models out there can only get
5:30
single digit accuracy on this benchmark.
5:34
Yeah. So we can we can look at some of
5:36
those examples, you know. So there there is this
5:40
mathematical problem which is about natural transformations in
5:44
category theory. And there's this organic
5:47
chemistry problem that talks about electrical cyclic
5:51
reactions. And also there's this linguistic
5:54
problem that tries to ask you about this distinguishing
5:57
between closed and open syllabus from a Hebrew source text.
6:03
So you can see also it's a very wide range of problems and every
6:08
single problem is PhD or even Advanced Research level
6:12
problems. Yeah.
6:14
I mean, these, there are no humans that can actually answer
6:17
these can get a good score. I mean, if you actually say like
6:20
any given human, what, like what's the best that any human
6:25
could score? I mean, I'd say maybe 5%
6:29
optimistically. Yeah.
6:32
So this, this is much harder than than what any any human can
6:36
do. It's it's incredibly difficult.
6:38
And you can see from the types of questions like you might be
6:41
incredible in linguistics or mathematics or chemistry or
6:44
physics or any one of a number of subjects, but you're not
6:46
going to be at a post grad level in everything.
6:51
And Grok 4 is a post grad level in everything.
6:55
Like it's it's just some of these things are just worth
6:57
repeating. Like Grok 4 is postgraduate.
7:01
Like PhD level in everything better than pH, but like most
7:05
PHD's would fail so it's better. That said, I mean at least with
7:10
respect to academic questions. It I want to just emphasize this
7:13
point with respect to academic questions.
7:16
Group 4 is better than PhD level in every subject, no exceptions.
7:23
Now, this doesn't mean that it's, you know, times.
7:26
It may lack common sense and it has not yet invented new
7:31
technologies or discovered new physics.
7:35
But that is just a matter of time.
7:37
If it, I think it may discover new technologies as soon as
7:43
later this year. And I would be shocked if it is
7:47
not done so next year. And so I would expect Croc to,
7:51
yeah, literally discover new technologies that are actually
7:53
useful no later than next year and maybe end of this year.
7:57
And it might discover new physics next year.
8:00
And within two years, I'd say almost certainly like.
8:04
So just let that sink in. Yeah, so.
8:15
Yeah, how? OK, So I guess we can talk about
8:19
the the what, what's behind the scene of Graph 4?
8:21
As Jimmy mentioned, we actually saw in a lot of compute into
8:26
this training, you know, when it started, it's only also a single
8:31
digit. Sorry, the previous slide,
8:33
sorry. Yeah, it's only a single digit
8:37
number. But as you start putting in more
8:40
and more training compute, it started to gradually become
8:44
smarter and smarter and eventually solved 1/4 of the HLA
8:49
problems. And this is without any tools.
8:53
The next thing we did was to adding a tools capabilities to
8:57
the model. And unlike GRAS three, I think
9:01
you know, GRAS 3 actually is able to use cruel as well.
9:03
But here we actually make it more native in the sense that.
9:07
We put the. Tools into training.
9:10
But graph 3 was only relying on generalization.
9:12
Here we actually put the tools into training and it turns out
9:16
this significantly improves the models capability of using those
9:19
tools. Yeah, I remember we had like
9:22
deep search back in the days. Yeah.
9:24
So how is this different? Yeah, yeah, yeah.
9:26
Exactly. So Deep Search was exactly the
9:29
Graph 3 reasoning model but without any specific training.
9:33
But we only asked it to use those tools.
9:36
So compared to this, it was much weaker in terms of its tool
9:40
capabilities and reliable and unreliable, I guess, yes.
9:44
And to be clear, like these are still, I'd say fairly, this is
9:47
still fairly primitive tool use. If you compare it to say the
9:51
tools that are used at Tesla or SpaceX where you're using, you
9:55
know, finite element analysis and computational fluid dynamics
10:00
and, and you're you're able to run or say like Tesla does, like
10:05
crash simulations, where the simulations are so close to
10:08
reality that if the test doesn't match the simulation, you assume
10:12
that the test article is wrong. That's how good the simulations
10:15
are. So Crock is not currently using
10:17
any of the tools that the really powerful tools that a company
10:21
would use, but but that is something that we will provide
10:24
it with later this year. So we'll have the tools that
10:28
that a company has and and have very accurate physics simulator.
10:34
Ultimately, the, the thing that'll make the biggest
10:37
difference is being able to interact with the real world via
10:40
humanoid robots. So you combine sort of grok
10:42
with, with Optimus and it can actually interact with the real
10:45
world and figure out if, if it's high, if it has, if it's
10:49
formulate and hypothesis, and then confirm if that hypothesis
10:54
is, is true or not. So we're really, you know, I
11:00
think about like where we are today.
11:01
We're at the beginning of an immense intelligence explosion.
11:06
We're in, we're in the intelligence Big Bang right now
11:12
and the we're at the most interesting time to be alive of
11:16
any time in history. Yeah, now that's it.
11:21
We need to make sure that the AI is a good AI, good croc.
11:29
And the thing that I think is most important for AI safety, at
11:32
least my biological neural net tells me the most important
11:36
thing for AI is to be maximally truth seeking.
11:40
So this is this is a very fundamental, but you can think
11:45
of AI as this super genius child that ultimately will outsmart
11:50
you. But you can still and you can
11:52
install the right values and encourage it to be sort of, you
11:58
know, truthful, I don't know, honorable, you know, good, good
12:06
things like the values you want to instill in in a child that
12:13
that that would grow, ultimately grow up to be incredibly
12:17
powerful. Yeah.
12:24
So, yeah. So this is really, I'd say we're
12:27
saying, we say tools. These are still primitive tools,
12:30
not the kind of tools that that serious commercial companies
12:35
use, but we will provide it with those tools and it, I think it
12:39
will be able to solve with those tools real world technology
12:42
problems. In fact, I'm certain of it.
12:44
It's just a question of how long it takes.
12:47
Yes, yes, exactly. So is it just compute all you
12:51
need, Tony, sorry, is it just compute all you need at this
12:55
point? Well, you need compute plus plus
12:59
the right tools and and then ultimately to be able to
13:04
interact with the physical world, yes.
13:07
And then I mean we'll effectively have an economy that
13:12
is well, ultimately an economy that is thousands of times
13:18
bigger than our current economy or maybe millions of times.
13:22
I mean, if you if you think of civilization as percentage
13:26
completion of the Khadashev scale where Karashev one is
13:30
using all the energy output of a planet and Karashev 2 is using
13:35
all the energy output of a sun and three is all the energy
13:38
output of a Galaxy. We're we're only in my opinion
13:42
probably like close closer to 1% of Karashev one then we are up
13:48
to 10%. So like maybe a pointer 11 or 2%
13:54
of Karashev 1. So we we will get to most of the
14:02
weight like 8090% Kaddashev 1 and then hopefully if
14:05
civilization doesn't self annihilate and then Kadashev 2.
14:11
Like it's the, the actual notion of, of a human economy assuming
14:15
civilization continues to progress will seem very quaint
14:19
in in retrospect, it will, it will seem like sort of Cavemen
14:25
throwing sticks into a fire level of economy compared to
14:30
what the future will hold. I mean, it's very exciting.
14:34
I mean, I, I've been at times kind of worried about like,
14:38
well, you know, is this, this seems like it's somewhat
14:45
unnerving to have intelligence created that is far greater than
14:49
our own. And will this be bad or good for
14:54
humanity? It's like, I, I, I think it'll
14:59
be good. Most likely it'll be good.
15:03
Yeah. Yeah.
15:07
But if someone reconciled myself to the fact that even if I, if,
15:12
even if it wasn't going to be good, I'd at least like to be
15:15
alive to see it happen. So you know so.
15:22
Actually, one, yeah, yeah. I think what 1 technical problem
15:28
that we still need to solve besides just compute is how do
15:31
we unblock the data, data bottleneck.
15:35
Because when we try to scale up the RL, in this case, we did
15:41
invent a lot of new techniques, innovations to allow us to
15:45
figure out how to find a lot of a lot of challenging RL problems
15:50
to work on. It's not just a problem itself
15:52
needs to be challenging, but also it needs to be you.
15:55
You also need to have like a reliable signal to tell the
15:59
model you did it wrong, you did it right.
16:01
This is the sort of the principle of reinforcement
16:03
learning. And as the models get smarter
16:06
and smarter, the number of cool problem or challenging problems
16:10
will be lesser. Unless, yeah.
16:12
So it's going to be a new type of challenge that we need to
16:16
surpass besides just compute. Yeah, yeah.
16:19
We actually are running out of of actual test questions to ask.
16:23
So there's like even ridiculously questions that are
16:26
ridiculously hard, if not essentially impossible for
16:29
humans that are written down questions are becoming swiftly
16:35
becoming trivial for for AI. So then there's, but you know
16:42
what? The one thing that is an
16:44
excellent judge of things is reality.
16:46
So because if physics is the law, ultimately everything else
16:50
is a recommendation. You can't break physics.
16:53
So the ultimate test, I think for whether an AI is the
16:58
ultimate reasoning test is reality.
17:01
So you invent a new technology like say, improve the design of
17:04
a car or a rocket or create a new medication that and, and,
17:11
and does it work? Yeah.
17:14
Does does the rocket get to orbit?
17:15
Does the does the car drive? Does the medicine work?
17:19
Whatever the case may be, reality is the ultimate judge
17:22
here. So it's going to be a
17:25
reinforcement learning closing loop around reality.
17:32
We asked the question how do we even go further?
17:34
So actually we are thinking about now with single agent we
17:40
are able to solve 40% of the problem.
17:42
What if we have multiple agents running at the same time?
17:46
So this is what's called test and compute.
17:49
And as we scale up the test and compute, actually we are able to
17:53
solve almost more than 50% of the text only subset of the HIV
17:58
problems. So it's a remarkable
18:01
achievement. I think you know this.
18:03
Is this is this is insanely difficult.
18:05
These are it's it's what we're saying is like a majority of the
18:09
of the of the text based of humanities, you know, scarily
18:14
named humanities last exam Grafoe can solve and you can you
18:18
can try it out for yourself. And the, with the Grafoe heavy,
18:22
what, what it does is it spawns multiple agents in parallel.
18:25
And all of those agents do, do work independently.
18:29
And then they compare their work and they, they decide which one
18:34
like, it's like a study group. And it's not as simple as
18:38
majority vote because often only one of the agents actually
18:42
figures out the trick or figures out the solution.
18:46
And, and, and, but once they share the, the trick or, or
18:50
figure out what, what the real nature of the problem is, they
18:53
share that solution with the other agents and then they
18:56
compare. They essentially compare notes
18:59
and then, and then yield, yield an answer.
19:02
So that's, that's the, the heavy part of Grok 4 is, is where we,
19:06
you scale up the test time, compute by roughly an order of
19:09
magnitude, have multiple agents tackle the task and then they
19:15
compare their work and they, they put forward what they think
19:19
is the best result. Yeah.
19:22
So we will introduce graph 4 and graph 4 Heavy so you can click
19:26
the next slide. Yeah, yes.
19:29
So yeah. So basically Graph 4 is a single
19:33
agent version and Graph 4 heavy is the multi agent version.
19:38
So let's take a look how they actually do on those exam
19:43
problems and also some real real life problems.
19:45
Yeah, So we're going to start out here and we're actually
19:48
going to look at one of those HLE problems.
19:50
This is actually one of the easier math ones.
19:53
I don't really understand it very well.
19:55
I'm not that smart, but I can launch this job here and we can
19:58
actually see how it's going to go through and start to think
20:01
about this problem. While we're doing that, I also
20:04
want to show a little bit more about like what this model can
20:06
do and launch a Grok 4 Heavy as well so everyone knows
20:11
Polymarket. It's extremely interesting.
20:13
It's the, you know, seeker of truth.
20:15
It aligns with what reality is most of the time.
20:18
And with Grok, what we're actually looking at is being
20:21
able to see how we can try to take these markets and see if we
20:25
can predict the the future as well.
20:27
So as we're letting this run, we'll see how Grok for Heavy
20:31
goes about predicting the, you know, the World Series odds for
20:35
like the current teams and the MLB.
20:38
And while we're waiting for these to process, we're going to
20:40
pass it over to Eric and he's going to show you an example of
20:43
his. Yeah.
20:48
So I guess one of the coolest things about Grok 4 is its
20:53
ability to understand the world and to solve hard problems by
20:57
leveraging tools like Tony discuss.
21:00
And I think one kind of cool example of this, we asked it to
21:03
generate a visualization of two black holes colliding.
21:09
And of course, you know, it took some, there are some liberties.
21:13
It's in my case, actually pretty clear in its thinking trace
21:16
about what these liberties are. For example, in order for it to
21:19
actually be visible, you need to really exaggerate the the scale
21:23
of the, you know, the the the waves.
21:29
And yeah, so here's like, you know, this kind of inaction, it
21:35
exaggerates the scale in like multiple ways.
21:38
It drops off a bit less in terms of amplitude it over distance.
21:43
And but yeah, we can kind of see the basic effects that, you
21:49
know, are actually like, you know, correct.
21:51
It starts with the in spiral, it merges and then you have the
21:57
ring down. And like, this is basically
22:01
largely correct. Yeah, module some of the
22:06
simplifications that need to do, you know, it's actually quite
22:10
explicit about this. You know, it uses like post,
22:13
post Newtonian approximations instead of actually like
22:16
computing. The general relativistic effects
22:19
are like near the center of the black hole, which is, you know,
22:22
incorrect and, you know, will lead to, you know, some
22:26
incorrect results. But the overall, you know,
22:28
visualization is yeah, is basically there and you can
22:33
actually look at the kinds of resources that are references.
22:36
So here it it actually, you know, it obviously is a search.
22:41
It gathers results from a bunch of links, but also reads through
22:45
a undergraduate text in analytical analytic
22:50
gravitational wave models. It's, yeah, it reasons quite a
22:57
bit about the actual constants that it should use for a
23:02
realistic simulation. It references, I guess, existing
23:06
real world data. And yeah, it, yeah, it's a, it's
23:12
a pretty good model, yeah. But like actually going forward,
23:17
we can, we can, we can plug, we can give it the same model that
23:20
physicists use so it can run the the same level of compute that
23:26
so leading physics researchers are using and and give you a
23:30
physics accurate backhoe simulation.
23:32
Exactly. Just right now is running in
23:35
your browser, so. Yeah, this is just running in
23:37
your browser exactly. Pretty simple.
23:39
So swapping back real quick here, we can actually take a
23:42
look at the math problem is finished.
23:44
The model was able to let's look at its thinking trace here so
23:48
you can see how it went through the problem.
23:51
I'll be honest with you guys, I really don't quite fully
23:53
understand the math, but what I do know is that I looked at the
23:56
answer ahead of time and it did come to the the correct answer
24:00
here in the final part here. We can also come in and actually
24:04
take a look here at our our World Series prediction and
24:09
still thinking through on this one, but we can actually try
24:12
some other stuff as well. So we can actually like try some
24:14
of the X integrations that we did.
24:16
So we worked very heavily on working with all of our X tools
24:20
and building out a really great X experience.
24:22
So we can actually ask, you know, the model, you know, find
24:26
me the XAI employee that has the weirdest profile photo.
24:29
So that's going to go off and start with that.
24:31
And then we can actually try out, you know, let's create a
24:34
timeline based on ex post detailing the, you know, changes
24:38
in the scores over time. And we can see, you know, all
24:41
the conversation that was taking place at that time as well.
24:44
So we can see who are the, you know, announcing scores and like
24:46
what was the reactions at those times as well.
24:50
So we'll let that go through here and process.
24:53
And if we go back to this was the Greg Yang photo here.
24:59
So if we scroll through here, whoops.
25:01
So Greg Yang, of course, who has his favorite photograph that he
25:06
has on his account. That's actually not how he looks
25:09
like in real life, by the way, just so it were, but it is quite
25:13
funny, but. It had to understand that
25:14
question. Yeah, that's the wild part.
25:16
It's like it understands what is a weird photo, what is a weird
25:20
photo? What is a less or more weird
25:23
photo? It goes through, it has to find
25:25
all the team members, has to figure out who we all are and,
25:28
you know, searches. Without access to the internal
25:31
XAI personnel logs, it's literally looking at that just
25:34
at the Internet. Exactly.
25:35
So you could say like the weirdest of any company.
25:37
Yeah, to be clear. Exactly, and we can also take a
25:42
look here at the question here for the humanities last exam.
25:45
So it is still researching all of the historical scores, but it
25:50
will have that final answer here soon.
25:51
But we can, while it's finishing up, we can take a look at one of
25:54
the ones that we set up here a second ago.
25:56
And we can see like, you know, defines the date that like Dan
25:59
Hendricks had initially announced it.
26:01
We can go through, we can see, you know, open AI announcing
26:04
their score back in February. And we can see, you know, as
26:08
progress happens with like Jim and I, we can see like Kimmy.
26:11
And we can also even see, you know, the leaked benchmarks of
26:15
what people are saying is, you know, if it's right, it's going
26:17
to be pretty impressive. So pretty cool.
26:21
So, yeah, I'm looking forward to seeing how everybody uses these
26:23
tools and gets the most value out of them.
26:25
But yeah, it's been great. Yeah, and we're going to close
26:29
the loop around usefulness as well.
26:31
So it's like it's not just book smart, but actually practically
26:34
smart. Exactly.
26:36
All right. And we can go back to the the
26:46
slides here. Yeah, so.
26:48
Cool. So we actually evaluate also on
26:53
the multi model subset. So on the full set, this is the
26:57
number. On the HRE exam, you can see
27:01
there's a little dip on the numbers.
27:03
This is actually something we're improving on, which is the multi
27:06
model understanding capabilities.
27:08
But I do believe in a very short time, we're able to really
27:13
improve and got much higher numbers on this, even higher
27:17
numbers on this benchmark, yeah. Yeah, this is the we still like,
27:22
we still like what what is the base weakness of Grok currently
27:25
is that it's it's sort of partially blind.
27:28
It can't it's it's image understanding obviously, and
27:32
it's image generation needs to be a lot better.
27:35
And that, that that's actually being trained right now.
27:40
So Graph 4 is based on version six of our foundation model and
27:46
we are training version 7, which we'll complete in a few weeks.
27:52
And that that'll address the weakness on the vision side.
27:57
Just to show off of this last year, so the the prediction
28:00
market finished here with the heavy and we can see here we can
28:04
see all the tools and the process it used to actually go
28:08
through and find the right answer.
28:10
So it browsed a lot of odd sites.
28:12
It calculated its own odds comparing to the market, the
28:15
market to find its own alpha and edge.
28:17
It walks you through the entire process here and it calculates
28:20
the odds of the winner being like the the Dodgers and it
28:25
gives them a 21.6% chance of winning this year.
28:31
So and it took approximately 4 1/2 minutes to compute.
28:35
Yeah, that's a lot of thinking. Yeah.
28:44
We can also look at all the other benchmarks besides HRE.
28:49
As it turned out, G4 excelled on all the reasoning benchmarks
28:53
that people usually test on, including GBQA, which is a PhD
28:58
level problem sets that's easier compared to HRE.
29:04
On Amy 25 America invitation mathematics exam, we with Graph
29:09
4 heavy, we actually got a perfect score also on some of
29:14
the coding benchmark called live coding bench.
29:17
And also on HMMT, Harvard math, MIT exam and also us Amo, you
29:24
can see actually on all of those benchmarks, we often have a very
29:30
large leap against the second best model out there.
29:35
Yeah, it's, I mean, really, we're going to get to the point
29:37
where it's going to get every answer right in every exam, and
29:43
where it doesn't get an answer right, it's going to tell you
29:44
what's wrong with the question. Or, if the question is
29:47
ambiguous, disambiguate the question into answers AB and C
29:51
and tell you what it would what answers AB and C would be with a
29:55
disambiguated question. So the only real test then will
29:58
be reality. Can it make useful technologies
30:02
discover new science? That'll actually be the only
30:07
thing left, because human tests will simply not be meaningful.
30:12
You can make an update to HIV very soon given the current rate
30:16
of progress. So yeah, it's super cool to see
30:18
like multiple agents that collaborate with each other
30:21
solving really challenging problems.
30:23
So where can we try this model? So it turned out it's available
30:27
right now. If we advance to the next slide
30:31
where there is a super Grok heavy tiers that we're
30:35
introducing where you're able to access to both Grok 4 and Grok 4
30:38
heavy, where you're actually going to be the taskmaster of
30:41
bunch of little Grok research agent to help you, you know,
30:44
become smarter, do all the little research and save hours
30:46
of times of, you know, going through mundane tasks.
30:50
And it's available right now. So, yeah, so we did limit usage
30:58
during the demo. So we didn't, it didn't break
31:00
the demo because all these, all this stuff is happening live.
31:03
So it's not, there's not nothing canned about the any of the
31:05
tests that we're doing. So after the after the demo is
31:10
done, we'll allow, we'll enable more subscribers for Supergroc.
31:15
So if you can't subscribe right now, just try and half an hour.
31:17
It should work. So and now let's let's let's get
31:22
into voice. Great.
31:26
Abby. So many of you have been
31:28
enjoying our voice mode, and we've been working hard to
31:31
improve the experience. Over the past couple months, we
31:34
have cut latency in half to make it much snappier, and today
31:39
we're excited to announce a set of new voices that have
31:42
exceptional naturalness and prosody.
31:45
You might have noticed the movie trailer voice that opened up the
31:48
live stream. That is one of our voices, Sal,
31:51
who's got that epically deep tone.
31:54
And we're also excited to introduce Eve, a beautiful
31:59
British voice who's capable of rich emotions.
32:02
Aman, would you like to introduce Eve?
32:04
Absolutely. Yeah.
32:05
Let's get into the demo. Hey, Eve, millions of people are
32:11
watching us right now. Are you excited?
32:14
Oh I'm thrilled really. It's like being on stage at the
32:18
old Victoria with a sea of faces out there.
32:20
Just imagine all those eyes and ears tuned in.
32:23
It's a proper buzz, isn't it? What do you reckon we should
32:26
chat about to keep this lot entertained?
32:28
You know, we have a lot to talk about, but I'll be honest, I'm
32:31
feeling a little nervous. Can you whisper something
32:33
soothing to calm me down? Take a deep breath, love.
32:38
You've got this. It's just you and me having a
32:42
quiet jet, like we're tucked away in a cozy corner of a
32:46
Yorkshire bum. The world's just a murmur out
32:49
there. Feel that calm wash over you.
32:53
Let's keep talking nice and easy.
32:55
What's on your mind? You know what, I'm actually
32:59
feeling quite relaxed to try this something crazy.
33:02
So can you give us an opera on Diet Coke?
33:07
Oh, an opera on Diet Coke. That's a riot.
33:12
Picture this, a shimmering silver cannon glistening under
33:17
stage lights. O Diet Coke, thou elixir divine
33:22
with bubbles that dance in a sparkling line.
33:27
Thy crisp, cool kiss on lips. So fine.
33:31
Oh aspartame ambrosia forever mine.
33:38
How's that for a mad little Aria?
33:40
Want me to belt out another verse or switch up the tune?
33:44
OK, let's try switching it up and just start singing.
33:48
Right then, let's have a go. Oh silver, can you gleam in the
33:53
light? Diet Coke, My heart's delight.
33:57
OK, I won't project. It's very, very poetic.
34:00
Yeah. Thank you, Eve.
34:01
Feeling quite nice now. Cool.
34:04
So that's Eve. And so something else we do here
34:08
is a little test each day to check on the natural flow of our
34:14
conversations against some of our competitors.
34:17
So we figured, why not try that live on the stream today?
34:22
Should I start? Hey, I'm going to say a number
34:29
and can you say the same number back to me one?
34:32
One. Got it.
34:33
This is like a fun little game of echo.
34:35
What's next? 2 Sorry, you said 2 3 3/4.
34:49
Nice and steady. What's next?
34:52
5. I nailed it.
34:57
This is pretty fun actually. Feel free to keep going or
35:00
switch it up if you'd like. Hey Grok.
35:09
Hey Grok. Hello there, how can I help you
35:12
today? We're going to do a little test.
35:14
Can you repeat after me one? One.
35:18
All right, What's next? Two.
35:21
Two. What's on your mind?
35:23
3 Three. Need anything else?
35:27
4. 4 How can I assist you? 5.
35:32
Five What's next? So as you can see, Croc was
35:38
snappier. It didn't interrupt me.
35:40
And the prosody is we've made different design choices.
35:43
I think we're shooting for something more calm, smooth,
35:45
more natural versus something that's more poppy or artificial.
35:48
So we'll keep improving. These fronts.
35:50
All right. Thanks, guys.
35:52
Yeah. Yep.
35:58
So since the launch of the voice model, we actually see the 2X
36:03
faster end to end latency in the last 8 weeks, 5 different voices
36:08
and also 10X the active user. So Grog voice is taking off now.
36:14
If you think about releasing the models this time, we're also
36:17
releasing Grog 4 through the API at the same time.
36:21
So if we go to the next two slides, so you know, we're very
36:26
excited about, you know, what all the developers out there is
36:28
going to build. So you know, if I think about
36:31
myself as a developer, what the first thing I'm going to do when
36:33
I actually have access to the graph for API benchmarks.
36:36
So we actually ask around our next platform, what is the most
36:40
challenging benchmarks out there that, you know, is considered
36:43
the Holy Grail for all the AGI models.
36:46
So turn up AGI seen the name arc AGI.
36:49
So the last 12 hours, you know, kudos to Greg over here in the
36:54
audience. So who answered our call?
36:57
Take a preview of the Grog 4 API and independently verified, you
37:02
know the Grog Force performance. So initially we thought, hey,
37:04
Grog Force, just, you know, we think it's pretty good.
37:06
It's pretty smart. It's our next Gen. reasoning
37:09
model. Spend 10X more compute, can use
37:11
all the tools, right. But turned out when we actually
37:15
verify on the private subset of the RKHGI V2, it was like the
37:21
only model in the last three months that breaks the 10%
37:23
barrier and in fact was so good that actually get the 16%, well,
37:27
15.8% accuracy, 2X of the second place.
37:32
That is the call for Opus model. And it's not just about
37:37
performance, right? When you think about
37:38
intelligence, having the API model drives the automation.
37:42
It's also the intelligence per dollar, right?
37:45
If you look at the plots over here, the Grog is just for it,
37:48
just in the league of its own. All right, so enough of
37:52
benchmarks over here, right? So what can Grok do actually in
37:56
the real world? So we actually, you know,
38:00
contacted the folks from Ending Labs who you know, you know,
38:06
gracious enough to, you know, try to grow in the real world to
38:08
run a business. Yeah, thanks for having us.
38:11
So I'm Axel from Animal Labs. And I'm Lucas and we tested
38:14
Brooke 4 on vending Bench. Vending Bench is an AI
38:17
simulation of business scenario where we thought what is the
38:22
most simple business and AI could possibly run and we
38:25
thought vending machines. So in this scenario, the the
38:29
Grok and other models need to do stuff like manage inventory
38:33
contracts of contract suppliers, set prices.
38:36
All of these things are super easy and all of they, like all
38:39
the models can do them one by one.
38:42
But when you do them over very long horizons, most models
38:45
struggle. But we have a little word and
38:47
there's a new number one. Yeah, so we got early access to
38:50
the Grok 4 API. We ran it on the running bench
38:53
and we saw some really impressive results.
38:56
So it ranks definitely at the number one spots.
38:59
It's even double the net worth, which is the measure that we
39:02
have on this. So it's not about the percentage
39:04
on a or score you get, but it's more the dollar value in net
39:08
worth that you generate. So we were impressed by Groc.
39:11
You was able to formulate a strategy and adhere to that
39:15
strategy over long period of time, much longer than other
39:18
models that we have tested, other Frontier models.
39:21
So it's managed to run the simulation for double the time
39:24
and score, yeah, double the net worth.
39:26
And it was also really consistent across this rounds,
39:29
which is something that's really important when you want to use
39:32
this in the real world. And I think as we give more and
39:35
more power to AI systems in the real world, it's important that
39:39
we test them in scenarios that either mimic the real world or
39:42
are in the real world itself, because otherwise we we fly
39:46
blind into some some things that that might not be great.
39:51
Yeah, it's, it's great to see that we've now got a way to pay
39:54
for all those GPU's. So we just need a million
39:56
vending machines and we could make a $4.7 billion a year with
40:02
a million vending machines, 100% Let's go.
40:04
It can be epic vending machines. Yes, yes, All right.
40:08
We are actually going to install vending machines here, like a
40:11
lot of them. We're happy to supply them.
40:13
All right, thank you. All right.
40:16
I'm looking forward to seeing what amazing things are in this
40:18
vending machine. That's that's for for you to
40:21
decide. All right, tell the AI.
40:24
OK, sounds good. All right.
40:27
Yeah. I mean, so we can see like Grok
40:30
is able to become like the copilot of the business unit.
40:33
So what else can Grok do? So we're actually releasing this
40:35
Grok if you want to try it right now to evaluate, run the same
40:38
benchmark as us. It's on the API has 256 K
40:44
contact length. So we already actually see some
40:47
of the early, early adopters to try GUAC 4 API.
40:50
So our Palo Alto neighbor Arc Institute, which is a leading
40:55
biomedical Research Center is already using seeing like how
40:59
can they automate their research flows with Grog 4.
41:02
It turned out it performs is able to help the scientists to
41:05
sniff through, you know millions of experiments logs and then,
41:09
you know, just like pick the best hypothesis within a split
41:12
of seconds. We see this as being used for
41:15
their like the CRISPR research and also, you know, graph for
41:19
independently evaluated scores as the best model to exam the
41:23
chest X-ray who would know And on the in the financial sector,
41:29
we also see you know, the graph for with access to all the tools
41:32
real time information is actually one of the most popular
41:35
AI's out there. So you know all graph for is
41:38
also going to be available on the hyper scalars.
41:40
So the X AI enterprise sector is only, you know, started two
41:45
months ago and we're open for business.
41:51
Yeah. So the other thing we talked a
41:53
lot about, you know, having Grog to make games, video games.
41:57
So Danny is actually a video game designers on X.
42:00
So, you know, we mentioned, hey, who want to try out some grog
42:05
for preview APIs to make games. And then he answered the call.
42:09
So this was actually just made first person shooting game in
42:13
the span of four hours. So some of the actually the
42:17
unappreciated hardest problem of making video games is not
42:21
necessarily encoding the core logic of the game, but actually
42:24
go out source all the assets, all the textures of files and
42:28
and you know, to create a visual appealing game.
42:32
So one of the core aspects Rockford does really well with
42:34
all the tools out there is actually able to automate these
42:38
like assets sourcing capabilities.
42:40
So the developers can just focus on the core development itself
42:44
rather than like, you know, so now you can run a, you know,
42:46
entire game studios with game of one, like one person.
42:52
And then you can have Grok 4 to go out and source all those like
42:55
assets to automating tasks for you.
42:58
Yeah, the now the next step obviously is for Grok 2 play be
43:04
able to play the game. So it has to have very good
43:06
video understanding so it can play the games and interact with
43:08
the games and actually assess what whether a game is fun and,
43:12
and and actually have good judgement for whether a game is
43:15
fun or not. So with the, with version seven
43:19
of our foundation model, which finishes training this month,
43:21
and then we'll go through post training RL and whatnot that
43:26
will, that will have excellent video understanding.
43:29
And with the, with a video understanding and the, and
43:32
improved tool use. For example, for, for video
43:34
games, you'd want to use, you know, Unreal Engine or Unity or
43:38
one of the, one of the, the main graphics engines and then
43:43
generate the, generate the art, apply it to a 3D model and then
43:49
create an executable that someone can run on APC or, or a
43:52
console or, or a phone. Like we expect that to happen
43:59
probably this year and if not this year, certainly next year.
44:04
So that's it's going to be wild. I would expect the first really
44:10
good AI video game to be next year and probably the first half
44:20
hour of watchable TV this year and probably the first watchable
44:27
AI movie next year. Like things are really moving at
44:31
an incredible pace. Yeah, when Graca is connecting
44:35
world economy with vending machines, but we just create
44:37
video games for human. Yeah, I mean, it went from not
44:40
being able to do any of this really even six months ago to to
44:45
what you're seeing before you hear and and from from very
44:49
primitive a year ago to making a 3A sort of a 3D video game with
44:57
with a few hours of prompting. Yep.
45:01
I mean, yeah, just to recap. So in today's live stream, we
45:04
introduced the most powerful and most intelligent AI models out
45:08
there that can actually reason from the first principle using
45:11
all the tools. Do all the research, go on the
45:13
journey for 10 minutes, come back with the most correct
45:15
answer for you. So it's kind of crazy to think
45:19
about just like four months ago we had Grog 3 and now we already
45:23
have Grog 4 and we're going to continue accelerate as a company
45:26
XAI, we're going to be the fastest moving AGI companies out
45:29
there. So what's coming next is that
45:32
we're going to, you know, continue developing the model
45:35
that's not just, you know, intelligent smart thing for a
45:38
really long time spent a lot of compute.
45:40
But having a model that actually boasts fast and smart is going
45:44
to be the core focus, right? So if you think about what are
45:47
the applications out there that can really benefit from all
45:50
those very intelligent, fast and smart models and coding is
45:54
actually one of them. Yeah.
45:55
So the team is currently working very heavily on coding models.
45:59
I think right now the main focus is we actually trained recently
46:04
a specialized coding model, which is going to be both fast
46:07
and smart. And I believe we can share with
46:11
that model with you guys without you in a few weeks.
46:14
Yeah. Yeah, that's very exciting.
46:17
And you know, the second after coding is we all see the
46:21
weakness of Grog 4 is the multimodal capabilities.
46:26
So in fact, it was so bad that you know, Grog effectively just
46:30
like looking at the world squinking through the glass and
46:33
I can see all the blurry, you know, features and trying to
46:36
make sense of it. The most immediate improvement
46:40
we're going to see with the next generation preachment model is
46:42
that we're going to see a step function improvement on the
46:45
models capability in terms of image understanding, video
46:47
understanding and audios, right? It's now the models able to hear
46:51
and see the world just like any of you, right?
46:54
And now with all the tools at this command, with all the other
46:57
agents it can talk to, you know, so we're going to see a huge
47:02
unlock for many different application layers after the
47:06
multimodal agents. What's going to come after is
47:09
the video generation. And we believe that, you know,
47:12
at the other day, it should just be, you know, pixel in, pixel
47:15
out. And you know, imagine a world
47:20
where you have this influence scroll of content in inventory
47:24
on the X platform where not only you can actually watch these
47:28
generate videos, but able to intervene, create your own
47:32
adventures if you're just going to be wild.
47:36
And we expect to be training our video model with over 100,000 GB
47:40
2 hundreds and to begin that training within the next 3 or 4
47:45
weeks. So we're confident it's going to
47:48
be pretty spectacular in video generation and video
47:51
understanding. So let's see.
47:58
So that's anything you guys want to say.
48:03
Other than that, I guess that's it.
48:06
Yeah, it's, it's a good model, Sir.
48:08
It's a good, yeah. Well, we're very excited for you
48:11
guys to try Grok 4. Yeah, thank you.
48:14
All right, Thanks, everyone. Thank you.
48:15
Good night. Hey, thank you so much for
48:17
listening today. I really do appreciate your
48:20
support. If you could take a second and
48:21
hit the subscribe or the follow button on whatever podcast
48:24
platform that you're listening on right now, I greatly
48:27
appreciate it. It helps out the show
48:29
tremendously and you'll never miss an episode.
48:32
And each episode is about 10 minutes or less to get you
48:35
caught up quickly. And please, if you want to
48:38
support the show even more, go to patreon.com/stage Zero.
48:44
And please take care of yourselves and each other.
48:46
And I'll see you tomorrow.