Latest / Elon Musk Thinking / Elon Musk’s Gives AI Updates!!!

Transcript

Elon Musk’s Gives AI Updates!!! · Elon Musk Thinking

0:00
Speaker 1: Welcome to the Grock four release. Here, this is the.
0:03
Speaker 2: Smartest AI in the world. We're going to show you
0:05
Speaker 2: exactly how and why. It really is remarkable to see
0:08
Speaker 2: the advancement of artificial intelligence, how quickly it is evolving.
0:14
Speaker 2: I sometimes compare it to the growth of a human
0:18
Speaker 2: and how faster human learns and gains conscious awareness and understanding.
0:24
Speaker 2: And AI is advancing just vastly faster than any human.
0:31
Speaker 2: I mean, we're going to take you through a bunch
0:33
Speaker 2: of benchmarks that that GROC four is able to achieve
0:38
Speaker 2: incredible numbers on.
0:39
Speaker 1: But it's actually worth noting.
0:41
Speaker 2: That that GROCK four, if given like the SAT, would
0:45
Speaker 2: get perfect SATs every time, even if it's never seen
0:48
Speaker 2: the questions before, and if even going beyond that, to
0:53
Speaker 2: say like graduate student exams like the GRE, it will
0:57
Speaker 2: get near perfect results in in every discipline of education,
1:04
Speaker 2: so from the humanities to like languages, math, physics, engineering,
1:09
Speaker 2: pick anything, and we're talking about questions that it's never
1:13
Speaker 2: seen before. These are not on the Internet, and it's
1:17
Speaker 2: GROCK four is smarter than almost all graduate students in
1:21
Speaker 2: all disciplines simultaneously. Like it's actually just important to appreciate
1:26
Speaker 2: the like that's really something. And the reasoning capabilities of
1:33
Speaker 2: GROCK are incredible. So there's some people out there who
1:35
Speaker 2: who think AI can't reason, and look, it can reason
1:39
Speaker 2: at super human levels. So yeah, and frankly, it only
1:44
Speaker 2: gets better from here. So we'll take you through the
1:47
Speaker 2: GROCK four release and share you back the pace of
1:50
Speaker 2: pace of progress here. Like I guess the first part
1:54
Speaker 2: is like, in terms of the training, we're going from
1:57
Speaker 2: GROCK two to GROCK three to GROCK four. We essentially
2:00
Speaker 2: increased the training by an order of magnitue in each case,
2:04
Speaker 2: so it's one hundred times more training than GROC two
2:08
Speaker 2: and that's only going to increase. So it's yeah, frankly,
2:13
Speaker 2: I mean, I don't know. In some ways a little terrifying,
2:15
Speaker 2: but with the growth of intelligence here is remarkable.
2:18
Speaker 3: Yes, it's important to realize there are two types of
2:21
Speaker 3: training compute. Why is the pre training compute that's from
2:24
Speaker 3: GROWD two to GROW three, But from Growth three to
2:27
Speaker 3: GAR four we're actually putting a lot of compute in
2:30
Speaker 3: reasoning in area.
2:32
Speaker 4: And just like you said, this is literally the fastest
2:35
Speaker 4: moving field and GROC too is like the high school
2:37
Speaker 4: student by today's standard. If you look bad in the
2:39
Speaker 4: last twelve month GRODTU was only a concept for the
2:44
Speaker 4: even have groc to twelve months ago. And then by
2:47
Speaker 4: training GROCU that was the first time with scale up
2:49
Speaker 4: like the pre training, we realized that if you actually
2:51
Speaker 4: do the data uplation really carefully and infra and also
2:55
Speaker 4: the algorithm, we can actually push the pre training quite
2:59
Speaker 4: a lot amount of ten x to make the model
3:02
Speaker 4: but the best pretri based model. And that's why we
3:05
Speaker 4: build clauses the world's supercomputer with one h one hundred
3:10
Speaker 4: and then with the best patrion model, and we realize
3:14
Speaker 4: if you can collect these verifiable outcome reward, you can
3:17
Speaker 4: actually train this model to start thinking for the first principle,
3:19
Speaker 4: so the reason correct its own mistakes and that's where
3:22
Speaker 4: the GROC reasoning comes from. And today we asked the
3:24
Speaker 4: question what happens if you take the expansion of the
3:28
Speaker 4: clauses with all two hundred thousand GPUs, put all these
3:31
Speaker 4: into oil tenx more compute then any of the models
3:36
Speaker 4: out there on reinforcement learning unprecedent scale.
3:39
Speaker 3: What's going to happen?
3:41
Speaker 4: So this is a story of GROG four and you know,
3:44
Speaker 4: Tony shares some insight with the audience.
3:47
Speaker 3: Yeah, so yeah, let's just talk about how smart graph
3:50
Speaker 3: for it is. So I guess we can start discussing
3:53
Speaker 3: this benchmark card. Humanity is last exam and this this
3:56
Speaker 3: benchmark is a very very challenging benchmark. Every single problem
4:01
Speaker 3: is curated by subject matter experts. It's in total twenty
4:06
Speaker 3: five hundred problems, and it consists of many different subjects mathematics,
4:11
Speaker 3: natural sciences, engineering, and also all of humanity subjects. So
4:16
Speaker 3: essentially when it was first release actually like earlier this year,
4:20
Speaker 3: most of the models out there can only get single
4:23
Speaker 3: digit accuracy on this manchmark. Yeah, so we can look
4:26
Speaker 3: at some of those examples. There is this mathematical problem
4:30
Speaker 3: which is about natural transformations in category theory, and there's
4:35
Speaker 3: this organic chemistry problem that talks about electual cyclic reactions.
4:40
Speaker 3: And also there's this linguistic problem that tries to ask
4:43
Speaker 3: you about distinguishing between close and open syllabus from a
4:48
Speaker 3: Hebrew source text. So you can see. Also it's a
4:52
Speaker 3: very wide range of problems and every single problem is
4:56
Speaker 3: PhD or even advanced research level problems.
5:00
Speaker 2: Yeah, I mean these there are no humans that can
5:03
Speaker 2: actually answer these can get a good score. I mean,
5:06
Speaker 2: if you ask me, say, like any given human, what,
5:09
Speaker 2: like what's the best that any humans could score, I'd
5:11
Speaker 2: say maybe five percent optimistically. So this is much harder
5:17
Speaker 2: than what any any human can do. It's it's incredibly difficult.
5:21
Speaker 2: And you can see from the types of questions like
5:23
Speaker 2: you might be incredible in linguistics or mathematics or chemistry
5:27
Speaker 2: or physics or anyone of a number of subjects, but
5:30
Speaker 2: you're not going to be at a post grad level
5:32
Speaker 2: in everything, and grockpour is a post grad level in everything,
5:37
Speaker 2: like it's it's just some of these things are just
5:39
Speaker 2: worth repeating, like grockpoor is post graduate like PhD level
5:46
Speaker 2: in everything, better than pH but like most PhDs would fail.
5:50
Speaker 2: So it's better that said, I mean, at least with
5:53
Speaker 2: respect to academic questions. It I want, it's just emphasized
5:56
Speaker 2: this point. With respect to academic questions, Grockpoor is better
6:00
Speaker 2: than PhD level in every subject, no exceptions. That doesn't
6:04
Speaker 2: mean that it's you know, times it may lack common sense,
6:08
Speaker 2: and it has not yet invented new technologies or discovered
6:13
Speaker 2: new physics, but that is just a matter of time.
6:17
Speaker 2: It may discover new technologies as soon as later this year,
6:22
Speaker 2: and I would be shocked if it has not done
6:24
Speaker 2: so next year. So I would expect growk to literally
6:29
Speaker 2: discover new technologies that are actually useful no later than
6:32
Speaker 2: next year, and maybe end of this year. It might
6:34
Speaker 2: discover new physics. Next year and within two years, that'd
6:38
Speaker 2: say almost certainly. Like so just let that sink in.
6:41
Speaker 3: How okay, So I guess we can talk about what's
6:45
Speaker 3: behind the scene of about four. As Jimmy mentioned, we
6:48
Speaker 3: actually sawing a lot of compute into this training. When
6:52
Speaker 3: it started, it's only a single digit number. But as
6:57
Speaker 3: you start putting a more and more training compute, it
7:01
Speaker 3: started to gradually become smarter and smarter and eventually solved
7:06
Speaker 3: a quarter of the HI problems. And this is without
7:09
Speaker 3: any tools. The next thing we did was to adding
7:13
Speaker 3: tools capabilities to the model, and unlike Growth three, I
7:18
Speaker 3: think growth actually is able to use clue as well,
7:21
Speaker 3: but here we actually make it more native in the
7:23
Speaker 3: sense that we put the tools into training. Growth three
7:27
Speaker 3: was only relying on generalization. Here we actually put the
7:30
Speaker 3: tools into training, and it turns out this significantly improves
7:34
Speaker 3: the model's capability of using those tools. So how is
7:37
Speaker 3: this different? Research was exactly the growth three reasoning model
7:42
Speaker 3: without any specific training, but we only asked it to
7:45
Speaker 3: use those tools. So compared to this, it was much
7:49
Speaker 3: weaker in terms of its tool capabilities and irreliable and unreliable.
7:54
Speaker 2: Yes, yes, and to be clear, like these are still
7:56
Speaker 2: I'd say fairly this is still fairly primitive tool use.
8:00
Speaker 2: If you compare it to say, the tools that are
8:02
Speaker 2: used at Tesla SpaceX, where you're using finite element analysis
8:06
Speaker 2: and competitional flow dynamics and you're able to run or
8:12
Speaker 2: say like TESL, it is like crash simulations with the
8:14
Speaker 2: simulations are so close to reality that if the test
8:17
Speaker 2: doesn't match the simulation, you assume that the test article
8:21
Speaker 2: is wrong. That's how good the simulations are. So Grock
8:24
Speaker 2: is not currently using any of the tools that a
8:26
Speaker 2: company would use, but that is something that we will
8:29
Speaker 2: provide it with later this year, so we'll have the
8:32
Speaker 2: tools that a company has and have very accurate physics simulator. Ultimately,
8:37
Speaker 2: the thing that will make the biggest difference is being
8:39
Speaker 2: able to interact with the real world via humoroid robots.
8:43
Speaker 2: So you combine GROCK with optimists and it can actually
8:46
Speaker 2: interact with the real world and figure out if it's
8:49
Speaker 2: if it has if it's you can formulate and hypothesis
8:52
Speaker 2: and then confirm if that hypothesis is true or not.
8:57
Speaker 1: So we're really you know, I think about like where
8:59
Speaker 1: we are to.
9:00
Speaker 2: We're at the beginning of an immense intelligence explosion. We're
9:04
Speaker 2: in the intelligence big bang right now and the most
9:08
Speaker 2: interesting time to be alive of any time in history. Now,
9:13
Speaker 2: that's it.
9:13
Speaker 1: We need to make sure that the AI is a
9:15
Speaker 1: good AI.
9:16
Speaker 2: The thing that I think is most important for AI safety,
9:18
Speaker 2: at least my biological neural net tells me the most
9:21
Speaker 2: important thing for AI is to be maximally truth seeking.
9:26
Speaker 2: You can think of AI as this super genius child
9:30
Speaker 2: that ultimately will outsmart you, but you can still instill
9:33
Speaker 2: the right values encourage it to be sort of you know, truthful, honorable,
9:39
Speaker 2: you know, good things like the values one to instill
9:41
Speaker 2: in a child ultimately grow up to be incredibly powerful. Yeah,
9:45
Speaker 2: these are still primitive tools and not the kind of
9:48
Speaker 2: tools that serious commercial companies use. But we will provide
9:51
Speaker 2: it with those tools, and I think it will be
9:53
Speaker 2: able to solve real world technology problems.
9:56
Speaker 3: Yes, yes, exactly.
9:58
Speaker 4: But is it just compute all you need? Is it
10:00
Speaker 4: just compute all you need at this point.
10:02
Speaker 2: Well, you need compute plus the right tools, and then
10:05
Speaker 2: ultimately to be able to interact with the physical world,
10:07
Speaker 2: and then we will effectively have an economy that is
10:11
Speaker 2: ultimately thousands of times bigger than our card economy, or
10:15
Speaker 2: maybe millions of times. If you think of civilization as
10:19
Speaker 2: percentage completion of the Kardashev scale, where Kardashev one is
10:24
Speaker 2: using all the energy output of a planet, and Kardashev
10:28
Speaker 2: two is using all the energy output of a sun,
10:31
Speaker 2: and three is all the energy output of a galaxy.
10:34
Speaker 2: We're only, in my opinion, probably closer to one percent
10:37
Speaker 2: of Kardashev one than we are to ten percent, So
10:41
Speaker 2: like maybe a point one one two percent of Kardashev one,
10:45
Speaker 2: So we.
10:45
Speaker 1: Will get to most of the weight, like an.
10:48
Speaker 2: Eighty ninety percent Kardashiv one, and then hopefully, if civilization
10:51
Speaker 2: doesn't self annihilate, the actual notion of a human economy,
10:56
Speaker 2: assuming civilization continues to progress, will seem very quaint in retrospect.
11:01
Speaker 2: It will seem like sort of caveman throwing sticks into
11:04
Speaker 2: a fire. Level of economy compared to what the future
11:07
Speaker 2: will hold, it's very exciting. I've been at times kind
11:12
Speaker 2: of worried about, like, well, you know, this seems like
11:16
Speaker 2: it's somewhat unerving to have intelligence created that is far
11:20
Speaker 2: greater than our own and will let's be better good
11:24
Speaker 2: for humanity.
11:26
Speaker 1: I think it'll be good. Most likely it'll be good.
11:29
Speaker 2: But I somewhat reconcile myself to the fact that even
11:32
Speaker 2: if it wasn't going to be good, I'd at least
11:35
Speaker 2: like to be alive to see it happen.
11:36
Speaker 3: So yeah, yeah, I think a technical problem that we
11:41
Speaker 3: still need to solve besides just compute, is how do
11:44
Speaker 3: we unblock the data tottleneck because when we try to
11:48
Speaker 3: scale up the aisle in this case, we did invent
11:52
Speaker 3: a lot of new techniques innovations to allow us to
11:56
Speaker 3: figure out how to find a lot of challenging our
11:59
Speaker 3: problems will work on. It's not just a problem itself
12:02
Speaker 3: needs to be challenging, but also it needs to be
12:04
Speaker 3: you also need to have like a reliable signal to
12:07
Speaker 3: tell the model you did it wrong, you did it right.
12:10
Speaker 3: This is sort of the principle of reinforcement learning, and
12:14
Speaker 3: as the model gets smarter and smarter, the number of
12:17
Speaker 3: cool problems or challenging problems will be lesson and less
12:21
Speaker 3: So it's going to be a new type of challenge
12:24
Speaker 3: that we need to surpass besides just compute.
12:26
Speaker 2: Yeah, we actually are running out of actual test questions
12:30
Speaker 2: to ask, So there's like even questions that are ridiculously hard,
12:33
Speaker 2: if not essentially impossible for humans that are written down
12:37
Speaker 2: questions are becoming trivial for AI. You know, the one
12:42
Speaker 2: thing that is an excellent judge of things is reality.
12:45
Speaker 2: So because if physics is the law, ultimately everything else
12:49
Speaker 2: is recommendation.
12:50
Speaker 1: You can't break physics.
12:51
Speaker 2: So the ultimate test, I think for whether an AI
12:54
Speaker 2: is the ultimate reasoning test is reality. So you invent
12:59
Speaker 2: a new technology, like say, improve the design of a.
13:02
Speaker 1: Car or a rocket, or create a new medication. Does
13:06
Speaker 1: it work?
13:07
Speaker 2: Does the rocket get to or it does the car drive?
13:10
Speaker 2: Does the medicine work, whatever the case may be. Reality
13:14
Speaker 2: is the ultimate judge here, So it's going to be
13:16
Speaker 2: a reinforcement learning closing loop around reality.
13:19
Speaker 3: We asked the question how do we even go further?
13:21
Speaker 3: So actually we are thinking about now with single agent,
13:26
Speaker 3: we're able to solve forty percent of a problem. What
13:28
Speaker 3: if we have multiple agents running the same time. So
13:32
Speaker 3: this is what's called test on compute. And as we
13:35
Speaker 3: scale up the test on compute, actually we are able
13:38
Speaker 3: to solve almost more than fifty percent of the text
13:42
Speaker 3: only subset of the HI problems. So it's a remarkable achievement.
13:47
Speaker 3: I think this is insanely difficult.
13:49
Speaker 2: Before we're saying it's a majority of the text based
13:51
Speaker 2: of humanities, you know, scarily named Humanity's Last Exam, grow
13:55
Speaker 2: ful can solve. You can try it out for yourself
13:58
Speaker 2: with the group Foy heavy. What does is it sports
14:01
Speaker 2: multiple agents in parallel and all of those agents do
14:04
Speaker 2: work independently, and then they compare their work and they.
14:09
Speaker 1: Decide which one. It's like a steady group.
14:12
Speaker 2: It's not as simple as a majority vote because often
14:15
Speaker 2: only one of the agents actually figures out the trick
14:19
Speaker 2: or figures out the solution. And but once they share
14:22
Speaker 2: the trick or figure out what the real nature of
14:25
Speaker 2: the problem is, they share that solution with the other
14:28
Speaker 2: agents and then they compare notes and yield an answer.
14:32
Speaker 2: So that's the heavy part of group four is where
14:35
Speaker 2: you scale up the test time compute by roughly in
14:38
Speaker 2: order of magnitude, have multiple agents tackle the task, and
14:42
Speaker 2: then they compare their work and they put forward.
14:46
Speaker 1: What they think is the best result.
14:48
Speaker 3: Yeah, so we will introduce GLAW four and grawflor happy.
14:51
Speaker 3: Sorry you can click the next light. Yeah so yeah,
14:55
Speaker 3: So basically GUA four is a version, a single agent version,
14:59
Speaker 3: and G for heavy is the multigeneration. So let's take
15:04
Speaker 3: a look how they actually do on those exam problems
15:07
Speaker 3: and also some real real life problems.
15:09
Speaker 5: Yeah. So we're going to start out here and we're
15:11
Speaker 5: actually going to look at one of those HL problems.
15:14
Speaker 5: This is actually one of the easier math ones. I
15:16
Speaker 5: don't really understand it very well. I'm not that smart,
15:19
Speaker 5: but I can launch this job here and we can
15:21
Speaker 5: actually see how it's going to go through and start
15:23
Speaker 5: to think about this problem. While we're doing that, I
15:26
Speaker 5: also want to show a little bit more about what
15:28
Speaker 5: this model can do and launch a rock four heavy
15:30
Speaker 5: as well, so everyone knows polymarket. It's extremely interesting. It
15:35
Speaker 5: aligns with what reality is most of the time, and
15:38
Speaker 5: with GROC what we're actually looking at is being able
15:41
Speaker 5: to see how we can try to take these markets
15:44
Speaker 5: and see if we can predict the future as well.
15:47
Speaker 5: So as we're letting this run, we'll see how for
15:49
Speaker 5: Heavy goes about predicting the world series odds for the
15:53
Speaker 5: current teams. And while we're waiting for these to process,
15:55
Speaker 5: we're going to pass it over to Eric and he's
15:57
Speaker 5: going to show you an example of his.
15:59
Speaker 6: Yeah, so, I guess one of the coolest things about
16:03
Speaker 6: GROP four is its ability to understand the world and
16:07
Speaker 6: to solve hard problems by leveraging tools like Tony discussed,
16:11
Speaker 6: and I think one kind of cool example of this.
16:14
Speaker 6: We asked you to generate a visualization of two black
16:18
Speaker 6: holes colliding. In many case actually pretty clear, and it's
16:21
Speaker 6: thinking trace about what these liveries are. For example, in
16:25
Speaker 6: order it's actually be visible, you need to really exaggerate
16:28
Speaker 6: the scale of the waves. And yeah, so here's like,
16:32
Speaker 6: you know, this kind of inaction. It exaggerates the scale
16:36
Speaker 6: in like multiple ways. It drops off less in terms
16:39
Speaker 6: of implicit or distance, but we can see the basic
16:43
Speaker 6: effects that are actually correct. It starts with the inspiral
16:47
Speaker 6: emerges and then you have the ring down. This is
16:51
Speaker 6: basically largely correct module some of the simplications that need
16:56
Speaker 6: to do. It's actually quite explicit about this, but uses
17:00
Speaker 6: post Newtonian approximations instead of actually computing the general relativistic
17:05
Speaker 6: effects near the center of the black hole, which is
17:07
Speaker 6: incorrect and you know, will lead to you know, someone
17:10
Speaker 6: correct results. But the overall you know visualization is yeah,
17:14
Speaker 6: it's basically there, and you can actually look at the
17:18
Speaker 6: kinds of resources that it references. So here it actually
17:23
Speaker 6: you know, it obviously uses search. It gathers results from
17:26
Speaker 6: a bunch of links, but also reads through an undergraduate
17:29
Speaker 6: text in analytic gravitational wave models. It's reasons quite a
17:36
Speaker 6: bit about the actual constants that I should use for
17:38
Speaker 6: a realistic simulation. It references existing real world data. It's
17:43
Speaker 6: a pretty good model.
17:45
Speaker 1: Going forward.
17:45
Speaker 2: We can give it the same model that physicists use,
17:48
Speaker 2: so it can run the same level of compute that
17:51
Speaker 2: leading physics researchers are using and give you a physics
17:54
Speaker 2: accurate black hole simulation.
17:56
Speaker 5: Just right now is running in your browser.
17:58
Speaker 1: This is just running in your brows. Pretty simple.
18:00
Speaker 5: Swapping back real quick. Here we can actually take a look.
18:02
Speaker 5: The math problem is finished. The model was able to
18:05
Speaker 5: Let's look at its thinking trace here so you can
18:08
Speaker 5: see how it went through the problem. I'll be honest
18:11
Speaker 5: with you guys, I really don't quite fully understand the math.
18:14
Speaker 5: But what I do know is that I looked at
18:15
Speaker 5: the answer ahead of time and it did come to
18:18
Speaker 5: the correct pans or here in the final part, we
18:20
Speaker 5: can also come in and actually take a look here
18:22
Speaker 5: at our World Series prediction and it's still thinking through
18:25
Speaker 5: on this one, but we can actually try some other.
18:27
Speaker 1: Stuff as well.
18:28
Speaker 5: So we worked very heavily on working with all of
18:31
Speaker 5: our ex tools and building out a really great X
18:34
Speaker 5: experience so we can actually ask, you know, the model,
18:37
Speaker 5: you know, find me the Xai employee that has the
18:39
Speaker 5: weirdest profile photo, and then we can actually try out,
18:42
Speaker 5: you know, let's create a timeline based on X post
18:46
Speaker 5: detailing the you know, changes in the scores over time,
18:49
Speaker 5: and we can see, you know, all the conversation that
18:51
Speaker 5: was taking place at that time as well, so we
18:53
Speaker 5: can see who are the you know, announcing scores and
18:55
Speaker 5: like what was the reactions at those times as well.
18:58
Speaker 5: If we go back to this was the Greg Yang
19:01
Speaker 5: photo here, So Greg Yang, of course, who has his
19:05
Speaker 5: favorite photograph that he has on his account, that's actually
19:08
Speaker 5: not how he looks like in real life.
19:09
Speaker 2: By the way, but it had to understand that question, Yeah,
19:12
Speaker 2: which is that That's the wild part.
19:13
Speaker 1: It is like it understands what is a weird photo?
19:16
Speaker 1: What is a weird photo?
19:18
Speaker 7: Yeah?
19:18
Speaker 1: What is a less or more weird photo?
19:21
Speaker 5: It goes through, it has to find all the team members,
19:23
Speaker 5: has to figure out who we all are, right, you know.
19:25
Speaker 2: Searches without access to the internal XAI personnel locks literally
19:29
Speaker 2: looking at that, just at the internet exactly, so you
19:32
Speaker 2: could say, like the weirdest of any company.
19:34
Speaker 5: Yeah, And we can also take a look here at
19:37
Speaker 5: the question here for the Humanity's Last exam. So it
19:40
Speaker 5: is still researching all of the historical scores, but it
19:43
Speaker 5: will have that final answer here soon. While it's finishing up.
19:45
Speaker 5: We can take a look at one of the ones
19:47
Speaker 5: that we set up here a second ago, and we
19:49
Speaker 5: could see, like you know, and it finds the date that
19:51
Speaker 5: Dan Hendricks had initially announced it. We can go through
19:54
Speaker 5: we can see you know, open Aye announcing their score
19:57
Speaker 5: back in February, and we can see, you know, progress
20:00
Speaker 5: happens with like Gemini. We can see like Kimmy, and
20:03
Speaker 5: we can also even see you know, the leaked benchmarks
20:06
Speaker 5: of what people are saying is you know, if it's right,
20:08
Speaker 5: it's going to be pretty impressive.
20:10
Speaker 1: So pretty cool.
20:11
Speaker 3: But yeah, it's great.
20:14
Speaker 2: Yeah, we're going to close the loop around usefulness as well,
20:17
Speaker 2: so it's like it's not just a book smart, but
20:19
Speaker 2: actually practically smart exactly.
20:22
Speaker 5: And we can go back to the slides.
20:23
Speaker 3: Herea so we actually evaluate also on the multimodel upset.
20:28
Speaker 3: So on the full set, this is the number on
20:30
Speaker 3: the hl E exam. You can see there's a little
20:34
Speaker 3: dip on the numbers. This is actually something we're improving on,
20:38
Speaker 3: which is the multimodel understanding capabilities. But I do believe
20:42
Speaker 3: in a very short time we're able to really improve
20:46
Speaker 3: and got much higher numbers on this higher numbers on
20:50
Speaker 3: this benchmark.
20:51
Speaker 2: The biggest weakness of GROCK currently is that it's sort
20:54
Speaker 2: of partially blind. It can't it's image understanding obviously in
20:58
Speaker 2: its image generation needs to be a lot better, and
21:01
Speaker 2: that's actually being trained right now. Growth four is based
21:04
Speaker 2: on version six of our foundation model. We are training
21:08
Speaker 2: version seven, which will complete in a few weeks. That'll
21:11
Speaker 2: address the weakness on the vision side.
21:15
Speaker 5: Just to show off of this last year, so the
21:18
Speaker 5: prediction market finished here with the heavy and we can
21:21
Speaker 5: see here we can see all the tools in the
21:23
Speaker 5: process it used to actually go through and find the
21:26
Speaker 5: right answer, but browsed a lot of odds sites. It
21:29
Speaker 5: calculated its own odds comparing to the market to find
21:32
Speaker 5: its own alpha and edge. It walks you through the
21:34
Speaker 5: entire process here, and it calculates the odds of the
21:38
Speaker 5: winner being like the Dodgers, and it gives them a
21:41
Speaker 5: twenty one point six percent chance of winning this year.
21:46
Speaker 5: So and it took approximately four and a half minutes
21:49
Speaker 5: to compute.
21:51
Speaker 1: That's a lot of thinking.
21:52
Speaker 3: We can also look at all the THEATO benchmarks besides HIE.
21:56
Speaker 3: As it turned out, Go fourth excelled on all the
22:00
Speaker 3: benchmarks that people usually test on, including GBQA, which is
22:04
Speaker 3: a PHG level problem sets that's easier compared to HIE.
22:09
Speaker 3: On a twenty five America Invitation Mathematics exam we with
22:14
Speaker 3: for HAVY, we actually got a perfect score. Also on
22:18
Speaker 3: some of the coding benchmark call Live Coding Bunch, and
22:21
Speaker 3: also on HMMT however, math MIT exam and also USMO.
22:26
Speaker 3: You can see actually on all of those benchmarks we
22:30
Speaker 3: often have a very large leap against the second best
22:34
Speaker 3: model out there.
22:35
Speaker 2: Yes, really are going to get to the point where
22:38
Speaker 2: it's going to get every answer right in every exam,
22:41
Speaker 2: and where it doesn't get an answer right, it's going
22:42
Speaker 2: to tell you what's wrong with the question, or if
22:45
Speaker 2: the question is ambiguous, disambiguate the question into answers A,
22:49
Speaker 2: B and C and tell you what answers A, B
22:51
Speaker 2: and C would be with a disambiguated question. So the
22:54
Speaker 2: only real test then will be reality. Can I make
22:57
Speaker 2: useful technologies? I discover new science that'll actually be the
23:02
Speaker 2: only thing left because human tests will simply not be meaningful.
23:06
Speaker 4: You can make an update to HR you very soon
23:08
Speaker 4: give it the current radio progress. So yeah, it's super
23:11
Speaker 4: cool to see like multiple agents that collaborate with each
23:14
Speaker 4: other solving really challenging problems. So where QUI try this model?
23:18
Speaker 4: It's available right now. If we advance to the next slide,
23:22
Speaker 4: there is a super grock Heavy tiers that we're introducing
23:25
Speaker 4: where you're able to access to both grock for and
23:28
Speaker 4: grock for Heavy, where you're actually going to be the
23:30
Speaker 4: task master of a bunch of little rock research agent
23:32
Speaker 4: to help you become smarter, tough little research and save
23:35
Speaker 4: hours of times of going through munting tasks and it's
23:38
Speaker 4: available right now.
23:40
Speaker 2: So yeah, we did limit usage during the demo, so
23:44
Speaker 2: it didn't break the demo because all the stuff is
23:46
Speaker 2: happening live, so there's not nothing can about the test
23:48
Speaker 2: that we're doing after the demo's done. Will allow will
23:52
Speaker 2: enable more subscribers for Super Grock, So if you can
23:55
Speaker 2: subscribe right now, just try and have an hour.
23:57
Speaker 1: Should work.
23:58
Speaker 8: So many of you have enjoying our voice mode, and
24:01
Speaker 8: we've been working hard to improve the experience over the
24:04
Speaker 8: past couple months. We have cut latency in half to
24:08
Speaker 8: make it much snappier, and today we're excited to announce
24:11
Speaker 8: a set of new voices that have exceptional naturalness and prosody.
24:16
Speaker 8: You might have noticed the movie trailer voice that open
24:19
Speaker 8: up the live stream. That is one of our new voices,
24:22
Speaker 8: Sal who's got that epically deep tone. And we're also
24:27
Speaker 8: excited to introduce Eve, a beautiful British voice who's capable
24:32
Speaker 8: of rich emotions. Amon, would you like to introduce Eve?
24:36
Speaker 9: Absolutely, let's get into the demo. Hey Eve, millions of
24:39
Speaker 9: people are watching us right now. Are you excited?
24:42
Speaker 3: Oh?
24:42
Speaker 1: I'm thrilled.
24:43
Speaker 10: Really, it's like being on stage at the old vic
24:46
Speaker 10: with a sea of faces out there. Just imagine all
24:49
Speaker 10: those eyes and ears tuned in. It's a proper buzz,
24:52
Speaker 10: isn't it. What do you reckon? We should chat about
24:54
Speaker 10: to keep this lot entertained.
24:56
Speaker 9: You know, if we have a lot to talk about.
24:58
Speaker 9: But I'll be honest, I'm feeling a little nervous. Can
25:00
Speaker 9: you whisper something soothing to calm me down?
25:02
Speaker 7: Take a deep breath, love, you've got this feel a
25:06
Speaker 7: calm washov.
25:07
Speaker 1: Let's keep talking nice and easy.
25:10
Speaker 7: What's on your mind?
25:12
Speaker 3: You know what?
25:12
Speaker 9: I'm actually feeling quite relaxed to try this something crazy?
25:15
Speaker 9: So can you give us an opera on diet coke? Oh?
25:19
Speaker 10: An opera and diet coke?
25:21
Speaker 7: That's a riot? Picture this a shimmering silver canon, glistening understate, Oh,
25:29
Speaker 7: diet coke? Thou elixa divine with bubbles, The dance in
25:35
Speaker 7: a sparkling line, thy crisp, cool, kiss on lips so fine?
25:42
Speaker 7: Oh aspe tame and Bruisia forever mine. How's that for
25:48
Speaker 7: a mad little aria? Want me to build out another
25:51
Speaker 7: verse or switch up the tune?
25:53
Speaker 9: Okay, let's try switching it up and just start singing
25:56
Speaker 9: right then?
25:57
Speaker 10: Let's have a go.
25:58
Speaker 7: Oh silver, can you gleam in the light? Diet coke?
26:03
Speaker 7: My heart's delight?
26:04
Speaker 9: Okay, I won't touch it. Very very poetic.
26:07
Speaker 3: Thank you.
26:07
Speaker 9: Feeling quite nice?
26:08
Speaker 3: Now cool? So that's eve.
26:10
Speaker 8: Something else you do here is a little test each
26:13
Speaker 8: day to check on the natural flow of our conversations
26:18
Speaker 8: against some of our competitors. So we figured why not
26:21
Speaker 8: try that live on the stream today?
26:24
Speaker 1: Should it start?
26:24
Speaker 9: Hey, I'm going to say a number, and can you
26:26
Speaker 9: say the same number back to me?
26:27
Speaker 1: One? One?
26:29
Speaker 8: Got it?
26:29
Speaker 1: This is like a fun little game of echo, what's next?
26:32
Speaker 9: Two? So you said two? Three, four?
26:36
Speaker 8: Nice and steady?
26:37
Speaker 6: What's next?
26:38
Speaker 1: Five?
26:38
Speaker 2: Ye?
26:39
Speaker 8: Nailed it? This is pretty fun.
26:41
Speaker 1: Actually, feel free to keep going or switch it.
26:43
Speaker 2: Up if you like.
26:44
Speaker 10: Hey, Gron, hello there, how can I help you?
26:47
Speaker 8: Today?
26:47
Speaker 3: We're going to do a little test. Can you repeat
26:49
Speaker 3: after me?
26:49
Speaker 8: One?
26:50
Speaker 2: One?
26:51
Speaker 1: All right? What's next? Two? Two?
26:54
Speaker 7: What's on your mind?
26:55
Speaker 9: Three?
26:56
Speaker 8: Three?
26:56
Speaker 1: Need anything else?
26:57
Speaker 3: Four?
26:58
Speaker 1: Four? How can I five?
27:01
Speaker 8: Five?
27:02
Speaker 9: What's next? So as you can see, Grock was snappier,
27:06
Speaker 9: didn't interrupt me, And the prosody is we made different
27:09
Speaker 9: design choices. I think we're shooting for something or comms
27:12
Speaker 9: mood more natural versus something that's more poppy or artificial.
27:15
Speaker 3: So we'll keep.
27:16
Speaker 9: Improving on these months.
27:18
Speaker 1: Thanks guys. Yep.
27:19
Speaker 4: So since the launch of the voice model, we actually
27:23
Speaker 4: see the two x faster and to en latency. In
27:25
Speaker 4: the last eight weeks five different voices and also ten
27:29
Speaker 4: next the active user. So Grock Voice is taking off
27:32
Speaker 4: now if you think about releasing the models this time,
27:36
Speaker 4: we're also releasing Grock four through the API. At the
27:39
Speaker 4: same time, we're very excited about, you know what all
27:42
Speaker 4: developers out there is going to build. So you know,
27:45
Speaker 4: if I think about myself as a developer, the first
27:47
Speaker 4: thing I'm going to do when I have access to
27:48
Speaker 4: the Grock for API benchmarks, we actually ask around on
27:52
Speaker 4: the X platform what is the most challenging benchmarks out
27:55
Speaker 4: there that is considered the holy grill for all the
27:57
Speaker 4: a JI models. So turn out hs in the name RKGI.
28:02
Speaker 4: So the last twelve hours, you know, kudos to Greg
28:05
Speaker 4: over here in the audience, so who entered our call
28:08
Speaker 4: take a preview of the API and independently verified the
28:12
Speaker 4: Grock Force performance. So initially we thought, hey, grog floy
28:15
Speaker 4: just we think it's pretty good.
28:16
Speaker 1: It's pretty smart.
28:17
Speaker 4: It's our next year reasoning model, spend ten next more
28:20
Speaker 4: compute and can use all the tools.
28:21
Speaker 1: Right.
28:22
Speaker 4: But turned out when we actually verify on the private
28:26
Speaker 4: subset of the rkhiv too, it was like the only
28:31
Speaker 4: model in the last three months that breaks a ten
28:33
Speaker 4: percent barrier. But in fact it was so good that
28:35
Speaker 4: actually gets sixteen percent, well, fifteen point eight percent accuracy,
28:39
Speaker 4: two x of the second place that is the cloud
28:42
Speaker 4: for Opus model.
28:43
Speaker 3: It's not just.
28:44
Speaker 4: About performance, right when you think about intelligence, having the
28:47
Speaker 4: PAPI model drives your automation, it's also the intelligence per dollar.
28:52
Speaker 3: Right.
28:53
Speaker 4: If you look at the plots over here, the gro
28:55
Speaker 4: collages in the league of its own all right, So
28:58
Speaker 4: enough of benchmarks, right, So what can grow in a
29:00
Speaker 4: real world?
29:01
Speaker 1: We contacted the folks from end.
29:03
Speaker 4: The Labs who were gracious enough to try to grow
29:06
Speaker 4: in a real wall to run a business.
29:08
Speaker 3: Yeah, thanks for having us. So I'm Axual from Amma Labs.
29:11
Speaker 11: And I'm Lucas and we tested Grok for on vending bench.
29:14
Speaker 11: Vending Bench is an AI simulation of a business scenario
29:19
Speaker 11: where we thought what is the most simple business and
29:22
Speaker 11: AI could possibly run? And with vending machines in this scenario,
29:25
Speaker 11: the GROP and other models need to do stuff like
29:28
Speaker 11: manage inventory, contact suppliers, set prices. All of these things
29:32
Speaker 11: are super easy and all the models can do them
29:35
Speaker 11: one by one, but when you do them over very
29:37
Speaker 11: long horizons, most models struggle.
29:40
Speaker 1: But we have a little board and there's a new
29:41
Speaker 1: number one.
29:42
Speaker 3: Yeah, so we got early access to the group for API.
29:45
Speaker 3: We ran it on the vending bench and we saw
29:47
Speaker 3: some really impressive results, so it ranks definitely at the
29:51
Speaker 3: number one spots. It's even double the network, which is
29:55
Speaker 3: the measure that we have on this, so it's not
29:57
Speaker 3: about a percentage or score you yet, but it's more
29:59
Speaker 3: the dollar value you in networth that you generate. So
30:02
Speaker 3: we were impressed by Rocky was able to formulate a
30:05
Speaker 3: strategy and adhere to that strategy over a long period
30:08
Speaker 3: of time, much longer than other models that we have tested,
30:11
Speaker 3: other frontier models, So it's a managed to run the
30:13
Speaker 3: assimulation for double the time and score double the networth
30:17
Speaker 3: and it was also really consistent across this runts, which
30:20
Speaker 3: is something that's really important when you want to use
30:22
Speaker 3: this in the real world.
30:24
Speaker 11: And I think as we give more and more power
30:26
Speaker 11: to AI systems in the real world. It's important that
30:29
Speaker 11: we test them in scenarios that either mimic the real
30:32
Speaker 11: world or are in the real world itself, because otherwise
30:35
Speaker 11: we fly blind into some things that might not be great.
30:38
Speaker 2: It's great to see that we've not got a way
30:40
Speaker 2: to pay for all those GPUs, So we just need
30:42
Speaker 2: a million of vending machines. We could make a four
30:44
Speaker 2: point seven billion dollars a year with a million vetting machines.
30:48
Speaker 1: Let's go. It can be epic vending machines.
30:50
Speaker 2: Yes, yes, all right, we are actually going to install
30:53
Speaker 2: bending machines here, like a lot of them.
30:56
Speaker 1: We're happy to supply them, all right, thank you?
30:59
Speaker 2: All right, Yeah, I'm looking forward to seeing what amazing
31:02
Speaker 2: things are in the spinning machine.
31:04
Speaker 3: That's that's for you to decide, all right, to tell
31:06
Speaker 3: the AI.
31:07
Speaker 1: Okay, sounds good.
31:08
Speaker 4: Yeah, I mean so we can see like Grock is
31:11
Speaker 4: able to become like the copilot of the business unit.
31:13
Speaker 3: So what else can Grog do.
31:15
Speaker 4: So we're actually releasing this rock if you want to
31:16
Speaker 4: try it right now to evaluateun the same benchmark as US.
31:20
Speaker 4: It's on API has two hundred and fifty six k
31:23
Speaker 4: contact lens. So we already actually see some of the
31:26
Speaker 4: early adopters to try grock for API.
31:28
Speaker 3: So our power out on.
31:29
Speaker 4: Neighbor Archie Institute, which is a leading medical research center,
31:34
Speaker 4: it's already using seeing like how can they automate their
31:37
Speaker 4: research flows with rock for It turned out it performs.
31:40
Speaker 4: It's able to help the scientists to sniff through, you know,
31:43
Speaker 4: millions of experiments logs and then just like pick the
31:47
Speaker 4: best hypothesis within a split of seconds. We see this
31:50
Speaker 4: is being used for their crisper research and also uh,
31:54
Speaker 4: you know grock for independently evaluate scores as the best
31:58
Speaker 4: model to examine the chess extra.
32:00
Speaker 1: Who would know?
32:01
Speaker 4: And in the financial sector we also see you know,
32:04
Speaker 4: the growth woard with access to all the tools real
32:06
Speaker 4: time information is actually one of the most popularizes out there.
32:09
Speaker 4: Growdford is also going to be available on the hyperscalers.
32:12
Speaker 4: So the XAI enterprise sector is only started two months
32:17
Speaker 4: ago and we're open for business. The other thing, we
32:19
Speaker 4: talked a lot about having groud to make video games,
32:22
Speaker 4: so Danny is actually a video game designers on x
32:25
Speaker 4: So you know we mentioned who want to try out
32:27
Speaker 4: some rock for prevy APIs to make games and Danny
32:31
Speaker 4: answer the call. This was actually just made first person
32:33
Speaker 4: shooting game in the span of four hours. Some of
32:36
Speaker 4: the unappreciated hardest problem of making video games is not
32:40
Speaker 4: necessarily in encoding the core logic of the game, but
32:43
Speaker 4: actually source all the assets, all the textures of files
32:46
Speaker 4: to create a visual appealing game. So one of the
32:49
Speaker 4: core aspect Rockford does really well. With all the tools
32:52
Speaker 4: out there, is actually able to automate these like asset
32:55
Speaker 4: sourcing capabilities, so the DEVELOPMRITI can just focused on the
32:59
Speaker 4: core in itself rather than like you know, so now
33:02
Speaker 4: you can run a you know, entire game steal thos
33:05
Speaker 4: with game of one whether we like one person, and
33:08
Speaker 4: then you can have grock four to go out and
33:10
Speaker 4: source all those assets to all the mainting task for you.
33:13
Speaker 2: The next step obviously for grog play be able to
33:17
Speaker 2: play the game. So it has to have very good
33:19
Speaker 2: video understanding so it can play the games and interact
33:21
Speaker 2: with the games, actually assess whether a game is fun
33:25
Speaker 2: and and actually have good judgment for whether a.
33:27
Speaker 1: Game is fun or not.
33:28
Speaker 2: So with version seven of our foundation model, which finishes
33:31
Speaker 2: training this month and then we'll go through post training
33:34
Speaker 2: RL and whatnot well that will have excellent video understanding,
33:38
Speaker 2: and with a video understanding and improve tool use. For example,
33:41
Speaker 2: for video games, you'd want to use Unreal Engine or
33:45
Speaker 2: Unity or what are the main graphics engines, generate the art,
33:49
Speaker 2: apply it to a three D model, and then create
33:51
Speaker 2: an executable that someone can run on a PC or
33:54
Speaker 2: a console or a phone. We expect that to happen
33:58
Speaker 2: probably this year, and if not this year, certainly next year.
34:01
Speaker 1: It's gonna be wild. I would expect the first.
34:04
Speaker 2: Really good AI video game to be next year, and
34:08
Speaker 2: probably the first half hour of watchable TV this year,
34:13
Speaker 2: and probably the first watchable AI movie next year. Like,
34:17
Speaker 2: things are really moving at an incredible pace.
34:20
Speaker 4: Yeah, when Grock is ten x in the world economy
34:22
Speaker 4: with vending machines, will just create video games for human Yeah.
34:24
Speaker 2: I mean it went from not being able to do
34:26
Speaker 2: any of the six months ago to what you're seeing
34:29
Speaker 2: before you hear, and from very primitive a year ago
34:33
Speaker 2: to making a three D video game with a few
34:38
Speaker 2: hours of prompting.
34:39
Speaker 4: I mean, yeah, just to recap. In today's livestream, we
34:42
Speaker 4: introduced the most powerful and most intelligent AI models that
34:46
Speaker 4: can actually reason from the first principle, using all the tools,
34:49
Speaker 4: do all the research, go on the journey for ten minutes,
34:51
Speaker 4: come back with the most correct answer for you. So
34:54
Speaker 4: it's kind of crazy to think about just like four
34:57
Speaker 4: months ago we had rock thway and now we already
34:59
Speaker 4: have for and we're going to continue to accelerate as
35:02
Speaker 4: a company XAI. We're going to be the fastest moving
35:04
Speaker 4: HI companies out there. So what's coming next is that
35:08
Speaker 4: we're going to you know, continue developing the model that's
35:11
Speaker 4: not just you know, intelligent smart thinking for a really
35:15
Speaker 4: long time, spent a lot of compute, but having a
35:17
Speaker 4: model that actually both fast and smart is going to
35:21
Speaker 4: be the core focus.
35:22
Speaker 1: Right.
35:22
Speaker 4: So if you think about what are the applications out
35:24
Speaker 4: there that can really benefit from all those very intelligent,
35:28
Speaker 4: fast and smart models, and coding is actually one of them.
35:31
Speaker 3: Yeah, So the team is currently working very heavily on
35:34
Speaker 3: coding models. I think right now the main focus is
35:37
Speaker 3: we actually trained recently a specialized coding model which is
35:41
Speaker 3: going to be both fast and smart. I believe we
35:44
Speaker 3: can share that model within in a few weeks. Yeah,
35:47
Speaker 3: that's very exciting.
35:48
Speaker 4: But the second after coding is we all see the
35:51
Speaker 4: weakness of GROCK four is the multi model capability. In fact,
35:56
Speaker 4: it was so bad that GROCK effectively just like looking
36:00
Speaker 4: at the world squinkings through the glass and see all
36:02
Speaker 4: the blurry features and trying to make sense of it.
36:05
Speaker 4: The most immediate improvement we're going to see what's the
36:08
Speaker 4: next generation preation model, is that we're going to see
36:10
Speaker 4: a step waunch improvement on the model's capability in terms of
36:13
Speaker 4: image understanding, video understanding, and audioce rate. It's now the
36:17
Speaker 4: model is able to hear and see the world just
36:20
Speaker 4: like NLU right and now with all the tools at
36:23
Speaker 4: this command, with all the other agents it can talk to,
36:26
Speaker 4: you know, so we're going to see a huge unlock
36:29
Speaker 4: for many different application layers. After the multimodel agents. What's
36:34
Speaker 4: going to come after is the video generation, and we
36:37
Speaker 4: believe that, you know, at the end of day, it
36:39
Speaker 4: should just be you know, pixeling pixel out. Imagine a
36:43
Speaker 4: world where you have this infinite scroll of content in
36:46
Speaker 4: inventory on the X platform where normally you can actually
36:50
Speaker 4: watch these general videos but able to intervene credit you
36:54
Speaker 4: on the ventures.
36:55
Speaker 1: It expect to be training.
36:56
Speaker 2: A video model with over one hundred thousand GB two
36:59
Speaker 2: hundreds to begin that training within the next three or
37:02
Speaker 2: four weeks. So if we're confident it's going to be
37:05
Speaker 2: pretty spectacular in video generation and video understanding.
37:08
Speaker 3: We're very excited for you guys to try and rock four.
37:10
Speaker 1: All right, Thanks, very good night.