Latest / In the Loop AI / OpenAI and Google Just Beat the World's Smartest Mathematicians

Transcript

OpenAI and Google Just Beat the World's Smartest Mathematicians · In the Loop AI

0:03
Ejaaz: All right josh the ai nerds are
0:06
Ejaaz: fighting again this past weekend there was
0:09
Ejaaz: a very prestigious competition called the international math olympiad which
0:14
Ejaaz: hosts some of the brightest smartest mathematicians of our time and they're
0:18
Ejaaz: typically high schoolers and basically they come together and they take a really
0:22
Ejaaz: hard math test this is like four to five hours and those that score the highest, get medals.
0:27
Ejaaz: You can get bronze, silver, and the highest scorers get gold medals.
0:31
Ejaaz: So what's this going to do with AI?
0:33
Ejaaz: Well, recently, over the last couple of years, the organizers of this International
0:37
Ejaaz: Math Olympiad decided to start inviting AI models to participate as contestants.
0:43
Ejaaz: And they did terribly. Like, no one's come even near the human geniuses.
0:50
Ejaaz: Except this year, Josh, where they came to play and not one,
0:53
Ejaaz: but two AI models achieved not silver, but gold medals, which is just an insane thing, right?
1:01
Ejaaz: So it should be all fun and games, right? What a fairytale story.
1:05
Ejaaz: Well, unfortunately, OpenAI and Google got into an online spat where they started
1:11
Ejaaz: accusing each other of cheating.
1:13
Ejaaz: Now, remember, these are trillion dollar companies. So essentially,
1:17
Ejaaz: Josh, I was teleported this weekend back to my high school days where I felt
1:20
Ejaaz: like the teacher had to come in, separate the kids from arguing over some kind
1:25
Ejaaz: of random homework problem and get them to chill out.
1:28
Josh: We will look back at this episode and laugh at it like it's a joke because these
1:31
Josh: AIs, they're competing against high schoolers. That's so lame.
1:35
Josh: Only high schoolers? Like, come on, and you're just barely getting gold.
1:38
Ejaaz: Well, in their defense, Josh, these are some pretty smart high schoolers,
1:42
Ejaaz: man. Like I was looking at some of these math problems.
1:44
Ejaaz: I don't know if you can see my screen here. I'm sharing the official site.
1:48
Ejaaz: And if you look at some of these problems, here we go.
1:53
Ejaaz: And then like, okay, so they have basically, they host this competition in a
1:56
Ejaaz: different country each year.
1:57
Ejaaz: And you can kind of like download the test yourselves after the fact to see
2:01
Ejaaz: how well you could do it. I had a look at this one, Josh from the Afrikaans.
2:07
Ejaaz: I basically don't understand anything. One second. All right,
2:10
Ejaaz: take a look at that. Take a look at this.
2:13
Josh: That looks like quite a bit of squiggly lines on a page.
2:17
Ejaaz: You know what? That could be mistaken for a piece of art in a gallery if you
2:22
Ejaaz: didn't peer too closely at it. This looks insane.
2:25
Josh: Okay, so I take it back. So the high schoolers are probably pretty smart then.
2:28
Josh: And I guess the AI performing as well as the high schoolers is probably a pretty big deal, right?
2:33
Josh: Because that looks like very complicated math problems that I'm assuming most
2:37
Josh: of the smartest people in the world cannot solve.
2:39
Ejaaz: Exactly. Yeah. This is like something that is technically set for high schoolers
2:44
Ejaaz: and sometimes college kids, but is meant to demonstrate prowess in the field.
2:50
Ejaaz: So there's a lot of university academics, which obviously do math degrees and
2:54
Ejaaz: they do PhDs, but those are in very specific problems. So you kind of like in
2:58
Ejaaz: science, you just need to kind of pick and choose your lane and then dedicate your life to it.
3:03
Ejaaz: High schoolers is kind of college kids are kind of like the last point before
3:07
Ejaaz: you jump into your specialization.
3:09
Ejaaz: So really, if you're the best at generalized maths, you're going to compete in this competition.
3:13
Ejaaz: And what's so interesting is typically AI models haven't been able to perform
3:17
Ejaaz: very well because they needed a lot of context beforehand about the problem, Josh.
3:22
Ejaaz: So they needed to know that, you know, there was certain, you know,
3:27
Ejaaz: X equals something and Y equals something.
3:30
Ejaaz: And they had to have defined parameters to kind of figure out the problem.
3:33
Ejaaz: But this was the first time that AI models basically were just given a blank
3:37
Ejaaz: sheet of paper or not a blank sheet of paper.
3:38
Ejaaz: But they stared at the problem just as we just looked at it just now and had
3:42
Ejaaz: to read the words, read the characters, interpret what that meant in the context
3:47
Ejaaz: of that situation and the way that the question was framed and then figure it out themselves.
3:51
Ejaaz: So it's as if the AI models had a camera that looked at a paper,
3:55
Ejaaz: similar way that we look at test papers as kids through our eyes and figure it out themselves.
4:01
Josh: So what changed? What happened in the last year that made it so much better?
4:06
Josh: Because it went from, what, basically zero of six to now six or five of six questions answered.
4:11
Josh: Now it's a gold medalist. So what happened?
4:14
Ejaaz: So listen, I'm not going to try and explain it, but maybe you and I can decipher
4:18
Ejaaz: it through the legends themselves that built these models, right?
4:22
Ejaaz: Okay, so let me paint the scene for you, Josh.
4:24
Ejaaz: It is Saturday evening.
4:27
Ejaaz: You know, normal people are usually out and about. They're having fun.
4:30
Ejaaz: They're probably having dinner, catching up with friends or chilling at home, watching a movie.
4:34
Ejaaz: And this guy called Alexander Wei, who is OpenAI's head of reasoning.
4:39
Ejaaz: Reasoning is basically this new fancy technique that AI models have typically
4:43
Ejaaz: demonstrated, which has brought them up to like the frontier level of AI models.
4:47
Ejaaz: Basically, if your model can do reasoning, it's typically a pretty smart model, right?
4:52
Ejaaz: And he posts this tweet saying, I'm excited to share that our latest OpenAI
4:57
Ejaaz: Experimental Reasoning LLM has achieved a longstanding grand challenge in AI,
5:02
Ejaaz: a gold medal level performance on the world's most prestigious math competition,
5:07
Ejaaz: the International Math Olympiad.
5:09
Ejaaz: And he goes on to describe, you know, how the model basically took on each problem
5:14
Ejaaz: in its own regard and solved it and how this is a massive success and win for
5:19
Ejaaz: AI models and how, most importantly.
5:21
Ejaaz: OpenAI was the first ever model to complete this.
5:25
Ejaaz: And not too long after he posts that tweet, Josh, Sam Altman jumps in here, right?
5:31
Ejaaz: And he goes, again, he kind of echoes similar thoughts. We achieved gold medal
5:34
Ejaaz: level performance on the 2025 IMO competition with general purpose reasoning.
5:38
Ejaaz: And then he kind of like shells GPT-5 at the end. Basically,
5:41
Ejaaz: it's like a promotive thing for OpenAI.
5:44
Ejaaz: And I will say that this is really cool because what they've achieved is something
5:49
Ejaaz: that hasn't been done before, right? So very impressive feat.
5:53
Ejaaz: And in terms of how this works specifically, Cheryl Su here gives a really good breakdown.
5:58
Ejaaz: She says, the model solves these problems without tools like coding or Lean,
6:04
Ejaaz: which is another coding tool.
6:05
Ejaaz: It just uses natural language. So as I said earlier, It kind of reads the paper
6:09
Ejaaz: and just kind of interprets what it thinks it means.
6:12
Ejaaz: And it also has the same amount of time to do the test as other kits, so 4.5 hours.
6:17
Ejaaz: And she says, we see the model reason at a very high level, trying out different
6:22
Ejaaz: strategies, making observations from examples, and testing different hypotheses out.
6:27
Ejaaz: And she says, it's crazy how we've gone from 12% on the AIME test,
6:32
Ejaaz: which is what GPT-4O, which is OpenAI's early model, got to IMO gold,
6:38
Ejaaz: International Math Olympiad gold medal in 15 months.
6:41
Ejaaz: So just to set that in context, Josh, that is a crazy leap in 15 months.
6:45
Ejaaz: Imagine going from eighth grade level math to the best.
6:51
Ejaaz: Mathematician in the world in 15 months. It's a pretty insane thing.
6:55
Ejaaz: Yeah, I'd say so. So essentially the breakthrough that Cheryl is highlighting
6:58
Ejaaz: here is number one, the model didn't need any context.
7:03
Ejaaz: Number two, it used really high level reasoning to figure out the problems from first principles.
7:08
Ejaaz: And number three, it was able to test out multiple hypotheses at the same time
7:14
Ejaaz: instead of trying to one shot the problem.
7:15
Ejaaz: Typically in the past when AI models have been given a prompt or a problem,
7:19
Ejaaz: it tries to just like give it its best shot and give you one solution, Josh.
7:23
Ejaaz: Whereas what these models, these reasoning models do really well is they are
7:27
Ejaaz: able to hypothetically entertain many different scenarios and then pick the
7:30
Ejaaz: best one of which it thought it was an answer.
7:32
Ejaaz: And it ended up with the gold medal, which is insane, right?
7:34
Ejaaz: But it wasn't entirely without a few glitches here and there, Josh.
7:38
Ejaaz: So if you look at this post from Jasper, he read through the entire kind of
7:42
Ejaaz: like problem set that OpenAI's model went through. and he points out that some weird anomalies.
7:49
Ejaaz: So he kind of like talks about like how it kind of like analyzed and a bunch of things.
7:52
Ejaaz: And he goes, however, the write-up is kind of messy. He goes,
7:55
Ejaaz: it overuses shorthand and sentence fragments.
7:58
Ejaaz: It introduces new terms without definitions, for example, forbidden and sunny partners.
8:04
Ejaaz: I have no idea what either of those terms could mean, but it was just apparently
8:10
Ejaaz: just interspersing these phrases during its analysis.
8:13
Ejaaz: And so as a reviewer, or as an examiner, they were reading this,
8:17
Ejaaz: they were like, sorry, wait, what is it talking about?
8:20
Ejaaz: It got to the right answer, but what is it talking about, right?
8:23
Ejaaz: The other key point from this post is it was unable to solve one problem, problem six.
8:29
Ejaaz: And I'm not even gonna try and get into why it failed on that problem,
8:33
Ejaaz: but it was just particularly hard for it to figure out.
8:36
Ejaaz: But it still scored a high enough percentage that it got a gold medal.
8:40
Ejaaz: So it's basically a win for OpenAI, but that's when the drama starts unfolding.
8:44
Ejaaz: So I've got this post up from Mikhail Samin, which kind of like sparks this entire fight, Josh.
8:51
Ejaaz: He goes, according to a friend, the IMO, which is the International Math Olympiad.
8:55
Ejaaz: Asked AI companies not to steal the spotlight from kids and to wait a week after
9:01
Ejaaz: the closing ceremony to announce the results.
9:04
Ejaaz: OpenAI instead announced the results before the closing ceremony. Yeah.
9:09
Ejaaz: And then he goes on to basically say how this is essentially like some kind
9:13
Ejaaz: of clout chasing move from OpenAI.
9:16
Ejaaz: And OK, I tried to evaluate this, Josh, from OpenAI's kind of perspective,
9:20
Ejaaz: which is they basically want to steal the limelight,
9:23
Ejaaz: but also say that they were the first AI model to ever achieve gold on this
9:27
Ejaaz: competition, which puts them in a good light and makes users want to choose
9:31
Ejaaz: OpenAI and solidify the branding that OpenAI is the best. right?
9:35
Ejaaz: But on the other side, you know, they're kind of like stealing the spotlight
9:39
Ejaaz: from the kids, as this post says. But that's not actually the main trope.
9:44
Ejaaz: The main trope here, Josh, is OpenAI wasn't the only model to achieve a goal, right?
9:50
Ejaaz: At the same time, during the same testing period, you had Google achieving the exact same score.
9:58
Ejaaz: So then the question becomes, okay, well, it was whoever was ethical about announcing their own result.
10:04
Ejaaz: This post from Demis Hassabis, which is Google's head of AI,
10:09
Ejaaz: basically posts, and I'll note two days later, Official results are in.
10:14
Ejaaz: Gemini, which is their flagship model, achieved gold medal level in the International Math Olympiad.
10:20
Ejaaz: An advanced version was able to solve five out of six problems.
10:23
Ejaaz: So same as OpenAI, same thing, struggled on the sixth problem.
10:26
Ejaaz: Incredible progress. Huge congrats to the team.
10:29
Ejaaz: And a tweet here says that Google
10:31
Ejaaz: basically had to wait for marketing to approve the tweet until Monday.
10:35
Ejaaz: But OpenAI shared theirs first at 1 a.m.
10:38
Ejaaz: On Saturday and stole the spotlight.
10:40
Ejaaz: And we see the screenshot from Demis Hassabis, which, you know,
10:44
Ejaaz: he further clarifies this, basically saying,
10:46
Ejaaz: by the way, as an aside, we didn't announce on Friday because we respected the
10:50
Ejaaz: IMO's board's original request that all AI labs share the results only after
10:55
Ejaaz: the official results have been verified.
10:57
Ejaaz: Now that we've been given permission to share, blah, blah, blah,
10:59
Ejaaz: he shares. So Demis is playing the like good Samaritan here.
11:02
Ejaaz: He's like, ah, you know, we also have the good model, but we,
11:06
Ejaaz: you know, we have some pride and some manners about how we deal with these things.
11:10
Ejaaz: That's where it starts to get a little uglier, Josh, because we have OpenAI
11:15
Ejaaz: chiming in to this tweet, which basically says, and this is some random commenting
11:21
Ejaaz: on OpenAI and this entire situation.
11:24
Ejaaz: So OpenAI basically has zero advantages except the size of the team,
11:30
Ejaaz: aka the OpenAI team was claimed to be smaller than Google Gemini's team.
11:34
Ejaaz: So what he's inferring here is there's no real difference between OpenAI's models
11:38
Ejaaz: and Google Gemini's models. You can pretty much use either or.
11:42
Ejaaz: OpenAI maybe has a smaller team to build that model, but who the hell cares?
11:46
Ejaaz: And then one of the AI model researchers at OpenAI basically comes in and says,
11:52
Ejaaz: well, I think it's also interesting that they they
11:55
Ejaaz: being google curated and provided useful context
11:59
Ejaaz: to the model which we did not feels like
12:02
Ejaaz: taking your tutor's cheat sheet with you into the exam so shots basically being
12:07
Ejaaz: fired from open ai saying hey um you cheated you gave context to your model
12:12
Ejaaz: and that was why it was able to achieve gold we open ai didn't provide any of
12:17
Ejaaz: that context and it was able to reason from first principles, there you have it.
12:21
Ejaaz: But then directly beneath it, Vinay Rameshes, who is a Google DeepMind AI researcher, responds,
12:27
Ejaaz: it's worth noting actually that a deep think system, which is Google's AI system
12:32
Ejaaz: with no access to this corpus, so no context, also got gold.
12:36
Ejaaz: Again, according to the official graders, and he puts this in brackets because
12:40
Ejaaz: OpenAI didn't wait for the official graders to mark their score,
12:44
Ejaaz: with exactly the same score.
12:45
Ejaaz: So basically, this is like a pissing contest between two of the top AI model providers.
12:52
Ejaaz: Here's my take, Josh. And then I really want to kind of lean into what you think
12:56
Ejaaz: about this whole debacle.
12:57
Ejaaz: Number one, this seems so childish to me.
13:00
Ejaaz: Like, eventually, AI models were eventually going to get smarter or smart enough
13:05
Ejaaz: to solve these mathematical problems.
13:07
Ejaaz: And I think you said this earlier on.
13:10
Ejaaz: This is something that they're going to probably laugh about 10 years from now,
13:14
Ejaaz: right? that they were able to solve whatever, the most complex mathematic problems
13:17
Ejaaz: for humans, mere humans.
13:19
Ejaaz: And now AI is off creating wonderful scientific discoveries for us that we would
13:24
Ejaaz: have never comprehended or figured out ourselves, right?
13:27
Ejaaz: So firstly, you're arguing over something that's so silly.
13:31
Ejaaz: But number two, this kind of seems desperate on the open AI side.
13:36
Ejaaz: And maybe I'm being biased, but I'm just going to give you my take.
13:39
Ejaaz: Open AI has kind of had a series of stumbles recently.
13:43
Ejaaz: They claimed that they were going to release gpt5 which
13:45
Ejaaz: is their brand new frontier model but they've delayed it many months
13:49
Ejaaz: now um they got outperformed by
13:52
Ejaaz: grok 4 from xai uh so now
13:55
Ejaaz: they have a new benchmark that they need to beat a new model that they basically
13:58
Ejaaz: need to outcompete uh they claimed that they were going to release a new open
14:02
Ejaaz: source model and then delayed it after a chinese open source model was released
14:07
Ejaaz: and had one trillion parameters and outperformed not just their model,
14:11
Ejaaz: but any other open source model out there.
14:13
Ejaaz: And so I feel like they're looking
14:16
Ejaaz: for a win, right? They released their agent this week or last week.
14:21
Ejaaz: And so, you know, that had mixed review, mixed feedback.
14:24
Ejaaz: So I feel like Sam is desperate for a win.
14:26
Ejaaz: People are criticizing consistently their moat, asking what has OpenAI got?
14:32
Ejaaz: They've lost a ton of researchers to Meta and other companies.
14:35
Ejaaz: I feel like their back's against the wall.
14:37
Ejaaz: Sam's scared and he basically needs to grab any kind of win.
14:41
Ejaaz: So it reeks of desperation.
14:43
Ejaaz: What's your take, Josh?
14:44
Josh: I do empathize with the team. They've been coming under fire from every single angle.
14:49
Josh: I mean, you have Zuck poaching all of their talent, and then all of the other
14:54
Josh: open-source AI models are beating them at their own game.
14:57
Josh: And they're just kind of, they're really getting beat up now.
15:00
Josh: And I think that they're looking to get some footing. I'm sure this probably plays a role in it.
15:04
Josh: But I'm sure behind the scenes, they're really trying to fight hard to put their
15:09
Josh: feet back on stable ground, to get GPT-5 out the door, to build Project Stargate
15:13
Josh: and make this big infrastructure network.
15:14
Josh: They need some wins. So sure, this was probably an attempt to get ahead,
15:18
Josh: make them look good, win over some more hearts and minds.
15:21
Josh: But I think the most interesting part of the whole story is less the drama and
15:25
Josh: more the fact that these models were able to accomplish a really impressive
15:28
Josh: feat over such a short period of time.
15:30
Josh: From what I understand, previously when they attempted to solve these problems,
15:35
Josh: they used a custom training data set.
15:37
Josh: They used custom tool sets. It was mostly a model trained on solving mathematical problems.
15:42
Josh: And with this version, both the OpenAI version and the Gemini models,
15:49
Josh: they were both general purpose models.
15:51
Josh: They were not trained specifically with the intention of solving mathematical problems.
15:55
Josh: These are the general models that people day to day are using.
15:58
Josh: They're just now able to solve these math problems using this new general intelligence.
16:02
Josh: So it's a really interesting breakthrough that I think we get from reinforcement
16:05
Josh: learning that now there is not so much of an advantage to training a model specific
16:11
Josh: to one's skill set when you could just make it great at everything.
16:14
Josh: There was one thing that I noticed that some people call it cheating, other people don't.
16:19
Josh: But so with the mathematical, with the actual test that high school was had
16:23
Josh: to take, they're not allowed to use tools and they have a limited amount of
16:26
Josh: time per question to answer.
16:28
Josh: The models that, the OpenAI model and the Gemini model, they had infinite amount
16:32
Josh: of time to answer and they were allowed to use tools.
16:34
Josh: So there still are small differences in these.
16:37
Ejaaz: Were they allowed to like use the internet?
16:39
Josh: I don't know the specifics. I would imagine at least calculators,
16:42
Josh: at most probably the full repertoire of what we have currently available to
16:46
Josh: us, which is full internet search, code writing abilities. They could do their
16:49
Josh: own mathematical checks.
16:51
Josh: So I would just assume the minimum amount of constraints possible.
16:54
Josh: So there was much less constraints on the models, But they did solve the questions.
16:58
Josh: And I think that's super impressive. They got five out of six right.
17:02
Josh: Which was gold and better than almost every student, if I'm not mistaken.
17:06
Josh: Only a few students got the six out of six completely correct.
17:10
Josh: It's just cool to see the rate of progress of these models getting better.
17:12
Josh: That over the course of the last 15 months or so, they went from horrible and
17:17
Josh: narrowly trained to incredible and generally trained.
17:20
Josh: And as long as that trend keeps going, I think the drama matters less than the
17:24
Josh: output, which is models are getting really good at solving really hard math problems.
17:28
Josh: And original ones too, that the world has never seen before.
17:31
Ejaaz: Yeah, well, that last point is actually the main takeaway that I had,
17:35
Ejaaz: Josh, which is it's original, never-before-seen problems.
17:39
Ejaaz: Typically, these AI models are trained on things that they've seen before, as you said, right?
17:44
Ejaaz: They're trained on data sets. So they've already seen the problem,
17:46
Ejaaz: and then they have to work out, they know the answer, and they have to work
17:49
Ejaaz: out how to get there, right? So they kind of have a leading factor.
17:52
Ejaaz: Here, it's just kind of like completely unknown.
17:56
Ejaaz: The other thing is, this is kind of like the culmination of a trend,
18:01
Ejaaz: Josh, which is these AI models are really good at doing kind of binary tasks.
18:07
Ejaaz: And I don't want to reduce mathematics to binary tasks, but technically it's
18:12
Ejaaz: numbers, sequential formulas, that kind of stuff, right?
18:18
Ejaaz: So if you can run enough compute at a thing, and if you can get that AI model
18:23
Ejaaz: to consider all different decision parts, It's going to eventually get to the answer, right?
18:28
Ejaaz: But it's always a specific answer at the end of that, right?
18:32
Ejaaz: Whereas when it comes to more subjective things, more human experiential things,
18:37
Ejaaz: AI has typically struggled to...
18:40
Ejaaz: Improve at the same rate that it has for like all these different scientific
18:43
Ejaaz: and math problems so i'm glad that we've reached this pinnacle feat i think
18:47
Ejaaz: ai models have are really good at one thing and not so great at other things
18:53
Ejaaz: and i'm excited to see how like they kind of like try to start leapfrogging
18:57
Ejaaz: each other over the next couple of years.
18:59
Josh: Yeah it's it's that directional progress that we like
19:02
Josh: math is clearly the first because you can write down
19:05
Josh: proofs and you could check your work and there is an actual verifiable solution
19:08
Josh: and i think that's why we're seeing a lot of the progress start early
19:12
Josh: in math and then hopefully go on to these other places but
19:15
Josh: what we are seeing is these first signs of
19:18
Josh: new knowledge breakthroughs where it's solving a
19:21
Josh: new and novel problem that hasn't been
19:24
Josh: released before based on its previous data set
19:28
Josh: so it's not just pattern matching like you mentioned earlier where it has
19:31
Josh: this data set of questions it's kind of finding the right examples and
19:34
Josh: then applying that logic to the question it's actually
19:37
Josh: reasoning and it's it's reasoning in many instances and
19:40
Josh: then it's comparing its work and it's it's coming to a conclusion
19:44
Josh: and we saw this with the grok heavy model last week too when
19:46
Josh: it released um where i think the the new
19:49
Josh: meta is many instances solving hard
19:52
Josh: problems and then comparing so you lower that error rate more
19:55
Josh: and more and more each time and what we're seeing is great progress so
19:59
Josh: i mean although open ai and google are fighting again they're both they're both
20:04
Josh: fighting over over exciting progress and sure maybe one tried to sweep in and
20:08
Josh: steal the valor but they both did an excellent job in actually completing these
20:13
Josh: problems and placing gold in a test that was previously not possible to do from an ai model you
20:19
Ejaaz: Know who the real winners are here out of this josh.
20:21
Josh: Who's that high school kids
20:24
Ejaaz: Who now have an AI model that can do all their math homework for them.
20:28
Josh: Isn't that incredible? Like, man, think about it.
20:30
Ejaaz: I wish I had that.
20:31
Josh: You have an AI model that is as smart as the smartest people on planet Earth
20:35
Josh: in high school. If it could solve those math problems, it could solve anything.
20:38
Ejaaz: It sounds human as well, Josh. So, like, your teacher is going to struggle unless
20:43
Ejaaz: they use AI themselves to figure out whether you just did that yourself or completely
20:48
Ejaaz: just ran that through GPT, your mom's GPT subscription.
20:51
Josh: It really forces you to re-evaluate the school model, right?
20:54
Josh: Because now that this information is so readily accessible, it's so easy to solve these problems.
20:59
Josh: Is that the actual thing worth learning? Or is it how to use these tools that's
21:04
Josh: more important to get to the answer?
21:06
Josh: And there's this there's this dual pronged approach and we see we see
21:09
Josh: developers and programmers talk about this a lot where as soon
21:12
Josh: as they start to rely too heavily on the tools they start
21:14
Josh: to lose their touch they start to lose their ability to to deeply
21:18
Josh: understand how it reaches conclusions um but
21:22
Josh: is that worth it in exchange for getting to the answer much quicker and then
21:25
Josh: being able to seek many more answers i don't know it's weird dynamic if i was
21:29
Josh: a teacher i'd be worried because i mean similar to what we saw with the calculator
21:32
Josh: it just replace the thinking process and just yield you an answer and
21:38
Ejaaz: The thing with the calculator is like you you're
21:42
Ejaaz: using the calculator so it figures out the answer for you but you kind of
21:44
Ejaaz: loosely understand how it is working right you
21:48
Ejaaz: know what numbers it's crunching to get to that answer and then typically you
21:52
Ejaaz: do a few things on a calculator and then you get to your eventual answer for
21:57
Ejaaz: whatever the original question was the issue with or the concern that you're
22:01
Ejaaz: highlighting here with AI is it's doing really complex problems,
22:07
Ejaaz: which kids don't even need to understand in the first place just to get an answer,
22:11
Ejaaz: which they can then give to their teacher, get a grade and then go to university.
22:15
Ejaaz: But the kids don't actually learn actively in that process.
22:19
Ejaaz: And it's going to be a concerning trend if we see kids just trying to go from
22:24
Ejaaz: zero to 100% without understanding anything in between.
22:29
Ejaaz: A trend to watch.
22:31
Josh: This is our episode from a few weeks ago. Is AI making you dumber?
22:34
Josh: Yes. And I think that's just going to continue to be the question.
22:37
Josh: Oh, God. And I think the answer is it's all dependent on how you choose to use
22:41
Josh: the tools that you're given.
22:42
Josh: And if you use these tools as further leverage. So I'm sure these math olympiads
22:46
Josh: who can actually complete the problems would love to have this model to check
22:50
Josh: the problems and to work through the problems and to figure out shortcuts on
22:53
Josh: solving these problems.
22:54
Josh: Where if you deeply understand it, then this becomes an amazing tool to check
22:57
Josh: your work, to generate new questions for you.
22:59
Josh: It's a great study, buddy. or if you are not an olympiad and you still want
23:04
Josh: to get to the answer well you just kind of cheat your way through and you just
23:07
Josh: ask it for exactly what you want so it's that it's that split again and it's
23:11
Josh: up to the person to take their own agency solve their own problems and try to
23:14
Josh: use these for for tools of leverage instead of just problem solving machines that
23:19
Ejaaz: Actually reminds me of this tweet i saw yesterday josh um so what you're looking
23:23
Ejaaz: at here is a tweet from dave white dave White is a very prestigious investment
23:28
Ejaaz: slash research advisor at this fund called Paradigm,
23:32
Ejaaz: which basically it's a crypto fund, but it is one of the wealthiest funds out there.
23:37
Ejaaz: So a lot of the investments they made were massive wins. And a lot of the reasoning
23:42
Ejaaz: of those wins was from Dave White's analysis.
23:44
Ejaaz: He is a deeply thoughtful mathematician at his core, and he is famed for doing
23:50
Ejaaz: a lot of analyses on companies, mathematical analyses that have ended up, you know.
23:57
Ejaaz: Determining whether a fund puts $100 million in a company or zero, right?
24:01
Ejaaz: So a very important job worth hundreds of millions of dollars, right?
24:04
Ejaaz: And what he says here, basically, is him having an identity crisis,
24:08
Ejaaz: because he has looked up to the IMO, the International Math Olympiad.
24:13
Ejaaz: And he goes on to say in this tweet that subconsciously, whenever he's met a
24:18
Ejaaz: gold medalist IMO champion, he's always subconsciously thought that they were
24:22
Ejaaz: smarter than him, that he is more respecting of them.
24:25
Ejaaz: And now with this news that AI models basically can do his job for him,
24:30
Ejaaz: can reason better than him at some of these math problems, he now has an identity crisis.
24:35
Ejaaz: He doesn't know kind of where to go from this. And if people like Dave White
24:39
Ejaaz: is having this kind of like disillusioned sentiment from how smart AI is,
24:45
Ejaaz: you can imagine how this is going to happen for everyone else in all of the
24:49
Ejaaz: other sectors, Josh, right?
24:50
Ejaaz: It doesn't matter if you're a mathematician or an investment research advisor,
24:54
Ejaaz: you could be a technician in some kind of engineering industrial role,
24:59
Ejaaz: or you could be a teacher, or you could be a kid or a high schooler.
25:02
Ejaaz: I think this disillusionment is going to spread. And I think it's super important
25:06
Ejaaz: for people to kind of like evolve their thinking, like you said,
25:09
Ejaaz: Josh, and learn how to leverage these tools versus just consume.
25:13
Josh: Yeah, this is, I mean, this is crazy. There's a lot of people that are going
25:16
Josh: to have to adapt to this new world order of intelligence, where if you build
25:21
Josh: up your entire identity around being intelligent, well, perhaps you're going to have to alter the way
25:26
Josh: present yourself as intelligent because the meaning of intelligence is becoming
25:29
Josh: commoditized among these tools that are now reduced down to a single chat box.
25:34
Ejaaz: Yep. Benchmarks are going to have to reset themselves completely.
25:37
Ejaaz: But folks, that is the end of this episode. Thank you so much for tuning in again.
25:42
Ejaaz: Josh and I are going hammer and tong at Limitless.
25:46
Ejaaz: Our goal is to get you the hottest and trending topics and news fresh out the
25:51
Ejaaz: door, give you our commentary, our thoughts, and hopefully some useful insights for you.
25:55
Ejaaz: If you enjoyed this episode if you enjoyed any of our previous episodes please
25:58
Ejaaz: continue to share and spread them with all your friends and family and whoever
26:02
Ejaaz: you think might be interested in this we are getting tons of feedback from you
26:05
Ejaaz: guys and with every episode that we release we're getting better so please remember
26:09
Ejaaz: to like subscribe follow us it's hugely appreciative and helpful for us and
26:13
Ejaaz: we'll see you on the next one.