Latest / In the Loop AI / "Kimi K2 Thinking" is China's Plan To End American AI Dominance

Transcript

"Kimi K2 Thinking" is China's Plan To End American AI Dominance · In the Loop AI

0:00
Ejaaz: The world's latest and greatest AI model is 100% free for you to download and run at home right now.
0:07
Ejaaz: Kimi K2 Thinking is the latest reasoning model from Moonshot AI Labs,
0:12
Ejaaz: which is a Chinese frontier AI lab.
0:14
Ejaaz: And it beats OpenAI's GPT-5, Anthropics Claude, and Google's Gemini across pretty much all benchmarks.
0:21
Ejaaz: But that's not even the most shocking part. The most shocking part is that it
0:24
Ejaaz: only costs $4.6 million to train and build, which is only a fraction of the
0:29
Ejaaz: billions of dollars spent by OpenAI to train GPT in the first place.
0:33
Ejaaz: It's also 100% open source, which means that you can download and run Frontier
0:37
Ejaaz: AI right at home where you're sitting right now.
0:41
Ejaaz: But of course, it begs two very important questions. Number one,
0:44
Ejaaz: is open source AI the winning strategy?
0:46
Ejaaz: We've been led to believe that closed source is typically the better strategy
0:50
Ejaaz: when you run a business, but China and the AI models are proving us wrong here.
0:54
Ejaaz: And the second question, the more ominous question is, will the US stock market bubble finally pop?
1:01
Ejaaz: Josh, what have we got here? What is this new model and why is it taking over
1:06
Ejaaz: social media everywhere I look?
1:07
Josh: They did it again. The Chinese did it again. They knocked it out of the park.
1:11
Josh: Grand slam, home run. It's an unbelievably impressive model.
1:15
Josh: And this happens every time.
1:17
Josh: We get this amazing flagship model out of the US.
1:20
Josh: A couple months later, we get the same thing, marginally better,
1:23
Josh: at one-tenth of the cost. Like full orders of magnitude cost less than what
1:28
Josh: it calls for the leading AI labs in the US today.
1:30
Josh: The specs are really impressive. We're going to get into everything.
1:32
Josh: We'll start with, I guess, just like the high level spec sheet.
1:35
Josh: State of the art on humanity's last exam, which is the reference point that
1:39
Josh: we kind of use in terms of benchmarks.
1:41
Josh: It scored the highest anyone's ever scored, 44.9%.
1:45
Josh: It has a bunch of these really cool breakthroughs, but the big thing that it
1:49
Josh: excels at, like it says in the post here, reasoning, agentic search, and coding.
1:54
Josh: Now, there's a few cool things that we could talk about here.
1:56
Josh: Ejaz, maybe we'll just get into the charts because i feel like that's an easy way to visualize
2:00
Josh: how much better this model really is than all of the others and what we're seeing
2:03
Josh: on the chart is that while gpt5 was the best kimmy k2 is now the new best and
2:07
Josh: this is as it relates to thinking and reasoning
2:10
Josh: and this again this is so impressive because
2:13
Josh: one this model is fully open source you can go download the model and run it
2:16
Josh: yourself locally for free what were your first thoughts when you saw this because
2:20
Josh: to me i was like oh my god i was like why would i use anything else?
2:24
Ejaaz: My first thought, if I'm being honest, Josh, was like to look at the stock market.
2:27
Ejaaz: I was like, is this going to crash the entire US stock market?
2:31
Ejaaz: Like when DeepSeek initially released the R1 thinking model,
2:35
Ejaaz: do you remember it was at the end of last year, people's kind of entire bubble
2:40
Ejaaz: and vision of how AI models were trained was completely burst.
2:43
Ejaaz: And since then, China has repeatedly delivered on breaking edge models,
2:47
Ejaaz: one of which is the Moonshot AI lab team, which built Kimi K2.
2:52
Ejaaz: It's such an impressive model for a few different reasons for me.
2:55
Ejaaz: Number one, it can now compete with all the best. And,
2:59
Ejaaz: personally, GPT-5 is something that I use pretty much every day,
3:03
Ejaaz: whether it's for like kind of casual prompts and requests or whether it's kind
3:07
Ejaaz: of like the deeper thinking and research and some of the lines of work that
3:10
Ejaaz: I do. So it's become kind of like...
3:12
Ejaaz: Quintessential for me. Now to have a separate model that I can download and
3:17
Ejaaz: run privately on my own computer at home that I'm showing on this tweet here that costs 60 cents,
3:25
Ejaaz: per million token input and $2.5 output is just an insane cost-cutting average
3:30
Ejaaz: where if I was running a business using an AI model.
3:33
Ejaaz: There's very little reason for me not to switch over to something like this
3:36
Ejaaz: aside from maybe like maintenance and setup and stuff like that.
3:40
Ejaaz: The other really impressive thing for me, Josh, was the team itself.
3:44
Ejaaz: Like, this is only a two-year-old startup, which reminds me of another two-year-old
3:51
Ejaaz: startup, which is Elon Musk's XAI, right?
3:54
Ejaaz: And there's a funny link between these two models, Josh, which is Kimi K2's reasoning,
4:00
Ejaaz: this thinking model, can do so because it does this like really neat little
4:04
Ejaaz: chain of thought experiment where it takes many steps to kind of think to a
4:08
Ejaaz: logical answer versus just kind of like splurging an answer for you.
4:12
Ejaaz: That's something that Grok Heavy 4 did that they pioneered when they launched their new product.
4:19
Ejaaz: So Kimi K2 is kind of like drawn on some of these learnings from XAI to produce a similar model.
4:24
Ejaaz: The other really cool thing is
4:25
Ejaaz: it does this thing called tool use or tool calling whilst it's thinking.
4:31
Ejaaz: So if you imagine, as I'm kind of like trying to think through a complex problem,
4:37
Ejaaz: I will leverage different tools to be able to help me get to the answer.
4:40
Ejaaz: So if I'm doing a maths exam, I can use a calculator. Or if I'm doing a deep
4:45
Ejaaz: research question, I might use Google.
4:47
Ejaaz: This AI model naturally does that and has access to over 200 to 300 different
4:52
Ejaaz: tool calls and tool uses whilst it does its thinking. So just overall,
4:56
Ejaaz: a very impressively new looking AI model.
4:58
Josh: Yeah, Ijaz, you mentioned the cost being 60 cents per million tokens.
5:03
Josh: And I just want to add a little bit of context as to how low that actually is.
5:07
Josh: I was looking at the GPT-5 Pro cost per inputs, and it is $15 per million tokens.
5:14
Josh: $15 for the GPT-5 Pro cost currently.
5:17
Josh: The output is $120 per million tokens.
5:22
Josh: Granted, this is the top of the top. If you're using GPT-5 standard,
5:25
Josh: input is $1.25 per million tokens, output is $10.
5:29
Josh: So any way you scrape it, it's at least a 2x multi-cost reduction up to like
5:36
Josh: 100x on the highest end, assuming it can compete with GPT-5 Pro,
5:40
Josh: which all those benchmarks suggest it very well can.
5:43
Josh: So the cost is really like, it's a big deal. And to get kind of dig more into
5:48
Josh: the point that you were making, Ejoz, and how it actually works.
5:50
Ejaaz: Well, we'll get there.
5:54
Ejaaz: Save the memes.
5:55
Josh: Don't spoil the memes yet. We got to get to the funny jokes next.
5:58
Josh: But basically, the way this works is like there's this very complicated diagram
6:01
Josh: on the screen. I'm not going to try to even explain what that is.
6:04
Josh: But there's this fun way that I like to describe it when I was describing it
6:08
Josh: to my friend earlier this morning.
6:09
Josh: Which is that like Kimi K2, it's like this giant school and it has these things called specialists.
6:15
Josh: And in fact, Kimi K2 has 384 specialists. You could think of these specialists
6:18
Josh: as like a math club or a history club, coding club, debate, whatever it is.
6:22
Josh: And when you ask it a question, it doesn't invite the whole school.
6:25
Josh: It doesn't invite all the clubs. It's just, EJS, if you ask for a math question,
6:29
Josh: it will query the math club.
6:31
Josh: And it chooses eight out of those 384 clubs to help combine their answers,
6:35
Josh: pick the experts and decide how it's going to solve this problem.
6:39
Josh: So it has a trillion parameters, but it only uses 32 billion of them at once.
6:44
Josh: And that's how we're able to get the huge cost reduction because it uses this
6:47
Josh: thing called mixture of experts. A lot of people describe it as MOE,
6:50
Josh: but basically what it is, instead of using the entire model's intelligence to
6:54
Josh: answer, what should I have for breakfast this morning?
6:56
Josh: It will take the chef club. It will take the health club.
7:00
Josh: It will combine those together and it will form an answer that should hopefully
7:04
Josh: give you just as good as a result if you took the entire model but it's much more efficient
7:08
Josh: in terms of cost in terms of energy and in terms of the amount of tokens they
7:12
Josh: could generate because it's so much cheaper across the board and i think that's
7:15
Josh: one of the big really exciting things that has been cool to see coming out of china
7:19
Josh: we saw with deep seek we see it with kimmy and it's this mixture of agents architecture
7:23
Josh: where they're really kind of modularizing the entire model
7:26
Josh: and only using the stuff that's important for the specific query
7:29
Ejaaz: They were put in a very constrained position which is they didn't have access
7:33
Ejaaz: to the latest GPUs or NVIDIA GPUs, there's been a bunch of U.S.
7:37
Ejaaz: Tariff restrictions on Chinese labs getting access to these kinds of things.
7:40
Ejaaz: So they've really needed to kind of like work within their bounds and means.
7:45
Ejaaz: And so coming up with an architecture like Mixture of Experts or the one that
7:48
Ejaaz: they did is super important.
7:50
Ejaaz: And it brings me to this meme, Josh, which is, what are we doing here?
7:54
Ejaaz: There is an obvious mismatch between American-made AI models and the,
8:00
Ejaaz: Chinese ones. You've got OpenAI, which is now projected to spend $1.4 trillion,
8:05
Ejaaz: over the next five years.
8:06
Ejaaz: That's trillion with a T versus Kimi training for $4.6 million.
8:12
Ejaaz: Now, I know there's a bit of like clickbaitiness here.
8:15
Ejaaz: That $4.6 million was relative to one training run and usually takes a few training runs.
8:20
Ejaaz: But let's say it took like 20 training runs, right?
8:24
Ejaaz: At $4.6 million, that's still only like 100 mil, right? Or less than that.
8:28
Ejaaz: So it doesn't really matter when you put it into the context that GPT-5 is rumored
8:34
Ejaaz: to have cost $1.7 to $2.4 billion for OpenAI to train.
8:38
Ejaaz: So there's a mismatch that I don't quite understand, Josh. And that's what makes
8:42
Ejaaz: me the most nervous when it comes to what American-made companies and Frontier Labs are doing.
8:47
Ejaaz: I feel like they're missing the mark. I don't quite know what it is,
8:50
Ejaaz: whether it's this mixture of experts thing, but there's someone's being sold
8:53
Ejaaz: a lie and I don't know whether it's me or whether it's me like looking at this
8:58
Ejaaz: Kimi K2 model and being like wow it's so amazing.
9:01
Josh: Yeah, when I think about the role that China plays versus the United States
9:04
Josh: in terms of like open source companies or closed source companies here in the US,
9:09
Josh: the thing that is reassuring to me, at least, is a lot of these innovative breakthroughs
9:13
Josh: that happen on the software level actually do happen in these private AI labs.
9:18
Josh: We do get like chain of thought and reasoning. And there's like this whole slew
9:22
Josh: of new innovation that becomes standard very quickly.
9:24
Josh: That all happens in the United States AI labs. And as far as we're concerned,
9:29
Josh: the AI labs in the US still have they're making the most progress the fastest
9:33
Josh: they are creating the most innovation and then what you kind of see like we
9:36
Josh: described earlier in the episode
9:37
Josh: is that innovation starts to trickle down whether it's voluntary or whether
9:41
Josh: it's stolen and it gets implemented into these new models and they just completely
9:46
Josh: cut out the bottom in terms of cost and efficiency because that's kind of all
9:49
Josh: they're able to do they don't have access to the resources of
9:52
Josh: like millions of gpus from jensen huang and nvidia they don't have the access
9:57
Josh: to 50 billion dollars of capex just to spend on
10:01
Josh: employees just to spend on salaries and compensation um so it seems to me like
10:05
Josh: i mean we're still doing very well it's just
10:08
Josh: China is very good at implementing the technology and applying it at scale in
10:12
Josh: a way that's open sourced and the open source thing there's there's a lot to say for that because
10:16
Josh: it's it's very impressive and it's kind of this community effort that we saw
10:20
Josh: early days with the united states but once they became better they closed it off so what happens is
10:25
Josh: you get innovation in one company like Kimi and then you see it implemented
10:29
Josh: in DeepSeek and then you see it implemented in Quen and then suddenly this technology
10:33
Josh: is it's kind of synchronously growing between the three because it's all open
10:36
Josh: source they're publishing all the code all the open weights
10:39
Josh: and it's much more easier for them to thrive whereas innovation in the United
10:41
Josh: States very much happens behind a closed wall and it's only leaked out
10:45
Josh: at the advent of a new model when they release it to the world and people kind
10:48
Josh: of reverse engineer how it works
10:50
Ejaaz: Um, I was reading an article in the Financial Times where they interviewed Jensen
10:55
Ejaaz: Huang, and he said verbatim that China will win the AI race if they continue
11:03
Ejaaz: down the pot that they're currently on.
11:04
Ejaaz: And if the US doesn't kind of ramp up their energy production,
11:07
Ejaaz: he was making a wider point that their open source strategy is pretty effective
11:11
Ejaaz: in the way that they're up, that they're building these new AI models with the
11:14
Ejaaz: constraints that you just mentioned.
11:16
Ejaaz: And kind of speaking more about the open sourceness and the benefits of this.
11:21
Ejaaz: I've got a tweet up here, which shows that Kimi K2 thinking this new model can
11:26
Ejaaz: basically run on two MacBook M3 Ultras, which is like a couple of thousand dollars
11:32
Ejaaz: worth of cost, which is an insane thing to do to run Frontier
11:36
Ejaaz: AI model at home, privately in your house, trained and fine tuned on any of your own private data.
11:42
Ejaaz: So you don't need to kind of like sell that data to Sam or Mono,
11:45
Ejaaz: whoever, just super cool and super cheap, right? Because you're running local
11:49
Ejaaz: inference at home. So you don't have to worry about anyone kind of like spying
11:52
Ejaaz: on any of your queries or your prompts or your research.
11:55
Ejaaz: It's just all at home, which I thought was super cool.
11:58
Ejaaz: The other part of the open sourceness, which I found interesting,
12:01
Ejaaz: Josh, was the fact that they had an MIT license with this new release or an adjusted MIT license.
12:08
Ejaaz: And we'll dig into that in a second. But the point being, when DeepSeek released
12:13
Ejaaz: their first major open source model and it took the world by storm,
12:16
Ejaaz: there wasn't any major licenses around that.
12:18
Ejaaz: So you could pretty much download and do whatever the hell you wanted for it.
12:21
Ejaaz: You could implement it into your own product, whether you were an American founder.
12:25
Ejaaz: And if, let's say, you scaled that up to a million users that used a feature
12:28
Ejaaz: that was leveraging that DeepSeek model, you wouldn't have to credit that team at all.
12:34
Ejaaz: Kimi K2 kind of like takes a step in a different direction here where they've
12:37
Ejaaz: released an MIT license where I think if you hit
12:40
Ejaaz: I think it's either 10 million or 20 million users for your product you need
12:44
Ejaaz: to show the Kimi K2 label and say that listen I'm using this model under the hood but there's
12:50
Ejaaz: there's some differences with this license right Josh can we can we dig into that?
12:54
Josh: I believe it's modified. I don't know to the extent that it is modified,
12:58
Josh: but I know that there is something different going on here. What does this say?
13:02
Josh: Our only modification part is that if the software or any derivative works thereof
13:07
Josh: is used for any of your commercial products or services that have more than
13:11
Josh: 100 million monthly active users or more than 20 million US dollars or equivalent
13:16
Josh: other currencies in monthly revenue,
13:18
Josh: you shall prominently display KimiK2 on the user interface of such product or service
13:23
Josh: that's a fun little marketing ploy
13:26
Ejaaz: Fair enough fair enough you know what it reminds me of josh um what's that it's
13:30
Ejaaz: what meta tried to do with their llama models right,
13:34
Ejaaz: so um meta is the only other major american company that i can think of that
13:39
Ejaaz: went down this open source ai route and the goal or the intended goal at the
13:44
Ejaaz: time was to basically level the playing field,
13:48
Ejaaz: between Meta and OpenAI and other frontier model AI labs, which had raced so far ahead.
13:52
Ejaaz: So if you released all this cutting edge AI tech for free and accessible to
13:56
Ejaaz: anyone, then it kind of drives down the cost premium that OpenAI and all these
14:01
Ejaaz: other frontier AI labs can charge you to access this thing.
14:04
Ejaaz: China's doing that as a vast hole on the American AI stock market,
14:09
Ejaaz: right? So that's why we saw like NVIDIA crash, I think 4.2% on the news getting released and such.
14:16
Ejaaz: I'm curious whether this kind of pops the bubble and the CapEx bubble in America, Josh.
14:21
Ejaaz: Is that a crazy thing to say? I mean, the markets reacted pretty viscerally to this news.
14:26
Josh: I don't think I have a problem with this. I don't think it's popping a bubble.
14:31
Josh: I don't think we're in trouble.
14:33
Josh: I think this is just totally fine so long as we continue to stay slightly ahead or at least at par
14:40
Josh: i think we're really excellent at making software distributing software creating
14:45
Josh: products i think china's really good at shamelessly
14:49
Josh: innovating and deploying without needing to go through all the hoops and intellectual
14:56
Josh: Problems that the united states mostly has um so i don't think this will lead
15:01
Josh: to any sort of bubble popping i think a lot of the frontier innovative stuff
15:04
Josh: still happens in the u.s the place where i will begin to start to get a little
15:07
Josh: worried is when this switches to embodied ai once we
15:11
Josh: start moving from large language models to implementing these into robots or
15:16
Josh: implementing these into physical hardware that's where i think we have problems
15:21
Josh: on the software front we're good we're crushing it everyone's spending tons
15:24
Josh: of money um on the hardware front we don't have the same lead and
15:29
Josh: over the last what 30 to 50 years we've kind of outsourced manufacturing capabilities
15:34
Josh: to other places and therefore are just kind of i mean everyone knows we just
15:37
Josh: can't really make things cost effectively here in the united states
15:40
Josh: if we are at a foot race with china when it comes to making embodied ai like
15:44
Josh: humanoid robots specialized robots whatever it may be that's where things start
15:47
Josh: to get a little bit scary because that's where there is a significant lead and
15:51
Josh: that lead is comes in the form of atoms which are much more difficult to move than bits
15:54
Josh: because you can steal some open source code create this slight innovation on
15:57
Josh: top roll it out to a billion users overnight and that's innovation that does not happen
16:02
Josh: between version two and version three of your humanoid robot you actually have
16:05
Josh: to build it with a factory with real materials and people and places and it's
16:08
Josh: it's very difficult and challenging to do and china very much stands to be the
16:12
Josh: largest winner in that so i think on the software front i feel really confident
16:16
Josh: and as of now that's all that we're battling on but in this near future where
16:19
Josh: things start to become embodied, where AI becomes physically manifested in the world around us,
16:24
Josh: That seems like a place where I would start looking at Chinese investments a
16:28
Josh: little bit more than the American ones.
16:30
Ejaaz: Okay. I think I might push back a little bit and say that there is reasonable
16:36
Ejaaz: evidence to be bearish on the software side before it gets embodied AI.
16:40
Ejaaz: I mean, so a few ways to think about it.
16:43
Ejaaz: There is such a gross discrepancy when it comes to capital expenditure for these things.
16:50
Ejaaz: On one side, you've got the US spending trillions of dollars literally to train
16:55
Ejaaz: AGI or the best AI models.
16:57
Ejaaz: And on this side, you're in like the hundreds of millions of dollars,
17:00
Ejaaz: which is like an order of magnitude less, right?
17:03
Ejaaz: So there's an obvious mismatch here that we aren't seeing.
17:08
Ejaaz: Whether it comes down to training architecture, training design,
17:11
Ejaaz: or just kind of like hardware manufacturing.
17:13
Ejaaz: I don't know where that kind of advantage is being played, but the Chinese have
17:17
Ejaaz: found it and they're able to kind of really push down on that lever to get ahead
17:21
Ejaaz: or on par with the U.S. And they've been able to successfully do this for years
17:25
Ejaaz: now at this point. DeepSeq was kind of like test case one.
17:28
Ejaaz: Now I've seen like, you know, at least 50 open source models come out of Chinese
17:32
Ejaaz: frontier AI labs since then.
17:34
Ejaaz: Number two, it's not like the U.S. government has kind of like not tried to constrain them.
17:41
Ejaaz: We've imposed a number of different sanctions, which include,
17:43
Ejaaz: you know, constraining which GPUs NVIDIA and other manufacturers within the
17:49
Ejaaz: U.S. can sell to China. But that still hasn't stopped them.
17:53
Ejaaz: They've been able to maintain and train these frontier AI intelligences despite
17:58
Ejaaz: all of these different things.
17:59
Ejaaz: So I think if I were to look on the other side of this, it would be,
18:04
Ejaaz: so what if you have an open source model that is super cool?
18:07
Ejaaz: Why aren't you using it right now? Like I'm not using Kimi K2 regularly,
18:10
Ejaaz: even though I use GPT-5 and it might be better than GPT-5. And the answer for me is pretty simple.
18:15
Ejaaz: I'm locked into an ecosystem in OpenAI that I'm pretty happy with,
18:19
Ejaaz: which is it has memory on me. It understands who I am.
18:23
Ejaaz: It has a context of all the previous chats that I have with it.
18:25
Ejaaz: But also most importantly, Josh, if there's an issue with something on my account
18:29
Ejaaz: or something that I'm trying to use, there's a community that I can access.
18:32
Ejaaz: There's a support team that I can speak to. There's a software ecosystem that supports me, right?
18:38
Ejaaz: Versus me jumping ship to kind of Kimi K2 setting it up on my own and then having
18:41
Ejaaz: to like troubleshoot it myself I think a lot of people will be disincentivized to do that.
18:48
Josh: It is difficult, but I mean, we're seeing market forces from both sides, right?
18:51
Josh: Like I saw you included a link here somewhere where Cursor and Windsurf's new
18:56
Josh: AI models, they were using some sort of Chinese models.
18:59
Josh: In fact, they were thinking in Chinese. And I found this really fascinating
19:02
Josh: that like American-made products are now thinking in the Chinese language.
19:05
Josh: So that's certainly a concern in terms of the commercial side where those API
19:09
Josh: costs really matter, where if you can get a million tokens for $0.60 versus
19:14
Josh: $10, that really affects the margins of your business.
19:17
Josh: For consumers like us, there's no real interest to use Kimi K2.
19:22
Josh: And the phenomenon you spoke about earlier, where you can actually run a quantized
19:26
Josh: version of Kimi K2 on two Mac studios running the M3 Ultra chips,
19:32
Josh: it generates tokens at like 13 to 15 tokens per second. So it's very slow.
19:36
Josh: Slow. You're getting like a sentence or two every second, which it's much slower.
19:43
Josh: It's going to feel groggy. It's not going to feel well.
19:46
Josh: There's a case to be made that that changes because this year and it's funny
19:49
Josh: that apple's really the only computer that that supports this now they're releasing
19:52
Josh: the m5 ultra which will be the new version
19:54
Josh: and um it's going to be interesting to see how it plays out what i found interesting
19:58
Josh: this one side note actually that i wanted to share with you just because you
20:00
Josh: might find it cool too is the version that runs on these apple computers the
20:05
Josh: apple studios um it's a it's a slightly quantized version and
20:10
Josh: I heard about this and i learned about this recently in the tesla earnings call
20:14
Josh: that they had the shareholder meeting recently and we're gonna have an episode
20:16
Josh: on this later this week but there's this interesting thing that elon mentioned
20:20
Josh: during the episode where he was talking about quantize versus floating point
20:23
Josh: ai and i was like what the hell is that like what why are you spending so much
20:27
Josh: time talking about this it doesn't make sense
20:28
Josh: and what i realized is a lot of ai models they they use like
20:32
Josh: many, many points after the decimal in terms of data to get more precise results.
20:37
Josh: And that is floating point. When you quantize a model, you remove all of the
20:40
Josh: data to the right of the model and you just go to single integers.
20:43
Josh: So you lose the variance of maybe up to like 60%, but you gain so much faster
20:50
Josh: efficiency, so much better speed improvements, cost improvements,
20:53
Josh: and you can actually run it locally on these things.
20:55
Josh: So I think it's interesting to see the different decisions that people are making
21:00
Josh: in terms of, well, how precise does the model have to be versus how cost effective
21:04
Josh: and how efficient does it need to be?
21:05
Josh: And what we're seeing with Kimi Gay too is it's very easy to over-index on the
21:09
Josh: efficiency, but maybe that's not the stated goal of OpenAI, where if they really
21:13
Josh: wanted to, they could sign up, quantize these models. They could go more to integer type compute.
21:18
Josh: And it was just something I was thinking about is how they approach them,
21:21
Josh: because it could just be, well, Kimi's just kind of optimizing for speed and
21:24
Josh: efficiency, and the downstream effect is it's also really fast,
21:27
Josh: whereas OpenAI kind of hasn't really optimized for that specifically yet.
21:31
Ejaaz: Right. And the counter argument to that point would be, well,
21:35
Ejaaz: Josh, it's crushing all the benchmarks that we've evaluated all the other American
21:39
Ejaaz: models on, right? Yeah. So surely it's much better.
21:42
Ejaaz: And my pushback on that would be like, well, benchmarks don't really materialize in real life use.
21:49
Ejaaz: So what if it crushes 50% on humanities last exam?
21:54
Ejaaz: Is it useful for me to use? Does it understand what I'm trying to say?
21:57
Ejaaz: Does it understand the context of the prompts that I'm putting into it?
22:00
Ejaaz: Um the other side of this um you know on the point of quantization josh is um,
22:05
Ejaaz: i think that a lot of frontier american ai labs like open ai google etc,
22:10
Ejaaz: actually have enough compute to give you the best experience the um the highest
22:16
Ejaaz: floating point uh experience um to put it to put into that context,
22:20
Ejaaz: but they're using the majority of that compute to train the next big model that
22:24
Ejaaz: we haven't even seen yet, right?
22:27
Ejaaz: There was news that broke last week that OpenAI is doing this,
22:30
Ejaaz: right? So technically they have enough compute to give you like amazing service
22:33
Ejaaz: all year round, but they're using 70% of that compute to train GPT-6.
22:38
Ejaaz: So I think it's just a matter of prioritization right now until we reach some
22:41
Ejaaz: kind of parity that these AI models are good enough.
22:45
Ejaaz: But I will say from all of the things that we've discussed on this episode so
22:49
Ejaaz: far, there is one clear winner and And that is the consumer.
22:54
Ejaaz: It's you, I, and everyone listening to the show, which basically gets access
22:58
Ejaaz: to frontier level intelligence for the cost of next to nothing.
23:02
Ejaaz: Download it completely free and run it privately at home.
23:05
Ejaaz: On this tweet that I have pulled up here, it basically says for every closed
23:09
Ejaaz: model, there is an open source alternative in it.
23:12
Ejaaz: And it goes through a list like Sonnet 4.5, you've got GLM 4.6,
23:15
Ejaaz: Grok Code Fast, you've got GPT OSS, GPT 5, you've got Kimi K2 Thinking.
23:20
Ejaaz: And it just goes on and on and on. And if we look at this kind of like a year
23:23
Ejaaz: and a half ago, maybe even two years ago, this list would be non-existent.
23:27
Ejaaz: It would just be Frontier AI Labs on the closed source side and zero open source side.
23:32
Ejaaz: So to see this kind of progress is really, really encouraging.
23:35
Josh: Yeah, it's going to be a race. It's going to be a battle between open and closed source.
23:39
Josh: And perhaps that's not even the battle. Perhaps it's open source until they
23:42
Josh: catch up to closed source.
23:44
Josh: And then it's closed source across the board. So it's going to be interesting
23:47
Josh: to see the developments.
23:48
Josh: We have a new batch of models that are coming. We're kind of in this weird limbo
23:52
Josh: where Gemini 3 is hopefully coming soon. We'll have some new benchmarks.
23:55
Josh: And one of the things that was this harsh truth to kind of wrap my head around,
23:59
Josh: which is what you just mentioned, Ejaz, and the fact that everyone's just compute constraint.
24:03
Josh: Like OpenAI could have made GPT-5, probably twice as impressive if they really
24:07
Josh: wanted to they just have no compute to serve that and it would have been way
24:10
Josh: too expensive and way too slow so it's not that it's
24:13
Josh: they can't it can't be done it's just that people don't have the resources to
24:16
Josh: do it so it's this constant balancing act and it's gonna be fun to see how
24:21
Josh: how companies kind of slot themselves into that that curve of like how much
24:24
Josh: they want to spend on compute versus cost versus just what they have available
24:28
Josh: to actually use to train these models and deploy them at scale to users
24:32
Ejaaz: And that's it for today folks um super fun episode.
24:36
Ejaaz: It is always surprising to me how quickly open source catches up with closed
24:41
Ejaaz: source centralized AI. I always think kind of like it's going to lag a few years
24:45
Ejaaz: and now it's come down to the fact that it's lagging a few weeks.
24:48
Ejaaz: We have a jam-packed week. We have potentially a new nano banana model being
24:53
Ejaaz: released by Google tomorrow. Fingers crossed.
24:55
Josh: I'm praying for that.
24:56
Ejaaz: Fingers crossed. I'm also praying for that as well. And we have a second episode
25:00
Ejaaz: based on tesla's investor day which had some really jam-packed exciting news now listen,
25:05
Ejaaz: If you want the U.S. to win this AI race, and make no mistake,
25:10
Ejaaz: it is a race, you need to subscribe to American AI YouTube channels, one of which is us.
25:16
Ejaaz: Please subscribe, hit the notification button, wherever you're listening to, give us a rating.
25:20
Ejaaz: We are helped by these so much. It is bringing up so much awareness.
25:25
Ejaaz: The algorithm is favoring us. We're getting all these wonderful views and new incomers.
25:28
Ejaaz: We've got a thousand of you from last week, which is just insane.
25:32
Ejaaz: Hello, welcome to the channel.
25:33
Ejaaz: We hope you enjoyed the content and we will see you on the next one.
25:37
Josh: Yeah before i let them off the hook i'm checking i'm doing the stat update 83
25:41
Josh: percent of the people that watched last week were not subscribed if you're watching
25:44
Josh: this on youtube don't come on guys
25:46
Josh: or go on spotify my preferred place of finding this podcast it's the best i'm telling you
25:50
Josh: i don't know how to describe this to people any better spotify is so good you
25:53
Josh: have the video you have the audio you could turn it off and lock your phone
25:56
Josh: without needing a premium membership please go go over there go leave a comment
25:59
Josh: over there because also the comment section is kind of popping too so yeah anyway
26:03
Josh: thank you for all this we do
26:04
Ejaaz: Not pick and choose wherever you listen go for it.
26:08
Josh: There you go all right we will see you guys in the next one thank you for watching
26:11
Josh: as always much appreciated um peace