Latest / In the Loop AI / Are Cheaper AI Models Better than Claude and ChatGPT?

Transcript

Are Cheaper AI Models Better than Claude and ChatGPT? · In the Loop AI

0:00
Josh: Just last week, in the span of about 48 hours, three of the most powerful AI
0:04
Josh: labs on the planet all shipped brand new models.
0:07
Josh: Elon shipped Grok 4.5, OpenAI took 5.6 Global, and Meta, for the very first
0:13
Josh: time in history, put a price tag on its own frontier model.
0:16
Josh: And here's where it gets interesting. For the last five years or so,
0:19
Josh: the deal in AI was that every time a model shipped, it got smarter and cheaper at the same time.
0:23
Josh: But that kind of died this year with Anthropics Fable 5 and these new frontier
0:27
Josh: launches like GPT 5.6. So today, we're asking the question that probably everyone should.
0:32
Josh: When frontier intelligence costs less than a cup of coffee, when they cost just
0:36
Josh: a few pennies per millions of tokens, what does that look like in terms of your
0:39
Josh: costs and how much you use these models?
0:41
Josh: I mean, this is a totally different paradigm now. We have a very clear separation
0:44
Josh: between Fable and 5.6 Sol and Grok 4.5 and Meta's new model, MuseSpark 1.1.
0:51
Josh: And it seems like these models are kind of diverging in a way that's really
0:54
Josh: interesting, particularly centered around price.
0:56
Ejaaz: For the last couple of years, the ultimate validation of whether your AI model is good or not.
1:02
Ejaaz: Is how intelligent it is, how smart it is. And no one really cared about cost.
1:08
Ejaaz: And then Fable 5 kind of came on the scene. And I think it was like $20 or $30
1:14
Ejaaz: input and like $80 output.
1:16
Ejaaz: And companies that were spending tens to hundreds of millions of dollars started
1:19
Ejaaz: to think, is this right? Does this make sense? Do I need the smartest model to do every single task?
1:25
Ejaaz: And so we hear the likes of XAI, Elon's company. We look at the likes of Meta.
1:31
Ejaaz: They released these models and we compare it to Fable 5, we're like,
1:33
Ejaaz: they're not actually that smart.
1:35
Ejaaz: But that's the whole purpose of the models that they're releasing.
1:38
Ejaaz: They're not trying to be as smart as Fable. In fact, they're betting on the opposite.
1:41
Ejaaz: They're betting that the cheapest model per unit intelligence is the model that
1:46
Ejaaz: will ultimately fill the middle ground.
1:48
Ejaaz: That'll be the ultimate model that is embedded in every single enterprise and
1:52
Ejaaz: used by every single user.
1:53
Ejaaz: Because the truth is you don't need the most intelligent model to do your task.
1:57
Ejaaz: Maybe if it is 90 to 95% of the intelligence of the smartest model ever,
2:02
Ejaaz: but it costs one-tenth of the price.
2:04
Ejaaz: It's a no-brainer that you're using these models at scale. And like you said,
2:08
Ejaaz: Josh, over the last week, there have been two particular companies,
2:11
Ejaaz: Meta, who we've known and spoken
2:14
Ejaaz: about a lot on this show, who have spent upwards of, I think, $35 billion.
2:18
Ejaaz: To try and build the world's best model, came out with their new Meta Mew Spark 1.1.
2:23
Ejaaz: And then you had SpaceX AI, who recently IPO'd, and there's a lot of pressure
2:27
Ejaaz: writing on them, release their new Grok 4.5 models. Now, are they as good as Fable? No.
2:32
Ejaaz: But are they cheap enough to use at scale when you're using or spinning up agents
2:37
Ejaaz: or when you're trying to figure out that long, complex task that's going to
2:40
Ejaaz: take tens of hours and you don't want to burn very expensive Fable tokens?
2:44
Ejaaz: These models might be the ones to choose. And I think it's probably worth covering a bunch of them.
2:48
Josh: Yeah, there's a new meta almost in town where it's like a new model doesn't
2:52
Josh: necessarily mean higher intelligence.
2:53
Josh: It could just mean higher efficiency. And I think that's where we're going to
2:56
Josh: start with with the Grok 4.5 release, because this is kind of like an opus class
3:00
Josh: claim, but at a third of the price, which is a pretty big deal.
3:03
Josh: So this new Grok 4.5 model, it's built on XAIs or I guess SpaceX AIs,
3:09
Josh: their version nine foundation model, which is about 1.5 trillion parameters.
3:13
Josh: And for reference, the version we've been using all year, if you've used Grok
3:16
Josh: at all this year, that is the version eight small, which is about 500 billion parameters.
3:20
Josh: So we're looking at about a three times multiple in parameter count,
3:23
Josh: which generally speaking is three times better, but probably a little bit more.
3:27
Josh: It seems like this is a serious increase relative to what we've been using.
3:30
Josh: And Elon has called this Opus class model. But instead of just being this like
3:35
Josh: highly expensive, very slow model, it's much more quick and it's much more efficient
3:40
Josh: when it comes to how many tokens you're able to generate for that same dollar.
3:44
Josh: And this is very clearly the route that you could see SpaceX AI has been trying to go for a long time.
3:49
Josh: They're very hardcore engineers. They love the engineering challenge.
3:52
Josh: And what they're trying to do now is figure out how you can kind of sculpt these
3:57
Josh: GPUs that they're training on to get as efficient as possible.
4:01
Josh: We spoke a few weeks ago, Ijaz, about the etched guys, like the startup who
4:04
Josh: is building their own training architecture chip stack. And they're basically
4:09
Josh: building their own servers.
4:10
Josh: And within that, they're pretty big to make one specific thing work,
4:13
Josh: which is the transformer.
4:14
Josh: And we talked about how gpus are not very efficient they don't actually use
4:17
Josh: like sometimes up to 60 percent of the gpu isn't used what grok and the spacex ai team are doing
4:23
Josh: is they are taking that code base they're really getting down to the bare metal
4:26
Josh: to figure out how to squeeze the most juice out of it and that's what this model
4:29
Josh: is that's what 4.5 is it has
4:32
Josh: two dollars in six dollars out per million tokens generated
4:36
Josh: And it seems like it's incredibly efficient when comparing it to other models like Opus.
4:41
Josh: It appears as if it's up to four point times fewer tokens needed in order to reach the same task.
4:47
Josh: So that's like an adjusted multiple of what it says, 17 times less than Opus.
4:51
Josh: That's like a really big deal for a model as it relates to cost, at least.
4:55
Ejaaz: The major unlock that you have with Grok 4.5 is it's a model meant for building.
5:00
Ejaaz: So if you're a hobbyist out there, if you're a software engineer and you want
5:02
Ejaaz: to tinker with some of these models and try and build something,
5:05
Ejaaz: but you know it's costly and expensive. You don't want to use the Fable 5 API.
5:09
Ejaaz: This is a really, really good model to use because it's so efficient.
5:13
Ejaaz: So some of the stats you were referencing right there, Josh,
5:16
Ejaaz: basically it uses four times fewer tokens than Claude Opus 4.8.
5:21
Ejaaz: What that results in is a 17x less price or cost to do the same task.
5:28
Ejaaz: And the way that they've been able to achieve this is arguably the funnest or coolest part.
5:33
Ejaaz: They spin up a bunch of different agents to solve different problems of the
5:37
Ejaaz: tasks that you've asked it to at the same time in parallel.
5:40
Ejaaz: And this is a growing trend you'll see with the other models that we're going
5:43
Ejaaz: to talk about on this show today, where agentic coding or agentic reasoning
5:47
Ejaaz: or using agents to solve your problem has been a major unlock for these cheaper
5:52
Ejaaz: models to be cheap in the first place.
5:55
Ejaaz: The other thing I want to mention about Grok 4.5 in particular,
5:57
Ejaaz: because it's very unique to them and SpaceX AI is they just acquired a company called Cursor for...
6:03
Ejaaz: You know, a low price of $60 billion. And the major unlock that cursor gave
6:08
Ejaaz: them is they have all this data around how users use coding models.
6:13
Ejaaz: Now, it's very specific. It's not what they're coding. It is how they use the models.
6:18
Ejaaz: And this data was very important in training Grok 4.5 to become smarter at routing people's requests.
6:26
Ejaaz: So let's say you ask it to build an app that can change your wardrobe into something
6:33
Ejaaz: way better. And you send it like pictures of your camera or whatever that might
6:35
Ejaaz: be. We talked about this as an example on yesterday's episode.
6:39
Ejaaz: It'll be able to know which model to use at what time for how long and calculate
6:45
Ejaaz: the costs preemptively to make sure that it's not burning your wallet.
6:48
Ejaaz: It's a really, really smart model. And Elon himself has said,
6:52
Ejaaz: listen, this isn't gonna be the smartest model. We're working on better models,
6:55
Ejaaz: but it is a really good daily driver. It's a really good workhorse.
6:59
Ejaaz: And if you're anyone at a company that doesn't want to burn, you know,
7:03
Ejaaz: $30 in and $80 out on Fable 5 or whatever the cost is, something that's like
7:07
Ejaaz: 10x more, you should use this model for 80 to 90% of the work that you're trying
7:12
Ejaaz: to do, and then use Fable to kind of orchestrate the plan or the design or whatever that might be.
7:15
Ejaaz: The final point I'm going to make in SpaceX AI's favor, because you might be
7:19
Ejaaz: thinking, okay, fine, whatever, but like, when's the next model going to come?
7:22
Ejaaz: They took like, whatever, nine months to release this upgrade.
7:25
Ejaaz: Elon is cooking up three more models that are in order of magnitude larger than
7:31
Ejaaz: the model that we're talking about today.
7:33
Ejaaz: So right now, in under a month, we're expecting to see Grok 5,
7:37
Ejaaz: and Grok 5.5 is also being cooked at the same time as well.
7:40
Ejaaz: And these are like 5 to 10 trillion parameter models. So it's feasible to say
7:44
Ejaaz: that we'll have a Mythos class model from SpaceX AI in a couple of months' time.
7:48
Josh: Which is awesome. Yeah, this is the part that's most exciting to me.
7:51
Josh: Is like it's very obvious that spacex ai is the best at the engineering part
7:55
Josh: of generating tokens they're
7:57
Josh: have the whole vertical integration now they have the ability to build out these
7:59
Josh: data centers they have the data centers running and now they have the harness
8:03
Josh: through cursor and they also have the data set that they've used through cursor in order to kind of
8:07
Josh: achieve this fully integrated stack and what's funny now is if they are continuing
8:11
Josh: to release models that are bigger and bigger
8:13
Josh: if they ever do run up against a compute wall they remember they have that deal
8:17
Josh: with anthropic and with google i believe and
8:21
Josh: they're gonna have to figure out who's who's gonna get the short end of that
8:24
Josh: stick because it doesn't seem like they're gonna have gpus for everyone but
8:27
Josh: like you mentioned the the exciting thing here is that like this is version
8:30
Josh: one of it feels like spacex ai 2.0 this is their
8:34
Josh: second try at getting to the frontier the first time they may have gotten there
8:37
Josh: for a couple days but it wasn't very long lived now we're getting new models
8:40
Josh: every single month and the goal
8:43
Josh: around august is a two trillion parameter model and then the goal for grok 5
8:48
Josh: which is coming hopefully not too long after is 6 to 10 trillion parameter models
8:52
Josh: like this is this is going to put them right up at the frontier and if they're
8:55
Josh: able to serve these tokens
8:57
Josh: at a fraction of the cost that say gpt6 is going to be or mythos 6 or whatever
9:01
Josh: comes next that's going to be a pretty serious
9:04
Josh: competitor in the ai space because they're going to be right at that frontier
9:07
Josh: with a very low cost model that i think a lot of people
9:10
Josh: Are going to find a lot of use for now that is the grok update there is a second
9:15
Josh: update that is just as noteworthy and probably even more noteworthy actually
9:19
Josh: because this is the first time that meta is charging for tokens
9:23
Josh: meta famously they have been the open source kings they have always wanted to
9:28
Josh: publish the models open source to move the needle forward to kind of make this
9:32
Josh: open source developer community thrive
9:34
Josh: unfortunately if you are a participant of that your time has ended because now
9:39
Josh: meta is closed source and they are releasing
9:41
Josh: these closed source models they are charging via the api it's not a lot of money
9:44
Josh: but it was big enough news for mark zuckerberg to come back on x after a what
9:49
Josh: was a three or four year hiatus and actually announce
9:51
Josh: muse spark 1.1 which he describes as a strong agentic and coding model at a
9:56
Josh: very low price it's available through our new meta model api and in the meta
10:00
Josh: ai so ejes the question i have for you
10:02
Josh: because you know i'm i haven't been the most excited about meta recently
10:06
Josh: their offerings have left a lot to be desired as it relates to AI.
10:09
Josh: Is this a serious model? Like, is this worth actually paying for relative to
10:14
Josh: all the other models that exist today?
10:16
Ejaaz: Short answer is yes. And this comes from a professional meta hater.
10:19
Josh: Which is shocking. Like this is this is a novel breakthrough.
10:22
Josh: This is an exciting announcement for meta.
10:24
Ejaaz: Well, I'm just happy to see something competitive enough for the,
10:29
Ejaaz: a lot of people to use in the AI market. And I'll tell you why they would use this model.
10:33
Ejaaz: Number one, the reason right at the top is it is the cheapest model per unit
10:40
Ejaaz: cost of intelligence. So what do I mean by that?
10:42
Ejaaz: You can have cheap models, but they're pretty crappy and you won't end up actually
10:46
Ejaaz: using it for serious work.
10:48
Ejaaz: This is a model that you'll end up using for serious work and it costs 25% of
10:53
Ejaaz: the price of the Frontier model. Now, can it do everything a Frontier model
10:57
Ejaaz: can do, like what Fable 5 does? No. And Mark Zuckerberg openly admits that.
11:02
Ejaaz: But it is an absolute workhorse. You can throw it at a task and it can work
11:07
Ejaaz: for hours, or you could throw it at multiple problems at once and it can figure it out.
11:12
Ejaaz: But there's a few other advantages that this model in particular has.
11:15
Ejaaz: So aside from the price, which is, by the way, $1.25 in per million tokens and $4.25 out.
11:22
Ejaaz: Now, if you want a comparison as to how crazy cheap that is,
11:26
Ejaaz: that is cheaper than GLM's model, GLM 5.2, which is the leading open source
11:31
Ejaaz: Chinese model right now.
11:32
Ejaaz: And they're known for being the cheapest model. So the fact that Mark Zuckerberg,
11:36
Ejaaz: there's some blissful irony there, actually, with the open source thing and
11:38
Ejaaz: the fact that, you know, he was competing with China and they beat him.
11:41
Ejaaz: To come back and offer the cheapest model is pretty amazing. But the second thing is.
11:46
Ejaaz: And like Grok 4.5, it uses agents very intelligently. So it has this thing called
11:51
Ejaaz: a master agent in this model.
11:53
Ejaaz: And the master agent reads your prompt and it thinks very diligently about a
11:57
Ejaaz: plan to answer and execute your prompt.
12:00
Ejaaz: Now that sounds very vague, right? You're like, oh, aren't the other models
12:02
Ejaaz: doing it? No, when you look at like Fable 5, when you look at GPT 5.6,
12:06
Ejaaz: it processes its entire model weights, which is gargantuan by the way.
12:11
Ejaaz: And that ends up being very costly. Meta found a sneaky little way to circumvent
12:16
Ejaaz: this, which is have a master agent, have it plan, and then have it delegate
12:19
Ejaaz: to a bunch of different agents.
12:21
Ejaaz: The third thing that is very impressive about this model is that it's an omni model.
12:25
Ejaaz: So it can take in video, it can take in images, it can take in text all at once
12:30
Ejaaz: and understand how to use that intelligently.
12:33
Ejaaz: So if you give it a video, you can extract that video, it knows that you want
12:37
Ejaaz: to list it or use it for a post that you have on Facebook Marketplace,
12:41
Ejaaz: and it'll be able to do that in one shot.
12:44
Ejaaz: And then the fourth and final thing that it's very good at is computer use.
12:47
Ejaaz: So this thing can take over your computer, take over your account or whatever
12:50
Ejaaz: that might be, and just know what you want it to do. Now, version one is very
12:54
Ejaaz: fine tuned to Meta's products, Instagram, WhatsApp, Facebook itself.
12:58
Ejaaz: But the idea is you can pretty much use this for any computer use work going
13:02
Ejaaz: forward. And I'm looking forward to version 1.2 and 1.3.
13:05
Ejaaz: They pulled this off, by the way, within a month of releasing Mew Spark 1.
13:11
Ejaaz: And they also released Mew's image and video in the same week,
13:15
Ejaaz: which tells me that the cycle of release similar to SpaceX AI is getting much quicker.
13:21
Ejaaz: And if you ask me why that is, it's because both Elon and Zuck have one thing
13:26
Ejaaz: that the two frontier labs, OpenA and Anthropic, don't have.
13:30
Ejaaz: A crap ton of compute. They have so much compute. They are the most aggressive
13:36
Ejaaz: hyperscalers and arguably the most successful hyperscalers. They have amassed the most GPUs.
13:41
Ejaaz: And if you still believe that compute scaling laws matter, Zuck and Elon are
13:45
Ejaaz: not out of the race. In fact, this proves that they're back into it.
13:47
Josh: One of the things that I found noteworthy of this is that this API presumably
13:51
Josh: runs on Meta's own silicon.
13:53
Josh: It's the chips that they have been making and producing and likely running in
13:56
Josh: fact there's a report that they are spending 250
14:01
Josh: billion dollars including chips over the next year in order to build out something
14:05
Josh: like 14 gigawatts of data center capability
14:07
Josh: and that is mostly going to be running their new mtia 400 chips which are basically
14:13
Josh: the in-house meta silicon which is 400 faster than the previous generation and
14:19
Josh: uses 51 percent more hbm so for the hbm folks
14:23
Josh: When we talk about all our investing videos um these chips are going to be using
14:27
Josh: a lot more of it and meta is now charging a quarter of the rival's prices which is a really
14:32
Josh: kind of interesting and noteworthy thing and that combined with the switch from
14:36
Josh: open source to closed source it kind of implies that like mark zuckerberg very
14:39
Josh: clearly thinks the model is now the product instead of the moat around
14:43
Josh: the model so i think traditional meta would have been no no we're going to release
14:47
Josh: the model we're going to build a moat around it and that is going to be product we monetize
14:51
Josh: this new shift implies like no actually the model is the product and we're going
14:54
Josh: to integrate it into all of our services but in order to do that we're going
14:57
Josh: to do this huge tremendous data center build out and they're going to
15:00
Josh: build their own proprietary chips and it seems like both those things are actually
15:04
Josh: going well and i have to ask it's like okay
15:07
Josh: what happens if they actually do it like what happens if there is 14 gigawatts
15:11
Josh: of compute running next year like that's that seems pretty considerable it's
15:15
Josh: particularly running on their own silicon
15:17
Josh: which we know has tremendous competitive advantages when it comes to training your own models.
15:21
Ejaaz: Well, here's the bet that like we're making, we've made on this show multiple
15:26
Ejaaz: times on previous episodes, is that the future of AI isn't you tapping a bunch
15:32
Ejaaz: of buttons and approvals every single second.
15:35
Ejaaz: It is you write a prompt, you send the prompt, and then the AI just kind of
15:39
Ejaaz: knows what to do. It autonomously works.
15:42
Ejaaz: If you believe in that world, then you believe in a world of AI agents.
15:46
Ejaaz: And if you believe in a world of AI agents, these agents are going to be making
15:50
Ejaaz: tens to hundreds of thousands of tool calls per month.
15:54
Ejaaz: You don't want to be there clicking approve the entire time.
15:56
Ejaaz: And those tool calls are pretty expensive.
15:58
Ejaaz: So you want to go the cheapest route when you're using an agent to get the work
16:03
Ejaaz: done. That's basically the entire thesis for why cheaper models are more effective.
16:08
Ejaaz: And therefore, the labs who have the most compute and can use it most effectively,
16:13
Ejaaz: you mentioned, you know, their custom chips.
16:15
Ejaaz: I believe Elon and SpaceX are also working on their own chips.
16:18
Ejaaz: Open Air is doing their own with Jalapeno. This is a growing trend.
16:22
Ejaaz: It makes sense that the people that have the most compute and the best chip
16:26
Ejaaz: architecture will end up winning. And that's basically the bet that Meta and Elon are going after.
16:32
Ejaaz: I have to say, if we ground ourselves for a second and look at the counter-thesis...
16:37
Ejaaz: I do think Zuck is heavily subsidizing this model.
16:40
Ejaaz: I don't think there's any chance in hell that it actually costs $125 and $425
16:45
Ejaaz: output. I think he's subsidizing this massively.
16:48
Ejaaz: And I think he needs to prove a point to his investors or shareholders that
16:51
Ejaaz: it is worth the AI CapEx spend that he's probably going to announce at the end of Q2.
16:56
Ejaaz: So that's my bet. I think it's strategic. I think it's the right move,
16:59
Ejaaz: but I don't think he's unlocked some kind of major architecture redesign just yet.
17:04
Josh: Regardless, these are two really solid models between Grok 4.5 and now MuseSpark 1.1.
17:09
Josh: And to kind of place them in a spot relative to others, we can go down the price
17:14
Josh: list of other models and kind of compare what they're like. So at the top of
17:18
Josh: this list is Claude Fable 5, and that's $10 in, $50 out per million tokens.
17:23
Josh: GPT 5.6 comes next at just a little bit, close to half at $5 in, $30 out.
17:29
Josh: Then Opus is $5 in, $25 out.
17:32
Josh: And then it kind of goes down the line until we get to Grok 4.5,
17:35
Josh: which is two dollars in six dollars out then we have muse spark 125 and 425
17:43
Josh: and at the very bottom believe it or not is open ai with a dollar
17:46
Josh: in and six dollars out for gpt 5.6 luna so the middle section and the upper
17:53
Josh: section is kind of where the war is it's like we have those two models at the top
17:58
Josh: we have claude and we have gpt 5.6 soul
18:00
Josh: Those are very much competing on the intelligence curve. But then below that,
18:04
Josh: where we have this like kind of cluster of, you can think Gemini and the smaller
18:08
Josh: GPT models and Grok and Meta 1.1, that's where we are seeing this like price competition.
18:14
Josh: And it's funny to see this kind of divergence in strategies.
18:17
Josh: And I think that's a good way of looking at the frontier when you evaluate these
18:20
Josh: new models is, okay, is this a frontier intelligence model or is this a frontier price model?
18:26
Josh: And those two things now are very different because they're going to be used
18:28
Josh: for a very different set of use cases. And I think one of the more interesting
18:31
Josh: applications that we're going to be following on the show is how people route
18:35
Josh: through these models to do different tasks.
18:37
Josh: Like you said, some agentic tasks don't require necessarily the highest frontier intelligence.
18:41
Josh: Maybe you can get away with using a MuseSpark 1.1 for a lot of that,
18:45
Josh: and perhaps using a Fable 5 for orchestration of those agents, things like that.
18:49
Josh: So we're going to see what I suspect is a new meta start to come into play of
18:53
Josh: this orchestration at a high level and using different models for more particular
18:58
Josh: things, more specific things.
19:00
Ejaaz: Josh, have you heard of something called the Silicon Token Expenditure Index?
19:07
Ejaaz: It's basically this index which tracks the spending of companies or enterprises.
19:13
Ejaaz: I think the Bloomberg ticker is SDLLMTK.
19:18
Ejaaz: But the point of this index is it captures how much money is being spent by
19:23
Ejaaz: the top Fortune 500 on AI specifically.
19:27
Ejaaz: So if you're listening to us talking about cheap models and And,
19:30
Ejaaz: you know, talking about this thesis of cheaper models will be used more and you don't believe it.
19:35
Ejaaz: Well, you have to look at the customers who are actually ingesting this AI and
19:38
Ejaaz: the movements and actions that they're taking.
19:41
Ejaaz: And if I show you the index, you'll notice a particular trend over the last
19:45
Ejaaz: couple of weeks and months, which is it's down 20%, which basically means people
19:52
Ejaaz: are spending much less on tokens or per token.
19:55
Ejaaz: And they're also spending much less overall. What that indicates is they are
19:59
Ejaaz: looking for cheaper alternatives.
20:02
Ejaaz: And we're seeing this from the headlines that we've seen from Uber,
20:04
Ejaaz: from Meta themselves, from Microsoft, who are now adopting Chinese open source
20:08
Ejaaz: models into their product, into their Copilot product.
20:11
Ejaaz: We are seeing this shift of enterprises realizing that it's not about using
20:15
Ejaaz: the smartest model for every single task.
20:17
Ejaaz: It's about finding that middle ground. It's about finding the right model or
20:21
Ejaaz: maybe the right types of models to use for the middle ground of tasks.
20:26
Ejaaz: And that brings me to another point which is.
20:29
Ejaaz: I don't think it's necessarily just going to be one cheap model that dominates everything.
20:34
Ejaaz: I think they're going to realize after they've shifted to a cheaper model that
20:38
Ejaaz: they could use certain models for specific tasks.
20:41
Ejaaz: And that takes us down the path of routing, which is what Cursor has infamously
20:46
Ejaaz: figured out and what SpaceX figured out and acquired them for $60 billion.
20:49
Ejaaz: So there's these really interesting conversations around this trend,
20:52
Ejaaz: but it's being validated by this index that companies are going to be looking for cheaper models.
20:57
Ejaaz: And the irony is, what is the paradox, Josh, that we spoke about early on in
21:03
Ejaaz: the AI thesis, where the cheaper something gets, the more you end up spending
21:07
Ejaaz: on it? What is the name of that paradox?
21:10
Ejaaz: Jevon's Paradox. Jevon's Paradox. That's it. So the cheaper something gets,
21:14
Ejaaz: I expect to see way more tokens being spent because the output will actually
21:18
Ejaaz: be worth it instead of spending a very expensive prompt and getting maybe a
21:23
Ejaaz: mediocre answer from it.
21:25
Josh: Yeah, and this is in line with expectations. Goldman, they published this prediction
21:28
Josh: that token consumption would multiply 24 times between 2026 and 2030.
21:33
Josh: That's like a tremendous amount. That's 120 quadrillion tokens per month, which is unbelievable.
21:40
Josh: So even if the price of these tokens does go down some extent,
21:44
Josh: a multiplier of that much is still increased spend, which I think is important to note.
21:49
Josh: It's like, I don't think we're going to see a Fable class model be 124th the price very shortly.
21:56
Josh: But they're expecting that much increased demand and every single kind of
22:01
Josh: Every single prevailing wind is pushing towards longer-term agentic coding sessions,
22:06
Josh: where I feel like rarely do people actually now, or people who are using it
22:10
Josh: for productive uses at least, are using it just as a chatbot.
22:12
Josh: They're using it as an agent to do longer and longer and longer-term tasks.
22:16
Josh: And what we notice with these cheaper models in particular is that they actually
22:20
Josh: do oftentimes consume more tokens to get the same output at a higher quality.
22:24
Josh: I know we were mentioning this with 5.6 the other day, where some of their lower
22:28
Josh: models, they actually, they cost less, but they do use a considerable amount
22:32
Josh: more tokens to get to the answer.
22:34
Josh: So we're in this weird crossroads where we have these, like,
22:36
Josh: forcing functions that are pushing people to want to generate more tokens.
22:39
Josh: Companies want these tokens to be highly intelligent because they don't want
22:42
Josh: subpar work if they're paying for it.
22:44
Josh: But then they have this like enterprise belt tightening that's kind of happening
22:48
Josh: where like amazon killed it's like i think they had their internal token leaderboard
22:51
Josh: and they totally removed it
22:53
Josh: um coinbase and walmart are setting up usage caps a lot of companies are kind
22:56
Josh: of like crunching down on usage of ai internally because they can't really figure out how to justify
23:03
Josh: the value that's been given to the company so there's a lot of these weird it
23:06
Josh: very much feels like we're at a crossroads now and there's this new paradigm
23:09
Josh: shift that is happening as it relates to cost in particular and it's going to
23:12
Josh: be really interesting to see how this plays out
23:14
Ejaaz: There is one metric that i will be tracking to see whether these cheap models
23:22
Ejaaz: are actually good enough and it's simple,
23:25
Ejaaz: are these companies making money? When do they turn profitable?
23:30
Ejaaz: With Meta, I'm almost certain that they're subsidizing their cost of their model.
23:35
Ejaaz: With SpaceX AI, I don't know, maybe they're doing something similar.
23:39
Ejaaz: But until these guys post a positive revenue earnings on their quarterly earnings
23:44
Ejaaz: or whatever that might be, I'm not going to be convinced that cheap models are the way forward.
23:49
Ejaaz: I think it will directionally play out, but that's going to be a big metric
23:52
Ejaaz: to track. Now, if you compare that to the likes of Anthropic,
23:55
Ejaaz: they're not a public company yet. They're rumored to IPO maybe later this year.
24:00
Ejaaz: But there are rumors that they have turned profitable in Q2.
24:06
Ejaaz: And if true or if confirmed, they'll be the first AI lab to do so.
24:11
Ejaaz: And the major way that they've been able to do that is not only creating amazing
24:15
Ejaaz: models that are used by every single enterprise in the world,
24:18
Ejaaz: but because it's expensive, because they pay for the cost that you're getting.
24:22
Ejaaz: And at the end of the day, if you're making a lot of money and people continue
24:25
Ejaaz: using it, maybe that is the right way to go through. So that that would be the
24:28
Ejaaz: only the counterpoint to this.
24:31
Ejaaz: The final thing I'll say on this topic, I think, is.
24:35
Ejaaz: What I was alluding to earlier, which is I'm now convinced that,
24:40
Ejaaz: number one, it's not just going to be OpenAI and Anthropic ruling the entire world.
24:44
Ejaaz: I think it's going to be multiple labs. And I think that's ultimately a very
24:47
Ejaaz: good future, right? Maybe five, six, maybe 10 labs, right?
24:51
Ejaaz: And I think they're going to be so many different models to use for so many
24:55
Ejaaz: different purposes. I mean, look at Metamuse Spark 1.1.
24:58
Ejaaz: It topped the health benchmark. Did you know that?
25:01
Ejaaz: Like a random social media company's model topped the health benchmark.
25:05
Ejaaz: Grok 4.5, really good at computer use and spinning up agents.
25:09
Ejaaz: Luna from OpenAIR, also really good at computer use. So all these different
25:13
Ejaaz: models would be good for various different tasks.
25:16
Ejaaz: And if you don't want to spend time thinking about which subscription to get
25:20
Ejaaz: or what model to use or when, I think routing companies.
25:24
Ejaaz: So what cursor basically built, which is you can type in a prompt and they decide
25:28
Ejaaz: which parts of the prompt gets used by which kind of models will be the killer platform or product.
25:33
Ejaaz: I don't know of any company that is building this that hasn't been acquired, aka Cursor.
25:38
Ejaaz: Maybe OpenRouter. We've had the founder, Alex Atala, on our show before.
25:42
Ejaaz: Maybe they pivot into some kind of a routing product. But I'm convinced now
25:46
Ejaaz: that it's going to be a layer that sits on top of these models, potentially.
25:50
Josh: Yeah. I think the way I'm looking at it is like, okay, now there's two markets.
25:53
Josh: This used to be a singular commodity, a singular race to the top.
25:57
Josh: Now there's these two things. And like cost per intelligence is the right lens
26:00
Josh: for looking at maybe 80% of the work that's become like kind of routine,
26:05
Josh: but it is the opposite and exactly the wrong lens for like the 20% where
26:09
Josh: capability and reliability and verification matter.
26:13
Josh: It's like if you are doing really complicated work where mistakes cost you a
26:15
Josh: tremendous amount of money and time, if you're doing frontier math or problem
26:19
Josh: solving, or you're building complicated things as a business,
26:23
Josh: there's almost no limit to the amount that you'll spend in order to accomplish your goals.
26:27
Josh: Because I'm sure many of these companies have a tremendous amount of ideas that
26:30
Josh: they want to implement to the market.
26:32
Josh: And the constraint is their workforce and knowledge base that could actually deploy that.
26:36
Josh: There is no limit to the amount that they will spend on these Fable models,
26:39
Josh: on these GPT 5.6 and GPT 6.0 models.
26:42
Josh: But for the rest of the world, maybe that's not the case. They are price sensitive.
26:45
Josh: They don't need the frontier intelligence for everything.
26:48
Josh: And that's where a lot of these models are going to really play a serious role.
26:51
Josh: And i think that is cool that like we no longer have intelligence as the singular
26:55
Josh: commodity it is now split between intelligence and price those two things both
26:59
Josh: matter and i think that's kind of where we are in the market right now it's
27:02
Josh: like we are at this crossroads the commodity has diverged into two
27:06
Josh: and we have some pretty serious players in the pricing game who have a very
27:10
Josh: serious trajectory of actually continuing to be serious threats spacex ai is
27:14
Josh: building a huge amount of data centers they have a very clear trajectory to a
27:17
Josh: 20 trillion parameter model meta is going to put 15 gigawatts on the ground power it on
27:23
Josh: and they're going to do that vertically integrated with their own silicon and
27:26
Josh: like that's a pretty serious threat to the pricing war because you have to imagine
27:29
Josh: they're going to get some
27:30
Josh: amazing efficiency from that so yeah i think that's the update for
27:34
Josh: The state of low-cost models. It's been a very interesting week for them.
27:38
Ejaaz: As we're recording this episode, I'm realizing that we need to make another
27:42
Ejaaz: episode on these trends that we've spoken about during this talk.
27:48
Ejaaz: Primary ones being, okay, if the most expensive model lab or the most intelligent
27:53
Ejaaz: model lab isn't necessarily going to win over the next couple of years and it's
27:57
Ejaaz: going to be this middle ground, this cheap middle ground, which companies are
28:00
Ejaaz: there? Where are the bottlenecks?
28:02
Ejaaz: How are people using these things?
28:03
Ejaaz: What will agents look like when they're working on long horizon tasks?
28:07
Ejaaz: And most importantly, which companies, which customers are going to be using
28:11
Ejaaz: this thing? I think, I mean, you let me know, as you as the audience that are
28:14
Ejaaz: listening to this right now, would that be an interesting episode to sort of unpack and dig into?
28:18
Ejaaz: All of that said, I think that cheaper models are here to stay.
28:23
Ejaaz: And I think they're here to stay in a very big way. And I think they're going
28:26
Ejaaz: to come out, thankfully, from some frontier Western labs.
28:29
Ejaaz: We're shifted away from building the most expensive and intelligent model that
28:34
Ejaaz: is being kind of blocked to anyone and everyone to a $1.25 in,
28:38
Ejaaz: $4.25 out model from Meta, which surprisingly works and is intuitive.
28:43
Ejaaz: I'm also excited to see what people do with these cheap models.
28:47
Ejaaz: I don't think they're necessarily going to build the same things that people
28:51
Ejaaz: using Fable 5 are going to do simply because they're not prohibited by cost anymore.
28:55
Ejaaz: And I think that kind of unlocks a new line of creativity when you're building
29:00
Ejaaz: these apps. You're like, oh, you know what? Maybe I will try this crazy thing
29:02
Ejaaz: because it's only going to cost me 10 bucks.
29:04
Ejaaz: Heck, I'll just do it. So if you're listening to this.
29:07
Ejaaz: And you're inspired by some of the things that were spoken about,
29:09
Ejaaz: go and try some of these models. They're super cheap to use.
29:12
Ejaaz: And tell us what you end up building with it. Tell us if there's anything creative
29:15
Ejaaz: that you haven't seen being built by Fable 5.
29:18
Ejaaz: We would love to see it, demo it, heck, maybe even show it on an episode in the future.
29:22
Ejaaz: But if you enjoyed this, please, please, please share this with your friends.
29:26
Ejaaz: If you're not subscribed, please subscribe.
29:28
Ejaaz: If you haven't left us a comment, we always want to hear from you.
29:31
Ejaaz: We love hearing from you guys.
29:32
Ejaaz: And if you're listening to us on Spotify or Apple Music, please,
29:35
Ejaaz: please give us a rating. it helps us out pretty massively josh any any final thoughts.
29:40
Josh: No that's it thank you guys so much for watching um on sponsor front we're chatting
29:44
Josh: with some people this has been going well so for the people who have reached
29:46
Josh: out thank you we are working our way through those we are
29:49
Josh: slowly working towards becoming a self-sufficient entity which has been amazing
29:52
Josh: so thank you so much for the support on that again if you know anyone please
29:55
Josh: refer them send them our way
29:57
Josh: either on x email and description whatever it may be
29:59
Josh: but that is another episode next coming up is the roundup which is i think our
30:03
Josh: favorite episode of the week we just throw everything that we haven't been able
30:05
Josh: to talk about into one and do a full recap on everything that has happened this week which is a lot
30:10
Ejaaz: We got to talk about the data center bands dude like new york is my city why are we banning the day so.
30:15
Josh: We can have a lot of conversation about that one so stay tuned stay tuned for
30:20
Josh: that grieving session um but anyways that is the episode today thank you all
30:24
Josh: so much for watching as always and we will see you in the next one