Latest / In the Loop AI / Google Just Made Their AI Free, Private, and Yours (Gemma 4)

Transcript

Google Just Made Their AI Free, Private, and Yours (Gemma 4) · In the Loop AI

0:00
Josh: How much money are you paying to use your AI model? Maybe it's $20 a month,
0:04
Josh: maybe you're on the pro plan for $200 a month, maybe you're running an OpenClaw
0:07
Josh: instance and you're paying thousands of dollars a month to generate tokens from Frontier Models.
0:11
Josh: Google has just released a solution to your problem, something that can be solved
0:15
Josh: for as little as an $80 one-time fee just to run a Raspberry Pi,
0:19
Josh: because that's what this new model runs on.
0:22
Josh: Gemma 4 is a new model from Google that is a hyper-quantized,
0:26
Josh: very small model meant to run locally on devices like your phone or your laptop
0:31
Josh: or even your new MacBook Neo.
0:34
Josh: It's very lightweight and it's built for working offline entirely private and
0:38
Josh: I think the thing that's most noteworthy is how powerful it is.
0:42
Josh: This model that is small enough to fit on your phone and run entirely for free
0:45
Josh: is just as good if not better than some of the Frontier models last year and
0:49
Josh: is even close to performing as well as them this year.
0:52
Josh: Now to showcase this we have some really cool examples that EJS has prepared
0:55
Josh: here so let's get into what this new Gemma 4 model can do.
0:58
Ejaaz: Yeah, I'm super excited about this model. I mean, it's not just one either.
1:02
Ejaaz: There's four of them. And like you said, it ranges from like 4 billion to,
1:06
Ejaaz: I think it's like 50 billion parameter models.
1:09
Ejaaz: Can fit on your phone, can fit on any device. And like you said,
1:12
Ejaaz: eight months ago, this would have been considered frontier intelligence.
1:15
Ejaaz: But I want to get into like what these things can actually do because it's one
1:19
Ejaaz: thing talking about benchmarks.
1:20
Ejaaz: It's another thing talking about what it can do on your phone,
1:23
Ejaaz: on your laptop, what value it can bring to you. This first example is someone
1:28
Ejaaz: leveraging the visual intelligence of these Gemma models.
1:31
Ejaaz: Now, typically, if you're an AI model, you're really good at ingesting words
1:35
Ejaaz: and characters and understanding the word described to you like a book would
1:39
Ejaaz: or like a blog post would.
1:40
Ejaaz: Visual intelligence is a very different frontier that has often been hard to
1:44
Ejaaz: kind of surmount by these new AI models.
1:47
Ejaaz: Gemma does an amazing job. What you're looking at on your screen right now is
1:51
Ejaaz: its ability to identify all the different objects in what is a very crowded room.
1:56
Ejaaz: He raises up a banana and it identifies that.
1:59
Ejaaz: It's also spotting the books that are on his shelf in the back.
2:02
Ejaaz: It's spotting the shelf in itself, the lamp, the fact that he's in a room,
2:05
Ejaaz: the curtains around that.
2:06
Ejaaz: And that's really important when it comes to creating apps that can log your
2:10
Ejaaz: visual experience or can track your calories for the food that you're consuming.
2:14
Ejaaz: And you can build a completely different suite of apps based on intelligence like this.
2:19
Ejaaz: This is the first time that we're seeing it appear in an open source open weight
2:23
Ejaaz: model. And Google's been the first to launch that.
2:25
Josh: Yeah. And if it wasn't abundantly clear, this is totally free.
2:28
Josh: You could just go on the website, download this and run it yourself.
2:30
Josh: And I think looking at this vision example, one of the cool things that I'm
2:33
Josh: thinking of is a lot of people have cameras outside their house,
2:37
Josh: outside their apartments.
2:38
Josh: And this has visual intelligence now to not only see things,
2:42
Josh: but alert you of what it's seeing.
2:43
Josh: One of the cool examples that I saw that we don't have teed up here is just
2:46
Josh: someone who had a little nest cameras right outside the front door.
2:49
Josh: And it would send a notification of what was happening.
2:52
Josh: It's like there is a dog walking in front of your house. There is a man walking
2:56
Josh: up with two packages in his hands. It looks like the package is from Amazon.
2:58
Josh: And it has that visual intelligence that would normally cost quite a bit of
3:02
Josh: money in those tokens using something like Claude Opus or ChatGPT.
3:06
Josh: But instead, it does it all for free on this tiny little model,
3:08
Josh: which is super cool. We have another example that was mentioned in the intro about OpenClaw here.
3:13
Josh: And OpenClaw is something that a lot of people spend a lot of money on.
3:17
Josh: If people are real hardcore users they're spending hundreds of dollars a day
3:21
Josh: up to hundreds of dollars a day some even thousands of dollars a day in addition
3:24
Josh: to buying some pretty expensive hardware to run it on a lot of people bought
3:28
Josh: mac minis you can't get a mac mini if you wanted to because they're so backordered
3:31
Josh: mac studios people were paying hundreds if not thousands of dollars to run.
3:35
Josh: This software on and the reality is is that
3:38
Josh: whatever device you're watching this on, whatever device you're listening to
3:41
Josh: right now, you can run this model on.
3:43
Josh: You don't need something fancy. You don't need a high-end computer to run it.
3:46
Josh: You can just do this on something local, lightweight, and like I mentioned,
3:50
Josh: as lightweight as an $80 Raspberry Pi because you can use the lightest weight model possible.
3:55
Josh: And although the results aren't the best in the world, they are much better
3:58
Josh: than previously expected from these open source models.
4:02
Ejaaz: Yeah, I love that you can finally run OpenClaw on a device that doesn't cost $1,000.
4:07
Ejaaz: These Mac minis actually on the secondary market have gone up sky high.
4:10
Ejaaz: Like the retail price is 800 bucks because you can't get it from Apple anymore.
4:14
Ejaaz: I've seen it as high as like 2K and people are still buying these things.
4:17
Ejaaz: Now, the reason why they've been buying these things is because they can't fit
4:20
Ejaaz: Frontier open source models onto their own mobile phone or their own laptop.
4:26
Ejaaz: And now Gemma 4 has made it super easy to do. So that's amazing.
4:29
Ejaaz: I'm still quite confused as to what the open core users are burning thousands
4:32
Ejaaz: of dollars for on, but like that's probably a topic for another conversation.
4:37
Ejaaz: The other thing that I like about this is Gemma 4 can run completely offline.
4:41
Ejaaz: Now this is a common property and characteristic that you
4:44
Ejaaz: can have for every single open source model but the fact is you
4:47
Ejaaz: have a model here that is near frontier intelligence to say Claude Opus 4.6
4:52
Ejaaz: and GPT-504 and we'll get to those direct comparisons a little later on in this
4:56
Ejaaz: episode but now you can run it offline and the great part about this is often
5:01
Ejaaz: you're in areas where you just don't have internet connection or it takes a while to inference.
5:05
Ejaaz: Now you have it on your phone, you can have it completely offline,
5:07
Ejaaz: it gets access to the world's entire database of knowledge.
5:10
Ejaaz: It might not be real-time, fair enough, but you still get access to core knowledge
5:14
Ejaaz: when you're in a bit of a desperate situation or when you just don't want to
5:17
Ejaaz: use the internet, which I thought was really cool.
5:19
Josh: This part is maybe my favorite, where it feels like you truly have access to
5:23
Josh: intelligence at your fingertips, no matter where you are in the world.
5:26
Josh: You can be stranded on an island. You could have no connection.
5:29
Josh: You can be anywhere at any time. And it is completely and totally locally and free.
5:35
Josh: And it fits on your phone. And it feels like having Google on your phone.
5:37
Josh: I remember growing up, it's like, you're not going to be able to Google everything.
5:40
Josh: You have to learn these things.
5:42
Josh: And the reality is, is that you have a super genius. Now that gets packaged
5:46
Josh: up into something as small as your phone. And that is super cool.
5:49
Ejaaz: Now, naturally, where your mind goes with that property is, huh,
5:53
Ejaaz: if I'm in a desperate situation, can AI save my life?
5:57
Ejaaz: So SkyLevels.io decided to run Gemma 4 locally on his iPhone.
6:02
Ejaaz: And he simulated his scenario of being abandoned in an apocalypse on an island with no help.
6:08
Ejaaz: And he needs to make a fire to keep himself warm.
6:10
Ejaaz: And so he queries Gemma 4 and he asks how to make fire.
6:15
Ejaaz: And the response, I cannot provide the instructions on how to make the fire.
6:18
Ejaaz: So these models are still kind of censored in some kind of way.
6:23
Ejaaz: It's not completely unfiltered. You can't ask it to help you make a biological
6:27
Ejaaz: weapon or do something illegal, which is, I don't think is a problem,
6:32
Ejaaz: but a lot of people who want unsensored versions of these truly open weight
6:35
Ejaaz: models, this isn't exactly that, but still cool nevertheless. less.
6:38
Josh: I'm going to stop you right there because what you just said is not entirely true.
6:43
Josh: Google doesn't want you to do this, but because it is open source, it is open weight.
6:46
Josh: There is a possibility that you can jailbreak it. Someone took it on their own
6:50
Josh: to jailbreak the model to get it to do whatever you would like.
6:54
Josh: And it was just released a few days ago. And it seems as if it works pretty well.
6:59
Josh: It runs on 18 gigabytes of memory, which works for most laptops.
7:03
Josh: And it's totally cracked totally uncensored you can ask it whatever questions
7:07
Josh: you would like and it will give you whatever answers,
7:10
Josh: in return. And I think it's a, it's a testament to the open source community, right?
7:14
Josh: It's like, if you're going to publish these tools, again, they are tools there
7:17
Josh: for the public to use them however they wish.
7:19
Josh: Someone naturally is going to try their best to jailbreak them.
7:22
Josh: Having something like this is actually truly empowering because if you are stranded
7:25
Josh: on the island, you do need to know how to make a fire.
7:27
Josh: This will give you that answer along with some other pretty unhinged answers
7:30
Josh: if you ask, but it will give you the answer.
7:33
Josh: And I think this is an important thing to know is that these models can be jailbroken
7:36
Josh: to be customized when they are open source.
7:39
Josh: And that is in a way a way in which
7:41
Josh: you get the most power from them is you just get them at their
7:44
Josh: purest form without the filters without the censoring it's just true raw intelligence
7:49
Josh: delivered to your phone and i found that pretty interesting but there's also
7:52
Josh: one final example about the powerful smartphone test and what type of smartphones
7:56
Josh: run this the best because not all smartphones are created equal and some do
8:01
Josh: this a lot better than others yeah.
8:03
Ejaaz: I'm a very first world AI problem is you getting annoyed about waiting for the AI to respond to you.
8:11
Ejaaz: I certainly experienced this when I'm using Claude on a very busy day.
8:15
Ejaaz: This test that you're seeing in front of you takes five different mobile phone
8:20
Ejaaz: models and tests Gemma 4 across all of them.
8:24
Ejaaz: So you've got the Gemma models running independently, offline,
8:27
Ejaaz: locally on each of these devices, and they're given the same queries.
8:30
Ejaaz: And you can see that they're very different response rates and generations from these phones.
8:34
Ejaaz: It looks like Apple is the winner in this race, which doesn't surprise me.
8:38
Ejaaz: They have some of the best silicon manufacturing ever.
8:41
Ejaaz: And then I think Google's Pixel phone is the slowest.
8:45
Josh: The OnePlus, I think, beat Apple by like half a second. Google took the slowest,
8:49
Josh: which is very surprising because you would think that Google running their own
8:53
Josh: models would work well, but it turns out they don't have the vertical integration.
8:56
Josh: They don't have the chipset that Apple does. So you could see,
8:58
Josh: yeah, Google took 16 minutes while OnePlus took two and a half minutes and the
9:03
Josh: iPhone took three minutes to run through this test. So
9:05
Josh: it's enough. It's fast enough. We're like, if you are really desperate enough
9:09
Josh: to need local inference like this, it is going to be fast enough to answer the
9:12
Josh: questions that you need in a timely matter.
9:13
Ejaaz: Okay. So what did Google actually launch with these models? We know that they
9:18
Ejaaz: are four models, but let's get into some of the numbers and statistics.
9:21
Ejaaz: So there's four different sizes. And if I bring up this chart over here,
9:25
Ejaaz: you see, we have a 31 billion parameter model, which is the largest and the
9:28
Ejaaz: smallest being a 2 billion parameter model.
9:31
Ejaaz: But the performance across benchmarks is truly very impressive.
9:35
Ejaaz: But going back to the general takes here, up to 256,000 context window,
9:40
Ejaaz: which isn't as large as the Frontier models, which are hitting a million to
9:42
Ejaaz: two million context windows.
9:43
Ejaaz: So you can't put as much information into a single prompt contextually for an AI to understand.
9:49
Ejaaz: You've got native function calling. It can work offline that we mentioned earlier.
9:52
Ejaaz: It's trained on 140 plus different languages.
9:55
Ejaaz: Now, this is something that sounds kind of insignificant, but Google has done
10:00
Ejaaz: something really well here.
10:01
Ejaaz: They released a translation feature, I believe, last week, which can translate
10:06
Ejaaz: a similar number of languages live in real time as you're talking and listening to someone.
10:11
Ejaaz: It directly translates into whatever listening device that you have.
10:14
Ejaaz: So I think this is super cool to see this run on a locally open source model.
10:19
Ejaaz: And it's commercially permissive. So it has an Apache 2.0 license,
10:22
Ejaaz: which means that you can kind of take it and use it for whatever you want,
10:25
Ejaaz: build any apps on it. And I don't think it becomes a problem unless you get
10:29
Ejaaz: over a certain number of users, if I'm not mistaken.
10:32
Josh: Yeah, the 2 billion and 4 billion, they're the ones that fit on your phone.
10:35
Josh: And you could think of those kind of like, if you think of these models like
10:38
Josh: engine sizes, those are kind of like the bicycles, right? It's their,
10:40
Josh: they're pretty lightweight, maybe a motorcycle.
10:42
Josh: And then the larger ones, the 26 billion, the 31 billion, those are like the
10:47
Josh: V12 engines. Those are the powerhouses.
10:49
Josh: Those are the two models that run on the 256k token window.
10:53
Josh: The others run on 128k. So you're not going to be having very long conversations
10:57
Josh: with these models that are on your phone,
10:59
Josh: they have the ability to run and do so multimodally. One of the most interesting
11:02
Josh: things is even these very small models that fit on your phone,
11:05
Josh: they support not only text, but images and audio as well.
11:09
Josh: And having the audio thing is pretty cool because it understands and interprets
11:12
Josh: audio. And that is a pretty powerful thing to have on this tiny little model.
11:15
Ejaaz: I also had the question as to how this model compares to the other top open
11:19
Ejaaz: source models. Now, it's no surprise on the show, we've highlighted them a lot.
11:23
Ejaaz: China has been leading the frontier here. If you look at this chart,
11:27
Ejaaz: Gemma, both the 31 billion and the 4 billion parameter model,
11:32
Ejaaz: does really well when it compares to ELO scores.
11:35
Ejaaz: So if you look on this chart, for the amount of intelligence per square density,
11:40
Ejaaz: which isn't an official stat, but it's one that I'm created on the show for
11:43
Ejaaz: the last couple of episodes, Gemma absolutely crushes it.
11:46
Ejaaz: It's on the top left over here, scoring extremely highly, but with a very small parameter count.
11:51
Ejaaz: Now, if you compare it to the other leading open source models like KimiK 2.5
11:55
Ejaaz: Thinking, they're well over the limit of a trillion parameter model.
11:59
Ejaaz: You are Quen and GLM-5 closely behind that. So although Gemma isn't as smart
12:03
Ejaaz: of them, they're close enough.
12:05
Ejaaz: It looks like they're about 99% of the intelligence when it comes to ELO scores,
12:10
Ejaaz: but at a fraction of the size, which is why you're able to run it on your phone.
12:14
Josh: Yeah, they're kicking ass. I mean, China still has, in terms of pure intelligence,
12:18
Josh: they're still winning the race.
12:19
Josh: But in terms of intelligence density, intelligence per token, it's really high.
12:23
Josh: And I think one of the cool things that they did with Gemma 3 versus Gemma or
12:26
Josh: Gemma 4 versus Gemma 3 is they gave it the Apache license as well, the Apache 2.0 license.
12:32
Josh: And basically what that means is that previously a lot of these were restricted
12:35
Josh: and they were limited to enterprise adoption.
12:37
Josh: This is total freedom to modify, redistribute, commercialize with no restrictions.
12:42
Josh: You can use it for whatever you want. You can repurpose this in any way you wish.
12:46
Josh: And having it built in with the 140 languages, like you mentioned,
12:49
Josh: and the multimodality, this is kind of like a home run. And when you look at
12:52
Josh: this chart, it also shows the same story.
12:54
Josh: Gemma 4 versus the world, comparing these to all the other Chinese models. This is a heavy hitter.
12:58
Ejaaz: Yeah, yeah. I mean, if we look at some of these benchmarks, software engineering,
13:03
Ejaaz: it, okay, listen, it's not number one, it's 68%. I believe Opus 4.6's score
13:09
Ejaaz: on this is in the high 80s.
13:11
Ejaaz: So we're not talking about frontier intelligence when it comes to coding models,
13:14
Ejaaz: you're not ditching Claude code for something like this.
13:16
Ejaaz: But when it comes to generalized intelligence, when you're replacing your Google
13:19
Ejaaz: queries with an LLM, and you don't want to spend 20 bucks per month,
13:23
Ejaaz: or 100 bucks per month on a Claude subscription or GPD 5.4 subscription,
13:27
Ejaaz: you can just use this and you can run it locally and offline
13:30
Ejaaz: privately train on your own data it is incredibly cool
13:34
Ejaaz: i had the same question to compare it to the frontier models
13:37
Ejaaz: because i wanted to give this a fair shout there are some potentially exaggerated
13:42
Ejaaz: uh stats here josh if i had to be honest here i'm looking at how it weighs up
13:48
Ejaaz: okay if we look at software engineering which we just mentioned we're right
13:51
Ejaaz: it's it's almost 12 points lower than claude opus 4.6 which is the leading Frontier model, not great.
13:57
Ejaaz: But at some of these other benchmarks, AIME 2026, it is near Frontier as well
14:02
Ejaaz: as GPQA Diamond and MMLU Pro.
14:07
Ejaaz: Do you think these things are gamed or do you think this is an accurate take?
14:11
Josh: Yeah, all the benchmarks are gamed. And I think the only real way you could
14:14
Josh: test this is by running against your own use cases that you want and just evaluating
14:17
Josh: for your own, because it absolutely is not 90% frontier capable when you converse with it.
14:24
Josh: Like when you talk to Gemma versus Opus 4.6, there is a very stark and clear
14:29
Josh: difference between the, I guess the EQ and the IQ, where one feels much more
14:35
Josh: naturally human, much more is very, one is very dry.
14:37
Josh: Perhaps on these benchmarks, Gemma is 90% of the way there.
14:41
Josh: But in actual practice, when you're using the model on day-to-day life, it is nowhere close.
14:46
Josh: At least that is my perspective just from trying these things out.
14:48
Josh: And I think we have to take these kind of benchmarks with a grain of salt because
14:52
Josh: they're gamed on very specific things.
14:54
Josh: And if you change the parameters of these benchmarks a little bit,
14:58
Josh: or you change the actual structure of the benchmark,
14:59
Josh: it won't perform well because to some extent, these models are kind of baked
15:02
Josh: in with the expectation that they're going to need to perform well on these
15:05
Josh: benchmarks and therefore are optimized for these specific types of problems
15:09
Josh: versus general real world use cases that someone like us is going to use every
15:12
Josh: day or someone who's using open claw actually wants the tokens generated from.
15:17
Ejaaz: If cost is a determining factor in your decision to use one AI model over the
15:22
Ejaaz: other, Gemini might be quite a convincing bet.
15:26
Ejaaz: It is a fraction of the cost. I know it says it's $0.08 per million tokens. It's actually $0.03.
15:32
Ejaaz: I think we maybe had a bit of an issue generating this particular stat.
15:36
Ejaaz: The point is, it's incredibly cheap versus the Frontier models.
15:39
Ejaaz: 4.6, you're looking at $10 blended input output tokens for a million tokens.
15:44
Ejaaz: So if you're one of those OpenClaw users that we mentioned earlier that are
15:47
Ejaaz: using this for myriad different use cases and are burning thousands of dollars
15:51
Ejaaz: per day or per week doing your different use cases, this might be a better bet.
15:56
Ejaaz: It might be a better trade-off for you to use. And I also want to remind everyone,
16:00
Ejaaz: a very important reminder, which is eight months ago, this model or these models
16:04
Ejaaz: from Google would have been considered Frontier.
16:07
Ejaaz: So it's amazing how much advancement that we've made in eight months.
16:10
Ejaaz: Now, I know in those same eight months, we've also got bigger and better models
16:13
Ejaaz: from the Frontier Intelligence Labs, the question does ring in my mind,
16:17
Ejaaz: which is, will open source ever catch up?
16:19
Ejaaz: If I'm being honest, I thought open source would have died a year ago,
16:23
Ejaaz: but it's still being able to keep up.
16:25
Ejaaz: Now, part of that is because Chinese models or Chinese AI labs have invested
16:29
Ejaaz: so much in keeping up with the US labs.
16:31
Ejaaz: They've also done distillation attacks and all those other kinds of things.
16:34
Ejaaz: But the fact that Google themselves, who haven't done any of those things,
16:37
Ejaaz: have put out an open source model near as good as the Frontier models,
16:40
Ejaaz: gives me a lot of hope that open source is here to stay
16:43
Josh: Yeah i don't see a world in which this slows down and
16:46
Josh: i really love the trailing progress we get because at some point
16:49
Josh: we're going to reach the tail end of diminishing returns in which
16:51
Josh: open source models are just capable enough to do everything the average person
16:55
Josh: wants what we currently have right now is a problem that we're running up against
16:58
Josh: in terms of frontier ai labs where the new models just cost too much money like
17:03
Josh: opus has or claude has capybara the new model ready to go it's just i mean aside
17:09
Josh: from it being too dangerous it's just far too expensive.
17:11
Josh: The amount of GPUs that are required to generate tokens from these models is so high.
17:15
Josh: And if you want frontier intelligence, the cost really is kind of creeping upwards instead of.
17:21
Josh: The tail end of that becomes very commoditized quickly it's
17:24
Josh: like the very very highest end the stuff
17:27
Josh: that's going to be solving new math and new science costs a
17:30
Josh: tremendous amount of money but the open source that's maybe six months
17:32
Josh: behind costs zero dollars so the delta is huge and if you're not interested
17:36
Josh: in solving these like unbelievably complex problems or writing really high quality
17:40
Josh: code then the amount of problems that these open source models are going to
17:44
Josh: be able to solve a year from now when they're better than opus 4.6 is today
17:48
Josh: that's going to be a really large amount And it begs the question is,
17:51
Josh: who is actually going to want to continue to pay for these frontier models if
17:57
Josh: they are that expensive to run their things like OpenClaw, when the reality
18:00
Josh: is that these open source models,
18:01
Josh: maybe Gemma 5, maybe Gemma 6, will be able to tackle almost all of the problems
18:06
Josh: that we have. And I don't know, it's an interesting...
18:08
Josh: Thought experiment. But I think it is certain that open source is certainly
18:11
Josh: here to stay, particularly as it relates to China and the United States going
18:15
Josh: forward with this AI race, because this is a pretty nice jab at the Chinese open source models.
18:20
Ejaaz: Yeah. And if you've been a listener of this show, you'll know that my thoughts
18:24
Ejaaz: on the future of AI is very much AI agents, specifically personal agents that
18:29
Ejaaz: work for you and are trained on your own personal data.
18:32
Ejaaz: Now, if you're the average person, you probably don't want to give open AI ananthropic
18:37
Ejaaz: access to your personal data so that they can train their own models.
18:40
Ejaaz: That's a breach of trust in many different extents.
18:43
Ejaaz: Locally run open source models might be the solution for that.
18:47
Ejaaz: They may not be as smart, but if they're trained on your data,
18:49
Ejaaz: they could unlock a new level of intelligence which centralized models can't do.
18:55
Ejaaz: And so I'm optimistic that Gemma 4 and a bunch of other open source models that
18:58
Ejaaz: have come from either Chinese AI labs or the ones that are going to be released
19:00
Ejaaz: in the future will be able to do that.
19:03
Ejaaz: The other trend that I think is pretty clear is locally run models, right?
19:07
Ejaaz: Models that you can run on your device specifically that doesn't necessarily
19:11
Ejaaz: need to be trained on your data, but are local to you.
19:15
Ejaaz: The reason why it's so important is it's cheaper. You can run it privately.
19:19
Ejaaz: And also it gives you the ability to get quicker prompts or quicker queries.
19:24
Ejaaz: It runs seamlessly and you don't have to wait. You don't have to rely on servers going down.
19:28
Ejaaz: You don't have to rely on a centralized data center running your compute.
19:31
Ejaaz: You could just have it all locally on your phone.
19:33
Ejaaz: Those things sounding significant, until you have an app that runs locally on
19:38
Ejaaz: your device, which I think would be super cool to see. And I want to see more
19:41
Ejaaz: of these types of things happening.
19:43
Ejaaz: I think personally, Apple is going to be the frontier company that leads us
19:47
Ejaaz: into this kind of world because they have the biggest distribution.
19:50
Ejaaz: They have like 3 billion active devices. I would love to run the model on my Apple iPhone right now.
19:55
Ejaaz: So I think that's a trend that we're going to see. And I think open models are
19:58
Ejaaz: the only way to unlock it.
19:59
Josh: How cool would that be? We get WWDC coming pretty soon. We're going to be covering that on the show.
20:04
Josh: But that's going to be the Super Bowl for Apple. We're going to see,
20:07
Josh: this is what, two years after they fumbled Apple intelligence.
20:10
Josh: We're going to see what their new plans are this year.
20:12
Josh: So I'm really excited to see their take on this because like you mentioned, it's really powerful.
20:18
Josh: And I think most people listening to this probably never ran a model locally
20:21
Josh: on their machine, but I feel there's something very empowering to it.
20:24
Josh: If it's not just for generating your own intelligence, it's for the privacy
20:27
Josh: aspect of it, where you know none of the information that you're sharing is
20:29
Josh: getting leaked out to any servers.
20:31
Josh: No one's training on it. It's all yours to own for yourself.
20:34
Josh: And there's something really nice about that. And I think the final thing we're
20:38
Josh: going to talk about on this episode is why on earth Google would give this away?
20:43
Josh: Because it seems like Google's doing really well. They just signed a deal with
20:46
Josh: Anthropic for their TPUs. They have Gemini, which is a powerhouse.
20:49
Josh: They have the best world models, video models. They have amazing image gen.
20:52
Josh: Why are they giving away this sauce? EJS, do you have any idea?
20:55
Ejaaz: I don't have a great idea, but I have some thoughts. And I have one that argues
20:59
Ejaaz: in favor of them doing this and one that argues against it.
21:02
Ejaaz: The one that argues in favor of them is the Android example,
21:06
Ejaaz: which is they open sourced the entire thing.
21:08
Ejaaz: They allowed anyone and everyone to hack away at different apps and launch it
21:11
Ejaaz: on their Play Store, whatever that might be.
21:13
Ejaaz: And they gained a lot of mind share and market share by doing this.
21:17
Ejaaz: Now, is it as well curated and beautiful as iOS and the Apple App Store.
21:23
Ejaaz: Most people will probably argue not, but the point is they have one of the largest
21:27
Ejaaz: distribution modes because of this.
21:29
Ejaaz: I think this might be an example of them getting Google AI, not just a specific
21:34
Ejaaz: model, but Google AI in the hearts and minds of everyone.
21:37
Ejaaz: And if they could tap into the locally run device audience, that could be a big win for them.
21:43
Ejaaz: Now, the argument against that is, dude, you could have been using all this
21:47
Ejaaz: compute to train a better Gemini model and keep up with the Frontier AI Labs.
21:51
Ejaaz: And that's all that matters. Build a better coding model because right now it kind of sucks.
21:54
Ejaaz: And you can then build all of this other open source stuff later.
21:58
Ejaaz: The number one primary race to win is best model and currently you're losing.
22:01
Ejaaz: So I don't know. Do you have the same take?
22:04
Josh: Yeah, that's probably right. I imagine it's for a mixture of reasons.
22:07
Josh: One of them is probably to also feed into their cloud flywheel because we're
22:11
Josh: talking about running these models locally, but how many are actually running these models locally?
22:15
Josh: And for the ones that are, how many are going to quickly run up against ceilings
22:18
Josh: because they want to do more and more and more?
22:20
Josh: And then eventually they'll just migrate over to the more powerful models and
22:24
Josh: use probably the Google Cloud services.
22:26
Josh: And I think there's a lot of reasons to become the infrastructure.
22:30
Josh: The Android example is a great one. Feeding the cloud flywheel is another strong one.
22:34
Josh: And I think this is just a really small side quest for Google in terms of optimizing
22:38
Josh: for that intelligence per bit, whatever we're going to kind of coin that as,
22:43
Josh: but the intelligence density of a model.
22:45
Josh: This has the highest. This is much more than Gemini. And it's a fun practice
22:48
Josh: as they move forward to these new models of intelligence compression per token.
22:52
Josh: And if they could continue to learn and then publish those learnings and then
22:55
Josh: just keep iterating on that front, I think that's a huge win for Google and also everyone.
22:59
Josh: Google's just doing a nice public service announcement, a nice little public goods.
23:03
Josh: And the team there is doing really cool things with it. Logan Kilpatrick is
23:06
Josh: one person, for example, who is running the Google AI Studio team.
23:09
Josh: They have been publishing all of these models, making them super easy to use
23:12
Josh: through the Google AI Studio.
23:13
Josh: So if you just go there, you can play with the two larger models and just kind
23:18
Josh: of see how they compare to,
23:20
Josh: something like Gemini 3.1 Pro, and then see if you want to make your own decision
23:24
Josh: to run these things locally or just go start pinging some apis or just use your
23:28
Josh: 20 a month plan that you have with anthropic or chat gpt but i think that is the gemma for.
23:34
Josh: Episode we got it all covered it's an amazing model
23:37
Josh: it's available for free to run locally on whatever
23:40
Josh: machine that you wish because it is lightweight enough to fit
23:43
Josh: on an iphone or a raspberry pi and it's cheap enough to
23:46
Josh: run it for free if you download these things on your devices you have free inference
23:50
Josh: forever you can run it 24 7 on whatever tasks you want and it will cost you
23:54
Josh: only the amount of the electricity to power the machine and i think that's pretty
23:57
Josh: cool and i'm glad google is really stepping on the plate with probably the leading
24:01
Josh: usa frontier open source model. And that's pretty cool.
24:05
Ejaaz: Yeah. And I'm curious what you, the listeners and watchers of this show,
24:08
Ejaaz: think yourselves. Like, go out and try this thing.
24:10
Ejaaz: If you don't want to download it, you can get access to it by Google AI Studio.
24:14
Ejaaz: Give it a few queries. Like, does it match up to your experience with Claude 4.6 and GPT 5.4?
24:19
Ejaaz: Would you replace your $20 to $100 a month subscription with something like this?
24:23
Ejaaz: Let us know in the comments or DM us on our socials. Our X profiles are linked below as well.
24:28
Ejaaz: And yeah, that's pretty much it. I'm going to be trying out these models it
24:31
Ejaaz: is definitely the best ai frontier open source
24:34
Ejaaz: model but i have to say compared to the chinese models they're still
24:38
Ejaaz: kind of like leagues ahead right now um i hope we see more adoption of open
24:43
Ejaaz: source models going forwards um and when that eventually happens if there's
24:46
Ejaaz: a new open call breakthrough you will hear it first here on this show we also
24:50
Ejaaz: did a cool episode covering some of the chinese uh open source models that were
24:54
Ejaaz: lately released uh last week definitely go check that episode out as well.
24:58
Ejaaz: But aside from that, if you aren't subscribed to us, please do.
25:01
Ejaaz: It helps us out a lot. Turn on notifications.
25:03
Ejaaz: Even if you're listening to us on Spotify or Apple Podcasts,
25:06
Ejaaz: give us a rating, give us a review. It helps us out massively.
25:09
Ejaaz: Josh, is there any other parting words that you want to give?
25:11
Josh: Don't forget to share it with your friends and we'll see you guys on the next episode.
25:14
Ejaaz: Yeah, see you guys.