Latest / In the Loop AI / THIS WEEK IN AI: Google TurboQuant, OpenAI Ends Sora, SpaceX IPO

Transcript

THIS WEEK IN AI: Google TurboQuant, OpenAI Ends Sora, SpaceX IPO · In the Loop AI

0:00
Ejaaz: Google might have just popped the AI bubble. They released a new algorithm yesterday,
0:05
Ejaaz: which shrinks the requirement of memory needed to run an AI model,
0:09
Ejaaz: which isn't great news for the top memory stocks, which had billions of dollars
0:13
Ejaaz: of their market cap wiped out over the last 24 hours.
0:16
Ejaaz: We're looking at SanDisk is down almost 7% today.
0:20
Ejaaz: Micron is down 18% over the past five days, the majority of that yesterday.
0:24
Ejaaz: And SK Hynix, another major memory provider is down 6% today.
0:29
Ejaaz: It's absolutely decimated the market. We're going to get into this story and
0:32
Ejaaz: so much more on this episode.
0:33
Josh: Yeah. And some important context to know before we get into this is the reason
0:37
Josh: why these memory stocks are so high is because memory has been the single largest
0:40
Josh: constraint in the world of AI.
0:42
Josh: If you have ever tried to build a PC or buy a PC in the last 12 to 18 months,
0:48
Josh: chances are it's almost doubly as expensive as it was the previous 18 months. That's because AI.
0:53
Josh: It's what happened when crypto stole all the GPUs. AI is taking all of the memory
0:57
Josh: and memory is the limiting factor memory is what serves inference whenever you
1:01
Josh: ask chat gpt or anthropica question,
1:04
Josh: When it serves you the answer, that uses inference, that uses memory.
1:08
Josh: So memory has been the single crux. It's the reason why companies haven't been
1:11
Josh: able to scale as fast as they want, because the costs have been out of control.
1:15
Josh: This algorithm changes things a little bit. This changes the math here just
1:19
Josh: a smidge, but enough to make it scare the market entirely, right?
1:24
Ejaaz: Yeah, it changes it in a huge way. And the wildest part is the Google research
1:29
Ejaaz: team made this public and available for everyone to read the blueprint of.
1:33
Ejaaz: So it's called TurboQuant, and they're describing it as a new compression algorithm
1:38
Ejaaz: that reduces LLM key value cache memory by six times and delivers up to 8x speedup.
1:45
Ejaaz: So let me translate what that means.
1:47
Ejaaz: When you type or speak to an AI model, it has a short-term memory.
1:52
Ejaaz: Think about like humans, they kind of hold things in their head for a bit,
1:55
Ejaaz: and then they go to bed, and then they commit some of those memories for long-term
1:58
Ejaaz: memory, and some of those they kind of discard. But LLMs work in a similar way,
2:02
Ejaaz: and it's called the cache memory.
2:04
Ejaaz: Now, the issue with LLMs are that cache memory, as you talk to an AI more and
2:09
Ejaaz: more, gets really quite big and bloated, which means that AIs are very expensive to run.
2:14
Ejaaz: That's why we spend so much money on GPUs. That's why these data centers cost
2:17
Ejaaz: billions of dollars to build out.
2:19
Ejaaz: It is part of the reason when you inference a model.
2:22
Ejaaz: Now, what Google did was they released an algorithm which said,
2:25
Ejaaz: hey, yeah, you know that thing that bloated up your AI model?
2:28
Ejaaz: We can cut that down pretty massively by six times, actually,
2:31
Ejaaz: and save you a ton of money.
2:33
Ejaaz: So the world is going crazy with this new research, because if you could scale
2:37
Ejaaz: this up to Claude or ChatGPT, you may not need as many GPUs as we originally
2:42
Ejaaz: thought to serve Frontier AI models to anyone.
2:45
Josh: Yeah, it's a really big deal. Like you mentioned, every time you message a chatbot,
2:49
Josh: the model stores your entire conversation in that KV cache.
2:52
Josh: And on a large model, like the 70 billion parameter model, if you run a long
2:55
Josh: conversation, that cache will eat up to like 40 gigabytes of space on your GPU,
3:00
Josh: which is more than the actual model itself.
3:03
Josh: So that's why this is such a huge issue. And the big breakthrough here is that
3:07
Josh: Google can do so without any loss functions.
3:09
Josh: If anyone's a Silicon Valley fan, the TV show, the crown jewel of the entire
3:14
Josh: show was this middle out compression algorithm that didn't have any loss function.
3:18
Josh: You can just compress the data without losing any of the quality.
3:21
Josh: And Google's announcing that they've actually done this.
3:23
Josh: They've actually solved this problem. And that's why the memory market is freaking out.
3:27
Josh: It's like, oh my God, wait, if we can get six times the compression,
3:31
Josh: eight times faster on the same GPUs, surely there will be less demand for memory.
3:35
Ejaaz: Well, despite these stocks being down massively, I think the market is kind of overreacting.
3:41
Ejaaz: This is a good thing for AI in general, and it will lead to more consumption of AI.
3:45
Ejaaz: It's great that the AI models become cheaper to inference and that it frees
3:49
Ejaaz: up space for us to compute other AI prompts and queries.
3:53
Ejaaz: Jevon's paradox is going to occur here in the same way that people were kind
3:56
Ejaaz: of crapping on GPUs and saying these things are too expensive and aren't going to get cheaper.
4:00
Ejaaz: The same thing is going to happen with memory here. It's not going to result in less demand.
4:04
Ejaaz: It's going to result in even more demand. The demand for AI products right now
4:07
Ejaaz: is insatiable, and I don't think this changes anytime soon.
4:11
Ejaaz: The other part is, I think that what Google has created here
4:15
Ejaaz: is something that's going to take time to scale out. This is not going to happen tomorrow.
4:20
Ejaaz: This is a research paper, which you and I were actually joking about before
4:23
Ejaaz: we started recording, was released 11 months ago.
4:26
Ejaaz: But because the paper was so complex and had so many algorithms and mathematical
4:30
Ejaaz: formulas, it didn't actually surface to the public until yesterday.
4:33
Ejaaz: And everyone was like, wow. So it takes time.
4:36
Ejaaz: Since those 11 months, we haven't scaled it out to major models just yet.
4:38
Ejaaz: This only applies to small to medium models.
4:40
Ejaaz: But I can imagine the scaling to frontier models at some point,
4:43
Ejaaz: which will be a win-win for both sides.
4:45
Josh: I mean yeah there's there's no reason why you wouldn't be focused solely
4:48
Josh: on this right it's like if you can serve six times the inference on the
4:51
Josh: same amount of gpus or tpus or whatever gpu accelerator you're
4:54
Josh: using that's a huge breakthrough for any company and it also
4:57
Josh: extends past the cloud it moves to local inference as
5:00
Josh: well it's like if you're running a mac mini or you're even
5:03
Josh: on your iphone the amount of intelligence you
5:06
Josh: can run on a model that is six times more efficient
5:09
Josh: eight times faster and has zero
5:12
Josh: loss function is a really big deal and i think google
5:16
Josh: may be moving towards this what i see this trend happening with
5:19
Josh: google is i mean they have the tpu they're building their own stack
5:22
Josh: they have the partnership with apple to get into all of the iphones they have
5:25
Josh: their like kind of android division they're they're publishing their research
5:29
Josh: publicly to disrupt the market as they go it's a really noteworthy strategy
5:33
Josh: from google and it seems like as always they're leading on the thought leader
5:37
Josh: front right they've always been the top tier of research we'll see how they
5:40
Josh: can implement this into actual products
5:42
Josh: But it seems like the market's wrong. I can't imagine a world in which there
5:46
Josh: is less memory demand as a result of this.
5:48
Josh: Like, as you said, there will only be more. If we get six times more intelligence,
5:51
Josh: we will gobble that up so quickly. So market seems like it's wrong.
5:55
Josh: Maybe it's just an excuse to dump because the stocks have gone up so much over the last 12 months.
6:00
Josh: But I wouldn't be too worried about this if you are a owner of any sort of memory.
6:04
Ejaaz: It's actually very bullish Google stock, in my opinion. I actually bought a
6:07
Ejaaz: bunch more when this came out because I was like, oh, this is going to result
6:10
Ejaaz: in massive efficiencies for AI.
6:12
Ejaaz: Another thing is like, lest we forget, Google has been like the pioneer of a
6:17
Ejaaz: lot of these different AI trends.
6:18
Ejaaz: Like one of the earliest LLM formations was from a bunch of Google or ex-Google researchers.
6:25
Ejaaz: TPUs they've been working on for over a decade now. So they saw the GPU trend
6:28
Ejaaz: like way, way, way in advance. And now they're doing the same here.
6:31
Ejaaz: My question to you is, why do you think they publicize this for everyone to see?
6:36
Ejaaz: Surely this, like i saw a few comments about this as well like surely they should
6:39
Ejaaz: have patented this because this is a pretty sick algorithm that now like any company can use
6:44
Josh: Yeah i don't think the patenting thing really works anywhere
6:47
Josh: at all because a lot of this stuff is just a couple lines of
6:50
Josh: code it's like andre wrote auto research and it's
6:53
Josh: 600 lines of code it's like someone will take it
6:56
Josh: someone will use it the the race is far too powerful
6:59
Josh: far too there's far too much of a monetary incentive to
7:02
Josh: do this and you could think of it almost like google's deep
7:05
Josh: seek moment where deep seek created this unbelievable
7:08
Josh: research dropped the bomb they got all the publicity but
7:10
Josh: then the rest of the market kind of rushed to tackle that this is google dropping
7:14
Josh: their their deep seek moments and i think it's it's noteworthy it's really impressive
7:19
Josh: for google google has been kind of at the forefront of all this stuff and they're
7:22
Josh: proving that their research team really are super impressive and have the ability
7:26
Josh: to continue to lead the frontier forward.
7:29
Ejaaz: So speaking of companies that are at the frontier, one of those companies are
7:34
Ejaaz: falling behind this week, Josh.
7:36
Josh: We got to do a little victory lap, slam dunk. I'm not really sure how to frame this.
7:41
Ejaaz: But this is about Sora. Yeah, let's start with the victory lap, I guess.
7:45
Ejaaz: So Sora, for those of you who don't know, is one of OpenAI's winning consumer apps.
7:51
Ejaaz: It is their AI video generation app that functions very similarly to TikTok.
7:56
Ejaaz: You kind of scroll through Reels, but all the videos are air-generated.
7:58
Ejaaz: But the coolest part is you can feature yourself in them.
8:01
Ejaaz: You can feature your friends or any kind of IP or characters.
8:04
Ejaaz: This week, they announced that they were sunsetting the entire project.
8:09
Ejaaz: Sora, which six months ago wasn't a thing, is now dead.
8:13
Ejaaz: Some fun stats from Sora's very short but mighty reign.
8:17
Ejaaz: Within the first day, they achieved over 5 million downloads, which is just insane.
8:22
Ejaaz: I don't think any other app has done that before. they also hit
8:25
Ejaaz: number one and stayed at number one of the apple app store and i
8:28
Ejaaz: think the android play app store as well for a number of different
8:30
Ejaaz: weeks but it wasn't without any kind of
8:33
Ejaaz: grimace the public was split hollywood is basically uh how do i say um dancing
8:39
Ejaaz: on their graves uh open ai is shutting down its ai video slot making platform
8:44
Ejaaz: sora and if you look at the comments from different people it's just like did
8:48
Ejaaz: we just win uh people are like saying like this is such a big W.
8:52
Ejaaz: So people are generally happy outside of their AI sphere. But I don't know,
8:57
Ejaaz: Josh, how do you feel about this? Are you happy?
8:59
Josh: Yeah, well, this was just a failed experiment in the way that Meta did.
9:03
Josh: I forget. I don't even remember Meta's. What was it?
9:06
Ejaaz: Vibes?
9:07
Josh: Vibes perhaps vibes where they tried to do the same thing it
9:10
Josh: was tiktok but for ai and open ai tried tiktok but
9:13
Josh: for ai again but through the sora platform and i
9:16
Josh: am i mean we said this was stupid i haven't opened the
9:18
Josh: app since the week of i think there's there's a
9:21
Josh: harsh reality that open ai is learning that they perhaps don't understand
9:24
Josh: just due to the nature of them being a new company sam really
9:27
Josh: this is his first large company that he's run but i mean
9:30
Josh: it seems like a very predictable outcome when you just look
9:33
Josh: at what standard consumer behavior is and you kind of compare to other platforms
9:36
Josh: because when you think about a platform like
9:39
Josh: meta right it has two billion users or something like that and particularly
9:43
Josh: on instagram how many people of those 10 billion actually want to create content
9:46
Josh: maybe 10 million maybe 0.5 percent of the people would actually be serious creators
9:51
Josh: and the reality is is that almost every single human just wants to scroll and
9:55
Josh: zone out instead of spending energy like conceptualizing and editing these complicated things,
10:00
Josh: The novelty factor of it was very fun, but people don't want this and it's not that good.
10:05
Josh: And I think that novelty wears off. It's like we created those videos that show
10:09
Josh: us and you in them, and it was really cool for like two hours,
10:12
Josh: but creating a dedicated application where you have to go and download it and
10:15
Josh: you have to use it outside of the ChatGPT OpenAI universe is very annoying.
10:20
Josh: Had they rolled this up into the same app that everyone's using every day with
10:23
Josh: ChatGPT, maybe that's a different story, but it's, I don't know,
10:27
Josh: it's tough to kind of reason why they would have gone through with this in the
10:30
Josh: first place the way they did.
10:31
Ejaaz: Let me try and argue the other side. I think what Sora did really well was they
10:36
Ejaaz: made it really easy if you were super lazy, but you kind of had an idea for
10:40
Ejaaz: a piece of content to make that content.
10:43
Ejaaz: Just write it in words, takes a couple seconds, maybe max a minute.
10:46
Ejaaz: And then a couple of minutes later, you have a fully fledged video.
10:49
Ejaaz: Now, obviously, Sora v1 wasn't that great.
10:52
Ejaaz: Sora v2 started to improve and become much better. Sora v3 was way better,
10:56
Ejaaz: but there was still some AI cringiness around this. I do think this improves
11:00
Ejaaz: with a bunch of now other AI models that still exist to do this.
11:04
Ejaaz: CapCut actually yesterday released their own version just as Sora was sunsetted
11:08
Ejaaz: that uses Seed Dance 2, which is a Chinese AI model.
11:11
Ejaaz: And my God, the content that creators can now create just through words looks insane there.
11:15
Ejaaz: And then you have Grok Imagine on the ex-AI side under Elon Musk,
11:19
Ejaaz: and they're piling so many millions and millions of dollars to try and improve that model.
11:24
Ejaaz: So there is something there. I just think that a lot of this reason behind OpenAI doing this is
11:29
Ejaaz: One, they're constrained on cash. They've spent a lot of money on GPUs,
11:34
Ejaaz: Stargate. They had to sunset that.
11:35
Ejaaz: And now they're figuring out that they need to focus all their cash and resources
11:38
Ejaaz: on building the best LLM to beat Anthropic and the best coding AI to beat Anthropic,
11:43
Ejaaz: who is, quite frankly, eating their lunch right now.
11:46
Ejaaz: But there is another bit of news, which I didn't realize immediately.
11:49
Ejaaz: But then I noticed Disney.
11:52
Ejaaz: OpenAI had signed a $1 billion deal with Disney through Sora.
11:58
Ejaaz: And the idea here was Disney would invest $1 billion in OpenAI,
12:02
Ejaaz: and OpenAI would also get access to Disney's IP and characters and make them
12:07
Ejaaz: super famous on Sora via the AI-generated type of video medium.
12:12
Ejaaz: It sounds like Disney didn't even know.
12:14
Ejaaz: I've seen multiple tweets like this, where on Friday, they were signing a deal
12:18
Ejaaz: with OpenAI, and then on Wednesday of this week, when they decided to announce
12:23
Ejaaz: the news that they were shutting down Sora, the Disney team had no idea.
12:26
Josh: It's tough. I think this is a good lesson. there is this quote from Johnny Ive
12:31
Josh: that I love that I think would be nice for Sam Altman to hear when he was asked
12:35
Josh: about what Steve Jobs taught him about focus.
12:37
Josh: And he says, this sounds really simplistic, but it shocks me how few people
12:41
Josh: actually practice this.
12:42
Josh: One of the things that Steve would say to me was because he was worried that
12:44
Josh: I wouldn't focus. He would say, how many things have you said no to?
12:47
Josh: And I would tell him, I said no to this and I said no to that.
12:49
Josh: And he would create these sacrificial things that he was saying no to.
12:52
Josh: But focus means saying no to something with every bone in your body you think is a phenomenal idea.
12:56
Josh: And you wake up thinking about it, But because you are focusing on something
12:59
Josh: else, you can't work on that one thing.
13:00
Josh: And OpenAI does not have that focus.
13:03
Josh: OpenAI is creating these sacrificial things. They are not deeply focused on
13:06
Josh: one thing. And that's why there's
13:08
Josh: four different apps you have to download to use all of their tools.
13:10
Josh: There's just all of these spread attempts at creating virality,
13:15
Josh: at manufacturing virality through TikTokification of it, through this like image
13:19
Josh: generation of the Studio Ghibli stuff.
13:21
Josh: None of it is just building their core thing. And I think when you look at Anthropica,
13:25
Josh: when you look at OpenAI, and the difference between the two is that one has
13:29
Josh: focus and one does not. When you look at OpenAI...
13:31
Josh: You see five different product SKUs. They're all kind of all over the place
13:35
Josh: with different intentions.
13:36
Josh: When you look at Anthropic, they're the best coding model in the world. And that's it.
13:39
Josh: Everyone can converge on that one point. And therefore their velocity is so
13:43
Josh: fast. That's why we just filmed an episode yesterday.
13:45
Josh: Everyone should go watch about the eight updates they published in eight weeks
13:48
Josh: that basically entirely replaced OpenClaw.
13:50
Josh: They have the focus. And I'm hoping that OpenAI can now kind of shift this focus
13:54
Josh: into this singular place and start to move towards this new organization that
13:58
Josh: they are calling AGI Deployment.
14:00
Josh: This is crazy. They're preparing for AGI already.
14:03
Ejaaz: Yeah, so the team is basically getting reshuffled under a product organization
14:08
Ejaaz: within OpenAI called AGI Deployment.
14:10
Ejaaz: And this kind of harkens to a wider strategy that OpenAI has assumed for the
14:14
Ejaaz: last couple of months now, which is to build the best LLM, build the best coding
14:18
Ejaaz: AI model, and also the best world model, which is actually where the Sora team
14:22
Ejaaz: researchers are moving to.
14:24
Ejaaz: They're going to focus on building a world model, which basically helps AI models
14:28
Ejaaz: see the world and understand the physics of the world. the advantage of this
14:31
Ejaaz: is you get to better understand how humans perceive and interact with the world,
14:35
Ejaaz: which is something that LLMs can't do.
14:37
Ejaaz: Think of LLMs as an AI model that sits in the library and reads all the books,
14:41
Ejaaz: but doesn't actually experience the world for himself.
14:43
Ejaaz: World models actually help you understand the world. That's what the Sora team
14:45
Ejaaz: is going to focus on right now. And that's super exciting for one major reason, which is robotics.
14:50
Ejaaz: Robotics is going to be a huge thing.
14:52
Ejaaz: OpenAI teased robotics in one of their major announcements a couple of weeks
14:56
Ejaaz: ago. So we know that they're going to focus heavily on that.
14:58
Ejaaz: And I do think that's a smart move.
15:00
Ejaaz: I'm bullish OpenAI after all of this. But just to recount, this isn't the only
15:04
Ejaaz: major pivot they've made over the last week.
15:06
Ejaaz: Over the last couple of weeks, they've done quite a few things.
15:08
Ejaaz: So they've sunsetted Sora.
15:09
Ejaaz: They've announced that their Stargate project, which was like their major project
15:12
Ejaaz: to scale GPUs across the globe, is now canceled one year later due to funding issues.
15:18
Ejaaz: And Altman said that like, you know,
15:19
Ejaaz: I'm never going to release ads on the platform. He ended up doing that.
15:23
Ejaaz: In-app shopping completely failed and flopped. And now they've got to redesign
15:26
Ejaaz: that entire thing but it's been put on pause for now until they focus on all
15:29
Ejaaz: this other stuff all of this to say
15:31
Ejaaz: I think OpenAI is going to IPO soon, Josh. And I don't know what you think the
15:34
Ejaaz: odds are for this, but we should probably like bet on that.
15:37
Josh: Well, if only we had a marketplace for it. And thanks to our friends at Polymarket, when will OpenAI IPO by?
15:43
Josh: And the answer is kind of surprising where there's only a 36% chance that they'll even IPO this year.
15:49
Josh: And I think earlier in the year, this would have been a little bit different,
15:52
Josh: but it appears as if there's been some trouble in paradise for Anthropic.
15:55
Josh: Maybe that is moving over to OpenAI.
15:57
Josh: The reality that we're starting to see now is SpaceX might be the big one, and that might be it.
16:01
Josh: Um which is a little disappointing because i was hoping this would be
16:04
Josh: the year of all of the ipos but according to polymarket 36 chance
16:07
Josh: and if you think it's happening this june there's four percent chance so that's
16:11
Josh: absolutely not happening spacex is going to be the one that will be taking that
16:15
Josh: crown uh polymarket thank you very much for sponsoring this part of the episode
16:17
Josh: and maybe we should just go right
16:19
Josh: into spacex and their ipo which now is rumored to be filing next week,
16:25
Josh: confidentially this will not be public but they will do their confidential s1
16:27
Josh: filing later this week, early next week, which would point them to a public listing,
16:33
Josh: around the first couple of weeks of June. This is a really big deal for the
16:38
Josh: world because, I mean, this is likely to be now over a $2 trillion IPO.
16:44
Josh: That's the biggest IPO by far. This is going to be the largest thing to enter the market.
16:49
Josh: It's going to be larger than the GDP of some countries, and it will just go
16:53
Josh: live on the New York Stock Exchange or the NASDAQ sometime in a few months from
16:57
Josh: now. We're getting close.
16:58
Ejaaz: What a sequence of events to get us to this point. we had the rumors of SpaceX
17:02
Ejaaz: and XAI emerging and then it happened.
17:05
Ejaaz: Now we have rumors of SpaceX and Tesla merging. But first, I think we're going to get the SpaceX IPO.
17:10
Ejaaz: Rumors also say that they're going to be raising $75 billion in this listing,
17:15
Ejaaz: which is just absolutely insane and probably the largest raise that's ever been
17:19
Ejaaz: done on an American public stock market.
17:21
Ejaaz: The other thing is this comes off the back of Elon announcing some pretty ambitious
17:25
Ejaaz: projects for SpaceX itself.
17:27
Ejaaz: They're going to be launching a TerraFab. We made an episode about this.
17:29
Ejaaz: Definitely go check that out, which is a gigantic AI chip factory.
17:33
Ejaaz: They're going to be launching 80% of those chips into space via SpaceX.
17:37
Ejaaz: So it's going to become this whole monopoly of like AI in space,
17:40
Ejaaz: which I can't wait to see. Mass drivers on the moon.
17:42
Ejaaz: Definitely go check out that episode earlier this week. But yeah,
17:45
Ejaaz: it's going to be one of the biggest IPOs, Josh. Are you buying it? Absolutely.
17:48
Josh: I cannot miss it. I currently own SpaceX. It's the only private company I own. I'm ready to buy more.
17:53
Josh: I think it's going to be the most valuable company in the world. rolled in with tesla and
17:56
Josh: i could not be more bullish more optimistic more excited for the future with
18:00
Josh: these companies if elon's predictions are right we'll be seeing nvidia at 10
18:03
Josh: trillion and spacex and tesla will be right there with them um the companies
18:07
Josh: are going to grow quick i think the spacex ipo is very much the starting gun
18:10
Josh: now there is another gun that's been fired and it's been a not not a good one
18:13
Josh: from apple not a good because they have been,
18:16
Josh: banging out all the applications of everyone who
18:19
Josh: wants to get in the app store now there's been a recent issue with the app store
18:22
Josh: is now the fact that anybody can make apps means that a lot more people are
18:25
Josh: submitting apps so for the larger companies and the smaller companies like if
18:29
Josh: you have an app in the app store it's taken a lot longer to get your app approved
18:32
Josh: whether it be for just a normal update or to release an actual app and now it
18:37
Josh: appears that apple has stopped,
18:39
Josh: allowing updates for popular Vibe-coded applications?
18:43
Ejaaz: Yeah, they've basically put a
18:45
Ejaaz: not official, but unofficial halt on anyone launching AI Vibe-coded apps.
18:51
Ejaaz: And I'm confused about this for a few different reasons.
18:54
Ejaaz: Number one, if you look at a chart of app launches on the App Store over the
18:59
Ejaaz: last year, before Vibe-coding became a thing, it had completely plateaued.
19:04
Ejaaz: No one was making apps anymore. And that was becoming an issue for the Apple
19:09
Ejaaz: app developer ecosystem. They're
19:10
Ejaaz: actually kind of trying to encourage developers to build more things.
19:13
Ejaaz: Then AI vibe coding came along and suddenly they had a flourish of different
19:17
Ejaaz: vibe coded apps. Now granted, 90% of them were crap,
19:20
Ejaaz: 10% of them were pretty good. We got the replets of the world,
19:23
Ejaaz: which helped other people kind of build apps just by typing words or talking to an AI on your phone.
19:28
Ejaaz: And replet as a company is worth, I think, $9 billion. As of last week,
19:32
Ejaaz: they did another massive raise.
19:34
Ejaaz: So we aren't talking about small fish here. These are pretty big fish.
19:37
Ejaaz: Apple just cut them at the head this week or last week over the last seven days
19:42
Ejaaz: by saying that they are not going to encourage AI vibe coding anymore.
19:46
Ejaaz: And it just signals that monopolies like Apple, which have a chokehold on the
19:51
Ejaaz: app ecosystem like this, that take a massive 30% cut off of fees.
19:55
Ejaaz: We saw this happen in the crypto world as well, where people wanted to allow
19:58
Ejaaz: transactions. Apple said, we'll allow it for a bit and then we don't want this.
20:01
Ejaaz: They're now doing it with Vibe coding. I don't get the issue for them unless
20:04
Ejaaz: there's like major security exploits. I just don't think this is a good move in general.
20:08
Josh: This topic actually makes me pretty sad as an Apple fanboy, not because of what
20:12
Josh: they're doing to the developers, which is messed up.
20:16
Josh: And i don't like but the idea that apple is going
20:19
Josh: to exist as the king as it stands now
20:21
Josh: is just an impossibility in this new world when you
20:24
Josh: think about how easy it is to generate one of these apps and how easy it is
20:27
Josh: just to sideload them onto your phone like anyone with a
20:30
Josh: test flight account can vibe code an app of whatever you've ever wanted an iphone
20:34
Josh: app to look like you can send it direct to your iphone you can't put on the
20:38
Josh: app store but you can send it directly to your iphone and you could have your
20:41
Josh: own version of your dream app and it can do whatever functionality you want
20:44
Josh: and you're one prompt away from updating it and editing it and changing it to do whatever you want.
20:48
Josh: And the fact that it's so accessible now, and Apple is...
20:51
Josh: Clearly not leaning into adopting this new paradigm shift, there's this clash
20:55
Josh: that's happening. And we're seeing it here with this vibe coded apps.
20:57
Josh: They're not the good guy anymore.
20:59
Josh: They're not the person who is helping developers do what they want,
21:02
Josh: empowering them for this new paradigm of engineering.
21:05
Josh: They are the bottleneck. They are the like hammer who is stopping people from innovating.
21:10
Josh: And that to me makes me really sad. And that has always throughout history been
21:14
Josh: a losing formula. So I hope they turn it around.
21:16
Josh: We just got a WWDC announcement, which is happening in the next month,
21:21
Josh: I think. I forgot the dates.
21:22
Josh: But that's going to be their time where they're going to showcase all of the new AI stuff.
21:26
Josh: That's when they initially announced Apple Intelligence. It was the biggest
21:28
Josh: flop of all time. They're going to try to do it again.
21:31
Josh: Hopefully, they'll come out with some new policies to address this.
21:33
Josh: And hopefully, they'll have a chance to turn the ship around.
21:35
Josh: But that is their last chance.
21:37
Ejaaz: So something tells me, Josh, correct me if you think I'm wrong,
21:40
Ejaaz: that this may not be an outlandish move by Apple.
21:45
Ejaaz: It might be a strategic one because we also got some other news that broke this
21:49
Ejaaz: week, that their deal with Google,
21:51
Ejaaz: the deal with Google, where they pay Google $1 billion and get access to Google's
21:56
Ejaaz: Gemini AI models, what we originally thought was going to be some kind of licensing, right,
22:01
Ejaaz: actually is Apple getting full access to Google's Gemini model weights,
22:06
Ejaaz: which means that they can fine tune and build the model in any way that they
22:10
Ejaaz: want, they get full access to a model that Google spent hundreds of millions
22:14
Ejaaz: of dollars training years training just for a billion dollars a year.
22:17
Ejaaz: I don't know what's going on here. And it just means that Apple's got an absolute
22:20
Ejaaz: steal of a model, a Frontier LLM from Google, and they can now run with it and
22:24
Ejaaz: build their own AI apps, which is presumably what I think they're going to do.
22:28
Josh: This is a huge win for Apple. And we have to ask the question,
22:31
Josh: why is Google doing this? And I
22:34
Josh: I think it kind of pairs to that first topic that we spoke about,
22:38
Josh: which was that research paper where they're kind of democratizing intelligence
22:43
Josh: as best they can with TurboQuant, right?
22:44
Josh: It's like they're increasing the efficiency, they have their own custom TPUs
22:47
Josh: to make the price per token down.
22:49
Josh: And when we consider what the most existential threat in the world of AI is,
22:53
Josh: it's that edge inference and local inference gets really good so that you don't
22:57
Josh: need to generate tokens from these megacloud providers.
23:00
Josh: And I think the reality is that it's becoming more and more true because we
23:05
Josh: have now these turbo quant models that are going to be six times more efficient.
23:10
Josh: We have Apple who is taking Gemini models which are leading edge and distilling
23:14
Josh: them down into something that runs on your iPhone and the majority of the people
23:17
Josh: don't need world-class intelligence.
23:19
Josh: They don't need the bleeding edge stuff that is solving new physics and math.
23:23
Josh: They just want it to solve the day-to-day stuff and this
23:27
Josh: is possibly a hedge against that for Google is they're
23:30
Josh: creating the problem by throwing out papers like turbo quant
23:33
Josh: but then they also have access to the solution which is apple i
23:36
Josh: mean when we think about where the most local inference is held it's just
23:39
Josh: on the entire apple network on your laptops on your iphones you can run these
23:43
Josh: unbelievable models and now apple has the ability to do that with gemini and
23:48
Josh: i think it's strategic it makes sense they're collecting a paycheck they have
23:51
Josh: a great relationship with apple they've been doing it with the browser forever
23:53
Josh: and this is just a natural extension of that apple.
23:56
Ejaaz: Has all the distribution.
23:58
Ejaaz: They have 2.5 to 3 billion live Apple devices active in the world right now.
24:03
Josh: The Mag 7 stocks too. When you look at the Mag 7 stocks who've been like suffering
24:07
Josh: the most, Apple suffered the least.
24:09
Josh: It's just out of the race. It's right there unaffected by all of it.
24:12
Ejaaz: Yeah, they stayed number two market cap in the entire world for a company without
24:18
Ejaaz: touching a single GPU expenditure, without a single AI CapEx expenditure.
24:22
Ejaaz: Just insane strategy from them.
24:24
Ejaaz: They hold a very rich and important moat. But looking at Google as well,
24:28
Ejaaz: they're smart because they know that Apple won't win any of this AI stuff unless
24:33
Ejaaz: they get access to a major frontier LLM.
24:36
Ejaaz: Google's bet here is compute plus data equals the best model in the world.
24:42
Ejaaz: Apple does not have the compute. They do not have the model.
24:45
Ejaaz: Google can supply that for them. And something tells me that whereas Google
24:48
Ejaaz: might not be getting paid as much by Apple, they're going to exchange or broker
24:52
Ejaaz: a deal where they you get access to some of Apple's user data,
24:55
Ejaaz: which they can then use to train a better model, and it becomes this synonymous thing.
25:00
Ejaaz: But now, if the future is actually locally run edge compute models that run
25:05
Ejaaz: on your laptop that are frontier, then Google's shot themselves in the foot,
25:09
Ejaaz: maybe shot themselves in both feet at this point. But it's a bet they're making.
25:14
Ejaaz: And I think, I don't know, I think Google might have this one, if I'm being honest.
25:17
Josh: Yeah, I'm rooting for them. I'm also rooting for Meta now, who has this like
25:21
Josh: unbelievable pay package that I just saw.
25:24
Josh: $9 trillion is the valuation they need to exercise all these pay packages.
25:29
Josh: Like, okay, Elon, okay, Tesla board. I've seen this before.
25:33
Josh: What's the deal with this milestone package here?
25:36
Josh: Because a $9 trillion valuation for Meta, that seems like a lot of money.
25:42
Ejaaz: Okay. So two things have happened over the last week, which are at extreme odds at each other in Meta.
25:49
Ejaaz: Number one, they are closing down their metaverse division and rumored to be
25:54
Ejaaz: laying off up to 20% of the company. Now, these employees range from low-level
25:59
Ejaaz: product employees, engineering employees, all the way up to senior director roles.
26:03
Ejaaz: But this week, news also broke that they are going to be rewarding or they've
26:07
Ejaaz: offered very attractive comp packages to their top executives to the tune of
26:13
Ejaaz: $800 million for one person, their CFO.
26:17
Ejaaz: But it's under one premise and condition, which is over the next five years,
26:23
Ejaaz: Meta needs to hit a $9 trillion.
26:25
Ejaaz: That's with a T, dollar market cap. Just for context, no company in the history
26:31
Ejaaz: of the world has ever hit that market cap.
26:32
Ejaaz: In fact, the biggest and largest and most important company in the world right
26:35
Ejaaz: now, NVIDIA, is currently sitting at $4.5 trillion.
26:37
Ejaaz: So we're talking about a 2x for that. And for Meta, who is currently sitting
26:40
Ejaaz: in, I don't know where, are they still in the Mag 7 at this point?
26:45
Ejaaz: If so, I think they're like six or seven. They've got a long way to go,
26:49
Ejaaz: but it's interesting seeing the incentive design that Zuck is setting up. He spent
26:55
Ejaaz: I think upwards of $30 billion hiring 200 people this year and hasn't released a single AI model.
27:02
Ejaaz: Llama, their open source model is dead in the water. All the apps that they've
27:06
Ejaaz: released have been crap.
27:07
Ejaaz: So Meta is kind of making a big bet here. And I don't know if they're incentivizing
27:12
Ejaaz: the right people, if I'm being honest.
27:14
Josh: I'm rooting for them. I hope it works. I mean, in five years,
27:17
Josh: there will be a $9 trillion company, possibly multiple of them.
27:20
Josh: So there is a world in which Meta can achieve this. The problem is that they
27:24
Josh: haven't actually done anything to make progress towards that recently.
27:27
Josh: Like they've spent all this money, they've hired all this talent,
27:30
Josh: they haven't released anything compelling.
27:31
Josh: They haven't actually proven that the money that they're spending is working.
27:35
Josh: In fact, the counter argument to that is true, where Meta, the company,
27:39
Josh: just crushed the division that was responsible for renaming the company Meta.
27:43
Josh: So clearly there has been a series of big swings that haven't worked,
27:47
Josh: and we've yet to actually see a big swing that has worked, everything's failed.
27:51
Josh: Like you said, the open source models have failed. The pivot to the metaverse has failed.
27:54
Josh: Everything besides the core product of Facebook, which is their,
27:58
Josh: algorithm and their home feed and their social media platforms hasn't worked
28:03
Josh: out. Even the hardware sucks.
28:05
Josh: So they have a lot to figure out if they want to make this work.
28:08
Josh: But I'm rooting for them.
28:09
Josh: Zuck's an amazing CEO. I am like very hopeful that he can figure this out.
28:13
Josh: And this incentive structure is the right way to do it. You want to incentivize
28:16
Josh: people to bring the value. I hope they can bring the value.
28:19
Ejaaz: Now, for the last story of this episode, I want to introduce the listeners and
28:25
Ejaaz: watchers of this show to a secret, a secret within Limitless that none of you
28:30
Ejaaz: have ever known or has been publicized before.
28:33
Ejaaz: And this secret is about our co-host, Josh. Josh is a part-time music producer,
28:38
Ejaaz: and he releases absolutely banging tunes for the world to hear.
28:43
Josh: Yes. And this part-time skill actually originated about two weeks ago with the
28:46
Josh: advent of Google Lear A3.
28:48
Josh: And the fact that now I can make music with a single prompt,
28:52
Josh: maybe a sentence or two. So that's what I did today.
28:55
Josh: I said, generate me a three minute song about the Limitless podcast and how
28:58
Josh: it's the absolute best number one podcast in the world. There's nobody better.
29:02
Josh: The AI is scared of our show because it's so good. This just sounded like an
29:05
Josh: R&B hip hop type vibe. And we've spoken about Lyria 3 before of the show.
29:10
Josh: Lyria 3 is the music generation software from Google where you can prompt it,
29:14
Josh: you say what you want the song to be about, you say if you want lyrics or not,
29:18
Josh: what type of lyrics you want to sound, or you just let it run wild and choose everything for you.
29:22
Josh: So that was all the instruction i gave and it generated me
29:25
Josh: a song named no logic for the soul
29:28
Josh: which is very elegant i love i love the artwork it has like this cloud and these
29:33
Josh: roots and it's it's very good but the song is good so i want to play you a little
29:36
Josh: clip here of this song i'm not sure if you've had a chance to really enjoy the
29:41
Josh: full thing neither of us have so we're going to experience it for the first
29:43
Josh: time together but this is our new original no logic for the soul enjoy okay.
30:30
Ejaaz: That has to be a new outro song,
30:32
Josh: Josh. That is a banger. It's unbelievable how good this music is. So good.
30:36
Josh: The lyrics, not only do they make sense, but they rhyme. They have the right timing and cadence.
30:41
Josh: The chorus is great. They had a full horn section in there.
30:44
Josh: I mean, it's amazing, right? Because you think about the previous generation
30:49
Josh: and how they enjoyed music.
30:50
Josh: We had the Rolling Stones, the Beatles, the Grateful Dead.
30:54
Josh: These were generation-defining artists and musicians.
30:58
Josh: And the reason they were so powerful is because there was convergence around
31:01
Josh: them you go on the radio it's playing their music you talk
31:04
Josh: to your friends they have their cds and their vinyl records you go
31:07
Josh: on the street this is what's being played because this is what's available and
31:10
Josh: it was great but it was really the best that was available and everyone could
31:14
Josh: kind of converge on that fact the idea that you can now generate good music
31:18
Josh: not great music but good music and good enough for people who don't really care
31:21
Josh: to seek out great music implies the fact that we might never have anything like that again.
31:27
Josh: There might never be another Grateful Dead, another Rolling Stones,
31:30
Josh: another generational defining artist because it's so accessible.
31:35
Josh: If you look at the top charts on Spotify, it's the top trending tracks on TikTok.
31:39
Josh: And there's this direct correlation between the two. And it's stripped out a
31:42
Josh: lot of the humanness of artistry. It creates a whole different paradigm.
31:48
Josh: And listening to this really hits hard. I'm like, okay, yeah, it's actually over.
31:53
Josh: The fact that I can do this with a two-sentence prompt. I did this in 30 seconds.
31:57
Ejaaz: How long is the track, Josh, as well?
31:59
Josh: And the track is three minutes long. It's two minutes and 57 seconds.
32:01
Ejaaz: That's a regular song. That's a song that could go viral and like take over the Spotify charts.
32:06
Josh: And like if we did some prompt engineering, if we really refined the prompt
32:09
Josh: from more than two sentences to like a proper setup where there's many,
32:12
Josh: many paragraphs, many, many details, you kind of outline the lyrics and you
32:15
Josh: outline the cadence and what you want.
32:17
Josh: You can get unbelievably detailed with this and it sounds good. That's a good song.
32:22
Josh: So it's tough. I mean, this is one of those bittersweet moments where you're
32:25
Josh: like, holy shit, this is amazing, but holy shit, this is pretty powerful stuff.
32:29
Ejaaz: Now, I know there are a bunch of audiophiles listening to this thinking,
32:32
Ejaaz: God, Josh and Ijaz have no taste.
32:34
Ejaaz: May I just remind you, this is the worst this model will ever be.
32:38
Ejaaz: It's the worst. It's only going to get better. And there's already been a bunch
32:42
Ejaaz: of AI soundtracks that have gone completely viral.
32:45
Ejaaz: In fact, last week, Spotify had to shut down an AI music band,
32:49
Ejaaz: which they didn't realize was an AI music band, because they just didn't like that it was AI.
32:53
Ejaaz: But it was absolutely plugging through a bunch of streams. I think it gathered
32:56
Ejaaz: like 300,000 streams over the last month.
32:59
Ejaaz: Just insane. So these things aren't going away. Another thing I wanna point out is
33:03
Ejaaz: OpenAI just sunsetted Sora, which was the video generation of AI video generation version of TikTok.
33:10
Ejaaz: And now we have Google Liria, which is kind of doing a similar thing for music.
33:13
Ejaaz: So I don't think artistry and that type of medium is going away.
33:18
Ejaaz: There is an insatiable amount of demand.
33:20
Ejaaz: It helps music producers themselves kind of express the type of sound or song
33:26
Ejaaz: that they have in their head. So I think as a medium of translation,
33:29
Ejaaz: these AI tools are really valuable.
33:31
Ejaaz: But to your point, Josh, we are going to end up in a world where maybe everything
33:35
Ejaaz: sounds kind of beige and consistent.
33:38
Ejaaz: But I'm going to take the optimistic side of that, which is I think it's going
33:42
Ejaaz: to force humans and AI alike to be even more creative and come up with better
33:46
Ejaaz: soundtracks. I'm excited.
33:48
Ejaaz: And to your point around the Grateful Dead, you can now, I don't know,
33:52
Ejaaz: technically license their IP and create a never-ending album of their music.
33:56
Ejaaz: So maybe that's an angle as well.
33:58
Josh: Deadheads are not liking this at all. Oh man, this is going to disturb a lot of people.
34:04
Ejaaz: But it's here.
34:05
Josh: And like you said, it's the worst it will ever be. It's up only from here.
34:09
Josh: The quality will get better.
34:10
Josh: Of all of these tools, as Meta goes to $9 trillion, as Google disrupts entire
34:14
Josh: industries, it's all happening so fast.
34:17
Josh: That is another week fully covered in the books. If you've made it this far
34:20
Josh: in this episode and you've listened to the three previous episodes from this
34:24
Josh: week, you are now fully caught up.
34:25
Josh: There's nothing you need to know that you don't already know you can
34:28
Josh: go enjoy your weekend go touch some grass go hang out with a friend with um
34:31
Josh: do some analog stuff maybe that's kind of cool because i'm sure
34:34
Josh: everyone's just been in the trenches on um on their
34:37
Josh: devices watching the war zone all week but
34:40
Josh: yeah thank you guys so much for watching as always if you enjoy this episode
34:43
Josh: don't forget to share it with your friends who might also enjoy it uh subscribe
34:47
Josh: to our newsletter which is awesome it comes out twice a week we publish on x
34:50
Josh: all the time pretty prolifically now we're getting lots of action here's our
34:53
Josh: handles that you can find on screen right now Each has any final party thoughts
34:56
Josh: before we head off for the weekend?
34:58
Ejaaz: Um yeah i want a few of you to generate uh both josh and i individually and
35:03
Ejaaz: together our own songs and jingle let's see if you can come up with the best
35:07
Ejaaz: limitless intro outro song and maybe we'll feature it on a
35:10
Josh: Future that would actually be cool and send it to us on on x
35:13
Josh: because you can embed it in the post and we could just save it
35:15
Josh: from there so if you generate something tag you guys and
35:18
Josh: i on x and we will listen to it and maybe find a new intro outro track that
35:22
Josh: would be kind of that would be or a theme song i love a theme song we'd be like
35:25
Josh: superheroes we could have a theme song before we get on on camera that'd be
35:28
Josh: sick well anyways i know this was a long one so if you made it through i mean
35:31
Josh: 40 minutes of this show thank you you're a real one much appreciated as always
35:35
Josh: have an amazing weekend and we'll see you guys soon.