Latest / In the Loop AI / The Good and Bad of AI Coding: Amazon Shuts Down, Autoresearch, Claude Code Review, Lovable

Transcript

The Good and Bad of AI Coding: Amazon Shuts Down, Autoresearch, Claude Code Review, Lovable · In the Loop AI

0:00
Ejaaz: Last week, Amazon's entire platform crashed for six hours. No one could shop,
0:05
Ejaaz: buy anything, they couldn't even see prices.
0:07
Ejaaz: The reason was because a junior developer had submitted an AI-generated piece
0:12
Ejaaz: of code which crashed the entire platform, and it cost them millions and millions of dollars.
0:18
Ejaaz: Now, Anthropic, the creators of Claude Code, which is what Amazon was using
0:22
Ejaaz: to create AI-generated code,
0:24
Ejaaz: also had a similar issue where their entire platform has been suffering from
0:28
Ejaaz: outages this entire week.
0:29
Ejaaz: They actually also released a new product called Code Review,
0:33
Ejaaz: which helps use AI to help fix the code problems that their own model is fixing.
0:39
Ejaaz: It's all getting incredibly complex right now. And Amazon tried to hide the
0:42
Ejaaz: entire thing from a Financial Times reporter. It's all pretty crazy.
0:46
Ejaaz: And it forces us to answer the new narrative, which is AI-generated code isn't going anyway.
0:52
Ejaaz: Demand is at an all-time high, but have we hit a wall? Has it become too dangerous to use AI to code?
0:58
Josh: Yeah, and there's this interesting phenomenon happening. As these AI coding
1:01
Josh: abilities in general become better, where currently they account for,
1:04
Josh: what, about 4% of total GitHub commits.
1:06
Josh: The expectation is at the end of the year, they will account for 20% of total GitHub commits.
1:11
Josh: And there is this increasing reliance on these AI tools, but that creates these
1:16
Josh: key choke points and points of failure that have a significant effect.
1:21
Josh: I mean, And this Amazon outage was a huge deal that was pushed by a single person.
1:24
Josh: And then within Claude Code, they've been having a lot of downtime.
1:27
Josh: And the Anthropic, I mean, we were trying to use the programs today and it was
1:30
Josh: down. The servers weren't working quite right. So there's a lot of these growing
1:33
Josh: pains that are happening.
1:34
Josh: And it seems like we're running into more issues faster than people are trying to mitigate it.
1:38
Josh: So that's where I suspect this Claude Code checking agent comes in that we're
1:42
Josh: going to cover later. But this Amazon story was pretty fascinating.
1:45
Josh: Like this cost Amazon billions of dollars.
1:48
Ejaaz: Yeah, so let me walk you through the timeline for this, because this actually
1:53
Ejaaz: isn't the first outage Amazon's experience because of an AI-generated piece of code.
1:58
Ejaaz: So back in early December, actually November of 2025, they took a really hard-line
2:04
Ejaaz: approach and a new policy was invoked, which was...
2:07
Ejaaz: I want 80% of all Amazon's code generated to be AI generated.
2:13
Ejaaz: And this was their goal to be achieved by the end of 2026.
2:17
Ejaaz: Now, this flips, completely flips from a company that I think employs like hundreds
2:22
Ejaaz: of thousands of engineers and wants them all to kind of like handwrite or hand type the code.
2:26
Ejaaz: So this is a pretty aggressive flip. Amazon's been laying off tens of thousands
2:29
Ejaaz: of people. So this kind of is in trend with what they wanted to do.
2:33
Ejaaz: In the middle of December 2025, they experienced their first outage.
2:37
Ejaaz: It was 13 hours back then.
2:39
Ejaaz: So this is all adding up, by the way, we're talking about 10s of millions of
2:42
Ejaaz: dollars, then in late 2025. So at the end of December, there was another outage.
2:46
Ejaaz: And then we have the outages that we're speaking about today.
2:49
Ejaaz: And the issue that this flags is, although AI is like a really useful tool to
2:53
Ejaaz: generate code, and it finds bugs, it actually might create more problems than you'd expected.
2:59
Ejaaz: Because the issue that they're seeing is junior developers that come in that
3:03
Ejaaz: don't understand Amazon's code base, just kind of use AI to run like code to
3:08
Ejaaz: run autonomously, figure out what they want to create, and then they just submit
3:12
Ejaaz: it without actually reviewing and understanding it.
3:14
Ejaaz: And if this goes unguarded, it creates and results in issues like this.
3:18
Josh: Yeah. And there's a problem that's starting to happen now where agents are creating
3:21
Josh: code far faster than humans can keep up and check it.
3:24
Josh: So it becomes this impossibility where if you want to move at the velocity that
3:28
Josh: AI enables, you are simply unable to keep up with the change log of what's happening.
3:32
Josh: So you have to defer some sort of trust level to this AI, to its ability to
3:36
Josh: check itself, to run tests, to verify that it actually works.
3:38
Josh: And in some cases, it doesn't. I mean, I'm sure some people listening to this
3:41
Josh: experienced the outage.
3:42
Josh: And it wasn't just an AWS outage. This is Amazon, the actual storefront,
3:47
Josh: where my dad actually texted me.
3:48
Josh: He was like, am I doing something wrong? Can you place this order for me?
3:51
Josh: Because it's not going through.
3:52
Josh: And then I went to check myself and the whole service was down. I went on Twitter.
3:55
Josh: I saw Casey Neistat was posting a bunch of photos about this. It
3:58
Josh: was this big deal because anyone who was trying to order anything from
4:01
Josh: amazon the entire storefront was just totally offline so
4:04
Josh: aws the web services runs a significant percentage
4:07
Josh: of the internet amazon the storefront runs a significant percentage of the e-commerce
4:10
Josh: and these things have been going down at an increasing rate due to these tools
4:14
Josh: like cloud code so they're moving much faster but they're they're breaking things
4:18
Josh: this is like the early zuck facebook mantra it's like move fast and break things
4:22
Josh: they're now taking it to the extreme and
4:24
Josh: As we become increasingly reliant on these tools, we might see this start to,
4:28
Josh: I mean, permeate even further than just Amazon.
4:30
Ejaaz: Well, the weirdest part about this is that AI kind of has moved from this assistive
4:35
Ejaaz: tool to now being the foundational bedrock for a lot of these different like
4:39
Ejaaz: services that you've just mentioned. Like AWS runs the entire internet.
4:43
Ejaaz: Every single business that you interact with online probably uses AWS on the
4:46
Ejaaz: back end. So if they go down, then your business goes down as well.
4:49
Ejaaz: That's where we get all these outages.
4:51
Ejaaz: Amazon had some pretty severe repercussions. They have now done a complete 180
4:55
Ejaaz: on their policy of generating 80% of their code via AI.
4:58
Ejaaz: And they've now said, if you're a junior developer or engineer working at Amazon,
5:02
Ejaaz: you now need to get your manager's permission to submit code.
5:07
Ejaaz: So this is so tough because you've now gone from trying to automate the entire
5:12
Ejaaz: thing to now making it incredibly worse.
5:15
Ejaaz: So you're using AI to kind of generate the code, but then you still have to
5:18
Ejaaz: rely on a human constraint. and there are fewer managers than they are like junior developers.
5:23
Ejaaz: So it all just becomes really bogged up as a pipeline to kind of shipping changes.
5:27
Ejaaz: So I think this is going to slow down Amazon massively.
5:30
Ejaaz: But if it's for the result of, you know, saving your company tens of millions
5:33
Ejaaz: of dollars, fair play. But yeah, it's a pretty hard line approach.
5:36
Josh: And this all seems kind of connected to the trends that we've been seeing,
5:39
Josh: right? Particularly around jobs and distribution.
5:41
Josh: It's like a lot of people are cutting jobs, which means they're increasing the
5:45
Josh: reliance on these AI tools. But if the result is that the actual output of these
5:49
Josh: ai tools is causing a lot of damage it creates this this tension and
5:53
Josh: And it seems like the AI labs themselves, Anthropic, OpenAI,
5:57
Josh: they cannot hire talented people fast enough.
5:59
Josh: They are looking to hire every engineer under the sun who is competent and capable
6:03
Josh: of building great products.
6:04
Josh: But a lot of other companies are deferring that workload to these AI labs,
6:09
Josh: which is starting to create this interesting dynamic where the job market is
6:13
Josh: suffering, but the actual productive output of these companies is not to an extent.
6:17
Josh: And that's operating under the assumption that these code tools will get better.
6:19
Josh: But if Amazon and a lot of other companies start putting in these thresholds
6:23
Josh: that, again, bottleneck and throttle that amount of AI production capability,
6:27
Josh: like that probably doesn't help the cause a whole lot.
6:30
Ejaaz: There's also the angle that AWS or just data centers in general are becoming
6:35
Ejaaz: like a really valuable or national security threat level asset, right?
6:42
Ejaaz: So if we look at the current Iran conflict that's being waged between the US
6:45
Ejaaz: and Iran, Iran actually targeted strikes specifically to Amazon's AWS centers
6:53
Ejaaz: out in the UAE and Bahrain.
6:55
Ejaaz: And this caused a bunch of the outages as well. So it's not even just like AI coding level threats.
7:01
Ejaaz: This is becoming a geopolitical weapon at this point.
7:04
Josh: It makes sense. Every single story we tell, like this Game of Thrones narrative
7:08
Josh: that's kind of existed throughout the history of Limitless,
7:10
Josh: it's just elevating itself higher and higher to the global stage,
7:14
Josh: where now it's impossible to have any sort of conflict or
7:17
Josh: large decision without ai being in the middle of it like when you think
7:20
Josh: about this conflict what are the bigger things it's this like anthropic
7:23
Josh: deal with the pentagon they're now striking the aws data centers it's coming
7:26
Josh: for the infrastructure that's building these tools that are so powerful but
7:29
Josh: this isn't just happening on the worldwide stage this is happening on the consumer
7:33
Josh: and commercial level too you were mentioning something about lovable and how
7:36
Josh: much they've grown recently could you just explain lovable for the people who
7:39
Josh: aren't familiar and what's what how big they have grown recently it's crazy
7:43
Ejaaz: Okay, so typically the people who code, this might shock some of you,
7:47
Ejaaz: are the people who have learned to code.
7:49
Ejaaz: They're very technically focused work. But there are people like you and I and
7:52
Ejaaz: a bunch of our friends that don't necessarily know how to code,
7:55
Ejaaz: but still want to do that.
7:56
Ejaaz: And that's what resulted in Vibe coding, right?
7:59
Ejaaz: Now, Claude Code and products like Cursor are still very oriented for technical folks.
8:05
Ejaaz: A platform like Lovable is for people who have zero experience coding,
8:09
Ejaaz: but still want to code things. So the Lovable platform is actually a really
8:12
Ejaaz: cool platform where you just type whatever you want.
8:14
Ejaaz: Like I want to create an app that tracks my fitness reps or whatever that might
8:17
Ejaaz: be or create or suggest cooking recipes for my type of cuisine.
8:21
Ejaaz: And it does so in a couple of minutes.
8:24
Ejaaz: And it's really intuitive. It makes it super easy.
8:26
Ejaaz: And they have the additional perk of being able to deploy it live as a website
8:30
Ejaaz: or an app that you can publish on the app store super easily.
8:34
Ejaaz: So they have this end-to-end experience.
8:36
Ejaaz: Now, Lovable, when it started, had like a rocket-type trajectory.
8:40
Ejaaz: Within the first 12 months, I think they hit $200 million ARR,
8:44
Ejaaz: which is just like an insane growth. All of these AI companies are blowing up.
8:47
Ejaaz: Now, in the last month alone, they've added an extra $100 million of AR.
8:53
Ejaaz: They were at $300 million. Now they're at $400 million.
8:56
Ejaaz: So that's like roughly like just over 30% growth.
8:59
Ejaaz: Just insane amount of demand. And that's the point I want to make.
9:03
Ejaaz: Like when I look at these Amazon outages and Amazon restricting their developers
9:07
Ejaaz: from using Vibe Coding to improve their product, I don't think this is necessarily
9:12
Ejaaz: a wave that they can stop.
9:14
Ejaaz: So they should stop trying to implement policies that are restricting people
9:17
Ejaaz: from using it and instead try to figure out what is a better way to implement
9:22
Ejaaz: AI-generated code because this thing's not going away.
9:24
Ejaaz: Lovable, Cursor, Claw code are all skyrocketing and we're not going to stop that.
9:29
Josh: And I think a lot of the, I mean, aside of Claude Code, right, is Lovable and Cursor.
9:32
Josh: They're kind of model agnostic. They'll use whatever model you put.
9:36
Josh: So I was looking at Polymarket earlier today because they have markets for which
9:39
Josh: one of these models is actually going to be the best in which time.
9:42
Josh: And this is interesting because if you looked at these charts towards the middle
9:45
Josh: of January, you'd notice that Anthropic was kind of the favorite.
9:48
Josh: Anthropic was looking like they were going to have the best model.
9:51
Josh: But the reality now is that by March 31st, there's a very clear divide that
9:54
Josh: has happened over the last six to eight weeks. in OpenAI having an 85% chance
9:59
Josh: of having the best model.
10:00
Josh: Does that mean there's going to be something new or is that just the current model? We don't know.
10:04
Josh: Currently they're at 5.4, which is fantastic. I think everyone is kind of unanimously
10:09
Josh: decided that it's the best for coding.
10:11
Josh: But then there's also a fun following market here about Claude going down because
10:14
Josh: it appears as if Claude is really just not doing as good as it needs to be.
10:18
Josh: It's not as stable. We were trying to use it this morning. It wasn't working.
10:21
Josh: They're clearly experiencing growing pains because they have all these new users
10:24
Josh: and they're having to throttle reasoning.
10:26
Josh: They're having to limit the amount of searches that people are able to use.
10:30
Josh: I'm sure the researchers who use these to train the models are not happy.
10:33
Josh: And it looks like based on Polymarket, we'll be seeing somewhere between five to nine more outages in
10:42
Ejaaz: The month of March. Yeah, basically every single day this month.
10:44
Josh: Yeah, like it seems like they're having problems. And this is proof.
10:48
Josh: I mean, you can actually bet on the problems that they're having and make some
10:51
Josh: money on it, right? This is what, $26,000 of volume.
10:54
Josh: So thank you, Polymarket, for sponsoring this segment. And I guess now we can
10:57
Josh: get onto the solution, which is what Anthropic is starting to roll out.
11:00
Josh: But they're doing it in a way that's a little less than desirable.
11:03
Ejaaz: Yeah. So the main instigators of this entire AI coding debacle is Anthropic.
11:10
Ejaaz: They created Claude Code.
11:11
Ejaaz: Amazon owns 21% of Anthropic, which is just crazy to say out loud.
11:17
Ejaaz: And so they use Anthropic's model, Claude Code, to do all their AI code generation.
11:22
Ejaaz: But Anthropic decided, hmm, instead of trying to make our AI coding models better,
11:27
Ejaaz: let's just release a new product called code review, which will review your AI generated code.
11:32
Ejaaz: So you now have Claude generating the AI code and then also reviewing the code
11:36
Ejaaz: as a separate product that you pay for.
11:38
Ejaaz: And it's all for the beautiful price of, I think it's like 15 to 20 bucks. Where is it? Yeah.
11:44
Ejaaz: 15 to 20 bucks per review, which some might say is quite expensive.
11:48
Josh: And they scale based on the pull request complexity. So that number can get higher.
11:53
Josh: And what I find it funny is they're selling the problem where they're selling
11:58
Josh: you a model, you use it to code, it's going to create problems.
12:00
Josh: And then they're selling you a separate package for the solution where,
12:04
Josh: oh, you have problems in our code.
12:06
Josh: Well, here for another $25, $50, whatever it may be, we'll actually do a code
12:10
Josh: review and we'll fix the problems that we've created.
12:12
Josh: And that creates this uniquely disincentivized incentive where there's no real
12:18
Josh: reason for Claude to want to fix problems when they could just sell this package
12:21
Josh: on top to remove the problem.
12:23
Josh: And this comes in the face of chat gpt
12:26
Josh: and open ai doing the polar opposite where they're actually offering these
12:30
Josh: for free and here's rohan from open ai if you
12:32
Josh: want ai code review but don't want to pay 25 per review check
12:36
Josh: out codex review it leverages frontier codex models finds complex issues and
12:40
Josh: 100 usage based most runs should cost one dollar or less so here we have another
12:46
Josh: head-to-head collision happening where chat gpt open ai is going head-to-head
12:50
Josh: with cloud code and And it seems like Cloud Code is the favorite for a lot of people,
12:53
Josh: but I've noticed a lot of the more technical people on my timeline who are building
12:58
Josh: pretty hardcore things have been really relying on 5.4 and using codecs much more.
13:03
Ejaaz: Yeah, so to set the precedence, OpenAI and Anthropic now are neck and neck at
13:08
Ejaaz: building the best coding models.
13:09
Ejaaz: And Codex 5.4 from OpenAI has taken the lead. A lot of engineers are now saying
13:14
Ejaaz: it's way better than Opus 4.6, as you just mentioned.
13:17
Ejaaz: In terms of these security code review products that each company offers,
13:21
Ejaaz: OpenAI is instinctively cheaper.
13:24
Ejaaz: If you have a $20 a month subscription, you now get access to this code review
13:28
Ejaaz: tool for no extra cost. So it's automatically a better tool to use.
13:31
Ejaaz: Now, in terms of quality, in terms of how good it is at code review,
13:35
Ejaaz: we don't know because they haven't made any of their datasets public.
13:39
Ejaaz: Anthropic has, and it's pretty damn good.
13:41
Ejaaz: Also, if you compare Anthropic's product to traditional SaaS vendors that have
13:46
Ejaaz: these kind of AppSec security tools, it is still way cheaper.
13:49
Ejaaz: A lot of people were in my comments, actually, because I tweeted about this
13:52
Ejaaz: saying it was cheaper. They were like, no, it's not.
13:54
Ejaaz: I could use this SaaS tool. And I'm like, yeah, but you're not factoring in
13:57
Ejaaz: the engineering hours, they spend like 30 to 60 minutes per bug.
14:01
Ejaaz: That adds up to about $100.
14:02
Ejaaz: So it's still technically cheaper, but I would rather use OpenAI's tool,
14:05
Ejaaz: which is just way better.
14:07
Ejaaz: Now, Anthropic itself are facing so many outages for Claude.
14:10
Ejaaz: You mentioned earlier that, you know, Claude is out for you this morning.
14:13
Ejaaz: It's also out for me this morning. And I think this is because OpenAI basically
14:17
Ejaaz: has all the money in compute to subsidize this cost for a tool like this,
14:22
Ejaaz: whereas Anthropic is struggling.
14:23
Ejaaz: They're adding a million users
14:25
Ejaaz: to their general database or to the general user base every single day.
14:30
Ejaaz: And they're not able to subsidize people's compute anymore. So they're just
14:32
Ejaaz: shutting people's access off to it. And it's kind of frustrating to see, to be honest.
14:36
Josh: Yeah, and the way you have to think about this is each one of these companies
14:39
Josh: has a limited amount of compute.
14:40
Josh: And that compute takes care of everything. It has to serve the customers.
14:44
Josh: It has to serve the developers, the researchers.
14:46
Josh: And a lot of times when you're developing these new technologies or you're training
14:50
Josh: new models, the training run consumes a tremendous amount of GPU power.
14:53
Josh: The researchers want a lot of GPUs to run tests and run trials on,
14:57
Josh: but you still have to find extra resources to serve all of the users who are querying this model.
15:02
Josh: Every single day. And I was recently listening to Dario on, I think,
15:06
Josh: the Dwarkesh podcast, where he was talking about how they think about how many
15:09
Josh: GPUs to order into the future.
15:11
Josh: Because they go to Jensen, they go to NVIDIA, they say, hi, Jensen,
15:14
Josh: we would like to place this amount of GPU purchases for this year.
15:18
Josh: And what he was mentioning that I found interesting is that they can't over
15:21
Josh: order because if they're off by just a small percentage, the incremental cost
15:26
Josh: of those GPUs will far outweigh the growth, the revenue that they get from growth.
15:31
Josh: And what it sounded like he did is he was just taking the current growth and
15:35
Josh: mapping it to the future to hedge themselves against this collapse from debt
15:40
Josh: that they would have from over-ordering.
15:41
Josh: But the reality is, is that since that episode was recorded and now they have
15:45
Josh: gone fully vertical, like that curve has steepened significantly more and they're
15:51
Josh: going to have to figure out a way to solve this because this is not an easy
15:53
Josh: thing. These orders get placed years in advance.
15:55
Josh: They're projecting for a certain curve and they're getting a different one.
15:58
Josh: So is this a short-term thing?
16:00
Josh: Is this going to be durable? I don't know, but it is noteworthy that Anthropoc
16:04
Josh: is having some growing pains.
16:05
Ejaaz: Okay, so we have two problems here. We have an amazing AI model that can code,
16:11
Ejaaz: and that is inevitably going to be the future. Lovable's demand proves that.
16:14
Ejaaz: But then we're also running into the issue where we have all these security
16:17
Ejaaz: flaws and we could lose tens to hundreds of millions of dollars.
16:20
Ejaaz: Amazon demonstrated that, right? So we're at this conundrum.
16:22
Ejaaz: How on earth do we solve this? Do you know what the solution might be, Josh?
16:27
Ejaaz: It might be AI. It might be AI itself.
16:30
Josh: That checks out.
16:30
Ejaaz: You have to use AI to solve both of these individual AI problems, I'm afraid.
16:35
Ejaaz: So, Andre Carpathy, the godfather of AI, as I like to call him,
16:39
Ejaaz: released this really cool experiment that is just completely blown up,
16:43
Ejaaz: which hints at what the future solution might be to solving this problem that
16:47
Ejaaz: we just explained on this episode, which is called auto-research.
16:50
Ejaaz: Now, the best way to think about this is he created an AI model that acts as an AI researcher.
16:57
Ejaaz: But the coolest part about it is it autonomously does research.
17:01
Ejaaz: Now, typically, when you use an LLM, you need to prompt it. And then it says,
17:05
Ejaaz: I don't really know what this means.
17:07
Ejaaz: Can you tell me what it means? And you have to keep working back and forth.
17:10
Ejaaz: This thing runs completely on its own overnight, and in some cases, for days at a time.
17:15
Ejaaz: He's done this overnight when he slept for like 9 to 12 hours,
17:18
Ejaaz: and then he's done it again for two to three days.
17:21
Ejaaz: And how the AI model works is he sets an objective. He says,
17:25
Ejaaz: I want you to try and improve this part of your model.
17:28
Ejaaz: So he's talking to the AI model and saying, you need to improve yourself in this particular way.
17:33
Ejaaz: Go away and figure it out. The model then runs an experiment every five minutes.
17:37
Ejaaz: If the experiment comes out with an improvement, it caches it in.
17:41
Ejaaz: It improves its own model weight.
17:42
Ejaaz: It improves its own internal insights, right?
17:45
Ejaaz: If it doesn't, it discards it
17:47
Ejaaz: and it tries again. And it runs these experiments on and on and on and on.
17:51
Ejaaz: In this experiment, I think he ran around 150 experiments overnight.
17:54
Ejaaz: And it improved itself pretty marginally, as is demonstrated by this graph.
17:58
Ejaaz: The loss aversion just went crashingly down.
18:00
Ejaaz: So it suggests that maybe Anthropic and Open Air, instead of having to rely
18:04
Ejaaz: on releasing new tools every day, constrained by humans, they could just let
18:07
Ejaaz: the AI improve itself and everyone's hunky-dory, maybe.
18:10
Josh: This seems like an early form of this takeoff situation in which AIs become
18:15
Josh: self-improving, because I haven't really seen this architecture before in such a
18:20
Josh: open and easily accessible way i mean this is
18:23
Josh: all open source this i mean andre says here it's 630 lines
18:27
Josh: of code and traditionally it's on the human to iterate on
18:30
Josh: the prompt to improve the prompt to improve the
18:33
Josh: training code and to work together with the ai but this actually
18:36
Josh: does it completely and entirely on its own it goes
18:38
Josh: off it runs its own experiments it ingests those experiments updates
18:42
Josh: its values that it has updates its view on the world and then creates a new
18:45
Josh: set of experiments and runs this iterative process and what we saw in that chart
18:48
Josh: is that there was these incremental improvements that andre just didn't see
18:52
Josh: and again it's hard to overstate how much of a powerhouse andre is in terms
18:59
Josh: of ai engineering and understanding do
19:02
Ejaaz: He has two decades of experience in.
19:04
Josh: This and like ai what like wasn't even
19:06
Josh: around two decades ago like he was the guy who was building the damn thing
19:09
Josh: yeah um so for him to come along and for
19:12
Josh: the first time ever experience this novel breakthrough in which an ai
19:15
Josh: isn't able to improve his code in ways that he didn't initially see
19:18
Josh: it's like when ai played go for the first
19:21
Josh: time or when ai is playing chess and it makes these erratic moves you would
19:24
Josh: not have known that it was optimal but the model is
19:27
Josh: is so precise has so much context and so
19:30
Josh: much intelligence that it's actually able to make these optimizations that
19:33
Josh: humans otherwise wouldn't have seen and this is one of the earlier instances where
19:36
Josh: even andre got humbled and the code that he
19:39
Josh: was running was significantly improved overnight by this
19:42
Josh: self-recursive thing and the most interesting thing about this is it's
19:46
Josh: open source it's available to run on your laptop on
19:49
Josh: your pc on a computer right now and i
19:52
Josh: have to imagine that ai labs are going to take this and run with
19:55
Josh: it at scale where if you're training a model why would you not and and this
20:00
Josh: model using this auto research can do better than andre i'm sure it could do
20:03
Josh: better than the average mid-tier like prompt engineer at anthropic or chat or
20:08
Josh: open ai and actually improve the entirety of the company so this feels like something that is
20:13
Josh: starting from the bottom where in terms of it's like openly accessible it's
20:16
Josh: open source and then we'll move its way up through everything and become this
20:20
Josh: like auto-aggressive self-improving loop it's incredible i want to try it on
20:23
Josh: something we need a problem that's hard enough to try this on where you can
20:26
Josh: truly let it run for a series of days and then come back with the answer that's
20:29
Josh: better than anything you could have ever imagined well maybe research
20:32
Ejaaz: On how to make limitless the best and number one podcast in the world maybe i i'd be down to run this.
20:37
Josh: That's probably we should try that we should try that we can make some assumptions
20:40
Josh: on what it would it would say right It would say, well, we probably need to
20:43
Josh: convince all of our listeners to go subscribe to the podcast on your favorite podcast player.
20:48
Josh: Give it five stars, share it with all of their friends, and then make sure that
20:52
Josh: they comment things they either like or don't like about it to keep the engagement
20:55
Josh: rate high. Those things will help out the show.
20:58
Ejaaz: That actually sounds pretty great. And if you're listening to this,
21:00
Ejaaz: remember, you are no more than organic LLM.
21:03
Ejaaz: So if you want to process that prompt that Josh just specified,
21:06
Ejaaz: please go do it. It helps us out a massive amount.
21:09
Ejaaz: But just to round up and wrap up this episode, it's this weird conundrum.
21:14
Ejaaz: AI is either zero or 100 at this point.
21:17
Ejaaz: It's either deteriorating your product and causing massive outages,
21:21
Ejaaz: or it's doing something like this and helping the world's leading expert of
21:25
Ejaaz: AI research actually do a better
21:27
Ejaaz: job, which is just, there's no middle ground. It's either zero or 100.
21:30
Ejaaz: And it's just super exciting to see where all of this is going.
21:33
Ejaaz: I tweeted this out the other day.
21:36
Ejaaz: My mind, I don't know about you, Josh, is just in a fog at this moment because
21:40
Ejaaz: I can't keep up with everything that's going on.
21:42
Ejaaz: Something's switched over the last 30 days where
21:45
Ejaaz: The exponential curve has just like racketed up. I don't know whether this is
21:49
Ejaaz: because I'm in my own echo chamber or not, but it just feels like we are getting
21:52
Ejaaz: new model releases every single week at this point. There's so many new Frontier AI labs.
21:57
Ejaaz: Jan LeCun, who is the former head of AI research at Meta, has now started his
22:02
Ejaaz: own thing and he's focusing on world models. And I'm like, what is going on here?
22:05
Ejaaz: It's just, it's so much, but you're going to hear all about it on Limitless.
22:08
Ejaaz: Josh and I are working 24-7 doing this. We're releasing four new episodes every week, up from three.
22:14
Ejaaz: The viewership has been crazy the comments on our last episode
22:17
Ejaaz: where we talk about uploading a fruit flies brain into a laptop the banger of
22:22
Ejaaz: an episode go check that out comments are like four times what we normally get
22:26
Ejaaz: we're responding to every single one we are the most dialed show on ai that
22:30
Ejaaz: you can potentially watch uh please like subscribe leave us comments give us feedback and josh yeah.
22:36
Josh: And important to know like the four episodes is just a
22:38
Josh: testament to how fast we're going um and just the need to cover
22:41
Josh: more things so again you will not miss
22:44
Josh: anything if you watch the show and we will continue to scale to match
22:47
Josh: the the necessary demand to like sufficiently cover all of this and that's what's
22:51
Josh: happening i mean it's just downstream of the craziness we are very much in this
22:54
Josh: singularity event in the vertical part of the curve so enjoy it like i'm just
22:58
Josh: trying to take it all in i can take it for what it is get excited about it share
23:02
Josh: with everyone so yeah thank you guys all for watching as always and we'll see
23:05
Josh: you in the roundup tomorrow