Latest / In the Loop AI / Claude Mythos: Anthropic's Leak That's Too Dangerous to Release

Transcript

Claude Mythos: Anthropic's Leak That's Too Dangerous to Release · In the Loop AI

0:00
Ejaaz: Three weeks ago, rumors broke that a major AI lab had built a model more powerful,
0:05
Ejaaz: more dangerous, and more expensive than any AI model that we had seen before.
0:09
Ejaaz: We didn't know which model lab it would be. We didn't know what the model was called.
0:13
Ejaaz: And then just a few days ago, Anthropic leaked a model called Claude Mythos,
0:18
Ejaaz: which is supposedly more powerful than any model that they've ever built before,
0:22
Ejaaz: a tier above Opus 4.6, which is what we see today.
0:26
Ejaaz: This model is actually so good that it is considered a cyber security threat
0:30
Ejaaz: and can't be rolled out to the public just yet.
0:33
Ejaaz: But it's not just Anthropic that's building a model that is close to AGI like this.
0:37
Ejaaz: OpenAI has a model codenamed Spud, Google has a model codenamed Agent Smith,
0:41
Ejaaz: and there's many more to come this year.
0:43
Josh: But the Anthropic leak wasn't intentional. This was discovered by accident last
0:48
Josh: Thursday, March 26th, by a Fortune reporter who discovered that Anthropic's
0:52
Josh: content management system had a configuration error.
0:54
Josh: And for those who aren't familiar, the content management system,
0:57
Josh: it's how the web server serves files.
0:59
Josh: And within that, there is a config error that leaked nearly 3,000 unpublished
1:04
Josh: assets sitting in this publicly searchable database.
1:07
Josh: Anyone could find them. So two independent security researchers,
1:10
Josh: they went through, they confirmed.
1:12
Josh: And among these files were two blog posts of two models named Claude Mythos
1:17
Josh: and a new tier named Capybara.
1:21
Josh: Anthropic immediately removed access to all of
1:24
Josh: this as soon as it came out but then later on an anthropic spokesperson
1:27
Josh: confirmed that it represents a step change in
1:30
Josh: ai performance and is the most capable model we've ever built so they confirmed
1:33
Josh: what we're seeing here is real now the problem is like this image suggests on
1:38
Josh: screen we're missing a lot of information this is a leak that something like
1:42
Josh: this exists but we don't we're not sure exactly what what we do know is that
1:46
Josh: there is the new model tier, Ejaz, like you mentioned, named Capybara.
1:49
Josh: It is the new tier that sits above Opus. So now the lineup will kind of look
1:53
Josh: like Haiku, Sonnet, Opus, and then Capybara at the top.
1:57
Josh: It doesn't really sound quite right. Maybe that's an experimental name.
1:59
Josh: They might find something better.
2:01
Josh: And then Mythos is the specific model name within that tier.
2:04
Josh: So you can think of Capybara as the weight class and Mythos as like the fighter.
2:07
Josh: It's the specific model.
2:08
Josh: Now, according to the leaked documents, this dramatically outperforms Claude
2:12
Josh: Opus 4.6 on basically everything, but particularly coding, academic reasoning,
2:17
Josh: and the cybersecurity benchmarks.
2:19
Josh: And I think the cybersecurity one is one of the more interesting points here,
2:22
Josh: because it's so powerful as cybersecurity that one of the main reasons why they
2:26
Josh: can't release it is to actually prevent people from using it maliciously. Is that right?
2:29
Ejaaz: Yeah. So actually, if we rewind to about a month and a half ago,
2:34
Ejaaz: Anthropik's head of AI security, who's actually a legend in the industry,
2:38
Ejaaz: gave a talk about Claude Opus 4.6, when it had just released.
2:43
Ejaaz: And his talk described how the model was pointed at five to 10 very popular
2:49
Ejaaz: open source code bases with no instructions given.
2:53
Ejaaz: And what the model did was very, very interesting.
2:56
Ejaaz: It scanned all those code bases and discovered 500 major security flaws.
3:03
Ejaaz: Expert human AI security researchers couldn't discover in decades that they'd
3:08
Ejaaz: been staring and using these exact code bases.
3:11
Ejaaz: So Claude did in a couple of hours what many security researchers couldn't do.
3:15
Ejaaz: We're talking about like millions of compute hours and time spent staring at
3:18
Ejaaz: these code bases, testing it.
3:20
Ejaaz: Claude, Opus 4.6 managed to figure this out. Now, this created a lot of excitement,
3:25
Ejaaz: but also a lot of concern.
3:27
Ejaaz: Now, because these AI security researchers had a good heart,
3:31
Ejaaz: they weren't using this maliciously.
3:33
Ejaaz: But if you could imagine that if this model had been placed to,
3:36
Ejaaz: say, a malicious actor, they could have exploited these for many different reasons.
3:40
Ejaaz: And so these exploits were surfaced and they were fixed. But the question now
3:44
Ejaaz: becomes, what if a more powerful model was made more readily available to anyone
3:49
Ejaaz: or an attacker, for example, a foreign adversary that could discover and exploit any future bugs?
3:55
Ejaaz: That's the concern that's around that i have personally around
3:58
Ejaaz: clode mythos or capybara this model is supposedly
4:01
Ejaaz: meant to be a tier above anything that we've ever seen before apparently it
4:05
Ejaaz: is amazing at discovering and exploiting exploits
4:08
Ejaaz: so if it is let's say two orders of magnitude let's be conservative two orders
4:12
Ejaaz: of magnitude better than opus 4.6 we could have a real problem on our hands
4:16
Ejaaz: and so what anthropic has done now is they've started to slow release this secret
4:21
Ejaaz: model mythos and capybara to cybersecurity experts first.
4:27
Ejaaz: Why? Because they want them to figure out how they can harden their own defense
4:30
Ejaaz: systems before they publicly release this model.
4:32
Ejaaz: And someone, maybe a nefarious attacker might use it for unachievable gain.
4:37
Josh: I think it's ironic that the company building what it describes as an AI with
4:40
Josh: unprecedented cybersecurity capabilities leaked it because someone misconfigured their blog.
4:45
Josh: Like the irony there is too strong. And you have to wonder, you have to really
4:49
Josh: ask yourself the question, well, what if this model so smart that it's leaking
4:52
Josh: itself if it's like poking holes to like let people secretly find it i don't know
4:57
Josh: The one thing for sure is that, one, this model is going to be incredibly expensive
5:01
Josh: to run currently, at least.
5:03
Josh: That's part of the reason why we're not seeing it now. But the second is it's
5:05
Josh: going to be unbelievably powerful.
5:07
Josh: And the progress that we've had in the last year is going to probably look like
5:11
Josh: nothing compared to what we're going to get for the next three quarters.
5:14
Josh: The market also very much felt the effects of this because, oh my God,
5:17
Josh: these stock charts look absolutely horrendous.
5:19
Ejaaz: Yeah, CrowdStrike, which is like the major cybersecurity firm,
5:23
Ejaaz: was down a couple billion on the news.
5:26
Ejaaz: And Palo Alto Networks, which is another similar company that competes in this
5:28
Ejaaz: firm, also suffered from this.
5:30
Ejaaz: Now, these two charts that I'm looking at right now for these specific companies,
5:35
Ejaaz: Josh, gives me a little PTSD or deja vu.
5:38
Ejaaz: Because we were talking about this, I think, four weeks ago when Anthropic released
5:43
Ejaaz: their security review clawed feature.
5:45
Ejaaz: Which, you know, wasn't anything to do about Mythos, but basically helped review
5:49
Ejaaz: the Vibe code that you produced using Claude. And so cybersecurity stocks dumped again.
5:54
Ejaaz: This is happening seemingly on a monthly basis at this point.
5:58
Josh: Even though these charts are down quite a bit, I'm not sure how concerned the
6:02
Josh: market needs to be immediately because it appears as if this new model that's coming,
6:06
Josh: this new cybersecurity specialist is really compute intensive,
6:10
Josh: so much so that it's almost going to be impossible for them to run across all
6:15
Josh: the accounts currently without some serious compression and iteration and figuring
6:20
Josh: out how to run this more optimally.
6:21
Josh: And it seems like we're starting to see those growing pains, right? It's like,
6:25
Josh: As they're training models like this, as they're running them on their own servers,
6:27
Josh: it's starting to affect the average user.
6:30
Josh: I know sometimes I'll wake up and I'll feel like my opus is running a little
6:33
Josh: bit dumber than it was the day before. And we actually have data that backs this up.
6:36
Ejaaz: Yeah, so basically over the weekend, Clawed servers basically went down or were majorly impaired.
6:42
Ejaaz: There were a bunch of different outages. People were reporting very,
6:46
Ejaaz: very reduced quality in their interactions with Clawed.
6:49
Ejaaz: And this has been kind of like a repeating trend over the last couple of weeks.
6:53
Ejaaz: And now we might have the answer why.
6:55
Ejaaz: Typically, major AI labs, the last public bit of information that we had was
6:59
Ejaaz: from OpenAI's 2025 run of a major model.
7:03
Ejaaz: They dedicated 30% of their available compute to a training run.
7:07
Ejaaz: Now, the rumors state that for Claude Mythos, they've dedicated even more and
7:11
Ejaaz: that's like the major architectural breakthrough that they've made.
7:15
Ejaaz: If they've done that, that might be the reason why we aren't being able to use
7:19
Ejaaz: the best version of Claude as consumers because they're too busy using the compute
7:23
Ejaaz: to train the next step or tier in model.
7:27
Ejaaz: I don't know if this is a good or bad thing, but one thing it definitely like
7:30
Ejaaz: screams at me is like, we need a ton more compute.
7:34
Josh: Big time. And it's amazing to think about how far we've come just in the last
7:38
Josh: three months leading up to this moment here.
7:40
Josh: I mean, when you think about over the winter break is when people really start
7:43
Josh: to take vibe coding seriously.
7:44
Josh: And since then, companies have gone from a very small percentage of code to almost 100% of code.
7:50
Josh: I mean, this is saying 80% plus of all code deployed is written by CloudCo just for Anthropic.
7:54
Josh: It's unbelievable we started with opus 4.5 which
7:59
Josh: was released in november and then opus 4.6 came
8:02
Josh: in february which took us from a 200 000 token contacts
8:05
Josh: went into a million and now whatever this new thing is is going to really drive
8:10
Josh: up the coding capabilities in a really big way and i think it's probably worth
8:15
Josh: checking in on which model is going to be the strongest model which company
8:18
Josh: has the best model through the end of june and thanks to polymarket we have
8:21
Josh: some interesting stats on this.
8:22
Josh: So the people are betting that Anthropic has a 66% chance of having the best
8:27
Josh: AI model in June, which is huge.
8:30
Josh: And that number has increased very significantly recently. If you look just
8:33
Josh: back in February, it was Google who was the heavy favorite with a almost 80%
8:38
Josh: chance or 70% chance of having the best model.
8:41
Josh: That has changed recently in a big way, perhaps because of this leak.
8:45
Josh: But I'm not sure if this is fully up to date and
8:48
Josh: it may be missing some information because we have some news on open ai and
8:53
Josh: google who are planning to release something really important too and thank
8:56
Josh: you for probably for sponsoring that part of the show but let's talk about open
8:58
Josh: ai there's a new code name spud model that's coming and this is probably going
9:03
Josh: to be the mythos competitor so what is this looking like yeah
9:05
Ejaaz: Um that's the issue we don't really know all of these models we don't have the
9:09
Ejaaz: the specs we need the specs to talk about them.
9:12
Ejaaz: There's a few trends or patterns that are happening amongst the hottest,
9:16
Ejaaz: or should I say, top two or three AI labs.
9:19
Ejaaz: We've got Anthropic Releasing Mythos, which is their AGI or pre-AGI model,
9:23
Ejaaz: a massive, massive leap ahead.
9:25
Ejaaz: OpenAI is working on the same thing. They've been secretively working on a larger model.
9:30
Ejaaz: This has gone through a few different names. If you remember,
9:32
Ejaaz: Josh, by the end of the year, I think it was referred to as codename Sprout.
9:36
Ejaaz: And now it's referred to as Spud. So I don't know if that implies that it's
9:39
Ejaaz: grown massively since then.
9:41
Ejaaz: It's growing. But these models are supposedly meant to be anywhere between 10
9:44
Ejaaz: to 20 trillion parameter models.
9:47
Ejaaz: Now, for context, the largest models that we currently look at right now is
9:51
Ejaaz: between one to two trillion.
9:53
Ejaaz: So this is a major order of magnitude larger model.
9:57
Ejaaz: They're going to be compute intensive. They're going to be very expensive to serve.
10:00
Ejaaz: So we need to figure out how to scale AI infrastructure and a bunch of other things.
10:04
Ejaaz: But OpenAI's model is codenamed Spud, and it's meant to be the competitor to
10:08
Ejaaz: Mythos. People are anticipating that it might be something like GPT 5.5 or rather GPT 6.
10:15
Ejaaz: So again, a tier above what we see today. It's going to be advanced in coding,
10:19
Ejaaz: reasoning, and a lot of the things Anthropics is as well.
10:22
Ejaaz: When I look at this, Josh, personally to me, this seems to be,
10:26
Ejaaz: one, a massive bid to try and leapfrog each other.
10:31
Ejaaz: And number two, maybe try and juice their numbers ahead of a potential IPA.
10:34
Ejaaz: I don't know whether your reaction to this is the same, but that's like my gut
10:37
Ejaaz: reaction when I read news like this.
10:39
Josh: Yeah, it's probably both. They want to juice up things before the IPO,
10:42
Josh: but they also just want to win.
10:43
Josh: And I have some pretty strong speculations just based on vibes of what this is going to look like.
10:48
Josh: I think we've been seeing this recent convergence around OpenAI,
10:52
Josh: particularly on focus and on really dialing in what they're focused on.
10:56
Josh: And we saw a big move last week when they removed Sora. They totally destroyed Sora.
11:01
Josh: They moved a lot of the teams together. They made their chief of product,
11:06
Josh: um the chief of like agi release and it appears as if they're building a mega
11:11
Josh: app based on the rumors so
11:13
Josh: Part of the reason why I have a difficult time using OpenAI's products is there's
11:17
Josh: kind of spread out everywhere.
11:19
Josh: There's like the Sora app was one, there's Codex, then there's their browser,
11:23
Josh: then there's ChatGPT, and there's a lot of different software.
11:26
Josh: And the same is true with their models, or it was at least, where there was
11:29
Josh: GPT 5.3 Codex, and there was 5.3 High, Mid, Low.
11:33
Josh: There's all these different models that really complicate things and confuse things.
11:37
Josh: With 5.4, they made a singular model. Now 5.4 does your coding and it does the reasoning all in one.
11:42
Josh: And what I suspect with this new model, Codename Spud, is going to be the kind
11:47
Josh: of pinnacle of this focus, where I'm hoping they release this with their new
11:50
Josh: application, with a singular model.
11:53
Josh: So there's one model that is all-knowing. There's one application,
11:56
Josh: similar to what Anthropic does with the Cloud Desktop app, that has all of the
11:59
Josh: functionality under one roof.
12:00
Josh: And I think they're going to probably use this as a point to really...
12:04
Josh: Lean into that focus instead of distributing this across a lot of different areas.
12:08
Josh: And I'm hopeful that that will meaningfully change OpenAI more so than it'll
12:12
Josh: change Anthropik because it actually changes the way that users interface with
12:16
Josh: the product and it becomes a much better product.
12:17
Ejaaz: Yeah, I think for the majority of last year, I was pretty upset with the way
12:22
Ejaaz: that SAM and OpenAI were focusing on so many different things.
12:25
Ejaaz: I was just like, just focus on creating a really good model.
12:28
Ejaaz: You're being left behind in coding, Anthropik's eating your lunch, like figure this out.
12:32
Ejaaz: And then since their code read of like, what was it, November last year,
12:35
Ejaaz: they've been like reallocating compute, money, data, and all their resources
12:39
Ejaaz: to focus on building the best general model and the best coding model.
12:43
Ejaaz: So we're starting to see the fruits of that labor.
12:45
Ejaaz: I have a lot of faith now in OpenAI that they're going to produce a really good
12:48
Ejaaz: product that will compete with the likes of Anthropic, which have been eating their lunch.
12:51
Ejaaz: When I look at like the last week, it seems like it's pretty negative for OpenAI.
12:55
Ejaaz: You mentioned that they killed Sora.
12:57
Ejaaz: They also killed the $1 billion deal that they had signed with Disney.
13:00
Ejaaz: And they also shut down ChatGPT adult mode and a bunch of like consumer shopping
13:04
Ejaaz: apps and their like app marketplace as well.
13:06
Ejaaz: They're just focused on these few things right now.
13:09
Ejaaz: But then the other thing is Sam is also kind of defaulting on a few of the major
13:14
Ejaaz: GPU and data center deals, right?
13:16
Ejaaz: So we had the OpenAI and Oracle Abilene deal fall through where they couldn't
13:20
Ejaaz: finance it for a variety of different reasons.
13:24
Ejaaz: Then the other thing is they're defaulting on purchasing up to 40% of the world's
13:28
Ejaaz: memory supply because they haven't figured out their finances right now.
13:32
Ejaaz: So I think that OpenAI is going through kind of like a puberty period where
13:36
Ejaaz: they're figuring their stuff out and where to reallocate resources.
13:39
Ejaaz: But I think they're going to pull through.
13:40
Josh: And it also seems like this is indeed a serious breakthrough.
13:43
Josh: I mean, Sam, in an internal memo that got leaked out to employees,
13:46
Josh: he said things are moving faster than many of us expected.
13:49
Josh: And he called it a very strong model that can really accelerate the economy.
13:53
Josh: That seems like pretty large claims to make internally with employees who are
13:59
Josh: also kind of in the know and aware of what's going on.
14:01
Josh: And I just think that a lot of us who are sitting outside these labs are not
14:06
Josh: entirely wrapping our head around how much progress is actually about to hit
14:10
Josh: us over the next couple of months with these new model releases.
14:13
Josh: It seems like they're step function improvements.
14:15
Josh: And one of the employees from OpenAI actually hinted that Spud contains a capability
14:19
Josh: that is very different from what we've seen before. So while there aren't specifics,
14:23
Josh: there are clearly a lot of these huge novel breakthroughs incoming,
14:27
Josh: which is worth looking out for.
14:29
Josh: There's one final model release, model leak that we have from Google,
14:33
Josh: who has been doing well, kind of chugging along slowly in the background.
14:36
Josh: And this is called Agent Smith.
14:38
Josh: It's a secret AI tool. Do you have any information on this one, EJS?
14:41
Ejaaz: Yeah, so there was like a leaked report from an insider at Google.
14:45
Ejaaz: Apparently, Google employees are using a new internal tool called AgentSmith
14:49
Ejaaz: that can automate tasks such as coding, according to three people that were familiar with it.
14:53
Ejaaz: The way that this product is supposed to work is within their Vibe coding platform
14:57
Ejaaz: called Antigravity, which exists today but hasn't really had a major upgrade
15:03
Ejaaz: for, let's say, a couple months now, which is like an eternity in the AI world.
15:07
Ejaaz: So they're releasing a new AI model called Agent Smith that is supposed to take
15:11
Ejaaz: a multi-agent approach and use an upgraded version of Gemini 3.1.
15:16
Ejaaz: So it's probably not going to be 3.1. It might be 3.5 or maybe even 4.
15:20
Ejaaz: Again, another order of magnitude leap up. So what we're seeing here is Google
15:23
Ejaaz: working on an AI coding model competitor to try and catch up to Anthropic and
15:27
Ejaaz: the likes of OpenAI's codecs.
15:29
Ejaaz: You've got OpenAI trying to reallocate resources and focus on building the best
15:33
Ejaaz: general model and catch up with Anthropic, which they have at coding.
15:36
Ejaaz: Then you have Anthropic trying to keep these two at bay and make the next order
15:41
Ejaaz: of magnitude up spending all their compute but coming at the expense of serving
15:44
Ejaaz: their existing users which they're adding like a million a day reporting you know
15:49
Ejaaz: Claude servers being down and reduced quality of usage. So this is a very,
15:53
Ejaaz: I can like feel the tension in the air between these three companies right now.
15:56
Ejaaz: I don't know what Mets is doing.
15:58
Ejaaz: I don't know where Grok is. I'm rooting for them. I hope they catch up.
16:01
Ejaaz: But it seems to be these three major competitors right now that are in the running for winning this race.
16:05
Josh: They're firing. I mean, in the last 90 days, since we started this year to now,
16:08
Josh: we went from 200,000 context windows to a million.
16:11
Josh: We went from these coding assistants to compiler writing
16:14
Josh: agents who are completely capable of writing a very small amount
16:17
Josh: now over a quarter of google's production software and 80 plus
16:21
Josh: of anthropic software everything we learned this week the frontier is
16:23
Josh: going to keep moving faster and faster so we're in
16:26
Josh: for a crazy q2 q3 q4 just a
16:29
Josh: crazy 2026 and as all these things happen as these
16:32
Josh: ipos start to happen and they get even more fundraising to deploy
16:35
Josh: these ai data centers at scale things are really going
16:38
Josh: to get weird in a hurry but we will be here to cover it as always um if you
16:42
Josh: enjoyed this episode please don't forget share it with your friends uh like
16:45
Josh: it on youtube don't forget to subscribe if you listen on a podcast player like
16:50
Josh: spotify or rss you could rate us five stars there it's always really appreciated
16:53
Josh: you just any final notes before we sign off for the day we've
16:56
Ejaaz: Been absolutely killing it on our side uh loads of new subscribers loads of
17:00
Ejaaz: new listeners thank you guys so much for for joining us um and yeah i have a
17:04
Ejaaz: request because we always like to give out homework at the end of the episode
17:07
Ejaaz: um if you're listening to this and you are a insider at anthropic open air or
17:13
Ejaaz: google and you are willing to give an anonymous tip to our accounts,
17:16
Ejaaz: please spin up an Anon account on x slash Twitter and DM us.
17:20
Ejaaz: I would love to hear from you.
17:21
Josh: That'd be great. Well, yeah, thank you guys for watching. We'll see you in the next one.