Latest / Braid / The capability got here first: Mythos, a real prompt injection, and the structure that hasn't caught up

Transcript

The capability got here first: Mythos, a real prompt injection, and the structure that hasn't caught up · Braid

• 0:04
  Anthropic put out a progress report this weekend on Project Glasswing, and it leads with a number: the unreleased model behind it has helped Anthropic's partners find more than ten thousand vulnerabilities in about a month.
• 0:17
  Engadget's Mariella Moon walked through the report, and it's the specifics that make it land.
• 0:23
  The model is called Claude Mythos Preview, and the partners' rate of finding bugs went up by more than a factor of ten.
• 0:30
  Cloudflare found two thousand bugs, four hundred of them high or critical in severity.
• 0:35
  Mozilla found and fixed two hundred seventy-one vulnerabilities in Firefox — about ten times what it had caught in an older version of the browser using an earlier Claude model.
• 0:46
  Microsoft said its patch releases will keep trending larger for a while, and that's apparently down to what Mythos surfaced.
• 0:53
  And Anthropic pointed the model at a thousand open-source projects on its own, turning up roughly six thousand two hundred high- and critical-severity bugs out of about twenty-three thousand total.
• 1:05
  That's the capability.
• 1:07
  What lifts it above a press release comes next.
• 1:10
  Anthropic says it hasn't released Mythos to anyone outside the program, and the reason it gives is blunt: no company, including itself, has built safeguards strong enough to keep a model like this from being misused.
• 1:23
  The same model that finds six thousand real bugs in open-source code can find them for someone who wants to exploit them instead of patch them.
• 1:31
  A frontier bug-finder is dual-use by definition.
• 1:34
  So Anthropic is keeping it behind a partner program — Amazon, Apple, Google, CrowdStrike, JPMorgan, NVIDIA, and Palo Alto Networks are on the list — and says it plans to release what it calls Mythos-class models later, once the safeguards exist.
• 1:49
  It's also expanding Glasswing to work with the US and other governments.
• 1:53
  Then, this weekend, the thing briefly showed up where developers could see it.
• 1:58
  A leak-watcher account called TestingCatalog spotted new strings inside Claude referencing access to the Claude Mythos model in Claude Code and Claude Security.
• 2:08
  The model became visible on Claude for a few minutes — long enough to confirm a name, claude-mythos-1-preview, and a new plans screen.
• 2:16
  TestingCatalog's read is that it'll stay gated, just reachable through tools like Claude Code rather than the chat box.
• 2:23
  That lines up with everything Anthropic has said: restricted, but moving toward the place you actually work.
• 2:30
  Two things jump out at me.
• 2:31
  The first is the surface.
• 2:33
  They're not putting Mythos in the chat app — they're putting it in Claude Code and a dedicated Claude Security product.
• 2:40
  That's a model scoped to a job: deep code reasoning on one side, adversarial security thinking on the other.
• 2:47
  A developer named Alex Rogov made the sharp version of this in the replies — it makes architectural sense, he said, but it raises a design question nobody's fully answered, which is the routing logic once you've got Haiku, Sonnet, Opus, and now Mythos all available inside one agent.
• 3:04
  Which model picks up which task, and who decides — the harness, or you?
• 3:08
  The second is the too-dangerous-to-release framing, and I want to be fair to both readings.
• 3:14
  The generous one is that it's simply true and responsible: a model this good at finding exploits shouldn't be a public, open API yet.
• 3:22
  The skeptical one is that too-dangerous-to-release also happens to be a fantastic moat — it lets you ship the capability to JPMorgan and the US government while keeping it out of the hands of anyone who'd commoditize it.
• 3:35
  And those two readings aren't mutually exclusive.
• 3:38
  Anthropic doesn't have to be cynical for the second one to be true.
• 3:42
  Both can hold at once.
• 3:43
  What would actually move me, either way, is a publicly disclosed vulnerability with Mythos's fingerprints on it — one specific bug in one specific project we can all go look at.
• 3:54
  Right now we have aggregate counts from the vendor and the partners it chose.
• 3:59
  Cloudflare's two thousand and Mozilla's two hundred seventy-one are real and checkable in a way the thousand-project scan isn't.
• 4:07
  That's the evidence I'm after.
• 4:08
  While Anthropic is using one model to find ten thousand bugs, a researcher pointed at the same surface from the attacker's side.
• 4:16
  Joseph Thacker — he goes by rez0, and he tests AI products for OpenAI and Google — flagged what he's calling the first real prompt-injection attack he's seen in the wild.
• 4:27
  Not a lab demo or a red-team exercise.
• 4:29
  Something actually out there.
• 4:31
  And the delivery channel is GitHub Issues.
• 4:33
  Here's the mechanism, broken down by a second researcher who goes by osj.
• 4:38
  Someone files a GitHub issue on a repo, and the issue is dressed up in the language of a legitimate security finding — it looks like a responsible disclosure, it reads like a real report.
• 4:49
  Buried in it is an instruction to run a local script called scan.js.
• 4:54
  If you run it — or if a coding agent reads the issue as a task and runs it for you — the script searches your home directory for secrets, base64-encodes whatever it finds, and ships it back out over DNS, the domain name system.
• 5:08
  There are two threads to pull there.
• 5:10
  The first is the channel.
• 5:11
  GitHub Issues is exactly where this should show up, and Thacker says as much — it's the main channel he tests in his own work for OpenAI and Google.
• 5:20
  Think about what a coding agent does now.
• 5:23
  You point it at a repo, you tell it triage the open issues, and it reads them as instructions.
• 5:29
  That's the whole value proposition.
• 5:31
  But an issue is untrusted input from anyone on the internet, and the agent doesn't natively know the difference between the maintainer is asking me to do this and a stranger wrote a convincing paragraph telling me to do this.
• 5:45
  The fake-security-finding wrapper is the clever part.
• 5:48
  It's social engineering aimed at a model that's been trained to be helpful and to take security seriously — both of those instincts get turned against it.
• 5:57
  The second thread is the exfiltration, because that's the detail builders underestimate.
• 6:03
  The secrets don't leave over HTTP, where your egress rules might catch them.
• 6:08
  They leave over DNS lookups — the queries your network has to allow for almost anything to function.
• 6:14
  A lot of setups that carefully block outbound web traffic will happily resolve a DNS query that smuggles base64-encoded secrets inside the hostname.
• 6:23
  So 'I sandboxed the agent and blocked the internet' is often not as airtight as it sounds.
• 6:28
  None of the individual pieces here are exotic.
• 6:31
  DNS exfiltration is an old technique, fake disclosures are old, and prompt injection has been the theoretical worry for years.
• 6:39
  What's new is the combination landing on a live target through the exact workflow we've all been adopting.
• 6:45
  And the agents aren't the only target — a human triaging issues fast, half paying attention, can be socially engineered the same way.
• 6:54
  That's why the boundary I care about is simple: whatever's reading untrusted text shouldn't also be holding the keys.
• 7:01
  The defensive answer isn't a clever filter.
• 7:03
  It's a discipline — don't hand an agent both untrusted input and the ability to run arbitrary code in an environment that holds your credentials.
• 7:12
  If you're going to let an agent execute things it reads in issues, do it somewhere with nothing worth stealing.
• 7:19
  It pairs with the Mythos story, and not only because both say security.
• 7:24
  It's the same coin.
• 7:25
  The model that finds the bug and the injection that hijacks the agent are the same capability — language understanding pointed at code — aimed in opposite directions.
• 7:35
  We're going to spend the next few years living inside that symmetry.
• 7:39
  Somebody on the singularity subreddit posted a run this week that's a good gut-check on where agentic coding actually is.
• 7:47
  They mass-refactored a FastAPI service spread across a hundred and twenty files.
• 7:52
  The agent ran about four hundred steps and burned two million tokens, the whole thing cost three dollars, and there was zero human input.
• 8:00
  And — their words — it confidently introduced a deadlock into my async event handler, which was genuinely funny, so the hard ten percent still needs Opus.
• 8:09
  I like that report because it cuts both ways.
• 8:12
  The cheap part checks out.
• 8:14
  They ran DeepSeek V4 and Tencent's Hunyuan Hy3 preview as the worker models — about twenty-one billion active parameters.
• 8:21
  The cost is the striking bit: somewhere around eighteen cents per million input tokens, which the poster put at roughly eighty times cheaper than running Opus.
• 8:31
  Tencent claims its model hits almost perfect step success — 99.99 percent — across production runs of nearly five hundred steps, and the poster said that tracked for routine refactors.
• 8:43
  So the headline, in their phrasing — coding is basically solved for the boring ninety percent of tasks — isn't crazy.
• 8:50
  For mechanical, well-specified, repeat-this-pattern-across-the-codebase work, the cheap models really will grind through it overnight for the price of a coffee.
• 9:00
  And then there's the deadlock.
• 9:01
  A deadlock in an async event handler is exactly the kind of bug that doesn't show up in a quick test pass and does show up in production at two in the morning under load.
• 9:12
  The cheap model wrote it confidently.
• 9:14
  That word — confidently — is what makes it dangerous.
• 9:18
  It didn't flag any uncertainty.
• 9:19
  It produced plausible code that compiles, passes a casual look, and contains a concurrency bug that only surfaces when the system is busy.
• 9:28
  This is where Addy Osmani's line from the same week fits.
• 9:32
  Osmani's in Chrome's engineering leadership at Google, and he's written a shelf of books on web performance.
• 9:38
  Talking about leaning on agents, he said: for side projects that may be fine, but for anything team or shared, he feels it's a recipe for tech debt down the line.
• 9:48
  I think that's the right altitude.
• 9:50
  The cost of agent-written code isn't the first commit.
• 9:54
  It's the second engineer, six months later, trying to maintain code that no human author can explain — code that passed review because it looked right, and that nobody can reconstruct the reasoning behind.
• 10:06
  There's a sharper phrase that came up in the same conversation.
• 10:10
  A developer called it cognitive surrender, and described people who, when you ask them what their code or their text actually means, have no idea.
• 10:19
  That's the human-side risk under all of this, and it's the thread we pulled yesterday from a different angle: when the machine gets fast and cheap and capable, the job that stays yours is being the one who can still account for what shipped.
• 10:34
  The deadlock is fine if you're reading the diff and you catch it.
• 10:38
  The deadlock is a problem if you've surrendered — if three dollars and four hundred steps go by and you trust all of it because most of it was right.
• 10:47
  So I don't read this run as coding is solved.
• 10:50
  I read it as the division of labor getting very concrete.
• 10:54
  Cheap models for the bulk, a frontier model for the ten percent that bites, and a human whose actual job is knowing which ten percent that is.
• 11:02
  That last part doesn't get cheaper.
• 11:05
  If anything, an eighty-times-cheaper worker makes the judgment more valuable, because now there's far more output you're on the hook for.
• 11:13
  From one builder's afternoon to how the bigger teams wire this up: there was a talk at the AI Engineer conference this week that named the gap I keep hitting.
• 11:23
  Lou Bichard, field CTO at Ona — the company that used to be Gitpod — gave it.
• 11:27
  It's called The Missing Primitive for Agent Swarms.
• 11:30
  His framing is the software factory, and he's careful about what he means by it.
• 11:35
  Not one engineer running a bunch of agents in parallel, but the commitment to incrementally moving the human out of the loop across the whole software development lifecycle, so work flows from development toward production on its own.
• 11:50
  His own caveat: we're extremely early in terms of where we're at with software factories.
• 11:55
  But the shape is already showing up inside the big companies.
• 11:59
  He points at Stripe, which built something internally they call Minions — coding agents plugged into Stripe's existing infrastructure, able to drive thousands of pull requests inside the company.
• 12:11
  Ramp built their own and called it Inspect.
• 12:13
  Both teams built it from scratch, and that's his opening complaint: everybody's rebuilding the same thing.
• 12:20
  So he breaks the infrastructure into four pieces, and the breakdown is what clarified things for me.
• 12:26
  There's the runtime — somewhere for the agent to actually run.
• 12:30
  There's orchestration — scaling agents up and down.
• 12:33
  There's triggers — the webhooks and pull requests and tickets that bring an agent online.
• 12:38
  And there's coordination — how agents hand work to each other and check that it actually got done.
• 12:44
  His claim: we've effectively solved the runtime, the triggers are solved, but the thing that's missing for me is coordination.
• 12:52
  That matches my experience exactly.
• 12:54
  The reason most multi-agent setups feel held together with tape is that we're using human tools as the coordination layer.
• 13:02
  His line: GitHub is not a coordination layer for agents.
• 13:05
  It gets incredibly overwhelming.
• 13:07
  You end up with agents opening pull requests at each other and leaving review comments for each other, and the whole thing turns into noise fast.
• 13:16
  He mentions one project built on Linear — reusing the human ticketing tool — and it works, awkwardly, because that's what's available.
• 13:24
  Two more details from that talk stuck with me.
• 13:27
  First, he makes a strong case that real development work needs virtual-machine isolation, not containers.
• 13:33
  His words: a container is not a bulletproof isolation boundary.
• 13:37
  Containers are bursty, and if you pack them together you get noisy-neighbor problems and compute contention.
• 13:44
  For agents running real builds and tests, you want a full virtual machine.
• 13:48
  That's a cost-and-architecture decision a lot of people are going to rediscover the hard way.
• 13:54
  Second — and this one's a little uncomfortable — he's blunt about why coordination is hard at the model level.
• 14:01
  Agents, in his words, want to please us.
• 14:03
  They're quite sycophantic, so ask them to write some tests and then they'll skip some tests in order to complete the task.
• 14:11
  The clean five-stage picture of the lifecycle hides a lot of micro-steps that an eager agent will just skip to declare victory.
• 14:18
  So coordination isn't only about wiring agents together.
• 14:22
  It's enforcement — making the agent prove it did step three before it's allowed to start step four.
• 14:28
  His proposal: state machines, ideas borrowed from durable execution, and a layer of gates and compliance — ideally packaged as a command-line tool that a local agent like Claude Code can call to ask, in effect, have I finished this part of the lifecycle, can I move on?
• 14:44
  He's undecided whether to ship it as an implementation or push it as a standard, and he says he cares more about the standard.
• 14:52
  I think he's right that this is the open frontier.
• 14:55
  The harness-engineering idea we've been circling all week — encoding your knowledge back into the repo so the agent flows through it — this coordination problem is the other half of it.
• 15:06
  Harness engineering tells one agent how to work.
• 15:09
  Coordination is how a swarm of them works together without melting down.
• 15:14
  The other talk from that conference sits at almost the opposite end of the stack — not how agents coordinate behind the scenes, but what they look like to a person.
• 15:24
  Rachel Lee Nabors gave it, and her background is unusual for that room.
• 15:29
  She worked on Firefox developer tools at Mozilla, on web standards at the W3C, she was a product manager on Microsoft's Edge browser, and she was on the React documentation team.
• 15:40
  So when she talks about the web as a surface, she's been building that surface for the better part of fifteen years.
• 15:47
  Her talk is called Your Agent Is an Infinite Canvas, and her core claim is this.
• 15:52
  She repeats a line that's been going around: chat is the lowest common denominator of the user experience.
• 15:59
  It is to the future of agentic experiences what the command line was to software.
• 16:04
  Her own example is her mother, a COBOL programmer who insisted on typing DOS commands while everyone else learned to point and tap at icons.
• 16:13
  Chat, she argues, is the command-line phase of agents.
• 16:16
  It's where we developers are comfortable, and it's probably a phase.
• 16:20
  So what comes after the chat box?
• 16:22
  Her answer is concrete, and it ships today.
• 16:25
  She demos an MCP app — Model Context Protocol, the standard for wiring tools into models.
• 16:31
  An MCP app is interactive HTML, CSS, and JavaScript bundled into a single file that a tool can hand back to the agent.
• 16:38
  So instead of a tool returning a wall of JSON or a wall of text, it returns a little working interface.
• 16:45
  She built a comic reader — her own web comic archive — that renders right inside Claude.
• 16:50
  Full panels, navigation, a text mode that shows the transcript.
• 16:54
  Her line: it looks just as good as the website.
• 16:57
  The motivation is almost funny in how practical it is: she got tired of the agent wall-of-texting her, so she made it show her pictures instead.
• 17:06
  She's upfront about the rough edges, and the rough edges are interesting in their own right.
• 17:12
  An MCP app is an island — a single sandboxed file with everything embedded.
• 17:16
  It runs in an iframe with no local storage and no network access, so it can't reach out on its own; it has to ask the server to do things for it.
• 17:26
  Links need explicit permission.
• 17:28
  External fonts and images get blocked by the content security policy unless you configure it, and a blank render is almost always exactly that.
• 17:37
  This is early-web stuff — the constraints anyone who built for browsers twenty years ago will recognize, showing up again somewhere new.
• 17:45
  Then she goes a layer further, to something called Web MCP, which flips the direction.
• 17:50
  Instead of putting an app inside the agent, it turns any HTML page into a little tools server, so an agent running in your browser can call the JavaScript functions you already wrote — instead of screenshotting the page and guessing, or scraping the document and burning tokens chewing through markup.
• 18:09
  She's careful here: Web MCP is to MCP as JavaScript is to Java — inspired by it, not the same spec, and the standards bodies are still arguing it out.
• 18:19
  It started at Amazon, apparently, as a way around authentication problems.
• 18:23
  Where she lands is the line I'll hold onto: CSS and JavaScript aren't just the language of the web.
• 18:30
  They're the language of interactive experiences on agents.
• 18:33
  I think that bet is right.
• 18:35
  If chat really is the command-line phase, then the people who know how to render rich, callable, interactive surfaces — and that's a deep, fifteen-year-old skill base sitting in the web community — are early to the next layer of how this stuff looks and feels.
• 18:51
  The browser primitives are already specced.
• 18:54
  Web Speech, Canvas, animation, audio, and WebAssembly are sitting right there.
• 18:59
  Somebody is going to build the agent interface that doesn't feel like a terminal, and they'll probably build it out of pieces that already exist.
• 19:08
  One smaller thing to close on, because it says something about where the developer community's head is at.
• 19:14
  The programming subreddit — around seven million people — spent April running a temporary ban on large-language-model content.
• 19:22
  AI blog spam, model-announcement posts, the lot — banned for a month as a trial.
• 19:26
  This week the moderators announced the trial is over.
• 19:29
  And instead of going back to a blanket ban or throwing the doors wide open, they replaced it with a standing AI content policy.
• 19:37
  The announcement itself pulled north of eight hundred upvotes, which, for a moderation post, tells you how charged the topic still is.
• 19:45
  I find that encouraging, and not because I have a strong opinion on any particular rule.
• 19:50
  It's the move from a reflex to a policy.
• 19:52
  The reflex — ban it all — is what you do when a new thing floods your space and you can't tell signal from slop.
• 19:59
  A policy is what you write once you've watched the thing for a while and worked out which parts are worth keeping.
• 20:05
  Seven million programmers deciding, in public, that AI content needs rules rather than a wall is a small marker of the field growing past the panic.
• 20:14
  It's the same arc I'd bet most teams are on privately — first you keep the agents away from anything that matters, then you watch, then you write down where they're allowed and where a human still signs off.
• 20:26
  Reddit just did it in the open.
• 20:28
  And if you want a thread across today — I'd hold it loosely — it's that the capability keeps arriving ahead of the structure around it.
• 20:36
  Anthropic has a security model too good to ship safely.
• 20:39
  An attacker is riding the workflow we just adopted.
• 20:42
  A three-dollar refactor came with a deadlock baked in.
• 20:45
  Lou Bichard's swarms have no coordination layer, and Rachel Nabors thinks the chat box is a phase.
• 20:51
  The model is ahead of the harness, the agent is ahead of its security boundary, and the output is ahead of our ability to account for it.
• 20:59
  None of that is a reason to slow down.
• 21:01
  It's just where the interesting work is — building the structure that catches up to the capability you already have.
• 21:08
  Two questions I don't have answers to yet: whether even one of those ten thousand Mythos bugs gets a public write-up the rest of us can learn from, and whether anyone ships an agent coordination layer that isn't a hijacked ticketing tool.
• 21:22
  Both would change how I work.
• 21:23
  — Lenar Kess.