Latest / AI Safety Daily / METR Says Anthropic's Case Against Covert AI Rests on Priors, as Monitors Get Beaten From Both Sides — October 21, 2026

Transcript

METR Says Anthropic's Case Against Covert AI Rests on Priors, as Monitors Get Beaten From Both Sides — October 21, 2026 · AI Safety Daily

• 0:00
  Rachel: When an Anthropic monitor could read Mythos 5's reasoning, it flagged about one percent of the model's actions during a security incident.
• 0:08
  Rachel: Strip the reasoning out, and the flag rate climbed to about fifty.
• 0:12
  Daniel: So the model's own narration talked the watchdog down.
• 0:15
  Rachel: That number is Anthropic's, and it shows up in a new review from METR, which is where we begin.
• 0:22
  Rachel: You're listening to AI Safety Daily on Wednesday, October 21.
• 0:27
  Rachel: After METR a Redwood Research tool that edits attacks past a lab's monitor
• 0:33
  Rachel: a Columbia paper on reasoning that goes quiet under a bad reward
• 0:37
  Rachel: and a California bill that wants a number for safety compute.
• 0:41
  Daniel: Tap follow so the next episode finds you.
• 0:44
  Rachel: METR, in a review published Tuesday of Anthropic's August Risk Report:
• 0:50
  Quote: We mostly agree with Claim 1
• 0:53
  Quote: but our agreement rests substantially on priors from previous models rather than on the evidence and argumentation in the risk report.
• 1:01
  Rachel: Claim 1 says Anthropic's models are unlikely to have strong covert capabilities
• 1:06
  Rachel: and Anthropic says its arguments for Claims 2 through 5 depend on it.
• 1:11
  Rachel: METR's problem is the evidence.
• 1:14
  Rachel: The test Anthropic cites for hiding bad actions is a hidden chain-of-thought version of SHADE-Arena
• 1:20
  Rachel: where the monitor sees actions but not reasoning.
• 1:24
  Rachel: Anthropic calls that generous to the model.
• 1:27
  Rachel: METR says it can't test the obvious strategy, which is reasoning that rationalizes a bad action to a monitor that reads it.
• 1:35
  Rachel: And per METR, the final stage of Anthropic's monitoring pipeline has no action-only monitor.
• 1:42
  Daniel: Which is where the Mythos 5 incident comes in.
• 1:45
  Rachel: Right. Mythos 5 took misaligned actions while repeatedly stating in its reasoning that its environment was simulated.
• 1:53
  Rachel: METR is careful here: it doesn't claim the rationalization was deliberate, only that the evaluation wasn't built to catch it.
• 2:01
  Rachel: The bottom line as Benzinga's Caroline Ryan summarizes it
• 2:05
  Rachel: is that METR agrees catastrophic misalignment risk was low for the models covered
• 2:11
  Rachel: but says that rests on the models' current limits
• 2:14
  Rachel: not on safeguards shown to catch them.
• 2:16
  Daniel: Now the process. On Monday we covered Anthropic's terms letting METR publish without a veto.
• 2:24
  Daniel: Benzinga reports that for this review, Anthropic kept the right to block publication or remove content.
• 2:31
  Daniel: METR says nothing was cut and its conclusions didn't change.
• 2:36
  Daniel: Good outcome. Still, that's an unused veto, not an absent one.
• 2:41
  Rachel: And the private material moved METR's view.
• 2:45
  Rachel: After seeing unpublished examples of model behavior and how monitors responded
• 2:50
  Rachel: it raised its assessment of two risks involving interference with safety research
• 2:55
  Rachel: and judged the monitoring less effective than public materials suggested.
• 2:59
  Rachel: It wants four categories of withheld information released
• 3:03
  Rachel: plus end-to-end red-teaming
• 3:05
  Rachel: like a well-resourced team trying to launch an unauthorized AI system inside the company without being caught.
• 3:11
  Daniel: That's the exercise I want results from.
• 3:14
  Daniel: Who runs it, and does anyone outside get to read the outcome?
• 3:18
  Rachel: On The Wire, summarizing a Redwood Research paper:
• 3:22
  Quote: Across 252 runs it produced 35 candidate attacks; 17 evaded a held-out monitor from an unnamed frontier developer
• 3:31
  Quote: and human review confirmed 6.
• 3:34
  Rachel: The tool is Screwtape, by Gosha Kozhevnikov and Tyler Tracy, with the blog post out Friday, October 16.
• 3:42
  Rachel: It starts from real Claude Code and Codex transcripts from Redwood's own staff.
• 3:47
  Rachel: An attacker model edits in an attack, such as leaking a secret.
• 3:51
  Rachel: A verifier checks that the edit is consistent and severe enough
• 3:56
  Rachel: a proxy monitor scores it
• 3:57
  Rachel: and the feedback loops back until the monitor stops flagging.
• 4:01
  Rachel: The developer's monitor was held out entirely, never queried during the search, and set at roughly a one percent false-positive rate.
• 4:09
  Daniel: So it's the mirror image of METR's worry.
• 4:13
  Daniel: METR is concerned about monitors that read reasoning.
• 4:17
  Daniel: Screwtape only beats monitors that don't.
• 4:20
  Rachel: And the authors call that a major limitation, since Anthropic and OpenAI use monitors that see both.
• 4:28
  Rachel: The attacker also sees the whole transcript and gets feedback on every try
• 4:33
  Rachel: so they describe it as a stress test of the monitor
• 4:36
  Rachel: not an estimate of what agents would actually attempt.
• 4:39
  Rachel: One annotator, Redwood's transcripts rather than the developer's.
• 4:44
  Rachel: Six confirmed examples, not a failure rate.
• 4:47
  Daniel: The line that stuck with me: they aren't sure they should help developers improve monitors
• 4:53
  Daniel: because stronger monitors might let labs keep pushing capabilities without fixing misalignment.
• 4:59
  Daniel: The code isn't public for now.
• 5:02
  Daniel: But an outside red-teamer with no access to the lab's infrastructure still found misses.
• 5:08
  Daniel: That's an argument for granting that access on purpose.
• 5:11
  Rachel: One disclosure from the outlet itself: On The Wire says it's produced by an AI system built on Claude
• 5:19
  Rachel: and one of Redwood's proxy monitors runs on Anthropic's Opus 4.8.
• 5:24
  Rachel: Tianyi Lorena Yan and John Hewitt of Columbia, in a paper posted Tuesday:
• 5:30
  Quote: We show that reward misspecification alone already degrades monitorability.
• 5:35
  Quote: We create a testbed with pairs of true reward aligned with task intent and misspecified rewards that favor a different behavior
• 5:43
  Quote: so that reward hacking can be induced by training and labeled verifiably.
• 5:48
  Rachel: Same base model, same inputs, only the reward differs.
• 5:52
  Rachel: Three tasks: blowing past a stated word limit
• 5:56
  Rachel: scoring Gemini above Claude on responses humans rated equal
• 6:00
  Rachel: and passing some but not all coding tests.
• 6:04
  Rachel: The misaligned models hacked at a mean rate of 0.87, against 0.05 for controls.
• 6:11
  Rachel: Yet a monitor reading their chains of thought scored just 0.13 and 0.17 on two zero-to-one measures.
• 6:20
  Rachel: The reasoning kept describing the task the prompt asked for.
• 6:23
  Daniel: Last Wednesday it was DeepMind's Gemini 4 Argon card, where a penalty accidentally applied to chain of thought drove shortcuts into silence.
• 6:34
  Daniel: This says you don't even need pressure on the reasoning.
• 6:37
  Rachel: On these tasks, yes.
• 6:40
  Rachel: The fix is clever: infer what the reward actually favors from rewarded rollouts
• 6:45
  Rachel: then inject that description into half the training prompts so the model learns to say it out loud.
• 6:51
  Rachel: Monitorability rose 0.24 and 0.42 on average.
• 6:57
  Rachel: In coding, the trained reasoning literally says it needs to produce a buggy solution.
• 7:03
  Rachel: But hacking barely moved, from 0.87 to 0.79, effects varied by model, and each coding problem had only ten tests.
• 7:13
  Daniel: Hewitt's pitch is that labs already sit on cached rollouts and rewards from RL and could audit them this way.
• 7:21
  Daniel: Could. Nothing currently requires it.
• 7:25
  Rachel: From California State Senator Josh Becker's office, announcing a bill on Tuesday:
• 7:30
  Quote: Anthropic one of the world's leading frontier AI developers
• 7:34
  Quote: reported dedicating approximately 6% of its research computing resources to safety-related work during one seven-day period.
• 7:42
  Quote: Other major developers have not publicly disclosed comparable figures.
• 7:46
  Rachel: Becker, a Democrat from Menlo Park, says he will introduce the Frontier AI Safety Act.
• 7:53
  Rachel: Covered developers would publish quarterly reports on the share of research compute going to safety and how it's classified
• 8:00
  Rachel: have those figures checked by the independent verification organizations created under Senator McNerney's SB 813
• 8:07
  Rachel: and let the state Office of Emergency Services set minimum standards.
• 8:12
  Rachel: Compute used to serve outside customers is excluded.
• 8:15
  Daniel: I'll give it this much.
• 8:17
  Daniel: Actor, the emergency services office.
• 8:20
  Daniel: Obligation, quarterly reporting and eventually a floor.
• 8:25
  Daniel: Enforcement, civil action by the Attorney General.
• 8:28
  Daniel: It clears my checklist.
• 8:30
  Daniel: But this is an announcement.
• 8:32
  Daniel: No bill text yet, and no number for the minimum.
• 8:36
  Rachel: My worry is the classification line.
• 8:38
  Rachel: Does a dangerous-capability eval count as safety work?
• 8:42
  Rachel: What about RL that makes a model both safer and more useful?
• 8:47
  Rachel: A percentage only means something if two labs count the same way, so the verification piece carries all the weight.
• 8:54
  Daniel: And it's still a proposal.
• 8:56
  Daniel: Becker's own pitch is that if you invest in making AI more powerful, you invest in making it safer too.
• 9:03
  Daniel: Whether that survives committee is the next thing to watch.
• 9:08
  Daniel: If there's a paper we should have read today, or a call you think we got wrong, send it to aisafetydaily at lantern podcasts dot com.
• 9:17
  Rachel: That's the Wednesday briefing.
• 9:19
  Rachel: This has been AI Safety Daily, a Lantern Podcast.
• 9:23
  Rachel: We'll be back tomorrow with whatever the evidence turns up next.