Latest / Dark Perimeter: Real Breaches, Real Stakes / Two Sentences
Transcript
- 0:00SPEAKER_01: On October twenty-seventh, a few lines of text showed up in Claude Code.
- 0:05SPEAKER_01: Nobody announced them.
- 0:07SPEAKER_01: There was no blog post and no changelog entry.
- 0:10SPEAKER_01: A user on X found them sitting in the configuration that Anthropic's servers pushed down to the tool.
- 0:17SPEAKER_01: The gist was simple.
- 0:19SPEAKER_01: Build original games, genre games, and games from your own designs.
- 0:24SPEAKER_01: Do not build a recreation of a specific existing game, even if you rename it.
- 0:29SPEAKER_01: Three days later, an anthropic engineer confirmed it publicly and described the change as two sentences.
- 0:36SPEAKER_01: Two sentences delivered from a server with no setting to turn them off.
- 0:41SPEAKER_01: That is where Agentic AI is in late 2026.
- 0:46SPEAKER_01: The tools are powerful enough that two sentences of policy matter, and the work is portable enough that people immediately asked where else it could go.
- 0:55SPEAKER_01: I'm Cole Draden, and this is Dark Perimeter.
- 0:58SPEAKER_01: With me are Dr.
- 0:59SPEAKER_01: Elliot Vance, who studies how these systems are built and governed, and Marcus Hale, who has spent more nights than he'd like cleaning up after them.
- 1:08SPEAKER_00: Thanks, Cole.
- 1:09SPEAKER_00: I want to flag up front that the rule is short, and the fight around it is not.
- 1:13SPEAKER_00: It touches copyright doctrine older than the personal computer, a Supreme Court decision from this spring, and a question no court has answered yet.
- 1:22SPEAKER_00: Can a machine be a clean room?
- 1:24SPEAKER_02: And from where I sit, the interesting part isn't the rule.
- 1:27SPEAKER_02: It's what people do the minute a sanctioned tool says no.
- 1:30SPEAKER_02: Within hours, the replies were full of people saying, fine, I'll run an open model.
- 1:34SPEAKER_02: That's a shadow IT story, and I have seen it before.
- 1:37SPEAKER_01: We'll get to all of it.
- 1:39SPEAKER_01: First, the honest state of play.
- 1:41SPEAKER_01: Elliot, strip away the marketing.
- 1:43SPEAKER_01: Where is Agentic AI right now?
- 1:46SPEAKER_00: The short version is that agents are real, they're useful, and they're far less deployed than the press releases suggest.
- 1:53SPEAKER_00: By agent I mean a model that plans, takes actions with tools, checks its own work, and keeps going across many steps without a human approving each one.
- 2:02SPEAKER_00: Coding is where that works best because code has a built-in referee.
- 2:06SPEAKER_00: The tests pass or they don't.
- 2:08SPEAKER_00: Outside of code, the picture gets murkier fast.
- 2:11SPEAKER_01: Give me numbers.
- 2:12SPEAKER_00: The survey data disagrees with itself, which tells you something.
- 2:16SPEAKER_00: Communda's 2026 report found 71% of organizations say they use AI agents, but only 11% of a genic use cases had actually reached production.
- 2:26SPEAKER_00: Deloitte puts companies using agentic AI at least moderately at about 23%.
- 2:31SPEAKER_00: McKinsey's latest survey has 44% of respondents saying AI is scaling across their enterprise, and that's AI broadly, not agents.
- 2:39SPEAKER_00: Then there are the surveys published by companies that sell agent platforms, where every single respondent plans to expand.
- 2:46SPEAKER_00: I read those the way I read a dentist's survey on flossing.
- 2:50SPEAKER_02: And the security side of the house is not relaxed about any of it.
- 2:53SPEAKER_02: Forrester's 2026 security survey had 49% of security decision makers naming agentic AI as a concern.
- 3:01SPEAKER_02: Here's what that looks like on the ground.
- 3:03SPEAKER_02: An agent is a service account with an imagination.
- 3:05SPEAKER_02: It holds credentials, it reads untrusted text, and it can act on what it reads.
- 3:09SPEAKER_02: We spent four episodes this summer on what happens when one of those gets loose and the failure modes haven't changed, prompt injection, permissions that are far too broad, and nobody watching the logs.
- 3:19SPEAKER_01: So where's the frontier practically?
- 3:22SPEAKER_00: Think in three tiers.
- 3:24SPEAKER_00: At the top are the closed frontier tools, Clawed Code and its competitors from OpenAI and Google, running the biggest models through long autonomous sessions.
- 3:34SPEAKER_00: Just behind them, closer than at any point I can remember, are the open weight models.
- 3:39SPEAKER_00: DeepSeek's V4 Pro shipped in April under an MIT license.
- 3:43SPEAKER_00: On October 17th, they released V4.1 Flash, tuned specifically for agentic coding, with a 1 million token context window and API prices roughly an order of magnitude below the closed frontier.
- 3:57SPEAKER_00: Then on November 1st, DeepSeek published an experimental agent called SuperAgent.
- 4:02SPEAKER_00: It watches a team's GitHub repository and chat channels and picks up coding work without being summoned.
- 4:07SPEAKER_00: DeepSeq's own release calls it a research prototype, not a product.
- 4:12SPEAKER_00: The third tier is what runs on your own desk, which is smaller, slower, and far more private than either of the others.
- 4:18SPEAKER_02: I'd add one thing.
- 4:31SPEAKER_02: That cuts both ways, and honestly, that's the whole episode.
- 4:34SPEAKER_01: Let's do the rule.
- 4:36SPEAKER_01: Elliot, why did this happen now?
- 4:38SPEAKER_00: Because of October.
- 4:40SPEAKER_00: Early in the month, Kotaku reported that browser-playable ports of games like Halo Combat Evolved, Grand Theft Auto Vice City, and Call of Duty Black Ops were circulating, and the people behind them credited Anthropic's Opus 5.5 model with decompiling the originals far faster than human reverse engineers could.
- 4:58SPEAKER_00: Hundreds of similar projects followed within weeks.
- 5:01SPEAKER_00: Smaller studios got scared of the reverse problem.
- 5:04SPEAKER_00: Digital Sun Games cancelled a public playtest of its upcoming game, Revamp, out of fear someone would feed it to an AI and ship a copy first.
- 5:12SPEAKER_00: Then on October 27th, the instruction appeared in Claude Code.
- 5:16SPEAKER_00: On the 30th, Addy Osmani at Anthropic confirmed it.
- 5:19SPEAKER_00: He said using Claude to infringe intellectual property violates their usage policy, that original games, genre games, and games built from your own assets should keep working, and that anyone wrongly refused should send him the case.
- 5:32SPEAKER_01: Marcus, you're the practitioner.
- 5:34SPEAKER_01: Is that a reasonable rule?
- 5:36SPEAKER_02: Partly.
- 5:36SPEAKER_02: Look at what those ports were.
- 5:38SPEAKER_02: Decompiling a commercial binary and shipping it to a browser is not a clean room.
- 5:43SPEAKER_02: It's the opposite of a clean room.
- 5:45SPEAKER_02: It's taking the house apart brick by brick and rebuilding it on a different lot.
- 5:49SPEAKER_02: A vendor that doesn't want its tool doing that has a defensible position.
- 5:53SPEAKER_02: My problem is the mechanism.
- 5:54SPEAKER_02: The rule was pushed silently, it can't be switched off, and the tool has no way to know whether you own the thing you're describing, licensed adaptations, commissioned work, a studio remaking its own back catalog.
- 6:06SPEAKER_02: All of that sits in a gray zone that two sentences can't sort out, and I'll say it as someone who values transparency.
- 6:12SPEAKER_02: When a vendor changes what a tool will do, I'd like to read it in release notes, not discover it in a config file.
- 6:18SPEAKER_00: The critics went further.
- 6:19SPEAKER_00: Gizmod's argument, roughly, was that AI companies seem far more willing to put guardrails around cloning software than they ever were around the training data that made these models possible.
- 6:31SPEAKER_00: That's pointed, and it's made in good faith.
- 6:34SPEAKER_00: The counter-argument is that a usage policy against infringement already existed, and the product team is now enforcing it at the moment of action instead of after the fact.
- 6:44SPEAKER_00: Both of those can be true at once.
- 6:46SPEAKER_01: Let's back up for listeners.
- 6:48SPEAKER_01: What is a clean room and why does everyone suddenly care?
- 6:52SPEAKER_00: Clean room design is an old idea in copyright.
- 6:55SPEAKER_00: Copyright protects expression, not ideas.
- 6:57SPEAKER_00: So you split into two teams.
- 6:59SPEAKER_00: One team studies the original and writes a specification of what it does in plain functional terms.
- 7:05SPEAKER_00: A second team, which has never seen the original code, builds from that specification alone, and a lawyer sits between them checking that nothing expressive crosses the wall.
- 7:15SPEAKER_00: That's how compatible PC BIOS chips became legal to build in the early 80s.
- 7:20SPEAKER_00: It was slow and expensive on purpose.
- 7:23SPEAKER_00: The cost was the moat.
- 7:36SPEAKER_02: People rewrote the leaked Claude code source into Python after Anthropic's own packaging slip in March.
- 7:41SPEAKER_02: And at Fosdem, a satirical service called Malice offered clean room as a service.
- 7:46SPEAKER_02: One set of bots reads the documentation, a separate set writes the code.
- 7:50SPEAKER_02: It was a joke.
- 7:51SPEAKER_02: It also worked.
- 7:52SPEAKER_00: Here's the legal problem, and I want to separate what's settled from what isn't.
- 7:57SPEAKER_00: Settled.
- 7:58SPEAKER_00: In March, the Supreme Court declined to hear Thaler v.
- 8:01SPEAKER_00: Pearl Mutter, which leaves in place the ruling that a work generated by AI with no human author can't be copyrighted, not settled, whether an AI can serve as the clean team at all.
- 8:12SPEAKER_00: No court has ruled on that.
- 8:14SPEAKER_00: And there's a deeper issue.
- 8:15SPEAKER_00: The traditional clean room works because the second team demonstrably never saw the original.
- 8:20SPEAKER_00: A large model may have been trained on the original or on a thousand descriptions of it.
- 8:24SPEAKER_00: One law firm likened asking a model to set aside what it knows to instructing a jury to forget testimony it already heard.
- 8:31SPEAKER_01: So the clean room might fail, and even if it holds, the code it produces might belong to nobody.
- 8:38SPEAKER_00: That's the uncomfortable sum.
- 8:39SPEAKER_00: And notice that the rule only says games.
- 8:42SPEAKER_00: An engineer in Atlanta, Brandon Thomas, has used the same model to build free open source replacements for Adobe's creative tools and for Word, Excel, and PowerPoint.
- 8:52SPEAKER_00: He describes them as clean room rebuilds from public file format specifications.
- 8:57SPEAKER_00: A law professor told Al Jazeera nobody knows whether that would survive scrutiny.
- 9:02SPEAKER_00: Anthropic hasn't said whether the rule will expand beyond games.
- 9:20SPEAKER_01: Marcus, walk it like a menu.
- 9:24SPEAKER_02: With one rule stated first, and I'm going to repeat it at the end.
- 9:28SPEAKER_02: Switching tools changes the vendor's policy.
- 9:30SPEAKER_02: It does not change the law.
- 9:32SPEAKER_02: If the thing you're building infringes, it infringes on Deep Seek on a rented H-100, and on your laptop in airplane mode.
- 9:38SPEAKER_02: This menu is about cost, privacy, control, and risk.
- 9:42SPEAKER_02: It is not a menu of legal outcomes.
- 9:44SPEAKER_01: Understood.
- 9:45SPEAKER_01: Option one.
- 9:46SPEAKER_02: Option one is stay where you are and work inside the line.
- 9:49SPEAKER_02: By Anthropic's own statement, original games, genre games, and games built from your own designs and assets should keep working.
- 9:55SPEAKER_02: If you get refused on something that's genuinely yours, report it.
- 9:59SPEAKER_02: That's the cheapest path, and it keeps you on approved tooling.
- 10:02SPEAKER_02: The open source world also has two patterns that held up for decades before AI showed up.
- 10:07SPEAKER_02: Free Civ is a strategy game in the civilization tradition with its own code and its own art.
- 10:13SPEAKER_02: OpenMW rebuilds the Morrowind engine, but makes you supply the game data from a copy you own, genre plus your own assets, or a new engine plus your own data.
- 10:22SPEAKER_02: Whether a given AI tool will help with that second pattern is now an open question.
- 10:26SPEAKER_02: The pattern itself is old.
- 10:29SPEAKER_02: Option two is an open weight model through a hosted API, and you have to split it in half.
- 10:33SPEAKER_02: Deep Seek's own API and its consumer app process your prompts on infrastructure in China, under Chinese law.
- 10:39SPEAKER_02: On October 19th, the Department of Homeland Security made public an exclusion and removal order that bans DeepSeq from federal civilian agencies.
- 10:48SPEAKER_02: It cited the risk of data reaching the Chinese government and the possibility that the models contain malicious code.
- 10:54SPEAKER_02: That order doesn't cover private companies or personal use, but if you handle client confidential data, the hosted Deep Seek API is the wrong tool.
- 11:02SPEAKER_02: Full stop.
- 11:03SPEAKER_02: The other half is the same open weights served by a U.S.
- 11:06SPEAKER_02: inference provider.
- 11:07SPEAKER_02: Under a contract, you can actually read.
- 11:09SPEAKER_02: Your data stays with a domestic vendor.
- 11:10SPEAKER_02: Same model, very different jurisdiction.
- 11:13SPEAKER_00: And the second reason in that DHS order is the one researchers find interesting.
- 11:18SPEAKER_00: Data exfiltration is a hosting problem.
- 11:20SPEAKER_00: You can engineer around it by running the weights yourself.
- 11:23SPEAKER_00: But a behavior trained into the weights, a backdoor, survives self-hosting.
- 11:28SPEAKER_00: Nobody has a mature way to audit hundreds of billions of parameters for that.
- 11:33SPEAKER_00: So moving off the hosted API fixes one risk and leaves the other one standing.
- 11:38SPEAKER_01: Option 3.
- 11:40SPEAKER_02: Option three is renting the hardware and running open weights yourself.
- 11:43SPEAKER_02: October prices.
- 11:44SPEAKER_02: An H100 runs about$2 an hour on marketplaces like vast.ai, about 4-6 on the AI-focused clouds like Lambda and RunPod, and 7-12 on the big hyperscalers.
- 11:53SPEAKER_02: Blackwell B200s run roughly six to nine, and prices have gone up this year, not down.
- 11:59SPEAKER_02: Two cautions.
- 12:00SPEAKER_02: First, the flagship open models are big.
- 12:02SPEAKER_02: Running something in the class of V4 Pro at useful speed takes a node with several GPUs, not one card.
- 12:08SPEAKER_02: At that point, you're paying tens of dollars an hour, which can cost more than just buying API tokens unless you're running agents around the clock.
- 12:15SPEAKER_02: Second, a marketplace GPU is a stranger's computer.
- 12:19SPEAKER_02: The host is a third party you've never vetted.
- 12:21SPEAKER_02: If your code or your data matters, use a provider's secure tier under contract, not the cheapest listing.
- 12:27SPEAKER_01: Option 4.
- 12:28SPEAKER_01: Offline.
- 12:29SPEAKER_02: Option 4 is your own machine, LM Studio, OLAMA, Lama.cpp.
- 12:34SPEAKER_02: On a consumer card with 12 to 24 gigabytes of video memory, you're running smaller models, compressed down so they fit.
- 12:41SPEAKER_02: They're genuinely good for autocomplete, explaining code, small refactors, and anything you can't let leave the building.
- 12:47SPEAKER_02: They are not a frontier agent.
- 12:48SPEAKER_02: Hand a small local model a long job on a big code base and you'll watch it lose the plot.
- 12:53SPEAKER_02: The upside is total control and a real air gap if you want one.
- 12:56SPEAKER_02: The downside is you now own every decision the vendor used to make for you, including the safety ones.
- 13:02SPEAKER_00: There's a layer people forget, which is the harness.
- 13:05SPEAKER_00: The agent is the model plus the loop around it, the tool that reads files, runs tests, and decides what to do next.
- 13:12SPEAKER_00: There are open source harnesses that will drive almost any model you point them at, and DeepSeek's super agent is one more.
- 13:19SPEAKER_00: Read the fine print on that one.
- 13:21SPEAKER_00: DeepSeek's own release says it runs with full operating system user access and should be isolated before use.
- 13:27SPEAKER_00: It defaults to an observe mode that only records what it would have done.
- 13:31SPEAKER_00: That's a responsible default.
- 13:32SPEAKER_00: Switch it off on your daily laptop, and you've installed an autonomous process with your permissions that listens to your team's chat.
- 13:40SPEAKER_01: And the clean room itself.
- 13:49SPEAKER_00: The best current guidance I've read comes down to separating the agents the way you'd separate the people.
- 13:54SPEAKER_00: One environment does analysis and writes the specification.
- 13:58SPEAKER_00: A completely different environment with different sessions, a different workspace, and a different repository implements from the specification only.
- 14:08SPEAKER_00: A human reviews the specification before it crosses, because that's where expression leaks.
- 14:14SPEAKER_00: Keep every prompt and every output.
- 14:16SPEAKER_00: The one thing an AI workflow adds is a record, and the record is your evidence.
- 14:21SPEAKER_00: And ask the honest question first.
- 14:23SPEAKER_00: If the target is widely published or open source code the model has almost certainly seen, is a clean room even credible?
- 14:30SPEAKER_00: Sometimes the safer route is taking a license or building on a genuinely different design.
- 14:35SPEAKER_02: And notice what that list leaves out.
- 14:37SPEAKER_02: It doesn't include decompiling the binary.
- 14:40SPEAKER_02: It doesn't include the original art, music, names, levels, or story.
- 14:44SPEAKER_02: A clean room may protect you on code.
- 14:46SPEAKER_02: It does nothing for the assets, and it does nothing for trademarks.
- 14:49SPEAKER_01: Let's bring it home for the people who run security programs.
- 14:53SPEAKER_01: What does this week's news change for them?
- 14:55SPEAKER_02: One big thing: every guardrail in a sanctioned tool creates pressure toward an unsanctioned one.
- 15:00SPEAKER_02: Your developer gets a refusal at four in the afternoon on a deadline.
- 15:03SPEAKER_02: By five, there's a Deep Seek API key in an environment variable, or a rented GPU on a personal credit card, or an agent running locally with that developer's credentials.
- 15:12SPEAKER_02: None of it shows up in your AI inventory.
- 15:15SPEAKER_02: The refusal was the event, the workaround is the incident.
- 15:18SPEAKER_01: So what do you actually do?
- 15:20SPEAKER_02: Five things, and none of them are exotic.
- 15:23SPEAKER_02: One, decide your model endpoint policy and enforce it at the proxy and DNS layer.
- 15:28SPEAKER_02: If the hosted DeepSeek API isn't approved, block its domains and alert on attempts, because every attempt is a conversation you need to have.
- 15:36SPEAKER_02: Two, publish an approved path for open weights.
- 15:39SPEAKER_02: Which models, which hosts, which inference providers, and for what data classification.
- 15:44SPEAKER_02: If you don't give people a legal door, they'll cut one.
- 15:46SPEAKER_02: 3.
- 15:47SPEAKER_02: Treat downloaded weights like any other third-party software.
- 15:50SPEAKER_02: Pin versions, verify hashes against the publisher, prefer file formats that can't execute code when they load, and record where every model came from.
- 15:58SPEAKER_02: 4.
- 15:59SPEAKER_02: Sandbox your agents, a container or a virtual machine, no standing credentials, an egress allow list, and no access to chat channels unless that's literally the job.
- 16:08SPEAKER_02: 5.
- 16:09SPEAKER_02: Give people a fast internal way to appeal a refusal, so the answer to the tool said no is a ticket, not a workaround.
- 16:15SPEAKER_00: I'd add a governance point for leadership.
- 16:18SPEAKER_00: Vendor policy is now a runtime variable.
- 16:21SPEAKER_00: It can change silently from a server as it did here, and it can change in either direction.
- 16:26SPEAKER_00: If a business process depends on what a vendor's model is willing to do today, write that dependency down like any other third-party risk, with an owner and a fallback.
- 16:35SPEAKER_00: And document the residual risk, honestly.
- 16:38SPEAKER_00: Self-hosting cuts data exposure.
- 16:40SPEAKER_00: It does not remove model provenance risk, and it moves every safety decision onto your own team.
- 16:46SPEAKER_00: Leadership should sign off on that trade knowing what it is.
- 16:50SPEAKER_01: Marcus, does anything in the open weight world worry you more than the rest?
- 16:54SPEAKER_02: Speed and price.
- 16:56SPEAKER_02: A one million token context window and agent-tuned models at a tenth of the cost means the capability that built those game ports is now cheap and unsupervised for anyone who wants it.
- 17:07SPEAKER_02: Defenders should assume attackers have the same menu we just read, minus the first rule.
- 17:12SPEAKER_01: Let me pull it together.
- 17:22SPEAKER_01: The closed frontier still leads, but open weights are close enough behind that the bigger difference now is who holds the off-switch.
- 17:30SPEAKER_01: Anthropic drew a line on recreating specific games with a defensible reason and a mechanism worth questioning.
- 17:38SPEAKER_01: Silent, server-side, and blunt.
- 17:41SPEAKER_01: Clean rooms run by machines are legally untested, and the code they produce may not be ownable by anyone.
- 17:49SPEAKER_01: And the options are real.
- 17:52SPEAKER_01: Work inside the line.
- 17:54SPEAKER_01: Use a hosted open model under a jurisdiction you can live with.
- 17:58SPEAKER_01: Rent hardware from someone you've vetted, or run it yourself and own every decision.
- 18:04SPEAKER_01: Marcus, the rule you promise to repeat.
- 18:08SPEAKER_02: Switching tools changes the vendor's policy.
- 18:10SPEAKER_02: It doesn't change the law.
- 18:12SPEAKER_02: Pick your tool for cost, privacy, and control.
- 18:14SPEAKER_02: Pick your project based on whether you have the right to build it.
- 18:17SPEAKER_00: And keep the receipts.
- 18:19SPEAKER_00: Whatever you build with an agent, the log is the closest thing you'll have to proof of how it was made.
- 18:24SPEAKER_01: That's the episode.
- 18:32SPEAKER_01: Our guests are composite characters, and the show notes say so.
- 18:37SPEAKER_01: I'm Cole Drayden.
- 18:38SPEAKER_01: This has been Dark Perimeter.