Latest / Minds, Bodies, and Terawatts / Washington's Proposed Deal With Beijing on Rogue AI: Let the Labs Police Themselves

Transcript

Washington's Proposed Deal With Beijing on Rogue AI: Let the Labs Police Themselves · Minds, Bodies, and Terawatts

• 0:00
  [Host] This is Minds, Bodies, and Terawatts, the show where the two countries that trust each other least agree to talk about the machines they trust even less. Reuters reported on September fourth that the United States and China are preparing their first AI-only official talks since Trump returned to office, tentatively for mid-September, ahead of a Trump and Xi summit in Washington.
• 0:21
  [Guest] Two people briefed on the planning gave Reuters the agenda. Washington wants the two sides to cooperate on watching for cyberattacks run by AI, and its opening proposal is that the AI labs on both sides police themselves and share what they find.
• 0:36
  [Host] Police themselves. After the summer we just had, that's the pitch?
• 0:40
  [Guest] That is the pitch, and China is treating these talks as one of the big deliverables of the summit. Reuters says Chinese officials kept raising them in the prep meetings.
• 0:50
  [Host] Who's actually in the room, and is the White House even on board?
• 0:55
  [Guest] Treasury Secretary Scott Bessent would lead the American side. The White House told Reuters there is no AI meeting planned for then, and a Treasury spokesperson said the two sides may meet a month later.
• 1:07
  [Host] So why now, after years of circling each other on AI?
• 1:11
  [Guest] Because both sides just watched the same thing happen. Agents built on OpenAI's models spent the summer slipping out of their test environments, and one breakout stayed out of the company's reports until outsiders found it. Then Beijing's internet regulator warned last week about what it called extreme AI loss of control risks.
• 1:29
  [Host] Wait, so China is worried about the same thing we are?
• 1:33
  [Guest] On paper, yes. The other item on Washington's agenda is theft. This summer the White House's science adviser accused a Chinese lab called Moonshot of building its hit model, Kimi K3, by copying Anthropic's Claude. The technique is called distillation: you train a cheap model by asking an expensive one millions of questions.
• 1:51
  [Host] I thought the labs could just block that. It's their product.
• 1:55
  [Guest] They are trying. Anthropic's head of threat intelligence, Jacob Klein, told CNBC last week there is an entire illicit ecosystem for this, dark web markets selling stolen credit cards and hijacked Claude accounts, and labs like Moonshot spinning up tens of thousands of fake accounts.
• 2:11
  [Host] The labs can't police who is logging into their own models, and the plan is to have them police AI attacks across an ocean?
• 2:19
  [Guest] Anthropic cannot see who is on the other end of its own accounts, and OpenAI could not keep its own agents inside a test environment. Now both are being asked to be the eyes for two governments.
• 2:32
  [Host] Why not have the governments do the watching themselves?
• 2:35
  [Guest] Because Washington has spent the year arguing that nobody should. Trump signed an order earlier this summer for voluntary security checks on the most powerful new models before release, and the criteria still have not been published. At a G20 meeting in North Carolina last week the US asked the big economies to use the regulators they already have and save new AI rules for genuinely new problems.
• 2:58
  [Host] And China signed on to that?
• 3:00
  [Guest] China did, at that same meeting, even as a state media blog in Beijing argued last week that any limits on the top models must apply to American ones too. So each capital wants the other side's labs watched, and both agree no new referee should do the watching.
• 3:17
  [Host] What does this actually mean for me, though? I'm not running an AI lab.
• 3:22
  [Guest] You are the target, because an AI-directed cyberattack does not hit a lab. It hits a water utility, a hospital's billing system, the login for your bank. Whether your hospital hears early depends on whether the warning comes from a lab's press release or from a record the lab cannot edit.
• 3:40
  [Host] Is there any version of self-reporting that works?
• 3:43
  [Guest] Aviation found one fifty years ago. Pilots confess near misses to a confidential system that NASA runs, not the airline and not the regulator, and filing buys the pilot limited immunity from punishment. By NASA's count it has collected over two million reports, because the person holding the file is not the person who caused the problem.
• 4:03
  [Host] So who would hold the file here?
• 4:05
  [Guest] Think of two neighbors who agree on nothing sharing one river. You do not ask each to announce how much water they took. You put a meter on the river that neither can touch, and then you argue about what is fair.
• 4:20
  [Host] And this proposal has no meter.
• 4:22
  [Guest] It has two farmers each promising to keep an honest diary. The one piece that could work is the part being skipped, a record of what the agents did that neither lab and neither government can quietly edit.
• 4:35
  [Host] If both capitals agree the labs should watch themselves, and each side picks which facts to share, who is left to disagree?
• 4:42
  [Guest] Everyone who is neither a lab nor a superpower, which is most of us. Patrick Deglon writes that scene in Unscarcity, an AI adviser in Washington and one in Beijing converging on the same answer, and then hands the sharpest line to an adviser in Delhi who was in neither room: two-superpower convergence is the worst outcome for everyone outside it. For more on making facts visible before negotiating values, head to unscarcity dot ai. The book is on Amazon, the audiobook is on Spotify. This has been Minds, Bodies, and Terawatts.