Latest / Minds, Bodies, and Terawatts / AI Executives Privately Expect a Catastrophe Within a Year. They Are Not Rehearsing How to Stop It. They Are Rehearsing What to Say the Day After.

Transcript

AI Executives Privately Expect a Catastrophe Within a Year. They Are Not Rehearsing How to Stop It. They Are Rehearsing What to Say the Day After. · Minds, Bodies, and Terawatts

• 0:00
  [Host] This is Minds, Bodies, and Terawatts, the show where the fire drill skips the fire and goes straight to the press conference. Axios reported Friday that top executives at Anthropic, OpenAI and other AI companies are privately gaming out what to do after a catastrophic AI event sets off a public and political revolt.
• 0:18
  [Guest] Not how to stop the event. What to do the day after. And the insiders talking to Axios aren't hedging on whether it happens: they say a major incident is coming within six to twelve months, and many of them call it inevitable.
• 0:34
  [Host] What does catastrophic mean to them? I hear that word from AI people a lot, and it usually means a slide deck.
• 0:41
  [Guest] This time it means a large-scale cyberattack. The scenario they keep returning to is one that shuts down financial services, internet access, or power and water. Your bank app dead and the tap dry.
• 0:53
  [Host] Hold on, every bank and every airline runs disaster drills on paper. Why is it news that an AI company does too?
• 1:01
  [Guest] Because of which way the drill points. A bank rehearses getting the systems back. Axios says these plans are about war-gaming the worst cases and educating members of Congress, so that when lawmakers reach for a response, the companies have already shaped what they reach for.
• 1:17
  [Host] So the rehearsal is for the lobbying. They're practicing the politics of a disaster they say they can't prevent?
• 1:24
  [Guest] That's the part a bank doesn't do. And Axios says the executives expect Democrats to come after the industry hard if they take Congress in the midterms next month.
• 1:34
  [Host] What do the companies say to all this?
• 1:37
  [Guest] OpenAI told Axios it runs preparedness exercises like many companies, and that the scenarios are not treated as inevitable. Anthropic, asked the same, declined to comment.
• 1:46
  [Host] Not treated as inevitable is a careful sentence. It doesn't say unlikely.
• 1:50
  [Guest] It doesn't, and OpenAI's chief executive said the quiet part days earlier. Sam Altman told Politico that the world should accept some bad things happening, his words, for the benefits of this technology and people having the agency. He wouldn't promise a world with no major hacks; the line he drew was at a serious loss of control to AI.
• 2:11
  [Host] Who gets to pick which bad things, and does anyone ask the people they land on?
• 2:17
  [Guest] The Associated Press polled the country and published Thursday: sixty-four percent say AI is moving too fast, and about eight in ten say keeping it under human control should be a top job for the government. A truck driver in rural Minnesota told the AP he can picture a train of self-driving trucks on the interstate, and that there's nothing he can do to control it. That's who the labs are planning around: people who already feel they have no say.
• 2:45
  [Host] Is anything on the books meant to stop the fire itself, rather than manage the day after?
• 2:51
  [Guest] What's on the books is a promise. Late last month the labs signed a voluntary safety pledge at the White House that the president called morally binding, which is the kind of binding that doesn't bind. So what exists before the fire is a promise, and what exists after it is a plan to shape the law.
• 3:11
  [Host] And the rest of us get what, whatever Congress writes in the panic?
• 3:16
  [Guest] That's the dangerous week, and France shows what gets written in it. After the Paris attacks it declared a twelve-day emergency. It ran seven hundred and nineteen days, extension after extension, and when it finally ended, Parliament folded most of the powers into ordinary law. The emergency ended; the powers didn't.
• 3:33
  [Host] So whatever gets written the morning after is what we live with?
• 3:38
  [Guest] Usually for keeps: the emergency declared right after September eleventh was renewed again last month, for the twenty-fifth straight year, according to the government's own Federal Register. Powers go in easily during a crisis and come out almost never, because ending them looks like relaxing vigilance.
• 3:54
  [Host] Then the labs aren't wrong to prepare. If the rules get written the day after, I'd want a say too.
• 4:01
  [Guest] They're not wrong, they're early. The companies will be in the room with a draft while the rest of us are on hold with the bank. The counterweight isn't to keep them out, it's to decide now that anything written in that week carries an expiry date, ninety days and then a mandatory gap before anyone can renew it, so the panic law has to earn a second life in daylight.
• 4:26
  [Host] Would anybody pass that before the thing happens? Sprinkler codes get written after fires.
• 4:31
  [Guest] Every time, and that's the case for writing the expiry rule first: it's the one piece of day-after law that is cheap to pass while nothing is burning, because it takes power from nobody today.
• 4:43
  [Host] So where does that leave the six-to-twelve-month clock?
• 4:46
  [Guest] Running, and the companies building this are rehearsing the apology and the amendments. The one drill nobody is running is the one that tests the sprinklers before the fire. For more on who writes the rules after the alarm, head to unscarcity dot ai. The book is on Amazon, the audiobook is on Spotify. This has been Minds, Bodies, and Terawatts.