Latest / Minds, Bodies, and Terawatts / A Federal Appeals Court Says the Pentagon Can Call Anthropic a National Security Risk for Building an AI That Refuses to Spy on Americans

Transcript

A Federal Appeals Court Says the Pentagon Can Call Anthropic a National Security Risk for Building an AI That Refuses to Spy on Americans · Minds, Bodies, and Terawatts

• 0:00
  [Host] This is Minds, Bodies, and Terawatts, the show where a company got blacklisted for what its product won't do. The Associated Press reported on Friday, September 25th, that a federal appeals court has upheld the Pentagon's decision to label Anthropic a national security supply chain risk.
• 0:15
  [Guest] The vote was two to one. The ruling lets the Defense Department keep pulling Claude out of its systems, and it bars contractors from using anything Anthropic makes on Pentagon work.
• 0:26
  [Host] Remind me what Anthropic did to earn that label. This isn't a Chinese router maker, it's a San Francisco company.
• 0:33
  [Guest] It told the Pentagon no. Early this year the department wanted Claude available for all lawful uses, and Anthropic agreed to loosen many of its rules except two lines it wouldn't cross: fully autonomous weapons, and mass surveillance of Americans. Within days the defense secretary posted that warfighters would never be held hostage by the ideological whims of Big Tech, and designated the company a risk.
• 0:55
  [Host] So the thing they refused to build was a tool for watching me, and the government says that refusal is the risk, which sounds backwards.
• 1:04
  [Guest] The majority opinion is more careful than that. It says the label turns on what Anthropic does, not why. Anthropic encodes restrictions into Claude, and the Pentagon reasonably feared those could reach into a mission the department considered authorized.
• 1:17
  [Host] Feared, or had it actually happen?
• 1:19
  [Guest] Had it happen, the judges say. Claude once refused prompts from the CDC, the federal disease agency, about stopping infectious disease, and there was a dispute over whether it could be used in an ongoing overseas military operation. The court's view is that a tool which might quit mid-mission is a supply chain problem, whatever the intentions behind it.
• 1:39
  [Host] Okay, I'll grant that much. If I'm the person in the field, I don't want my software growing a conscience at the wrong moment.
• 1:47
  [Guest] That is the Pentagon's strongest case, and the majority said plainly that Anthropic's intentions are noble. It just held that in our republic the president and the secretary decide how to weigh an AI that refuses against an AI that picks the wrong target.
• 2:02
  [Host] Then what did the dissenting judge see that the other two didn't?
• 2:06
  [Guest] She saw the next contractor coming. Judge Henderson wrote that the law was built for hostile or deceptive interference, foreign governments and saboteurs, not for a vendor's openly published rules. Her worry is that whoever replaces Anthropic gets told to permit any function the department deems necessary, or share the same fate.
• 2:24
  [Host] And nobody has to actually blacklist that next company for the threat to work.
• 2:29
  [Guest] Nobody has to, the possibility does the job. Every AI vendor now knows the ceiling on what it may refuse is set by whoever runs the Pentagon, and that shapes what gets built before anyone is asked to change anything.
• 2:42
  [Host] What has this cost Anthropic so far?
• 2:45
  [Guest] Reuters says the company puts it at billions in lost business, just as it prepares to sell shares to the public. Anthropic says it respectfully disagrees and is weighing further review, and it points to a San Francisco judge who last month struck down a parallel designation as unlawful retaliation for its views on AI safety.
• 3:04
  [Host] Two courts, two answers, so which one is actually in charge?
• 3:08
  [Guest] Both are, under different laws. The California ruling blocks the administration's broader government-wide freeze on Anthropic, while Friday's ruling keeps the Pentagon's own door shut, and only that door.
• 3:18
  [Host] Where does the share sale fit? A court-stamped risk label is not what you want in the middle of pitching investors.
• 3:25
  [Guest] The day before the ruling, Reuters relayed a report that Anthropic is asking its shareholders to hand Dario Amodei and his six co-founders a hair over half of the voting power on most matters, for as long as three of them still hold enough shares. The model is Palantir's, the defense data contractor, and the point is that the company's judgment stays with the founders after the public buys in.
• 3:49
  [Host] So the same week the government says Anthropic's judgment is a risk, the founders move to make sure new shareholders can't outvote that judgment.
• 3:57
  [Guest] That is the shape of it, and it shows what everyone in this fight agrees on: the machine shouldn't decide on its own what it refuses. Some human writes the no, and the whole argument is about which humans.
• 4:10
  [Host] The court says the secretary, the founders say the seven of them. Who speaks for the people that surveillance line was drawn to protect?
• 4:18
  [Guest] Nobody, in either document, and the listener whose data that line was drawn around isn't a party to the case and won't hold the special shares. A rule that lives at one official's pleasure, or inside one share class, is a rule the next official or the next investor can rewrite.
• 4:36
  [Host] So the no exists today, but it isn't anybody's law?
• 4:39
  [Guest] A court deferring to a secretary on one side, a share structure on the other, and neither is a rule the public wrote. For more on who decides what a machine refuses, and what it takes to keep a no in place once it's written, head to unscarcity dot ai. The book is on Amazon, and the audiobook is on Spotify. This has been Minds, Bodies, and Terawatts.