Latest / ELECTE's Podcast: AI Frontiers / Who controls AI?

Transcript

Who controls AI? · ELECTE's Podcast: AI Frontiers

• 0:00
  SPEAKER_00: The protagonist is Claude, the model developed by Anthropic.
• 0:04
  SPEAKER_00: On twenty seventh, february twenty twenty six, the Pentagon placed the company on a blacklist previously reserved for Kaspersky, Huawei, and suppliers linked to rival powers.
• 0:14
  SPEAKER_00: The official justification risk to the national security supply chain.
• 0:20
  SPEAKER_00: Pursuant to 10 USC, Paragraph thirty two hundred and fifty two, a provision designed to counter foreign sabotage, never before used against an American company.
• 0:30
  SPEAKER_00: The reason?
• 0:32
  SPEAKER_00: Anthropic refused to remove two clauses from its$200 million contract with the Department of Defense.
• 0:38
  SPEAKER_00: The first prohibited the use of Claude for mass surveillance of American citizens.
• 0:42
  SPEAKER_00: The second prohibited its use in fully autonomous weapons without human supervision.
• 0:48
  SPEAKER_00: On 26th February, CEO Dario Almoti had written, fully autonomous weapons could also prove crucial to our national defense.
• 0:57
  SPEAKER_00: But at present, cutting-edge AI systems are simply not reliable enough to power them.
• 1:02
  SPEAKER_00: We will not knowingly supply a product that puts American servicemen and civilians at risk.
• 1:08
  SPEAKER_00: The day after the blacklisting, the United States launched airstrikes against Iran as part of Operation Epic Fury, and Claude was still there.
• 1:17
  SPEAKER_00: Active within classified military systems, it was set to remain there for months thanks to a 180-day removal window provided for by the designation itself.
• 1:27
  SPEAKER_00: According to various media reports, the system contributed to the classification and selection of military targets.
• 1:33
  SPEAKER_00: CENTCOM confirmed the use of advanced AI tools.
• 1:37
  SPEAKER_00: The Pentagon's CIO, Kirsten Davies, told the Senate, the system is currently active.
• 1:43
  SPEAKER_00: Essential and dangerous, same system, same week.
• 1:47
  SPEAKER_00: This isn't just a story about a contract gone wrong.
• 1:49
  SPEAKER_00: The question with no owner.
• 1:51
  SPEAKER_00: It is the story of a question that no institution in the world today has the formal right to resolve.
• 1:57
  SPEAKER_00: Who decides what an artificial intelligence system can do?
• 2:01
  SPEAKER_00: The company that built it?
• 2:02
  SPEAKER_00: The government that bought it?
• 2:04
  SPEAKER_00: Parliament?
• 2:04
  SPEAKER_00: An international treaty?
• 2:06
  SPEAKER_00: No one?
• 2:07
  SPEAKER_00: In practice, the answer is far less noble than it seems.
• 2:10
  SPEAKER_00: It depends on the contract.
• 2:12
  SPEAKER_00: Case by case, clause by clause, negotiation by negotiation, without transparency, without binding precedence, without citizens having a say.
• 2:25
  SPEAKER_00: The anthropic Pentagon case has simply brought to light what until yesterday remained in the shadows.
• 2:30
  SPEAKER_00: The manufacturer says it's not ready.
• 2:34
  SPEAKER_00: Anthropic's argument is technical rather than ethical.
• 2:38
  SPEAKER_00: An AI system has measurable error rates.
• 2:41
  SPEAKER_00: If those rates are incompatible with a particular application, for example, weapons that select and engage targets without human intervention, the manufacturer has an engineering responsibility to say so.
• 2:54
  SPEAKER_00: If Boeing were to inform the Pentagon that an aircraft is not certified for a certain mission profile, the normal response would be a technical review, not a blacklist.
• 3:04
  SPEAKER_00: Anthropic has done the equivalent in AI terms, but the story doesn't end there.
• 3:10
  SPEAKER_00: Boeing would not time a technical communication to coincide with the deadline of a political ultimatum.
• 3:16
  SPEAKER_00: It would not present it as a battle for democratic values, and it would hardly see its own product leap to the top of the app store the very next day.
• 3:25
  SPEAKER_00: Furthermore, the limits declared by Anthropic are selective, as revealed by the affidavits filed in the federal litigation.
• 3:32
  SPEAKER_00: The company accepts the use of Claude for foreign intelligence.
• 3:35
  SPEAKER_00: It accepts partially autonomous systems.
• 3:38
  SPEAKER_00: The refusal concerns two specific lines mass domestic surveillance, full lethal autonomy without human supervision.
• 3:47
  SPEAKER_00: It is not a refusal of military use.
• 3:50
  SPEAKER_00: It is a calibrated refusal.
• 3:52
  SPEAKER_00: The government says, We decide.
• 3:55
  SPEAKER_00: The Pentagon's position follows a mirror image logic.
• 3:59
  SPEAKER_00: In a democracy, military operations are authorized by Congress, directed by the president, and subject to judicial review.
• 4:06
  SPEAKER_00: The required standard, any lawful use, means any use permitted by current law.
• 4:11
  SPEAKER_00: The strategic memorandum of 9 January 2026 formalized this approach.
• 4:16
  SPEAKER_00: Those who accepted, joined.
• 4:18
  SPEAKER_00: OpenAI signed a classified contract just hours after Anthropic was blacklisted.
• 4:22
  SPEAKER_00: XAI had already signed on 23rd February.
• 4:25
  SPEAKER_00: The difference is not in principle.
• 4:27
  SPEAKER_00: Both declare limits.
• 4:29
  SPEAKER_00: The difference lies in the mechanism.
• 4:31
  SPEAKER_00: Anthropic wanted binding contractual prohibitions.
• 4:35
  SPEAKER_00: OpenAI safeguards refer to existing law, and this means that the government can use those models for anything that is not already illegal.
• 4:43
  SPEAKER_00: As MIT Technology Review noted, we've essentially gone back to square one, allowing the Pentagon to use its own AI for any legitimate purpose.
• 4:53
  SPEAKER_00: But legitimate is not a universal category.
• 4:56
  SPEAKER_00: It varies from country to country, the global regulatory vacuum.
• 5:01
  SPEAKER_00: The dispute between anthropic and the Pentagon cannot be resolved within the current regulatory framework.
• 5:06
  SPEAKER_00: There is no clear system for determining when a military AI is safe enough.
• 5:11
  SPEAKER_00: DOD Directive 3000.
• 5:14
  SPEAKER_00: Aero 9 requires appropriate levels of human judgment and the use of force.
• 5:18
  SPEAKER_00: But it was not written with cutting-edge language models in mind.
• 5:22
  SPEAKER_00: And according to the affidavits filed in court, once installed in a classified and air gapped environment, Claude cannot be modified remotely, no direct access, no kill switch, no intervention without Pentagon authorization.
• 5:35
  SPEAKER_00: A private company ends up having de facto power to limit what a state can do with legally acquired technology.
• 5:42
  SPEAKER_00: A state may blacklist a company for imposing those limits, and no law says who is in the right.
• 5:48
  SPEAKER_00: The problem is not just an American one.
• 5:50
  SPEAKER_00: What is lawful under the US visa may be incompatible with the European GDPR, the EU AI Act prohibits certain forms of mass biometric surveillance, and this is just the Euro American front.
• 6:02
  SPEAKER_00: The global picture is worse.
• 6:03
  SPEAKER_00: Summit year signatories OSA China type of commitment.
• 6:07
  SPEAKER_00: RIAM LIA TWIN2350 Plus Call to Action, nonbinding.
• 6:11
  SPEAKER_00: REAM SOL 202460, Blueprint for Action, Nonbinding.
• 6:16
  SPEAKER_00: REAM A Corunia 202635 Pathways for Action, Nonbinding, Fewer Signatories.
• 6:22
  SPEAKER_00: Fewer commitments, less consensus.
• 6:24
  SPEAKER_00: Just as technology is accelerating, it is true that the United States and China continue to engage in dialogue in smaller scale formats, but the practical outcome remains the same.
• 6:35
  SPEAKER_00: There is no shared framework.
• 6:37
  SPEAKER_00: At the UN, the debate on lethal autonomous weapons has been ongoing for nine years.
• 6:42
  SPEAKER_00: Secretary General Guterres has described them as morally repugnant and has called for a treaty by 2026.
• 6:49
  SPEAKER_00: In December 2024, 166 countries voted in favor of a resolution on laws, opposed Belarus, North Korea, and Russia, but the negotiating group operates by consensus.
• 7:00
  SPEAKER_00: A single state is enough to block everything.
• 7:03
  SPEAKER_00: In the absence of specific treaties, the rules governing military AI are laid down in commercial negotiations.
• 7:10
  SPEAKER_00: Not in parliaments, not in treaties, in contracts.
• 7:15
  SPEAKER_00: The European silence.
• 7:17
  SPEAKER_00: No European institution has formally commented on the case, yet the issue affects anyone building critical infrastructure on AI platforms negotiated in Washington.
• 7:27
  SPEAKER_00: The EU AI Act regulates the use of AI in Europe.
• 7:31
  SPEAKER_00: It does not regulate how those models are designed in the United States.
• 7:35
  SPEAKER_00: If a system is designed to accept any lawful use, Europe can only intervene afterwards.
• 7:40
  SPEAKER_00: Ex post control, not architectural constraints.
• 7:44
  SPEAKER_00: And if your company uses AI, this story affects you more than you might think.
• 7:49
  SPEAKER_00: Where we are now?
• 7:51
  SPEAKER_00: On 9 March 2026, Anthropic filed two federal lawsuits, one in the Northern District of California, alleging violations of the First and Fifth Amendments and the Administrative Procedure Act, the other before the DC Circuit Court of Appeals.
• 8:05
  SPEAKER_00: On 24th March, at the preliminary hearing before Judge Rita F.
• 8:09
  SPEAKER_00: Lynn, the tone was clear.
• 8:11
  SPEAKER_00: Lynn described the designation as an attempt to cripple anthropic.
• 8:16
  SPEAKER_00: She asked the government why?
• 8:18
  SPEAKER_00: If the issue was the integrity of the chain of command, the Pentagon had not simply stopped using Claude, instead of invoking a rule designed for hostile foreign suppliers.
• 8:29
  SPEAKER_00: When the government invoked national security, the judge replied, If an IT supplier merely needs to be stubborn over contractual terms to be declared a risk, then the threshold is dangerously low.
• 8:40
  SPEAKER_00: Microsoft filed an amicus brief, a voluntary submission in support of one of the parties to the case, citing serious consequences for the entire technology sector.
• 8:51
  SPEAKER_00: Around fifty employees from OpenAI and Google DeepMind did the same in a personal capacity.
• 8:56
  SPEAKER_00: Elizabeth Warren described the designation as retaliation.
• 9:00
  SPEAKER_00: A preliminary ruling is expected shortly, but even if the judge grants an injunction, the designation would only be suspended, not overturned.
• 9:08
  SPEAKER_00: The trial on the merits could last a year or more.
• 9:11
  SPEAKER_00: The question that remains unanswered.
• 9:13
  SPEAKER_00: One company said, Our system isn't ready for this.
• 9:17
  SPEAKER_00: It was blacklisted.
• 9:18
  SPEAKER_00: The system remained active in combat because it was too useful.
• 9:22
  SPEAKER_00: The United Nations debates without producing binding agreements.
• 9:26
  SPEAKER_00: A federal judge is deciding whether a private company can technically restrict its own product.
• 9:31
  SPEAKER_00: No institution was designed to answer the question who controls AI?
• 9:35
  SPEAKER_00: The Pentagon is designed to win wars.
• 9:37
  SPEAKER_00: Companies want to sell technology.
• 9:39
  SPEAKER_00: The UN is designed to build consensus.
• 9:42
  SPEAKER_00: The courts are designed to enforce the law.
• 9:44
  SPEAKER_00: The question exists in the space between these institutions.
• 9:48
  SPEAKER_00: Currently, there is no one in that space.
• 9:51
  SPEAKER_00: AI governance is not decided in parliaments, nor in treaties, nor in courts.
• 9:57
  SPEAKER_00: It is decided in contracts.
• 9:59
  SPEAKER_00: Technology moves at the speed of capital.
• 10:02
  SPEAKER_00: Rules, however, move at the speed of political consensus.
• 10:05
  SPEAKER_00: The gap between the two is becoming the place where the future is decided.
• 10:09
  SPEAKER_00: The next time you choose an AI model for your company, you are not just choosing a supplier.
• 10:14
  SPEAKER_00: You are also deciding which concept of power will enter your infrastructure.
• 10:18
  SPEAKER_00: The newsletter will be taking a break next week for Easter.
• 10:21
  SPEAKER_00: We'll be back on 9 April.
• 10:23
  SPEAKER_00: Fabio Larchia, CEO and founder, electee every week.
• 10:26
  SPEAKER_00: We explore AI without the hype, using data, analysis, and an independent perspective.
• 10:33
  SPEAKER_00: Sources Anthropic Pentagon case.
• 10:36
  SPEAKER_00: CNBC, Anthropic Sue's, Trump Administration over Pentagon Blacklist, 9 March 2026, 2.
• 10:43
  SPEAKER_00: CNBC, Judge Press's DOD on why Anthropic was blacklisted, 24 March 2026, 3.
• 10:51
  SPEAKER_00: NPR, Anthropic Suez Trump Administration over blacklisting decision, 9 March 2026, 4.
• 10:57
  SPEAKER_00: TechCrunch, Elizabeth Warren calls Pentagon's decision to bar anthropic retaliation, 23rd March 2026, 5.
• 11:06
  SPEAKER_00: E Week, Warren calls Anthropic Blacklist, Retaliation, Ahead of Key Pentagon Hearing, 24 March 2026, 6.
• 11:14
  SPEAKER_00: Modern Diplomacy, Anthropic Challenges, Pentagon Blacklisting, 11 March 2026, 7.
• 11:20
  SPEAKER_00: Android Headlines, AI at War, Anthropic Fights, The Pentagon's Unprecedented Blacklist in Court on 25 March 2026, U.S.
• 11:28
  SPEAKER_00: Regulatory Framework.
• 11:29
  SPEAKER_00: 10 USC, Paragraph 3252, Supply Chain Risk Statute 2, DOD Directive 3000.
• 11:36
  SPEAKER_00: 09, Autonomy and Weapons Systems, 2012, Updated 2023, Autonomous Weapons and International Governance.
• 11:43
  SPEAKER_00: United Nations, GGE on Lethal Autonomous Weapons Systems 2262.
• 11:49
  SPEAKER_00: UN News, UN Chief Calls for Global Ban on Killer Robots, May 2025, 3.
• 11:55
  SPEAKER_00: ASIL, Lethal Autonomous Weapons Systems and International Law, Growing Momentum 2025, 4.
• 12:01
  SPEAKER_00: CCW GGE, Rolling Text on Laws, Status 18 December 2025, PDF 5.
• 12:07
  SPEAKER_00: Stop Killer Robots, REM 2026 Press Release, February 2026, ED REM Summit.
• 12:13
  SPEAKER_00: The Defense Watch, U.S.
• 12:14
  SPEAKER_00: and China refused to sign military AI declaration at REM 2026 and 6 February 2026 and 2.
• 12:21
  SPEAKER_00: Just security artificial urgency reflecting on AI hype at the 2026 REM Summit, March 2026, 3.
• 12:29
  SPEAKER_00: Mexico Business News, 35 Nations Back Military AI rules as U.S.
• 12:33
  SPEAKER_00: China opt out, 12th February 2026, 4.
• 12:37
  SPEAKER_00: U.S.
• 12:38
  SPEAKER_00: Department of State Political Declaration on Responsible Military Use of AI and Autonomy Fees.
• 12:43
  SPEAKER_00: Government of the Netherlands.
• 12:45
  SPEAKER_00: RAM 2023.
• 12:47
  SPEAKER_00: Endorsing countries.
• 12:48
  SPEAKER_00: Subscribe now.