Latest / In The Loop / Why the department of war banned Claude

Transcript

Why the department of war banned Claude · In The Loop

0:00
So last week Anthropic, so the maker of Claude, fell out with
0:04
the US government and the Department of War.
0:07
This is a huge high profile fallout.
0:09
Trump even took to true social to claim that Anthropic and
0:14
Claude are enemies of peace and the government and huge woke
0:19
liberals and all because Claude said no to some really quite
0:24
terrifying use cases that the army wanted to implement.
0:27
And then the very same afternoon, Open AI announced a
0:30
high profile partnership with the Department of War.
0:33
And since then, we've seen Open AI uninstalls rise by 295%, one
0:40
star reviews spike and double day over day.
0:44
And then, despite being banned by the government, Anthropic's
0:47
cord was then used to conduct air strikes against Iran.
0:51
What a crazy turn of events. And today I'm covering it all.
0:55
What the actual contracts say and the fine print now, the
0:58
things that most people don't know about the situation, the
1:01
specific details that really do matter and trying to understand
1:06
the answer to a key question. When technology becomes so
1:10
powerful, who has the right to decide how to use it?
1:14
This is in a Leaf with Jack Horton.
1:16
Hope you enjoy the show. So on Thursday evening, on
1:30
February 27th, Sam Altman sent an internal memo to every single
1:36
employee Open AI. So the Wall Street Journal broke
1:39
the story. And in it, Altman actually said
1:42
the open AI had long believed that AI should not ever be used
1:46
for mass surveillance or autonomous lethal weapons.
1:50
And this is a case where it's important to me.
1:53
And I quote, this is by Sam Altman.
1:55
This is a case where it's important to me that we do the
1:58
right thing and not the easy thing that looks disingenuous.
2:02
And less than 24 hours later, Open AI then signed a deal with
2:06
the Pentagon and therefore the Department of War to deploy its
2:10
models in throughout its security operations.
2:13
So to understand how we got here, I think you need to
2:16
understand what happened in the days before this agreement,
2:19
because Anthropic had been embedded in the Pentagon system
2:22
since, you know, June 2024, and it was the first frontier AI
2:27
model to get there. They got into this partnership
2:30
via Palantir and AWS and actually got security level
2:35
clearance at level 6, which is classified as secret clearance.
2:40
And by the summer of 2025, they had a contract worth up to $200
2:43
million. And Claude was being used by
2:47
CENTCOM for intelligence assessments, identifying
2:50
targets, simulating battle scenarios.
2:53
And it was reportedly used during the capture of the
2:55
Venezuelan president in January 2026.
2:58
And, you know, this wasn't a company sitting on the sidelines
3:00
here being super ethical. Anthropic was already very deep
3:04
inside the US military establishment, far more than any
3:07
other AI lab. Now the dispute that really
3:10
kicked off publicly and had Trump weighing in, calling
3:13
Anthropic an enemy of the people, actually was triggered
3:16
by Palentia. Palentia tried to get Claude or
3:20
Anthropic to agree to what they call an all lawful use standard,
3:23
meaning that essentially the military could use claw for
3:26
anything they wanted that wasn't explicitly illegal.
3:30
And Anthropic said, yeah, that's absolutely fine, apart from 2
3:32
exceptions that it wouldn't budget on, which was #1 mass
3:36
surveillance or mass domestic surveillance of the American
3:40
people and #2 was fully autonomous weaponry that would
3:43
take humans completely out of the loop.
3:46
Everything else, as I said, was completely on the table.
3:49
All fine. Every other military use case
3:51
you can think of, just not those two.
3:53
Now on Tuesday the 25th of February, the Defence Secretary
3:58
of the US government met with the CEO of Anthropic, Dario
4:02
Amadei at the Pentagon and essentially they gave each of
4:05
their deadline which was Friday. By 5:00 PM we have to have
4:08
agreed to all lawful use or lose the contract completely.
4:11
So lose that massive contract with the US government.
4:14
And that night the Pentagon sent what it called it's last and
4:17
final offer. So Anthropics team had reviewed
4:19
the offer that was given to them and essentially their conclusion
4:22
was, and all the lawyers conclusion was that there was
4:25
lots of wording that frames the agreement as a compromise, but
4:29
ultimately left all the legal loopholes to just do whatever
4:34
they want anyway. So essentially we had to just
4:36
disregard all safeguards. And it got that heated that the
4:40
CEO of Anthropic, Amede, published his response on the
4:44
Anthropic website. You can go see that today.
4:45
His arguments was specific and not ideological.
4:49
He said on weapons, current frontier AI models are just not
4:52
reliably good enough to engage in targets, and deploying them
4:55
are just going to endanger American fighters and civilians.
4:59
And on surveillance, he said that there is no federal law
5:01
that stops AI conducting mass surveillance on millions of
5:05
people. But federal agencies haven't yet
5:08
caught up with laws to protect people from mass surveillance.
5:12
You know, data brokers can legally sell mass data to the US
5:16
government and then use those models to quite literally
5:20
conduct huge mass surveillance. And if you think about AI
5:23
models, what they're fantastic about is turning huge amounts of
5:26
data into the ability to zone in on particular people, groups of
5:32
people or situations. You know, before AI, think about
5:35
this, an agency that buys and sells people's data, it was
5:38
basically like pouring data into a big bucket.
5:41
And it's really difficult to make sense of that data.
5:43
Yes, you can use it, and it's really powerful for machine
5:46
learning models, and there's loads of nefarious, dangerous
5:49
use cases there. But with AI, that same data
5:51
becomes an insane surveillance system.
5:54
You know, you can cross reference, you can automatically
5:57
build profiles that flags individuals or tracks any
6:00
pattern you want across millions and millions of people
6:04
instantaneously. The technology is what makes
6:06
that data really, really dangerous.
6:08
And that's the gap that Anthropic took a stance on.
6:10
And I have to say, I've been a big cloud user for a long time.
6:14
I couldn't even imagine using open AI.
6:16
I've not got an open AI subscription.
6:18
I tell everyone to not use open AI for a number of reasons.
6:21
We'll discuss those, I'm sure. But it makes me respect the
6:25
company a bit more as well because Amadei wrote and I
6:28
quote, these threats do not change our position.
6:31
We cannot in good conscience accede their request.
6:35
You know, anyone that's been a entrepreneur that has their own
6:39
business or is running high profile, high pressure teams,
6:43
for example, knows that the pressure to say yes to contract
6:47
can be pretty overwhelming at times.
6:49
Because although a company from the outside looks amazing,
6:52
they're always, as we all know, internal stresses and pressures.
6:55
And sometimes doing that right thing is not easy, especially
6:59
when you're throwing away hundreds and hundreds and
7:01
hundreds of millions and probably billions over a couple
7:03
of years of potential revenue. And he said no, which I'm so
7:08
relieved up. So yeah, that's that's all the
7:10
situation that led up to Donald Trump.
7:12
And I quote, calling them left wing nut jobs, putting American
7:16
lives at risk and threatens actually the full power of the
7:19
presidency with major civil and criminal criminal consequences
7:24
if they didn't agree to cooperate.
7:26
And you know, a few minutes later, they called Anthropic a
7:28
complete supply chamber that must be removed from every
7:31
system. That same evening, about 8-9
7:50
hours before the US launched strikes on Iran, Sam Altman
7:53
announced on X that Open AI had reached an agreement with the
7:56
Pentagon. Surprise, surprise.
7:57
So yeah, let's, let's get into what Open AI's agreement looks
8:00
like, because as I've said, that they're seeing huge backlash
8:03
right now from their customers and user groups.
8:06
On the face of it, both companies are claiming identical
8:09
principles. The difference is the mechanism,
8:12
and the mechanism is quite literally everything.
8:14
So Anthropic wanted specific contractual prohibitions, so
8:19
language in the contract itself that specifically banned
8:22
surveillance and autonomous weaponry regardless of their
8:26
legal status. So whether or not that is made
8:29
legal in the next two, 5-7 years, they did not want AI to
8:33
be used for that. And obviously that's what the
8:34
Pentagon refused to accept. Their position was that no
8:37
private company should ever have veto power over how the military
8:41
uses technology. You know, the Constitution and
8:43
existing laws are the appropriate constraints in the
8:46
view of the Pentagon. And, you know, there's an
8:48
argument to say that it's, you know, and this is what was on
8:51
all American news channels on CBS, saying on some levels, you
8:55
have to trust the military to do the right thing.
8:57
And the geopolitical argument behind, you know, that stance is
9:01
real. But there are real threats all
9:03
over the world. And China's AI labs are
9:05
definitely not negotiating red lines with their government.
9:08
Their integration is moving incredibly fast because there's
9:11
0 friction. And so the Pentagon will view
9:14
this as an arms race with China. And you can't let from their
9:19
perspective, the government's perspective and the Pentagon's
9:21
perspective, the vendors set the rules, whereas Open AI have
9:24
taken a very different approach. Instead of any contractual
9:28
prohibitions, they cited existing laws and policies.
9:31
So the 4th Amendment, the Foreign International
9:34
Surveillance Act, which is a Pentagon directive against
9:38
autonomous weaponry, their position is essentially, you
9:41
know, these laws already cover all of our concerns.
9:44
The government has agreed to follow them, and we build
9:46
technical safeguards into the model as an additional layer of
9:50
protection. But this is where, if you read
9:51
the fine print where, you know, there's a gap that opens up.
9:55
So Jessica Tillotman, which is she's the associate Dean for
9:59
government procurement law and George Washington University,
10:02
she analysed the published contract and she concluded that
10:05
it doesn't give open AI an anthropic style ability to stop
10:09
the government using the technology on those use cases.
10:13
Though it does state that the Pentagon cannot use the
10:16
technology when breaking laws. Which sounds reassuring until
10:20
you remember that the entire reason Anthropic drew its lines
10:23
was that no existing laws prohibit the things that we
10:27
should be worried about because law has not caught up yet with
10:30
technology. You know, the 2023 Pentagon
10:33
directive that was against autonomous weaponry, The open AI
10:36
citing, you know, it doesn't actually ban them.
10:38
All it is is creating guidelines on how it should and should not
10:42
be used. And there's always a waiver that
10:45
essentially is a clause stating that urgent military needs means
10:49
that you can do anything you want.
10:50
And Altman even admitted in an AMA.
10:54
So it asked me anything on XA couple days later.
10:56
The deal was very, very rushed and the optics definitely don't
11:00
look good for open AR. And he even went further and
11:03
said on Monday that we shouldn't have rushed to get this out on
11:06
Friday. Even open a eyes own team and
11:08
people noticed. Leo Gow, an alignment researcher
11:12
of the company even said that. You know, it's so obvious that
11:15
this is weak and the government can now just technically use it
11:18
for anything that is deemed lawful, but there are now
11:21
missing laws to deem what is or is not lawful when it comes to
11:24
things like mass surveillance. Now that there is always a
11:47
counterpart, you know, which is opening eye eyes argued that if
11:51
you don't trust the government to follow the law, why would you
11:53
trust it to honour contract terms either, You know, if the
11:56
Pentagon is going to do mass surveillance regardless, you
11:59
know, align and drop its contract wouldn't stop them
12:02
doing it. And there definitely is
12:03
something to that. However, contract term shape
12:06
behaviour and it creates oversight mechanisms and
12:09
establishes political consequences for actions and
12:12
violations. You know, a line in the sand
12:14
that someone might just step over anyway is better than no
12:17
line in the sand. And anyway, I mean, there's been
12:19
huge fallout as you can just imagine.
12:21
The council ChatGPT movement has, it's been going on since
12:24
January. I mean, I actually cancelled
12:26
mine in January because I found Greg Brockman was the biggest
12:31
donor to Trump, donating over $25 million to his action
12:37
committee. And I think the Pentagon
12:40
situation has turned this into something much, much, much
12:43
larger. You know, Chat GPTS daily
12:46
download growth fell by 13% and one star review spiked over 775%
12:52
on Saturday and doubled again on Sunday.
12:55
Claw downloads are climbed by 81% and hit #1 on Apple's US App
13:01
Store with over 500,000 downloads in a single day.
13:04
You know, people aren't just debating policy online.
13:07
I think they're they're voting with their fingers and their
13:10
funds. See, I think there's going to be
13:12
a bunch of things to watch for here.
13:13
First, whether the supply chain risk designation.
13:16
So when the government announced the cord is a supply chain risk
13:20
actually gets fully formalized. It isn't just a tweet so far,
13:24
it's just a social media post. It's not a legal proceeding and
13:28
legal experts called it, you know, basically more political
13:31
theatre than than an actual show of force.
13:33
Second, Claude is still the only model on the Pentagon's
13:37
classified network, and the Wall Street had reported that it was
13:40
even used in the Iranian strikes just hours after the ban was
13:44
announced. The military would have six
13:45
months to swap the model under that directive, but many people
13:49
have already come out in the military saying it's going to be
13:51
a huge pain in the arse. And 3rd, the what could be
13:54
described as the chilling effect.
13:55
So, for example, Google has quietly removed its long
13:58
standing ban on AI weaponry development from its AI
14:02
principles. We did that in February 2025
14:05
right before signing an agreement with the Pentagon.
14:08
And you know, every tech company is probably watching what
14:10
happens to your Anthropic, which is if you step out of line,
14:13
you'll be burnt. So yeah, to conclude, I think
14:37
there's been a bit of a crazy turn of events.
14:39
I think there's been one of the most high profile fallings out
14:42
between AUS company and the US government.
14:46
But what's really important I think, is that, you know, I
14:48
can't help but coming back to the fact that Dario Amadei's
14:51
favorite book, the one that he used to buy for every single new
14:55
employee and frofic was The making of the Atomic Bomb.
14:59
He wanted people at the company to understand that what they
15:02
were building could and probably would eventually become so
15:07
important that the government would want to control it.
15:11
And that when that moment arrived, the people who built
15:14
the technology would have to decide whether they got a say in
15:17
how it was used. And that moment arrived last
15:20
Thursday. One company said no, another
15:22
said yes. And the person who wrote the
15:24
memo about doing the right thing then signed the deal the next
15:27
day. I think that says a lot about
15:29
Sam Altman, and it says a lot about Open AI.
15:31
And I hope the cancelled ChatGPT movement continues.
15:35
Anyway, that's it. I hope you enjoyed today's
15:37
episodes. I hope you learned something and
15:40
I'll see you next week.