Latest / The Fourcast / Trump Iran Israel: Is this the first AI war?

Transcript

Trump Iran Israel: Is this the first AI war? · The Fourcast

0:00
AI does not solve all problems. It will never solve all of the
0:03
problems. So you may be able to generate
0:05
all the targets in the world based on remarkable AI tools,
0:09
but that may not matter if you don't have.
0:10
Any munitions What? Might a kind of AI superpower
0:14
war look? Like huge numbers essentially
0:18
of, of robots potentially going against each other or going
0:20
against some hybrid of, you know, humans and weapons across
0:23
domains, land, sea, space and also, of course, cyberspace.
0:29
It's not something that is a good idea to think about right
0:33
before going to bed. Hello and welcome to The
0:40
Forecast. I'm Kieran Jenkins.
0:42
Across the battlefields of Ukraine, Gaza and now Iran, a
0:46
new technology is quietly reshaping how wars are fought.
0:51
Artificial intelligence is helping militaries process
0:53
intelligence, identify targets, and make decisions at rapidly
0:58
accelerating pace. Some believe this is the
1:01
beginning of an AI revolution in warfare, one that could
1:05
eventually lead to autonomous weapons and algorithmic
1:09
battlefields. So how far has this
1:11
transformation already gone? Is Iran the first AI war, and is
1:17
the world ready for what's coming?
1:19
Well, to discuss, I'm joined by Amelia Probasco, who was a Navy
1:23
officer, later worked in the Pentagon and is now a senior
1:27
fellow at the Center for Security and Emerging
1:29
Technology, and also Arthur Holland, Michelle, a journalist
1:33
and AI researcher. Arthur, first of all, are we
1:37
seeing AI in this conflict in Iran on a scale that just hasn't
1:40
been used before on the battlefield?
1:42
We have already seen some AI used in a somewhat similar scale
1:47
in Israel's campaign in Gaza starting in 2023.
1:53
I mean this is a larger campaign more generally.
1:57
So you know, I think the use of AI may be scaling with that and
2:01
something crucial that has happened in that interim since
2:04
2023 is that we are seeing now these chatbot based or LLM based
2:10
systems being used. That is of course novel.
2:15
That was not something that was happening in 2023.
2:17
It started happening a bit in 2024, but the scale there is, is
2:23
perhaps bigger, although we don't have a full picture of how
2:26
extensively that AI, specifically that subset of AI
2:30
is being used in these operations.
2:32
It's definitely a step up. I, I, I will certainly say that.
2:36
Amelia, what's your take on the scale of what we're seeing in
2:39
terms of AI being deployed in this conflict?
2:42
Yeah, I think Arthur has really explained well that the
2:45
militaries have been trying to adopt AI for a long time.
2:48
It can improve their operations. I think it's important to
2:51
understand that without these AI tools, there are people passing
2:55
Excel spreadsheets and PowerPoint presentations around
2:58
and having multiple meetings. So they are quite keen to try
3:02
and adopt these sorts of tools. What we're seeing now is it is
3:06
quite different. The, as Arthur mentioned, the
3:08
incorporation of large language models, it is very new.
3:12
The system that was talked about most in the United States, the
3:15
incorporation of clawed into the Maven smart system didn't happen
3:18
until the end of 2024. And the adoption, as I have
3:22
understand it was not didn't take off right at the beginning.
3:25
It's taken some time, but as it turns out, the United States
3:28
Central Command, which has responsibility for this current
3:31
conflict, is the furthest ahead in trying to understand the
3:35
technology and trying to find productive and responsible ways
3:39
to use it. So I think we are seeing
3:42
something pretty remarkable right now.
3:43
But before we get into exactly how AI might be being used right
3:48
now, can we just talk about where the war is right now?
3:50
Because for all America's power and for all it says it's
3:54
destroying Iran's capabilities, for all its AI, Iran seems to be
3:59
pretty effective at causing an awful lot of destruction and
4:03
disruption as well. The resilience has has been
4:06
impressive, I think, though it comes as no surprise to anyone
4:11
that both sides of this conflict were preparing for this sort of
4:15
a conflict for a long time. And so as I understand that this
4:19
could be part of their strategy is to induce more, more chaos in
4:22
the region in an attempt to attrite the will of of the
4:27
United States and others to continue the campaign.
4:30
I suppose the question is, Emilia, with all, with all your
4:32
military knowledge, you know, can America's AI power
4:38
realistically do anything to reopen the Strait of Hormuz, for
4:43
instance, which is like so vital for global trade and which is
4:46
strangling the global economy right now?
4:48
AI does not solve all problems. It will never solve all of the
4:52
problems. It is a tool in the toolbox of
4:56
diplomatic, economic, military that must be put to bear in
5:00
terms of coming to a resolution. So while AI can do some pretty
5:04
incredible things, can help you understand and maybe predict how
5:08
and where to respond, at the end of the day, there's so many
5:12
other important things. And, and one of the things that
5:14
we talk about in the United States in particular right now
5:16
are munitions, of which there are a limited number.
5:18
So you may be able to generate all the targets in the world
5:21
based on incredible satellite imagery and remarkable AI tools,
5:26
but that may not matter if you don't have any munitions.
5:28
Right. And Arthur, you've got the
5:30
American AI models with like quintillions of computations per
5:35
second versus Iranian mines, which are just causing an
5:40
absolute nightmare for ships trying to get from A to B.
5:44
Yeah, it's, I mean, it's a very old story where technological
5:49
wizardry finds itself foundering at times in the face of, you
5:56
know, a much more rudimentary set of military practices.
6:02
We've seen this in, you know, every form of asymmetrical war,
6:07
certainly in, in, in, in the last decades where, you know,
6:12
militaries with less technology and less money can show in some
6:18
cases an alarming amount of resilience against, you know,
6:26
formidable powers. AI is not going to necessarily
6:31
change that dynamic. You know that there is questions
6:35
in terms of the logistics, how many systems you have flying in
6:40
the air that are actually going to be able to match these
6:43
targets that the AI is potentially or supposedly, you
6:47
know, proposing at these lightning fast speeds.
6:53
I I think another factor here also though, is that we have to
6:57
temper some of our expectations about just how much of an
7:01
advantage AI can give. You know, despite all the
7:05
information that these systems bring in, despite their enormous
7:08
computational power, they're still not an all seeing eye.
7:12
You've got to keep in mind that you're dealing with a dynamic
7:15
enemy. As people always say, the enemy
7:18
gets a vote in these things. So they're going to be moving
7:21
around, they're going to be adapting, they're going to see
7:24
how, you know, US and Israeli tactics are evolving in the
7:27
battle space. They're going to be finding
7:30
vulnerabilities within those tactics, whether they're AI
7:33
related or not, and they're going to adapt.
7:36
You don't have a static enemy that will just throw its hands
7:39
up and say, Oh well, you have AI, you have these capabilities,
7:42
sort of there's nothing else we can do.
7:43
They're going to keep trying different things.
7:45
If, if I could just pile on to one comment that that Arthur
7:49
made, you know, in addition to the fog of war is currently,
7:52
it's, you know, what most folks talk about when we talk about
7:55
the mistakes, the uncertainty that military commanders have to
7:58
deal with that fog of war in the, the idea, the hope of every
8:03
military commander is that AI, among other tools will reduce
8:06
it. So it will be more, it'll be
8:08
clearer, they will be able to make better decisions.
8:10
And I think to some degree that is true.
8:13
At the same time, there is, you know, this concept that we talk
8:16
about in AI of irreducible uncertainty.
8:18
There is a point at which we cannot tell you with any
8:20
certainty what will happen. It is irreducible, and so the
8:24
you may have some probability that something may happen, but
8:27
we're not sure, and that is where human judgement is
8:30
currently maintained and must be maintained.
8:31
There will be no ability to perfectly predict the future,
8:35
regardless of the tools. There's a question of
8:37
accountability there as well. Then too, Amelia, in terms of AI
8:42
presenting a human being with a set of targets, how?
8:46
How do we know how that decision, that presentation was
8:50
reached and where is the accountability at the end of the
8:52
day? In my opinion, the
8:55
accountability question is quite clear.
8:57
It is the responsibility of the military commander who launches
8:59
the attack, and that is a part of a military ethos, certainly
9:04
in the United States, where I was trained.
9:07
The sense of accountability, a sense of responsibility.
9:10
I think that there are, I'm sure a lot of people right now
9:14
thinking very hard about what has happened in Iran and, and
9:17
how that might have happened. And they will take, you know, I
9:20
know it will affect people personally as it should in terms
9:25
of how the computer may have presented some amount of
9:28
information. You're hitting on a related, and
9:32
it's a very important issue, which is this notion of
9:34
transparency. So when you look at the result
9:37
from an AI system, do you have a sense for how it arrived at that
9:41
answer? Because that's very important to
9:43
assessing whether or not the answer is legitimate and there
9:46
are issues that have been documented with large language
9:49
models in their ability. This is the black box problem
9:52
that their inability to communicate exact or to have a
9:56
human completely understand how a decision might have been come
10:00
to a recommendation. That is an ongoing area of
10:02
research. But regardless, at the end of
10:05
the day, the commander has to take responsibility for that
10:08
action. And I think most military
10:09
officers and I think internationally that's still the
10:12
norm. That a live ethical question
10:15
though, Arthur, in this conflict, can there be human
10:19
accountability if there have been millions and millions of
10:23
computer processes ahead of human involvement in that
10:26
decision? There's often a misconception
10:28
that accountability is a binary, that there is either
10:31
accountability in a military operation or there is 0
10:35
accountability in. In fact, the process of
10:40
assigning responsibility and thus accountability for
10:44
unintended harms in warfare is a complex process.
10:48
It involves a whole structure of sort of political and legal
10:52
considerations that also take into account how that harm was
10:58
caused, whether it was caused by a tool that is perhaps easier to
11:02
understand or one that's a little more complicated or
11:05
indeed much more complicated, like AI.
11:09
The concern ethically, and for me, the discussion that's
11:12
happening around this ethically is, is really around whether AI
11:17
erodes further. What is a system of
11:20
accountability that is already at risk, a system of
11:25
accountability that already does not work perfectly.
11:29
A system where sometimes things happen and that chain of
11:35
assigning blame, if you will, sort of runs cold and, and each
11:41
erosion of that is, is is particularly a concern.
11:44
Even if for any given instance with an AI related harm, you can
11:50
say, ah, yes, this, this commander, you know, should have
11:53
known or could have done better or should have done something
11:56
differently, You know, it, it the process is just harder to
12:01
actually ascertain that. And as the use of these systems
12:05
gets closer and closer and closer to the actual, the actual
12:09
decision on the use of force, that's where I think this
12:13
question of accountability comes a lot trickier.
12:15
At the moment, AI is often participating in roles that are,
12:18
as they say, sort of upstream from the actual, you know,
12:22
button that presses, you know, the, the order to strike, if you
12:26
will, or, or pulls the, the trigger.
12:28
But when you have that person looking at a target and AI is
12:32
actually, you know, significantly influencing that
12:36
tactical decision, I think that's where you could
12:39
potentially, if it's not handled right, see a, a, a significant
12:44
erosion of, of accountability because there are just so many
12:49
technical things that can potentially go wrong in complex
12:52
ways. Let's just get on to how AI
12:55
might be being used then by the US right now in Iran, you got
13:00
Admiral Brad Cooper said recently the American military.
13:03
Humans will always make final decisions on what to shoot and
13:06
what not to shoot and when to shoot.
13:09
But advanced AI tools can turn processes that used to take
13:12
hours and sometimes even days into seconds.
13:16
So it is being used Amelia, how much so I think?
13:19
Let's go back to what Admiral Cooper said, because I think
13:22
many people don't understand how complex the targeting process
13:25
is. There is no moment where you
13:27
target, your button gets hit and it's all, it's all done.
13:29
That's at the end of a very long process.
13:32
It begins with the collection of intelligence.
13:34
And there's no, there's almost never just one source of
13:37
intelligence. There will be multiple sources.
13:38
So you'll get imagery from satellites, you'll get imagery
13:41
from drones, you'll get electromagnetic emission data.
13:46
There's lots and there's human intelligence of course as well.
13:49
Lots of different sources of data have to come together.
13:52
That data is then assembled in a way to understand what
13:55
correlates to what particular potential target.
13:59
Once you've sort of got that amount of information, which now
14:02
we're talking about that step alone includes potentially
14:05
thousands and thousands of satellite images, then you start
14:09
to work through a process of is this a legitimate target?
14:12
Is this a target that is in the priority that assigns with the
14:16
rules of engagement of the the instructions that I've been
14:18
given by my military commander? Then is it a target?
14:22
Where does it fit in my priority list?
14:24
What unit should be assigned to strike that target?
14:27
What units most appropriate, What weapons most appropriate?
14:30
Do I have enough fuel for those weapons?
14:32
Then finally you will get to a moment where something is
14:36
launched potentially or you may reassess it once it is launched,
14:41
then you have another step, which is called battle damage
14:43
assessment. So that's why when Admiral
14:45
Cooper says this could say very long process, it it is.
14:49
It's an incredibly long process. In Iraq where they were doing
14:51
strikes, 2000 members were part of the Combined Air Operations
14:55
Center responsible for doing the large volume of strikes in Iraq.
14:59
So this is very time intensive. And so you can imagine many
15:03
different points along that, you know, fairly bureaucratic
15:06
process when you might be able to apply AI in a more in a
15:11
fairly responsible way. So for example, going through
15:13
all that satellite imagery. So some of those have been done
15:16
for quite a while, looking at satellite imagery and using
15:18
computer vision. That's that's fairly standard at
15:22
this point. But there are other potential
15:24
ways, for example, to say, if this is my priority target, does
15:27
it align with my rules of engagement?
15:29
Does it align with the instructions from higher
15:31
command? Those are the sorts of ways that
15:33
you can imagine militaries are beginning to include these tools
15:38
to accelerate and shorten that targeting cycle.
15:40
Has it massively. Accelerated what is possible.
15:43
So for instance the 1st 24 hours of the war, I think the US
15:49
targeted 1000 different sites, 1042 an hour that is.
15:54
Would that have been possible without AI speeding up this
15:57
process that you just outlined? I think.
15:59
It's hard to know. Again, you can imagine that
16:04
United States has been looking at potential Iranian targets for
16:06
quite a long time. So it's not like, you know, just
16:09
earlier this month they decided that they would go look for
16:11
targets and they generated 1000. So it's hard to know exactly
16:16
what might be a related to AI of how much of that quantity.
16:19
It's certainly a lot. I, I don't think we can as yet
16:23
comment to your other point about it's evidence that it is
16:26
accelerated. I did a report on the 18th
16:29
Airborne's adoption of the Maven Smart system, which is the same
16:31
system that's being used today at Centcom, though it's it's
16:35
gone through numerous improvements.
16:37
And what we found was that the 18th Airborne was able to
16:40
achieve with the Maven Smart system tool a rate of fires that
16:44
was equivalent to what we saw in Iraq with that 2000 person crew.
16:49
They did it with 20 people. So these tools are really pretty
16:53
incredible in terms of how they are able to accelerate
16:57
operations. And that was in 2024, right?
16:59
And Arthur, I mean, do you, do you think it's, it's, it's a
17:02
problem if AI is now watching the drone footage that is coming
17:07
in and potentially identifying what it's seeing rather than
17:10
human operators? It's really the hinge point of
17:15
whether it's a problem. The the hinge point of whether
17:18
it is additive or whether it's actually going to cause more
17:22
harm is everything that happens around the AI, You know, what is
17:26
the, what is the digital infrastructure?
17:29
You know, is the, the data that that's going into the AI system
17:33
good data? Is it reliable data?
17:36
Is it truthful? Are there things in that data
17:39
that might not actually represent what the reality on
17:43
the ground or indeed is someone an adversary perhaps trying to
17:48
manipulate that data in order to get the AI to do something bad
17:53
or unpredictable on the other side?
17:56
It really comes down to the the, the, the sort of human scaffold
18:00
like the the decision scaffold that that AI is going to operate
18:06
within. So if a human is given, you
18:11
know, an alert from an AI system that says there is a, you know,
18:15
missile launcher in X coordinate and they press the button.
18:20
And this is a kind of simplification.
18:22
I mean, might roll her eyes. That's how I'm simplifying this
18:25
this process. But you know, if the the human's
18:29
response to that AI alert is to blindly press the button.
18:34
So yeah, I mean, this is a smart AI system.
18:36
This is doing millions of computations.
18:38
So I'm just going to trust it then.
18:41
Yeah, there, there, there is a potential, there's a a massive
18:44
potential there for for all sorts of things to go wrong.
18:48
If on the other hand, that same human receives the alert from
18:52
the AI system that says this is a potential site of a potential
18:57
target, and the human does their due diligence on that, it's
19:01
still saving some amount of time, right?
19:03
But they're, they're not just going to spend 20 seconds on it.
19:07
They're really going to slow down and think, OK, what's the
19:10
other sources of information? How good is the data in this
19:13
particular instance? Am I getting different
19:15
information from another source that I need to sort of reconcile
19:18
with this one? Then you know that there is a
19:21
potential that actually you can really reduce the possibility of
19:26
of of those types of, you know, targeting mistakes that are so
19:29
much concern for for everybody. Help help.
19:33
Us as lay people understand then Amelia, let let's imagine that
19:37
there is a strike that goes wrong and hundreds of civilians
19:41
die. Is it possible to strip back the
19:45
AI processes behind the human decision, or is it just all too
19:50
complex and opaque? You know, I think there's good
19:54
and there's bad here. On the one hand, you know, we,
19:57
we still do not fully understand sometimes how decisions get made
20:00
inside of the AI. On the other hand, because we
20:03
have created these AI systems, we now have a much better handle
20:06
on data. And again, you know, in the
20:09
past, passing around documents, it would be very difficult to
20:12
reconstruct where the mistake was made.
20:14
Now that all this data is in a system and that system is
20:17
tracking who made what decision exactly when in a computer way,
20:21
we are able to achieve accountability and and sort of
20:24
understand mistakes in a whole in a whole new way.
20:26
And I think that is excellent news internationally cause
20:30
Arthur. People are asking questions,
20:31
aren't they, about the school in Manab that was hit on the first
20:35
day of the war and whether there was any involvement in that
20:39
strike from the. Initial reporting from the, you
20:42
know, the, The New York Times, it seems like the, the, the, the
20:47
issue there was actually in the data source that there was some
20:51
outdated data that's a bit of a mouthful, but some outdated data
20:56
relating to the identity, you know what that, what that
21:00
structure was actually doing and what it was for.
21:03
And, you know, given the proximity to this naval base and
21:08
of course there is very little in the way of transparency as to
21:11
what role, if any, AI played in the specific set of decisions
21:18
that connected that faulty data with a human decision, which was
21:24
inevitably 1 imagines a human decision actually, yeah, fire,
21:28
fire a missile at it. So it's it's a little hard to
21:31
say beyond what you know has has been reported the.
21:36
U.S. military says it's still investigating.
21:38
Of course, Amelia. I mean, there were concerns, and
21:42
there are concerns from no less than Anthropic, which has been
21:46
in this extraordinary fight with the US government.
21:50
Yes. So first of all, I think it's
21:52
important to be clear the concerns with Anthropic happened
21:55
before the war, before the US launched against Iran.
21:59
So I have not heard any statements from them.
22:01
More recently, their concerns in in their came up in a
22:06
contractual dispute with the Department of War where they the
22:12
the distinction was between what the department said was all
22:15
lawful use and and anthropics concerns around mass
22:19
surveillance and autonomous weapons.
22:22
And I think, you know, the concerns that Anthropic phrased
22:24
are concerns that many people have.
22:28
And at the same time, you know, there's a lot it's difficult to
22:31
say that a private company should be able to dictate the
22:33
way that the United States military launches operation.
22:36
So both sides had pretty important points and Thropic's
22:40
core concern I think was 2 fold. 1 is that this technology is not
22:44
yet at a level of maturity where it should be delegated certain
22:48
decisions. And so they, in my
22:50
understanding, wanted to ensure that the department was only
22:53
deploying the systems into situations in which they were
22:56
well suited to carry out orders, to follow the commands, to align
23:00
with human intent. So I think that that notion of
23:05
the reliability of these systems was a was a big concern.
23:09
The second one is that we have always, and I think people
23:13
realize this in their everyday lives.
23:15
Our data is everywhere, whether it's advertising companies that
23:19
have it or one of the social media platforms that have it.
23:23
Data on individuals is everywhere.
23:25
It's been here for a while, but we didn't have the tools to be
23:29
able to sift through it quite so efficiently.
23:31
And so, you know, many of us felt comfortable that
23:33
information was being collected by our governments because we
23:36
knew the governments couldn't actually connect all the dots
23:38
very well for lots of bureaucratic reasons.
23:41
And I think one of the points that Anthropic is trying to make
23:43
is that's no longer so much of a problem.
23:45
It is much better these AI tools are able to connect dots in a
23:49
way that raises new questions about privacy and mass
23:52
surveillance. And so I think they're trying to
23:54
bring those those two particular really should be a legislative
23:59
discussion, a sort of more public discussion to the fore
24:02
it. Is a little bit frustrating that
24:05
the the the company by making taking this specific moral stand
24:11
and in regards to autonomous weapons, I think the
24:13
surveillance, the domestic surveillance side is sort of
24:16
somewhat separate. It may have even been a a bigger
24:19
factor in their decision. But if we just park that aside
24:22
for a second, their specific red line on autonomous weapons in a
24:27
similar way to how AI companies are constantly telling us to
24:30
focus on the the risks of super intelligence, and that is a
24:34
distraction from present day risks.
24:37
It, it is there is a risk here that in anthropic signalling
24:42
that the one crucial red line is no autonomous weapons without
24:48
human oversight. They are kind of obfuscating and
24:51
diverting attention away from some of the really very real,
24:56
very present, and perhaps a little bit more granular risks
25:00
that we see even before you get anywhere close to that line.
25:04
You know, even when you have AI supporting decisions way
25:07
upstream in the, in, in, in the, in the chain of decision
25:11
making's decision making, that leads to, to, to the use of
25:14
force. And I think Anthropic is
25:17
potentially in a position to activate a conversation about
25:20
what it would look like to have a constructive collaborative,
25:24
you know, effort involving government and private industry
25:28
around those, those low hanging fruit risks, those risks that
25:31
are happening today where there's already been a
25:33
tremendous amount of progress on them.
25:34
But there's still a lot of unresolved questions.
25:37
How you make these systems more transparent, more interpretable,
25:40
how you have human decisions on the basis of outputs that are
25:43
responsible decisions. And if you have that
25:46
conversation in a responsible and, you know, sort of
25:50
collaborative spirit where you're not drawing these
25:53
somewhat arbitrary, you know, red lines that are good for
25:56
grabbing headlines but not for much else.
26:00
I think you can really see progress in avoiding
26:03
particularly specifically the types of catastrophes that we
26:06
know are possible when you have rapid targeting campaigns, as
26:10
we've seen in the last few weeks.
26:13
So I'm falling into the trap now, am I?
26:15
Arthur, by asking you if autonomous weapons are already
26:19
within sight? I mean they're.
26:21
Very much within sight on a technological level.
26:25
If you talk about the specific ingredients that are necessary
26:29
for a weapon to look at 2 potential buildings and make a
26:34
quote, UN quote, decision as to which it's going to smash into
26:39
those technological affordances exist that you know that that,
26:43
that that's pretty well established.
26:46
What is less clear is whether the technology is at a point
26:51
where militaries would actually see any particular advantage in
26:57
handing that specific decision over to an autonomous weapon.
27:02
As we've seen in this campaign, you can do a tremendous amount
27:06
of damage in a very short amount of time with human decisions
27:08
that are assisted not only by AI but all sorts of other, you
27:12
know, digital and computerized systems.
27:15
So it's a little less clear where autonomous weapons would
27:20
fit into that, especially given the fact that AI is still not at
27:24
that 100% predictability and reliability.
27:27
So, you know, in, you know, sort of ceding that control to the
27:31
weapon, you know, ultimately you're going to do what no
27:36
commander, and I'm not a military person, I mean, they
27:38
can correct me, No commander wants to do, which is give up
27:41
control. You know, all commanders want
27:42
specific, direct, deliberate control over what they're doing.
27:46
That said, that's for this conflict and that's where things
27:50
are right now. The real concern is if you get
27:52
to a conflict where the benefits of ceding that control despite
27:58
the risks are of so tantalizing and so of such sort of strategic
28:04
and tactical importance that militaries will actually
28:07
seriously consider the possibility of deploying
28:10
weapons. I'm not saying now, but in some
28:13
future and, and now I'm, I'm really talking about the
28:15
potential of a conflict between 2:00 near PS states, for
28:18
example, two superpowers where everything's happening so
28:21
quickly. And there you might see that
28:23
calculus, that risk calculus shift and that's potentially a
28:25
big worry and that. Brings us to Russia and China.
28:30
Emilia and a sort of AI warfare arms race, yes.
28:35
So certainly this has been talked about a lot and much of
28:40
it has been at the theoretical level.
28:42
With this current conflict in Iran, we're starting to see how
28:46
AI might be used. I think it's still theoretical
28:50
largely to talk about how it would be used in a near pure
28:53
conflict. There's certainly some of the
28:55
concerns Arthur raised or the the concerns that folks might
28:57
have. I've just completed, we
29:00
completed an analysis of 9000 People's Liberation Army, the
29:04
Chinese Army's request for proposals for technical
29:08
solutions to challenges that they had identified.
29:11
And 9000 of the ones that we looked at had something to do
29:14
with AI. So not necessarily large
29:16
language models, but computer vision, natural language
29:18
processing. They are doing widespread
29:21
experimentation. They are trying every potential
29:25
use of these tools. I would tell you that, you know,
29:29
they are also using it for potential targeting for target
29:31
development much you know much like has been talked about in
29:35
Ukraine and has been talked about in with Russia and with
29:38
United States and with an Israel, but different kinds of
29:43
targeting as I understand it. The ones that I found frankly a
29:46
bit more concerning they had AI applications in space, so
29:50
orbital targeting algorithms and trying to do conduct space
29:53
warfare. And then the other one that I
29:56
find quite concerning has to do with the deployment of cognitive
29:59
operations. So this is influence or
30:02
information operations where they've developed or they're
30:05
requesting, I should say deep fake management systems.
30:08
So the ability to collect thousands of images, audio and
30:12
then construct deep fakes to target at particular individuals
30:16
as required. Those are the sorts of things
30:19
when we start to talk about near peer competitors that may not be
30:21
what you thought of. You might only think of
30:24
targeting on land, but in the near pure kinds of context you
30:28
have to start also thinking about massive influence
30:31
operations that might prevent a particular war, or what would be
30:36
truly catastrophic, which would be more offensive offensive
30:39
warfare in space warfare. In space.
30:41
Well, Amelia's already spelled it out a bit there, Arthur, but
30:44
I mean, what might a kind of AI superpower war look like?
30:50
It's not something that is a good idea to think about right
30:54
before going to bed because it's a little bit nightmarish the way
31:01
it's talked about in theoretical, in the theoretical
31:05
space. I mean, it's very, you know,
31:06
absolutely right to say that it's still very much in theory.
31:09
Is that you? Everything happens at machine
31:14
speed as it's described, you know, where the, the, the the
31:19
human is no longer able to add sort of to push the process
31:28
along that the, the human becomes the friction point, the,
31:31
the human becomes the bottleneck.
31:34
So in addition to all of the applications that Amelia has has
31:38
has referred to, there is also a a lot of interest in in what's
31:44
called mass. It's a simple word, but just
31:46
having a lot of weapons. So thousands of drones,
31:50
thousands of loitering munitions, which are these sort
31:53
of hybrid between a drone and a missile, thousands of robotic
31:58
systems. And they are able to overwhelm
32:02
the enemy not only with mass, but also their capability of
32:07
coordinating in a machine speed like fashion that no sort of
32:13
human set of commanders would necessarily be able to to match.
32:18
So, you know, just huge numbers essentially of, of robots
32:24
potentially going against each other or going against some
32:26
hybrid of, you know, humans and weapons and inhabited weapons
32:29
and uninhabited weapons across domains, land, sea, space, and
32:36
also, of course, cyberspace. And, and it's sort of a total
32:42
warfare picture multiplied by this idea of just superhuman
32:46
speed. Again, not saying you
32:48
necessarily want to think about if you're trying to have a good
32:51
night's sleep, Amelia. Just before we go, can you also
32:54
give us the best case scenario? Yeah.
32:57
Well, actually, if I could give maybe 2 glimmers of hope or, or
33:01
or two things that make folks perhaps don't think about as
33:03
often. First of all, I was the
33:06
beneficiary of an autonomous weapon system when I was in the
33:08
Navy. It's called the Aegis weapon
33:10
system. There's another one called the
33:12
close in Weapons system, and these two systems have been on
33:16
Navy ships for decades, and they can go autonomous.
33:20
They don't have to be autonomous, but you can turn
33:22
them into an autonomous mode when someone is shooting at you.
33:26
And so having sat in one of those ships and known how those
33:29
systems worked, I was very thankful to the engineers who
33:32
had constructed it such that if I was ever put in a situation, I
33:36
could turn it on and it would be my sort of my last line of
33:39
defense from an incoming missile.
33:41
So there are defensive systems that, frankly, especially if
33:44
we're talking about hypersonic missiles, have to operate at
33:47
faster than human thought. But those are defensive systems.
33:50
And so it's important to put them in that category.
33:52
The other thing that I think, you know, as I mentioned before,
33:55
the potential to use all of the data that is now available and
33:59
these AI systems to be more accountable for the decisions
34:03
that we make. I think it's incredibly
34:05
impressive what the press has been doing to try and gather
34:09
data to reveal what's happening. Nonprofits also have this
34:12
opportunity to gather the data and use these same tools to hold
34:17
people accountable, to identify civilian structures, and to try
34:21
and use what we are developing not just to wage war, but to
34:26
protect civilians when we are in the midst of a conflict.
34:29
So I would put those as two potentials that are available
34:32
and we need to seize those opportunities.
34:35
OK. Emilia Probasico, former Navy
34:37
officer, now senior fellow at the Center for Security and
34:41
Emerging Technology. Arthur Holland, Michelle,
34:43
journalist and AI researcher. Thank you both very much.
34:46
That was really insightful. That's it for the forecast.
34:49
See you next time.