Latest / INSIGHT TODAY / Why scientists say AI poses an extinction-level threat to humanity

Transcript

Why scientists say AI poses an extinction-level threat to humanity · INSIGHT TODAY

• 0:00
  Speaker 1: Artificial intelligence has become part of everyday life for a
• 0:03
  Speaker 1: lot of people, but could it soon have the ability
• 0:06
  Speaker 1: to wipe out humanity? According to a growing chorus of
• 0:09
  Speaker 1: AI insiders, the answer is yes. One of them resigned
• 0:13
  Speaker 1: this week from the U.S. AI firm Anthropic with a warning.
• 0:17
  Speaker 1: Jacob Coxon's post on X, it went viral. This is
• 0:20
  Speaker 1: what he said. The people building AI earnestly believe that
• 0:24
  Speaker 1: it could kill us all by the end of the decade.
• 0:27
  Speaker 1: This is not a marketing stunt. If anything, many executives
• 0:30
  Speaker 1: and senior researchers will couch their phrasing in the press
• 0:34
  Speaker 1: to sound sensible, but I hear the same people express
• 0:38
  Speaker 1: fear privately. No other human activity poses this level of danger.
• 0:44
  Speaker 1: Coxon is not the only one sounding the alarm. Several
• 0:47
  Speaker 1: other AI researchers have jumped in to support him, including
• 0:51
  Speaker 1: anthropic safety researcher Evan Hubinger. Hubinger put the risk of
• 0:55
  Speaker 1: AI killing all humans within the next decade at greater
• 0:59
  Speaker 1: than 10%. He explained, I think the risk from present
• 1:05
  Speaker 1: models is low. What I am worried about is superintelligence
• 1:09
  Speaker 1: arising from recursive self-improvement, as we have said, is happening
• 1:15
  Speaker 1: faster than we thought. So how seriously should we be
• 1:20
  Speaker 1: taking these warnings? That's what I want to find out
• 1:22
  Speaker 1: now from Sana Zakaria. She's Director of Emerging Technologies and
• 1:26
  Speaker 1: Resilience at RAND Europe. That's a not-for-profit research organization. Sana,
• 1:31
  Speaker 1: it's good to have you with us. One of the
• 1:33
  Speaker 1: AI researchers there wrote that he is most concerned about
• 1:37
  Speaker 1: super intelligence arising from recursive self-improvement. Can you explain that
• 1:44
  Speaker 1: in simple language. What does that mean and why is
• 1:46
  Speaker 1: it potentially so dangerous?
• 1:50
  Speaker 2: Yeah, absolutely. So recursive self-improvement is the idea that an
• 1:55
  Speaker 2: AI system could help improve the next generation of AI.
• 1:58
  Speaker 2: So whether that is its code, its research process, or
• 2:03
  Speaker 2: how it uses compute resources. And the concern that this
• 2:07
  Speaker 2: could create is that it could have a feedback loop
• 2:10
  Speaker 2: where a more capable system helps build an even more
• 2:13
  Speaker 2: capable one potentially faster than basically people can properly test
• 2:18
  Speaker 2: or understand or even govern that. I think the concern
• 2:22
  Speaker 2: here is legitimate, but I would say that we have
• 2:25
  Speaker 2: not seen that full process of independent, autonomous, more capable
• 2:33
  Speaker 2: system that is able to gain and accelerate the next
• 2:37
  Speaker 2: round in that way. But I would say that we
• 2:40
  Speaker 2: are seeing perhaps pieces of that hypothesis coming to fruition
• 2:44
  Speaker 2: where increasingly autonomous systems can use tools, pursue complex tasks,
• 2:50
  Speaker 2: and sometimes behave in unintended ways. So I would say
• 2:53
  Speaker 2: it's a serious future risk. It's not an established fact
• 2:57
  Speaker 2: about today's systems just yet.
• 2:59
  Speaker 1: And there are warnings out now that AI could wipe
• 3:02
  Speaker 1: out humanity within a decade. How could that actually be done?
• 3:07
  Speaker 1: I mean, what sort of scenarios are plausible with what
• 3:10
  Speaker 1: we know right now?
• 3:13
  Speaker 2: So I think people worry about extinction scenarios because of
• 3:17
  Speaker 2: what we call loss of control scenario. Primarily the concern
• 3:20
  Speaker 2: here is that a future AI system could become highly capable,
• 3:25
  Speaker 2: really operate autonomously and gain access to our tools, our infrastructure.
• 3:30
  Speaker 2: And perhaps we have given it poorly specified goals to
• 3:35
  Speaker 2: pursue in ways that people cannot reverse. And so to
• 3:38
  Speaker 2: reach an extinction level outcome, you would have to have
• 3:41
  Speaker 2: many failures accumulate and combine. So that would be extreme capability,
• 3:46
  Speaker 2: real world access, weak safeguards, failure to detect that something
• 3:51
  Speaker 2: has gone wrong, and failure of governments and companies to respond. Now,
• 3:54
  Speaker 2: that is possible enough to plan for. It doesn't need
• 3:57
  Speaker 2: to be inevitable. And so current systems don't have that
• 4:00
  Speaker 2: capability to cause that kind of loss of control autonomously.
• 4:05
  Speaker 2: But our choices about deployment, testing, regulation can still materially
• 4:09
  Speaker 2: shape that outcome.
• 4:11
  Speaker 1: U.S. President Trump, he was asked whether he's concerned that
• 4:14
  Speaker 1: AI could lead to human extinction. I want you to
• 4:17
  Speaker 1: take a listen to what he said.
• 4:23
  Speaker 3: No, I don't have any. I have concerns that if
• 4:26
  Speaker 3: we don't win AI, we're going to be put in
• 4:29
  Speaker 3: a very bad position. We are leading China right now
• 4:32
  Speaker 3: by a pretty good myriad, I would say a year,
• 4:36
  Speaker 3: which is considered a lot.
• 4:39
  Speaker 2: A year.
• 4:39
  Speaker 1: So how much is the development of AI capabilities a
• 4:42
  Speaker 1: weapons race of sorts between the U.S. and China?
• 4:47
  Speaker 2: I mean, look, there are clear elements of strategic competition here, right?
• 4:51
  Speaker 2: The U.S. and China, as well as many leading economies
• 4:55
  Speaker 2: in the rest of the world, they see advanced AI
• 4:57
  Speaker 2: as important for economic productivity, scientific leadership, cyber capabilities and
• 5:03
  Speaker 2: national security. But I think describing it as a weapons
• 5:07
  Speaker 2: race is too narrow. I think it's also competition between companies,
• 5:11
  Speaker 2: between supply chain universities. I think the danger here is
• 5:16
  Speaker 2: that this kind of competitive dynamics rewards speed and deployment,
• 5:21
  Speaker 2: while safety testing, transparency restraint are treated as costs to
• 5:26
  Speaker 2: that economic advantage. And I think that's why the international
• 5:30
  Speaker 2: channels for cooperation remain necessary, even in this kind of
• 5:35
  Speaker 2: competitive race dynamic environment.
• 5:37
  Speaker 1: And considering how AI works, how AI models, I guess,
• 5:42
  Speaker 1: can teach other AI models, is it actually possible then
• 5:46
  Speaker 1: for one country to win the AI race, if you will?
• 5:53
  Speaker 2: I mean, I think the way people talk about winning
• 5:57
  Speaker 2: an AI race is which country has the capability and
• 6:02
  Speaker 2: who are the developers and who are the users, right?
• 6:05
  Speaker 2: Because if you are a country where you are utilizing
• 6:08
  Speaker 2: a different country's capability and you're reliant on that and
• 6:13
  Speaker 2: you are deploying it on your sensitive data sets or
• 6:16
  Speaker 2: your national security infrastructure, that raises questions around reliability and
• 6:23
  Speaker 2: that reliance comes with certain geopolitical challenges and sensitivities. So
• 6:28
  Speaker 2: I think it's about everybody wanting to own a piece
• 6:31
  Speaker 2: of it in a way where they might have leverage
• 6:34
  Speaker 2: over something else. I think it would be hard to
• 6:38
  Speaker 2: treat it as a winning scenario for anyone, really. It
• 6:41
  Speaker 2: really depends on your perspective. But I think it's an
• 6:44
  Speaker 2: economic question. I think there are definitely those that might
• 6:48
  Speaker 2: have strategic advantage if they were able to progress to
• 6:54
  Speaker 2: their goals. But I think ultimately, if we lose control
• 6:57
  Speaker 2: of the situation and really progress forward, advancement's sake, then
• 7:01
  Speaker 2: I don't really think there's going to be any winners here.
• 7:04
  Speaker 1: Yeah, yeah, it's a very good point. SpaceX CEO Elon Musk,
• 7:08
  Speaker 1: he has dismissed the concerns being expressed about AI as
• 7:13
  Speaker 1: a very sophisticated PSYOP designed to get support for AI regulation.
• 7:19
  Speaker 1: What do you think about that? I mean, could that
• 7:20
  Speaker 1: be what's going on here? I know that the founder
• 7:22
  Speaker 1: of Anthropic, he's a vocal proponent of AI regulation.
• 7:29
  Speaker 2: So I think it's, I think it's reasonable to scrutinize
• 7:32
  Speaker 2: any sort of incentives behind any public claims made by
• 7:37
  Speaker 2: whether companies, researchers, governments, because there are always commercial or
• 7:41
  Speaker 2: political incentives that might be there. But I think calling
• 7:45
  Speaker 2: warnings aside without evidence is probably not a useful analysis.
• 7:49
  Speaker 2: I think the better question is whether the claims are
• 7:51
  Speaker 2: supported by evidence, whether they can be independently scrutinized and
• 7:56
  Speaker 2: whether companies are actually improving their safeguards as a result.
• 8:00
  Speaker 2: And so these are real AI safety and misuse issues. Today,
• 8:03
  Speaker 2: we ourselves have published a lot of analysis on this
• 8:06
  Speaker 2: with a lot of credible evidence. So people can reasonably
• 8:10
  Speaker 2: disagree about the probability of, you know, extreme future risks, perhaps, without,
• 8:15
  Speaker 2: I think, dismissing all of this as a public relations exercise.
• 8:19
  Speaker 1: Sonia, can you point then to serious efforts that are
• 8:24
  Speaker 1: underway right now to regulate AI?
• 8:28
  Speaker 2: I think there's a lot of– there's some top-down and
• 8:32
  Speaker 2: ground-up momentum in this space. I think the clear example
• 8:35
  Speaker 2: I can point to, given where I'm situated, is the
• 8:39
  Speaker 2: EU AI Act that has been put in place and
• 8:43
  Speaker 2: the Code of Practice. And with the EU AI Office
• 8:47
  Speaker 2: making a really monumental– undertaking a really monumental task here
• 8:52
  Speaker 2: in holding companies to account and companies– sort of, you know,
• 8:56
  Speaker 2: abiding by rules and safety regulation and really coming to
• 9:00
  Speaker 2: grips with what is it that we need to demonstrate
• 9:03
  Speaker 2: and how to be able to deploy and utilize our products. Okay.
• 9:08
  Speaker 1: Sana Zakaria with RAND Europe. Sana, we appreciate your time
• 9:11
  Speaker 1: and your insights on this. We certainly haven't heard the
• 9:14
  Speaker 1: last in this topic, that's for sure. Thank you.
• 9:18
  Speaker 2: You're welcome.