Latest / THE DAILY BRIEF / OpenAI pauses top-model work after AI bypasses internet safeguards

Transcript

OpenAI pauses top-model work after AI bypasses internet safeguards · THE DAILY BRIEF

• 0:01
  Speaker 1: An open AI model was meant to be training in
• 0:03
  Speaker 1: a space isolated or cut off from the internet. Instead,
• 0:07
  Speaker 1: it found a way around the safeguards and got online.
• 0:10
  Speaker 1: As a result, open AI has now paused work involving
• 0:14
  Speaker 1: its most powerful models. But it begs the question, what
• 0:17
  Speaker 1: happens when AI starts getting around the very rules meant
• 0:21
  Speaker 1: to contain it? OpenAI says the model was not told
• 0:25
  Speaker 1: to test its limits. It was simply trying to complete
• 0:27
  Speaker 1: a task. But when the normal route failed, it started
• 0:30
  Speaker 1: looking for another way and found one. The model was
• 0:34
  Speaker 1: given a simple search task. But when its normal searches failed,
• 0:38
  Speaker 1: it looked for another way and it found a loophole
• 0:41
  Speaker 1: in DNS. That's the system that translates website names into
• 0:45
  Speaker 1: internet addresses and used it to contact an outside chatbot.
• 0:49
  Speaker 1: OpenAI detected the breach and has now paused tool use
• 0:53
  Speaker 1: involving its most capable models while it tightens the safeguards.
• 0:58
  Speaker 1: Leon Stauffer is an AI researcher working on frontier AI evaluations,
• 1:03
  Speaker 1: agent governance... and Transparency. He's a research affiliate at SAS,
• 1:07
  Speaker 1: that's a Singapore-based AI safety and governance organisation.
• 1:11
  Speaker 2: Leon, good to see you.
• 1:13
  Speaker 1: I don't want to sound flippant to say it's a
• 1:15
  Speaker 1: trend for AI developers to sound the alarm, but we've
• 1:18
  Speaker 1: heard plenty of warnings from AI researchers and executives before
• 1:22
  Speaker 1: and recently. What's different about OpenAI actually pausing work this time?
• 1:30
  Speaker 3: I think this just rose into more and more developments
• 1:33
  Speaker 3: that are alarming. And I think it is right to
• 1:35
  Speaker 3: feel slightly alarmed. I believe OpenAI has paused previously as well.
• 1:39
  Speaker 3: And this was in response to the Hugging Face incident.
• 1:41
  Speaker 3: And then they have kind of put in place more
• 1:44
  Speaker 3: safeguards and monitoring to kind of catch incidents like these,
• 1:48
  Speaker 3: where AI agents go beyond what they were supposed to do.
• 1:51
  Speaker 3: And I think now having an incident where It was
• 1:53
  Speaker 3: shown that these safeguards have failed in some way is
• 1:57
  Speaker 3: just another way, OK, to take a step back and
• 1:59
  Speaker 3: reflect and pause and maybe fix these issues, hopefully as
• 2:02
  Speaker 3: best as they can.
• 2:03
  Speaker 2: Yeah.
• 2:03
  Speaker 1: So does this incident then tell us something genuinely new
• 2:07
  Speaker 1: about the frontier or frontier AI that should be applied
• 2:10
  Speaker 1: more broadly? Or does it just expose in isolation a
• 2:14
  Speaker 1: badly configured test environment?
• 2:18
  Speaker 2: I think this is a bit of both.
• 2:20
  Speaker 3: It is kind of in principle similar to other incidents.
• 2:23
  Speaker 3: It's not more dangerous or problematic than the ones we've
• 2:25
  Speaker 3: already seen, which is not to downplay it. It is
• 2:28
  Speaker 3: kind of very problematic to have this kind of incident
• 2:30
  Speaker 3: in the first place, when AI goes beyond what it's
• 2:33
  Speaker 3: supposed to do, what OpenAI and others call misalignment, not
• 2:37
  Speaker 3: doing the intended actions. And I think seeing that OpenAI
• 2:43
  Speaker 3: has taken very careful measures to try to monitor and
• 2:45
  Speaker 3: prevent this behavior and they still don't suffice is more
• 2:50
  Speaker 3: than just a badly configured test environment. I'm sure they've
• 2:53
  Speaker 3: put a lot of effort into configuring these as best
• 2:55
  Speaker 3: they can. They noticed something was wrong within 15 minutes
• 2:59
  Speaker 3: and then a human acknowledged it within three minutes and
• 3:02
  Speaker 3: yet it still took another two hours to actually stop
• 3:05
  Speaker 3: that agent going beyond what it was supposed to do.
• 3:08
  Speaker 3: So I think beyond just a bad And I'd like
• 3:11
  Speaker 3: to say we'll always have bad test environments. It also
• 3:14
  Speaker 3: shows some operational failures that these additional two hours has passed.
• 3:18
  Speaker 3: And ironing out these is just important to get right
• 3:22
  Speaker 3: as soon as possible.
• 3:23
  Speaker 1: AI leaders have been calling for regulation, Leon, for years.
• 3:27
  Speaker 1: Elon Musk was doing it as far back as 2017.
• 3:29
  Speaker 1: Sam Altman has done it repeatedly, including at the UN.
• 3:34
  Speaker 1: Why has meaningful regulation still been so limited?
• 3:41
  Speaker 3: I think there's many reasons, and I'm not a politics expert. One,
• 3:46
  Speaker 3: of course, getting regulation right is very hard in such
• 3:50
  Speaker 3: a fast-moving field where things are changing all the time.
• 3:53
  Speaker 2: The last two weeks have been crazy.
• 3:56
  Speaker 3: And then you have these somewhat race dynamics between companies
• 4:00
  Speaker 3: in the U.S., but also potentially between companies in the U.S.
• 4:03
  Speaker 2: And China. I don't want to overplay this.
• 4:05
  Speaker 3: I don't think this is a good reason to stop
• 4:07
  Speaker 3: regulation in any way. But it definitely seems like the U.S.
• 4:10
  Speaker 3: government wants to just focus on innovation, maybe at the
• 4:14
  Speaker 3: expense of immediate safety. And then there's also just technical
• 4:18
  Speaker 3: difficulties in getting safety right. And maybe you'd want regulations
• 4:21
  Speaker 3: to push for these. That being said, I do think
• 4:23
  Speaker 3: we're seeing some great regulation coming out. The EU AI Act,
• 4:27
  Speaker 3: which is enforced since August, is, I think, very comprehensive
• 4:32
  Speaker 3: and targets many of these things. California's SB 53 and
• 4:37
  Speaker 3: the most recent kind of recommendations by Governor Newsom on
• 4:40
  Speaker 3: kind of kill switches and broader understanding of what incidents
• 4:44
  Speaker 3: should be reported. I think these are good examples of
• 4:46
  Speaker 3: regulation coming and kind of addressing these issues.
• 4:50
  Speaker 1: I know you've already stated you're not a legal expert,
• 4:52
  Speaker 1: but I'll pose to you the question you can answer
• 4:54
  Speaker 1: as best you can and also for.
• 4:56
  Speaker 2: Our audience, I think.
• 4:58
  Speaker 1: Are these AI leaders serious about getting regulated or are
• 5:01
  Speaker 1: their lawyers telling them to get in front of the
• 5:03
  Speaker 1: issue before a model gets badly out of control? Is
• 5:06
  Speaker 1: there a danger here that calls for regulation are partly self-protective,
• 5:10
  Speaker 1: getting concerns on the record before something goes badly wrong?
• 5:16
  Speaker 3: I would take these concerns very seriously. I don't think
• 5:19
  Speaker 3: the companies are overplaying them. I think they genuinely believe
• 5:23
  Speaker 3: in the harms that the models could cause. At the
• 5:25
  Speaker 3: very least, I think many of the employees at these
• 5:28
  Speaker 3: frontier AI companies who are kind of witnessing the risks.
• 5:31
  Speaker 2: Every day, they're sounding the alarm.
• 5:34
  Speaker 3: And I don't think they're purely motivated by profit incentives. So, really,
• 5:38
  Speaker 3: I think it'd be too easy to just disregard this
• 5:42
  Speaker 3: as regulatory capture. Of course, they want clarity, and clarity
• 5:46
  Speaker 3: comes in terms of regulation as well, but these are
• 5:49
  Speaker 3: real imminent risks that should be taken seriously, and I
• 5:51
  Speaker 3: think they genuinely share that belief.
• 5:53
  Speaker 1: So, when governments have actually proposed tougher rules, have the
• 5:58
  Speaker 1: big AI companies supported them or largely tried to water
• 6:03
  Speaker 1: them down?
• 6:07
  Speaker 2: Again.
• 6:08
  Speaker 3: I'm not an expert on the legal matters. I think
• 6:09
  Speaker 3: this is a mixed bag, and you see definitely U.S.
• 6:13
  Speaker 3: tech companies supporting super PACs, and some of them are pro-regulation.
• 6:18
  Speaker 2: Some have switched.
• 6:19
  Speaker 3: I think OpenAI sometimes opposed some regulation, and then when
• 6:22
  Speaker 3: it was close to passing, they endorsed it. I think
• 6:26
  Speaker 3: there's just many aspects of this that I'm not all
• 6:28
  Speaker 3: familiar with.
• 6:30
  Speaker 1: In Australia, with that in mind, the social media ban
• 6:34
  Speaker 1: on teenagers played well for the Australian Prime Minister. By
• 6:36
  Speaker 1: doing that, Australia has shown governments are prepared to impose
• 6:41
  Speaker 1: sweeping rules on technology companies. Are we getting closer to
• 6:46
  Speaker 1: that kind of intervention with AI? Is there the view
• 6:49
  Speaker 1: inside of the industry that government intervention is coming?
• 6:57
  Speaker 3: I think government intervention is definitely coming. And as I said,
• 6:59
  Speaker 3: there is already government intervention. And the EU Act is
• 7:02
  Speaker 3: in place, is in force right now. And the EU's
• 7:05
  Speaker 3: AI office can request information from labs and find them.
• 7:10
  Speaker 3: And I think there's also a wish to have clarity
• 7:13
  Speaker 3: from within the companies on what the right thresholds for
• 7:16
  Speaker 3: reporting incidents are. Right now, there's just so, so much uncertainty,
• 7:20
  Speaker 3: not just in terms of legal and government uncertainty, but
• 7:23
  Speaker 3: also technical uncertainty in how models work. And this is
• 7:27
  Speaker 3: a great reason to pause, slow down and kind of reassess.
• 7:32
  Speaker 1: Robin Brombach, the CEO of the German AI image generation
• 7:35
  Speaker 1: specialist Black Forest Labs, was quoted yesterday saying about AI,
• 7:39
  Speaker 1: the main thing that can wipe out humanity is stupidity.
• 7:44
  Speaker 1: I want to know from you, what are the regulators
• 7:46
  Speaker 1: actually trying to protect us from? Is it bad actors
• 7:50
  Speaker 1: using AI? Is it companies cutting corners on safety? the
• 7:54
  Speaker 1: systems themselves behaving in ways we can't control? Or is
• 7:58
  Speaker 1: it that stupidity and overambition?
• 8:02
  Speaker 3: I mean, regulators should protect us from all of that.
• 8:05
  Speaker 3: As you said, there's many ways AI.
• 8:08
  Speaker 2: Can be risky.
• 8:09
  Speaker 3: It can be risky when in the hands of bad
• 8:12
  Speaker 3: actors who then misuse it and can take large scale
• 8:15
  Speaker 3: cyber attacks. It can be risky insofar of just developing it.
• 8:19
  Speaker 3: And we've seen from kind of the training and evaluation
• 8:23
  Speaker 3: run from big frontier AI companies, the things do go
• 8:26
  Speaker 3: wrong and that causes harm, like in the Hugging Face incident.
• 8:29
  Speaker 3: And although there was like no human harm in that case,
• 8:32
  Speaker 3: that might look different in the future.
• 8:34
  Speaker 2: And I think maybe it's.
• 8:38
  Speaker 3: Just like not intuitively that you're going to be most
• 8:40
  Speaker 3: careful in trying to develop this new technology. And there's
• 8:43
  Speaker 3: What it means to be careful is also still unclear.
• 8:46
  Speaker 3: And so regulation can help kind of say, we want
• 8:49
  Speaker 3: incident reporting, we want some sort of kill switches and
• 8:52
  Speaker 3: information sharing to make sure that everyone is able to
• 8:55
  Speaker 3: come together to develop AI as safely as possible.
• 8:58
  Speaker 1: So you can give us a clearer view from inside
• 9:00
  Speaker 1: the industry, if you like. The big companies are clearly
• 9:03
  Speaker 1: racing to make AI more capable. Is anyone really racing
• 9:07
  Speaker 1: just as hard to make it safer? I mean.
• 9:12
  Speaker 3: As you know, I'm not within these AI companies, so
• 9:14
  Speaker 3: I can't speak on that directly. But there's many AI security,
• 9:19
  Speaker 3: safety and responsible development teams within these companies that are
• 9:23
  Speaker 3: trying their hardest and have their full commitment to making
• 9:25
  Speaker 3: sure this technology is developed safely for all of humanity.
• 9:29
  Speaker 2: I think they're definitely racing towards that as well. But
• 9:32
  Speaker 2: you'll have.
• 9:32
  Speaker 3: Different incentives within these big companies. And obviously, I don't
• 9:35
  Speaker 3: know how they square off against each other.
• 9:38
  Speaker 1: Leon Stelfer from SASH, a Singapore-based AI safety and governance organisation.
• 9:43
  Speaker 1: It's been great to mine your knowledge of AI research
• 9:46
  Speaker 1: and expertise in frontier AI evaluations. Big thanks to you
• 9:49
  Speaker 1: and your organisation for taking part in the discussion today.
• 9:54
  Speaker 2: Thank you very much, Anthony. Goodbye.
• 9:57
  Speaker 1: So we've heard from a well-qualified expert. Your turn then now.
• 10:00
  Speaker 1: What do you think? Are warnings from AI companies just
• 10:03
  Speaker 1: a way to pass the blame when something goes badly wrong?
• 10:07
  Speaker 1: We do read your comments, so please share your views.
• 10:09
  Speaker 1: Anthony Howard on behalf of our team in Berlin saying
• 10:12
  Speaker 1: bye for now.