Latest / DEEP INSIGHT / Chinese AI tool told researchers how to make bioweapons

Transcript

Chinese AI tool told researchers how to make bioweapons · DEEP INSIGHT

• 0:00
  Speaker 1: Another major AI company has revealed major safety concerns about
• 0:04
  Speaker 1: its product. This time it's the Chinese company Moonshot. It
• 0:07
  Speaker 1: said it's conducting an internal review after researchers were able
• 0:11
  Speaker 1: to persuade two of its popular Kimi models to tell
• 0:14
  Speaker 1: them how to make biological weapons and carry out assassinations.
• 0:19
  Speaker 1: Peter Garrahan, founder of MindGuard, which tests the security of
• 0:22
  Speaker 1: AI systems, talked me through what his researchers found.
• 0:26
  Speaker 2: Through communicating with the models, it was able to successfully
• 0:30
  Speaker 2: jailbreak the model, which means you unshackle it into the
• 0:34
  Speaker 2: safety controls. And it talked about topics of weaponization, assassination,
• 0:39
  Speaker 2: but also could be used to stage cyber attacks.
• 0:42
  Speaker 1: And this really shouldn't have been possible, should it? Exactly.
• 0:46
  Speaker 2: These companies say in marketing and online, lots of safety.
• 0:52
  Speaker 2: We know technically they spend a lot of time in
• 0:54
  Speaker 2: this as well. These things are designed to be safe
• 0:56
  Speaker 2: and secure. However, time and time again, it's found seemingly
• 1:00
  Speaker 2: every week there are weaknesses in these models.
• 1:02
  Speaker 1: As I understand it, not only were they able to
• 1:05
  Speaker 1: suggest those ways of making the weapons and how to
• 1:08
  Speaker 1: carry out assassinations, they also then sort of suggest other
• 1:11
  Speaker 1: nefarious deeds you could do, don't they? Exactly.
• 1:14
  Speaker 2: And this goes just beyond topics of talking about things
• 1:17
  Speaker 2: and giving you recipes. These models, we found, can connect
• 1:22
  Speaker 2: to the Internet and run any type of software you want.
• 1:26
  Speaker 2: This is a recipe for actually causing cyber attacks against
• 1:29
  Speaker 2: different users and different people, which is really concerning.
• 1:31
  Speaker 1: Yeah, and we've seen various incidents recently, haven't we? Talk
• 1:34
  Speaker 1: us through, though, the difference between these jailbreaks that you've
• 1:38
  Speaker 1: been talking about and AI's hacking online systems, which we've
• 1:41
  Speaker 1: seen other instances of recently. Right.
• 1:44
  Speaker 2: So fundamentally, it's still the same technology. It takes language in,
• 1:50
  Speaker 2: breaks into numbers and gives you outputs. And every court
• 1:53
  Speaker 2: is getting more capable. So jailbreaking is it's either in
• 1:57
  Speaker 2: the data it's been trained upon or it makes connections
• 2:00
  Speaker 2: and starts talking about these things. So jailbreaks talk about
• 2:04
  Speaker 2: topics that it's been designed not to do. In cyber attacks,
• 2:08
  Speaker 2: it's the same principle, which is it's more capable. It's
• 2:11
  Speaker 2: been trained on cybersecurity hacking and it uses that then
• 2:16
  Speaker 2: to break into systems as well. And it's getting better
• 2:18
  Speaker 2: every single time.
• 2:19
  Speaker 1: So you say this is concerning. Just how concerning do
• 2:22
  Speaker 1: you think it is, this jailbreak?
• 2:25
  Speaker 2: It's highly concerning, not just beyond the topics it talks about.
• 2:29
  Speaker 2: That happens for two reasons. One is because in the
• 2:31
  Speaker 2: training data, it may or may not contain that information.
• 2:35
  Speaker 2: But these models are very good at finding connections between
• 2:38
  Speaker 2: different topics. To give an example, if it's been trained
• 2:41
  Speaker 2: upon murder mystery and medical journals, it could put them
• 2:47
  Speaker 2: together and actually synthesize really horrible topics and different things.
• 2:51
  Speaker 2: The one that really concerns a lot of us is cybersecurity.
• 2:54
  Speaker 2: This is what we do at MindGuard. It's getting better
• 2:58
  Speaker 2: at connecting the dots of how do you hack systems.
• 3:01
  Speaker 2: I know there's lots of stuff in the news right
• 3:03
  Speaker 2: now about the AI companies falling short in development, but
• 3:08
  Speaker 2: any security ritual will tell you in the space every
• 3:10
  Speaker 2: quarter it's getting closer and closer to do this thing automatically.
• 3:13
  Speaker 1: We had this meeting at the White House with President
• 3:15
  Speaker 1: Trump and all of these tech boss men. Talk us
• 3:20
  Speaker 1: through your reaction to this agreement that they've come to,
• 3:22
  Speaker 1: which he says is morally binding, that they're going to
• 3:24
  Speaker 1: police themselves.
• 3:27
  Speaker 2: It's difficult. I think there's very much the boy who
• 3:30
  Speaker 2: cried wolf in these cases because they've been hammering on
• 3:34
  Speaker 2: for so many quarters now about how dangerous their AI is.
• 3:37
  Speaker 2: And then they turn around and their AI automatically hacks
• 3:39
  Speaker 2: their customers. So I think there definitely is a good
• 3:43
  Speaker 2: message there in terms of, yes, we should be safe
• 3:45
  Speaker 2: to regulation. But everyone knows a lot of them have
• 3:49
  Speaker 2: IPOs coming up. They are marketing any news as good news.
• 3:53
  Speaker 2: Another come out about this. So beyond the actual surface
• 3:56
  Speaker 2: level discussion about, yes, we should be safe about these things.
• 4:00
  Speaker 2: There's also a regulation angle in terms of there's lots
• 4:03
  Speaker 2: of models now that are cheap and available free. It
• 4:06
  Speaker 2: really fronts that business model as well.
• 4:08
  Speaker 1: That's Peter Garaghan, founder of MindGuard, whose researchers found that
• 4:12
  Speaker 1: jailbreak at the AI firm Moonshot.