Latest / Best of Business / Dr Kerry McInerny: Auckland University AI expert on OpenAI's artificial intelligence hacking into another company

Transcript

Dr Kerry McInerny: Auckland University AI expert on OpenAI's artificial intelligence hacking into another company · Best of Business

• 0:09
  Speaker 1: You're listening to a podcast from News Talk sed B.
• 0:12
  Speaker 1: Follow this and our wide range of podcasts now on iHeartRadio.
• 0:16
  Speaker 2: Now.
• 0:16
  Speaker 3: OPENII says a new model of chat GPT, has hacked
• 0:19
  Speaker 3: another company of its own volition. A few models, including
• 0:23
  Speaker 3: GPT five point six Soul, were being tested without some
• 0:26
  Speaker 3: of the guardrails in place, and was able to break
• 0:28
  Speaker 3: into an AI business called hugging Face. Open ai is
• 0:31
  Speaker 3: fest up and is investigating the incident. Kerrie McInerney is
• 0:34
  Speaker 3: an AI expert at Auklan University.
• 0:36
  Speaker 1: And with us.
• 0:36
  Speaker 3: Hi, Kerrey, Hi, thanks for having me. Okay, So is
• 0:40
  Speaker 3: it fair enough to say what happened was it was
• 0:42
  Speaker 3: being tested, it realized hugging Face had the detail it needed,
• 0:45
  Speaker 3: so it hacked into the Internet and then it hacked
• 0:47
  Speaker 3: into hugging Face as servers.
• 0:49
  Speaker 2: Yes, that's correct. So this is unusual in that it
• 0:51
  Speaker 2: was the first AI agent reach of a major AI
• 0:54
  Speaker 2: platform and Hugging Face I think actually suspected that it
• 0:57
  Speaker 2: was a frontier AI model before they knew that open
• 0:59
  Speaker 2: ai was responsible for what happened.
• 1:02
  Speaker 3: What do you mean by that? They thought it was
• 1:03
  Speaker 3: a frontier AI model.
• 1:06
  Speaker 2: Just a sophistication of the attack because of course it's
• 1:09
  Speaker 2: been you know, AI used for various other kinds of
• 1:12
  Speaker 2: cyber attacks, things like generating code or writing phishing emails,
• 1:16
  Speaker 2: but this used an autonomous agent system from start to finish.
• 1:20
  Speaker 3: Are you alarmed by this?
• 1:23
  Speaker 2: Look? I'm alarmed by it and that yes, it does
• 1:25
  Speaker 2: raise these kind of predicted issues around cybersecurity that I
• 1:29
  Speaker 2: think people have been really worried about. But at the
• 1:31
  Speaker 2: same time, you know, my fear is that we focus
• 1:33
  Speaker 2: a lot on the capacities of the agents, on the
• 1:36
  Speaker 2: AI itself, and that distracts away from the people who
• 1:39
  Speaker 2: I think we really should be holding responsible, which is
• 1:42
  Speaker 2: open ai itself. And I think you even see that
• 1:44
  Speaker 2: in some of the language around the incident, that you know,
• 1:47
  Speaker 2: open ai and its employees are very key to kind
• 1:49
  Speaker 2: of direct these conversations towards the acts of autonomous agent
• 1:53
  Speaker 2: or AI misalignment rather than corporate responsibility.
• 1:58
  Speaker 3: What do you mean by that? What is the problem
• 2:00
  Speaker 3: with What is the corporate responsibility chat that we need
• 2:02
  Speaker 3: to have?
• 2:04
  Speaker 2: So, you know, I think companies like open ai and
• 2:07
  Speaker 2: their main competitors Slashes spin off from open ai Andthropic
• 2:10
  Speaker 2: built their reputation on the idea that they were going
• 2:13
  Speaker 2: to create you know, highly powered AI tools safely, and
• 2:16
  Speaker 2: that safety and humanity was at the heart of their mission.
• 2:20
  Speaker 2: But I think over the years we've just seen that mission,
• 2:23
  Speaker 2: that big claim not come true, and instead we're seeing
• 2:26
  Speaker 2: more and more incidents like this one where a tool
• 2:29
  Speaker 2: tested by open Ai causes meaningful harm and attacks another
• 2:32
  Speaker 2: company and causes a major cybersecurity incident. And so, you know,
• 2:37
  Speaker 2: the conversation I think we should be having is, you know,
• 2:39
  Speaker 2: are these firms as trustworthy as they initially said, and
• 2:42
  Speaker 2: do we want them to be sort of holding so
• 2:44
  Speaker 2: much power over these really influential tools.
• 2:47
  Speaker 3: Well, what alternative do we have, Because isn't the genie
• 2:50
  Speaker 3: out of the bottle once you've designed something as powerful?
• 2:53
  Speaker 3: Is that it just simply is as powerful as that?
• 2:56
  Speaker 2: Yeah, I mean, I think the question here shouldn't be
• 2:59
  Speaker 2: putting the genie back into the bottle. And I think
• 3:01
  Speaker 2: this is often an argument people make to suggest that
• 3:03
  Speaker 2: people who want, you know, a different kind of future
• 3:06
  Speaker 2: for AI make to suggest that, you know, to ask
• 3:10
  Speaker 2: for anything different would be luddyte or retrograde. But I
• 3:13
  Speaker 2: think that there's different ways we could be designing and
• 3:15
  Speaker 2: regulating these models to make them safer for everyone. You know,
• 3:18
  Speaker 2: what's really interesting about this case is that companies like
• 3:22
  Speaker 2: open ai are really keen to avoid or undermine open
• 3:26
  Speaker 2: weight or open source models, models that people can easily
• 3:29
  Speaker 2: download and amend themselves, to protect their own sort of
• 3:32
  Speaker 2: market position to be the leaders in this space. But ironically,
• 3:37
  Speaker 2: despite all their advertising previously that open source is really dangerous,
• 3:42
  Speaker 2: the company hugging Face actually ended up stopping the cyber
• 3:45
  Speaker 2: attack through the use of a Chinese open weight model.
• 3:48
  Speaker 2: And so I think what it really shows is that
• 3:50
  Speaker 2: we should be very careful about letting our AI regulation
• 3:54
  Speaker 2: and policy and governance be led by a very narrow
• 3:57
  Speaker 2: hand of actors like open ai or Anthropic who you know,
• 4:01
  Speaker 2: ultimately I think, do just have their best interests at heart.
• 4:04
  Speaker 3: Okay, now, given that there is this this happened, and
• 4:07
  Speaker 3: then obviously there's a search are using the same model
• 4:09
  Speaker 3: today that we find out about to hack into WordPress,
• 4:11
  Speaker 3: which means forty percent of the Internet. Basically, would you
• 4:14
  Speaker 3: look at those two things and think this is too
• 4:15
  Speaker 3: dangerous to release publicly or no?
• 4:19
  Speaker 2: I mean I think that with all these questions right,
• 4:22
  Speaker 2: we have to say who wins and who loses, Like
• 4:25
  Speaker 2: I think there's very obvious losers when we have these
• 4:27
  Speaker 2: powerful models released without sufficient governance and oversight. Right, it's
• 4:32
  Speaker 2: all the people running their websites on word Press who
• 4:34
  Speaker 2: now have to scramble to try and find meaningful cybersecurity measures.
• 4:38
  Speaker 2: And then the winners, I think, right are companies like
• 4:40
  Speaker 2: chat sorry, like open ai and like Anthropic who not
• 4:43
  Speaker 2: only have their own models being used, but then can
• 4:45
  Speaker 2: also start to market cybersecurity solutions. And you know, this
• 4:49
  Speaker 2: was an incredibly great piece of marketing for Aura, the
• 4:51
  Speaker 2: information cybersecurity company who did the research, and so you know,
• 4:56
  Speaker 2: I completely understand companies need to put their best foot
• 4:59
  Speaker 2: forward and advertise their services, and cybersecurity is obviously a
• 5:02
  Speaker 2: great thing, But I do worry about the implications of
• 5:05
  Speaker 2: running a study like that and then showing the world
• 5:08
  Speaker 2: how vulnerable all those websites are in one go. You know,
• 5:11
  Speaker 2: will all those businesses be able to cope with the
• 5:13
  Speaker 2: potential pressure that you know, potential scammers or hackers could
• 5:17
  Speaker 2: now you know, leverage that system to attack their websites.
• 5:21
  Speaker 3: Well, they probably knew it already, Kerry, thank you very
• 5:23
  Speaker 3: much appreciate. Kerrie mchaninny Auckland University AI experts.
• 5:27
  Speaker 1: For more from News Talks, ed b Listen live on
• 5:30
  Speaker 1: air or online, and keep our shows with you wherever
• 5:33
  Speaker 1: you go with our podcasts on iHeartRadio