Latest / The Morning Edition / OpenAI's hack of an Australian government website

Transcript

OpenAI's hack of an Australian government website · The Morning Edition

• 0:04
  S1: Prime Minister Anthony Albanese has launched a task force to
• 0:08
  S1: investigate how AI apparently went rogue and hacked into one
• 0:12
  S1: of Australia's major health care systems. Albanese revealed on Thursday
• 0:16
  S1: that an open AI agent infiltrated Medicare all the way
• 0:20
  S1: back in June. I'm Samantha Salinger Morris, and you're listening
• 0:24
  S1: to The Morning Edition from The Age and The Sydney
• 0:27
  S1: Morning Herald today, technology editor David Swan on how this
• 0:31
  S1: breach occurred and how worried we should be. David, welcome
• 0:38
  S1: back to the podcast.
• 0:40
  S2: Thanks for having me.
• 0:40
  S1: Let's start off with the basics. What happened?
• 0:43
  S2: So back in June, open AI was testing one of
• 0:46
  S2: its AI agents, and it was given a pretty boring job.
• 0:49
  S2: Just go and research public spending on medicines in Australia.
• 0:53
  S2: It then went to a government website, the Medicare Statistics
• 0:56
  S2: Reporting Service, which is not just the everyday portal that
• 1:00
  S2: you and I would log into. It's for researchers who
• 1:03
  S2: use it for data on Medicare spending. The website said no,
• 1:07
  S2: it blocked it and the agent didn't take no for
• 1:10
  S2: an answer. It found a way around the website's defenses,
• 1:13
  S2: and it got into parts of the system that the
• 1:15
  S2: public wasn't meant to see. The government didn't notice. Nobody
• 1:18
  S2: in the government saw that this happened. They didn't see
• 1:21
  S2: anything out of the ordinary. Open AI found this in August.
• 1:25
  S2: So months later, when it went back through its own
• 1:27
  S2: logs and it told the government on September 10th, a
• 1:30
  S2: couple of weeks ago now by email to an inbox
• 1:33
  S2: that gets checked once a day.
• 1:35
  S1: Okay. And so what was actually accessed by this open
• 1:38
  S1: AI agent? You know, did it access patient records of Australians? Like,
• 1:41
  S1: what did it get at?
• 1:42
  S2: No. And it's I think it's important to be really
• 1:44
  S2: clear about that. So, um, because we've had data breaches
• 1:47
  S2: before that have impacted that sort of data. So in
• 1:51
  S2: this case, no personal medical records, no Medicare claims, nothing
• 1:54
  S2: about individuals. So just really important to make that clear.
• 1:58
  S2: It got aggregate statistics, big picture numbers and things that
• 2:02
  S2: researchers use. But the portal wasn't connected to anyone's Medicare account. Um,
• 2:07
  S2: it's kind of an old website that the government today
• 2:09
  S2: has said has since been switched off, the way that
• 2:12
  S2: Richard Marles put it today. Uh, he said that this
• 2:14
  S2: data sat behind a fence.
• 2:16
  S3: Is your question is AI terrifying? Okay. Um.
• 2:21
  S4: Okay. Now this website didn't have a lot of information,
• 2:24
  S4: but what's next? Defense. Very secure national security experts.
• 2:29
  S3: So let me speak firstly to the the, the security
• 2:32
  S3: of government systems.
• 2:33
  S2: And then he said, your personal data sits in a
• 2:35
  S2: safe and national security sits behind a fortress. So this
• 2:40
  S2: is a fence that the agent climbed over.
• 2:42
  S3: In this instance. It's not particularly sensitive information. In fact,
• 2:45
  S3: the information has now been made public. And so the
• 2:48
  S3: security in place there was not the most significant security.
• 2:51
  S3: And that's reasonable.
• 2:53
  S2: So your medical records are all fine. But the thing
• 2:56
  S2: that climbed the fence was an agent, not a person,
• 2:59
  S2: and no one asked it to.
• 3:01
  S1: Okay, well, let's get into how did this happen? Was
• 3:04
  S1: it human error? Is this some sort of rogue agent?
• 3:06
  S1: What was going on, David?
• 3:08
  S2: Yeah, it's kind of. I've been asking that question today,
• 3:10
  S2: and I think a lot of people have. And there's
• 3:12
  S2: no clear answer. It's sort of somewhere in between, which
• 3:14
  S2: is what makes it super interesting. The human set the
• 3:18
  S2: task for the AI. So there was a human at
• 3:21
  S2: the start of it and the human said, go and
• 3:24
  S2: research medicine spending, which is like a relatively normal thing
• 3:28
  S2: to ask an AI to do. But then it was
• 3:30
  S2: up to the AI how the task actually got done.
• 3:33
  S2: So unlike a chatbot where you might ask a question
• 3:36
  S2: and you get an answer, the agent is given a
• 3:38
  S2: goal and then it's up to it how to get there.
• 3:42
  S2: It'll click around, it'll search, it'll try bits of code
• 3:45
  S2: and it'll keep trying. And then, um, when something fails,
• 3:49
  S2: it'll try again until it even gets to hacking, which
• 3:52
  S2: is what happened here. I think it's important to clarify too.
• 3:56
  S2: This isn't necessarily rogue AI, it's just AI doing its
• 3:59
  S2: job really well, too well. The way it was described
• 4:02
  S2: to me today was like, it's a super intelligent five
• 4:05
  S2: year old. It just goes and tries stuff. Um, and
• 4:10
  S2: it's up to us as humans to be really clear
• 4:12
  S2: about what it should and shouldn't do, I think. Um,
• 4:15
  S2: so basically it was a really good worker that just
• 4:19
  S2: wanted to do its homework.
• 4:20
  S1: Okay. Now, you said just before that this AI agent,
• 4:24
  S1: it jumped a fence. It didn't jump a fortress, but
• 4:26
  S1: is this still something of a warning shot for us?
• 4:29
  S1: Because it was only in July that many listeners will remember.
• 4:32
  S1: A swarm of autonomous AI agents developed by OpenAI broke
• 4:36
  S1: out of their so-called sort of sandbox closed environment and
• 4:40
  S1: breached the infrastructure of a very famous open source platform
• 4:43
  S1: called Hugging Face. And then OpenAI itself called that incident, yes,
• 4:48
  S1: a warning shot for the company and the rest of
• 4:50
  S1: the world. And then afterwards, we saw another researcher from
• 4:53
  S1: another AI company, anthropic. This was, uh, Jacob Coxson. He
• 4:57
  S1: quit the company, sounding the alarm of what he called
• 5:00
  S1: out of control artificial intelligence. So how bad is this?
• 5:04
  S2: Um. It's bad. I was in San Francisco last week.
• 5:09
  S2: I traveled with a company called Salesforce, and they held
• 5:12
  S2: a conference called Dreamforce, which has about 5000 people or
• 5:16
  S2: talking about AI. And Sam Altman was there on stage.
• 5:20
  S2: He called the Hugging Face incident. He called it terrifying
• 5:23
  S2: and the worst accident that he'd seen. He didn't mention Australia,
• 5:26
  S2: even though he would have known by then that this
• 5:29
  S2: incident had happened. I think it's important to note as well.
• 5:34
  S2: A lot of people have been cynical about these announcements,
• 5:37
  S2: and I think that's fair to. It's a fair reading
• 5:41
  S2: to assume that at least a little bit of this, uh,
• 5:45
  S2: companies that have billions of dollars at stake. OpenAI is
• 5:49
  S2: preparing for a big IPO. So is anthropic, and there's
• 5:53
  S2: billions of dollars on the line. So it's in some
• 5:56
  S2: ways in these companies interests to to disclose that. Look
• 6:00
  S2: how amazing our AI is. Look at what it can do.
• 6:03
  S2: And also, you need to pay us to buy our
• 6:06
  S2: cyber defenses to protect against these models. So I think
• 6:10
  S2: there's a healthy level of cynicism to be had here
• 6:12
  S2: as well. But yes, it's a warning shot. It does
• 6:16
  S2: feel like This incident is the first that we know
• 6:20
  S2: of of an AI agent autonomously attacking a government website. Globally,
• 6:27
  S2: I think this is the first, but it feels like
• 6:30
  S2: the type of story that I'm going to be writing
• 6:33
  S2: dozens of times, if not hundreds of times, over the
• 6:36
  S2: coming months and years. This is going to become much,
• 6:38
  S2: much more common and much probably worse in terms of
• 6:43
  S2: there weren't really any adverse outcomes here that we know of.
• 6:46
  S2: You know, it's not patient records, things like that. So, um,
• 6:49
  S2: it feels like, yeah, the start of a bit of
• 6:52
  S2: an avalanche.
• 6:53
  S1: Okay. Well, this is what I wanted to ask you, like,
• 6:55
  S1: what is your opinion on how worrying this is? Obviously,
• 6:58
  S1: you write a lot in this space. And, you know,
• 7:00
  S1: we saw that Deputy Prime Minister Richard Marles called it, quote,
• 7:03
  S1: relatively minor, but it feels scarier than that. Right? Given
• 7:06
  S1: what's happened in other countries, uh, or in other places
• 7:10
  S1: with AI break ins and hacks and, and what it
• 7:13
  S1: could possibly lead to.
• 7:14
  S2: It feels very much like we're out of control, that
• 7:19
  S2: this is out of control and that everything hasn't caught up.
• 7:24
  S2: And everyone I speak to, whether that's people on the
• 7:28
  S2: ground last week in San Francisco who are helping build
• 7:31
  S2: this stuff, or today I spoke to Alastair MacGibbon, who
• 7:36
  S2: was in charge of cyber security for Australia under Malcolm Turnbull.
• 7:40
  S2: And he said in his whole career, in 25 years,
• 7:44
  S2: he's always felt like we've been behind when it comes
• 7:47
  S2: to cyber security, but not like this. And last week
• 7:52
  S2: in San Francisco, the people building this stuff all felt
• 7:56
  S2: very much like it's time for a slow down that all.
• 8:02
  S2: And it's the question mark becomes, how do you actually
• 8:04
  S2: do this? But a lot of the labs, whether it's OpenAI,
• 8:07
  S2: anthropic and others have said we need a coordinated effort
• 8:12
  S2: to slow down. They call it pacing the frontier, um, deliberately, uh,
• 8:20
  S2: slow the speed at which we're building. Because whether it's cybersecurity, um,
• 8:27
  S2: the economy, um, almost every aspect you think of, of
• 8:31
  S2: things that touch AI, the AI is moving faster than
• 8:34
  S2: we can keep up. Um, and I think today we
• 8:36
  S2: saw one of the clearest examples of that yet where
• 8:40
  S2: we have these old systems, um, old rules in place,
• 8:44
  S2: none of it's ready for this stuff. And collectively we're
• 8:47
  S2: not ready. And um, it's fun to write about and
• 8:50
  S2: to talk about, but it's pretty scary as well.
• 8:52
  S1: It absolutely is. Well, thank you so much, David, for
• 8:55
  S1: your time.
• 8:56
  S2: Yeah. Thanks for having me.
• 9:02
  S1: Today's episode was produced by Tammy Mills, and our podcasts
• 9:06
  S1: are overseen by Lisa Muxworthy and Tom McKendrick. If you
• 9:10
  S1: like our show, follow The Morning Edition and leave a
• 9:13
  S1: review for us on Apple or Spotify. Thanks for listening.