Latest / In the Loop AI / ChatGPT Just Caught a Spy Stealing From OpenAI

Transcript

ChatGPT Just Caught a Spy Stealing From OpenAI · In the Loop AI

• 0:00
  ChatGPT just discovered and arrested a spy in OpenAI's own HQ.
• 0:05
  And that's the least craziest thing I'm going to say on this entire episode,
• 0:09
  because this whole story is crazy. It starts off with an unknown individual
• 0:13
  planting an audio listening spyware device in the back of a television wrapped in toilet paper.
• 0:19
  Why it's wrapped in toilet paper, I have no idea. But the important part is
• 0:22
  the conference room. Why do we care?
• 0:24
  Well, a bunch of really important executives and business partners at OpenAI
• 0:28
  meet to discuss proprietary information, secret product plans.
• 0:32
  So you can imagine if that information got into the wrong hands of a person,
• 0:36
  a company, or maybe even another country, that could probably become a bit of a problem.
• 0:41
  So OpenAI handed over the case to the FBI and their own internal model,
• 0:45
  who ended up ratting on the individual who had planted that device.
• 0:50
  Kash Patel, the head of the FBI, said this is the Bureau's utmost priority at
• 0:55
  the moment, that AI has finally become an importance or asset of national security and risk.
• 1:01
  Imagine if James Bond, 007, had access to Fable 5.1 back in the day.
• 1:08
  He'd be a bit of a beast. But also imagine if his enemies also had access to this model.
• 1:13
  That's what we're going to uncover in today's episode of In The Loop,
• 1:16
  brought to you by Qualcomm.
• 1:21
  So the story starts off two weeks ago, September 30th, when an OpenAI employee
• 1:27
  discovered a device plugged into the back of a television in an executive conference room.
• 1:33
  And it was wrapped in toilet paper for God knows what reason.
• 1:36
  But upon recovering this device, it was handed over to the security team at OpenAI.
• 1:39
  They started looking at it and they were thinking, hmm, this looks like a USB
• 1:43
  stick, but it has this weird circuitry that's kind of attached to it.
• 1:47
  So it seems like a bit of a hack job. What on earth is this device?
• 1:50
  And so they started to examine it and they asked their own internal AI model
• 1:54
  to uncover what this device might be used to.
• 1:56
  And they discovered three particular things. One, it had very heavy cryptography
• 2:01
  designed around its entire thing.
• 2:03
  So accessing the information that this device had was near impossible.
• 2:08
  Number two, the device was designed to be continuously recording audio,
• 2:13
  that it would also be triggered to start recording audio when someone would start speaking.
• 2:17
  So it seems very intentional that this device was to try and spy on conversations
• 2:21
  that were happening in that particular room.
• 2:23
  And then number three, any kind of data that got extracted from this device
• 2:27
  that could be transferred to, you know, your computer to view it or review it
• 2:31
  or listen to it was under heavy encryption as well, which meant that the data
• 2:35
  transfer layer had been very meticulously designed.
• 2:38
  And so OpenAI kind of like looked at this and thought, this seems like an attempt to spy on us.
• 2:43
  So we should probably hand this case over to the FBI, give them access to this
• 2:47
  internal model or work with them at least to try and figure out what the hell is going on.
• 2:51
  So opening i continued their internal investigations whilst they gave it to the fbi
• 2:55
  and very soon using their own internal model of i don't know gpt 6.5 gpt 7 call
• 3:01
  it whatever that latest model is the one that solved all the math stuff definitely
• 3:04
  go check that episode i made that very same model named someone.
• 3:09
  Traced back the entire circuitry design of this device and who was most likely
• 3:13
  to have planted the device and uncovered a name.
• 3:17
  So Open Air had an idea of who it might have been, but let the FBI continue
• 3:21
  on their investigation. But while they were doing this, they also decided to
• 3:25
  plant a camera in almost the exact same position where the USB stick was.
• 3:30
  So in the event that someone who planted it came back to look at it,
• 3:35
  they would be able to capture the suspect red-handed.
• 3:39
  And that's exactly what happened a few days later when the suspect returned had a look behind the tv.
• 3:45
  And the camera captured exactly who the culprit was and on the same day that
• 3:50
  this culprit was captured on this camera security team discovered a bag left
• 3:54
  behind by the culprit by accident
• 3:56
  in the printing room they went through it and they discovered a second device
• 4:01
  that looked very familiar to the one that they found behind the tv
• 4:04
  so they just took the bag put it in lost and found but they took the device.
• 4:08
  They were trying to capture this individual wet-handed. They also wanted to
• 4:12
  figure out what on earth was going on, what this individual might be using this
• 4:15
  information for. So they, again, were keeping hush and quiet.
• 4:18
  And they handed over the device and the information that they had to the FBI.
• 4:22
  Now, this individual discovered that his bag had gone missing and went to Lostom
• 4:27
  Property to try and find it. And he found the bag. So he felt pretty good about
• 4:31
  it, except that he noticed that the second device was missing.
• 4:34
  And he also noticed that the device behind the TV when he went to go check it
• 4:38
  was also missing. So he reported that there had been a theft of his own property.
• 4:42
  But by that time, the FBI and the OpenAI team was already on him and issued
• 4:47
  an arrest or warrant for his arrest the exact next day.
• 4:51
  Upon arresting this guy, they discovered that the suspect, his wife,
• 4:56
  and another family member of the suspect had a one-way ticket to Saudi.
• 5:02
  So he was going to fly out with his family and likely never return because.
• 5:07
  Presumably he thought he'd been busted so the guy got arrested and he has been
• 5:10
  internally investigated and being put through court or whatever all that stuff
• 5:15
  is by the fbi and cash patel
• 5:17
  who's the leader of the fbi and his team has been handling it since then but
• 5:21
  open ai has come up with an official statement kind of giving a high level idea
• 5:25
  of what's been going on but never named any names
• 5:28
  who has been naming names is pretty much every other media publication out there
• 5:32
  which gets to the next section, who the hell did this and why did they do this?
• 5:37
  The man's name is Abdulrahman Al-Fozan.
• 5:41
  He's a 35-year-old Saudi national who's lived in the US since 2020.
• 5:46
  But he's also worked at OpenAI for over a year on the product team.
• 5:50
  So he's like a product slash engineer. He's worked on multiple product releases
• 5:54
  at OpenAI, likely a core employee, been there for a while as the valuations
• 5:59
  have increased, so likely his equity is worth quite a bit.
• 6:01
  This is a person that people actually know at the company.
• 6:04
  If you look at his LinkedIn history, he's worked on things like AI agents,
• 6:08
  which is important for the next bit of this story.
• 6:10
  What was he doing? How did he create this device? And why was he trying to use
• 6:14
  this device in that secret room? Let me take you back six weeks prior to this
• 6:18
  device being discovered.
• 6:20
  This individual, Abdul Rahman, queried an internal personal agent at OpenAI.
• 6:25
  Guess what he asked that agent?
• 6:27
  Hey, where do high-powered execs at OpenAI?
• 6:30
  Have meetings? Also, business partners. Where do these execs have meetings with
• 6:35
  those business partners?
• 6:36
  Can you tell me the exact rooms? This guy was literally asking an internal version
• 6:42
  of ChatGPT, which the security team and any OpenAI employee or anyone with a
• 6:47
  brain cell could get access to and see who was querying it.
• 6:50
  That's the device that he, or rather, that's the agent that he used to try and
• 6:54
  target and locate an executive room. So he finds this room. He finds the perfect
• 6:58
  room. It seats six people and more.
• 7:01
  They only have discussions about proprietary research, designs,
• 7:04
  product releases, things that you don't want getting out to the public.
• 7:07
  But there's a problem. He can't book the room. He can't get access to the room.
• 7:11
  So that's only bookable by execs and their secretaries. So he tries to then
• 7:15
  devise a plan as to how he can be a listen-in on the conversations there.
• 7:20
  And he comes up with a bright idea.
• 7:22
  I need to create an audio device that can spy and listen in to the conversations
• 7:26
  that are happening in there, but I don't know how to build that device.
• 7:30
  You know what might be useful? Let me ask AI.
• 7:34
  So he goes to his wife, who has a chatbot account, and he goes on to ask it
• 7:40
  a series of very specific prompts, which is,
• 7:43
  I want to create an audio device that continuously listens to the surrounding
• 7:48
  conversation around it.
• 7:49
  But also when it stores this data, I want it to be encrypted such that only
• 7:54
  I have access to this data.
• 7:55
  And in the event that someone gets a handle on this device and is able to extract
• 7:59
  the data, I still want it to encrypt the data so that no one can get access to this thing except me.
• 8:05
  But I also need to design this device so that it plugs into a television in a room.
• 8:11
  But I don't want the television to detect the device because that would give it away.
• 8:14
  So could you create the device such that it only appears to the TV as a power
• 8:19
  device, which means that it's charging the device, but it doesn't know what
• 8:22
  the device is being used for.
• 8:23
  So a series of back and forth prompts over the next like four weeks from that
• 8:28
  initial six week kind of like starter, ask my agent, which rooms should I plant this device in?
• 8:33
  He starts to plan this device and presumably he ends up building it.
• 8:37
  This is the thing that is wrapped around toilet paper.
• 8:40
  Investigators looked into what it took to make this device.
• 8:43
  It cost all but $24. So a $24 device cost this guy presumably a large chunk
• 8:49
  of his life. I looked into the prison sentence. It's like up to five years for
• 8:53
  the particular case. But before I get into that.
• 8:55
  The idea here is a $24 device basically caused this guy and the entire company
• 9:00
  and this entire kind of spy mission, a whole world of pain.
• 9:04
  Okay, so then we get to September 28th, two days before the device is discovered.
• 9:09
  Abdul Rahman enters the OpenAI Office HQ building. He walks up to his desk.
• 9:15
  He chills. He's working.
• 9:16
  He's going about his day, seemingly. Keeps glancing over to the executive conference
• 9:22
  room, where he keeps wandering around, scouting the area.
• 9:25
  He enters the building at 2 18 p.m and he's there till about 6 30 p.m so whatever
• 9:29
  a regular working day in his eyes but he's caught on cctv because of course
• 9:33
  investigators are now onto him and they can see him in hindsight
• 9:37
  scouting the entire area where he then pops into the room
• 9:41
  when presumably someone exits and he's able to hold the door open this is something
• 9:45
  straight out of a spy movie this is ridiculous also why the toilet paper i don't
• 9:50
  understand okay so then fast forward two days Zeta September 30th and it's the story I just told you.
• 9:56
  Someone discovers the device at the back of the television, presumably because
• 9:59
  the guy was caught on CCTV being shady around the office or the conference room in the first place.
• 10:05
  Everything comes down around him. He's caught, he's arrested,
• 10:08
  and they're currently working on what his sentence might be for his particular perjury.
• 10:13
  Now, before I get into what the
• 10:15
  ramifications of this is, I think it's important to spend a short moment
• 10:20
  how OpenAI's internal model discovered the guy because it got access to the
• 10:25
  device and it basically reverse engineered the entire thing.
• 10:29
  And it was able to uncover two things.
• 10:31
  Number one, how it was particularly designed and in accordance with the CCTV
• 10:37
  footage, which OpenAI also fed it.
• 10:39
  They could then figure out who the person likely was.
• 10:43
  And then when they had a suspect, which was this Abdul Rahman guy,
• 10:46
  they started to look at his activity secretly without warning him.
• 10:51
  And they discovered his conversation with the internal personal agent that he
• 10:55
  had six weeks prior, basically asking.
• 10:59
  Hey, where do all the execs have super important information and high priority
• 11:03
  product release information so that I can plant a listening device?
• 11:06
  I'm not a spy, but Spy 101, one.
• 11:08
  Don't talk to the tools that your companies own if you're trying to pull off this kind of a heist.
• 11:13
  It's crazy. But then they took it an extra step further.
• 11:16
  And this is the part where I'm still trying to figure out how they did it.
• 11:20
  But they were able to discover that he was having a conversation on another personal chatbot.
• 11:26
  And he then used this to prompt engineer what that $24 device would look like.
• 11:33
  He built it himself or he got someone to build it for him.
• 11:35
  And then it led to him planting it in the back of the room. So the guy used
• 11:40
  AI to plan and build and execute the entire thing.
• 11:45
  And then AI that he used ended up becoming the exact point of reason for his
• 11:49
  downfall, discovery and arrest.
• 11:51
  It's just this insane loop where you could have used the AI to do the bad thing,
• 11:55
  but you can use AI to do the good thing as well.
• 11:57
  So it's kind of like, it's not a wholesome story, but it does give me more confidence
• 12:02
  when, you know, we talk about fearmongering around AI cyber attacks and AI is
• 12:06
  going to wreak a lot of havoc, AI can be used for a lot of good as well.
• 12:09
  So it's just this insane story. And I have a lot of takes on this.
• 12:13
  So let's get into them on today's episode of In The Loop brought to you by Qualcomm.
• 12:20
  But the analogy that I would use here is he effectively walked into the building
• 12:25
  and what would be the analogy here?
• 12:27
  He asked the office receptionist where the most secretive conversations were
• 12:33
  happening amongst execs, found out the information from this receptionist,
• 12:37
  and then went to a colleague and asked them,
• 12:41
  hey, how do I build a listing device that I could plant in this room?
• 12:44
  He basically gave the cards over to people or employees or agents of the building.
• 12:49
  And he's honestly stupid to have not thought that people wouldn't investigate
• 12:53
  the tools that he would use, that he wouldn't leave a digital trace.
• 12:57
  And I guess that's like one of the larger points about this whole thing.
• 13:00
  Putting the politics and the crazy espionage theories aside,
• 13:04
  everyone's digital footprint when they use AI has become massively enlarged.
• 13:09
  Remember, when you're talking to your chatbot, it may seem like it's quite a
• 13:12
  personal thing, but all your data is being tracked by OpenAI Anthropic or whoever
• 13:18
  the model lab is, unless you're using open models hosted at home on your own.
• 13:22
  That's not necessarily a bad thing. It's a good thing in this case that they
• 13:25
  were able to trace it back.
• 13:26
  If this was an open model, probably not. The second thing is around the sentence.
• 13:31
  So I think right now, rumors or reports are saying at least that they have him
• 13:36
  on attempt to overhear or record something.
• 13:40
  I forget the term that they were I think it's like oral conversations,
• 13:45
  something very generic like that.
• 13:47
  Nothing that would indicate espionage, spying, or a breach of national security,
• 13:52
  which would presumably result in a much larger sentence.
• 13:57
  But right now, the max sentence that he can have for recording of oral conversations
• 14:01
  is five years, which seems honestly surprising to me.
• 14:05
  But he should get the charge for maybe like spy stuff. But again,
• 14:08
  details are unraveling. FBI is on it.
• 14:10
  That Kash Patel, head of the FBI, is fully committing resources to try and figure
• 14:14
  out what the hell this is.
• 14:15
  Which brings me to my third take. AI has officially become.
• 14:20
  An issue of national security. Now, I've said this many times on previous episodes
• 14:25
  on this channel, and it's crazy to think that this is happening now,
• 14:31
  and there's real-life examples of this happening.
• 14:33
  I think of a conversation that Jacob Coxon, who is the crazy safety and alignment
• 14:38
  researcher, remember the one that we spoke about on an episode previously where
• 14:42
  he left OpenAI, joined Anthropic, then left after three weeks because he did
• 14:45
  not have any faith that either the company were taking model alignment seriously.
• 14:51
  He went on a podcast last week and he said, there's this clip of him that went viral.
• 14:55
  He said that there is a 100% or sorry, 90% chance that there are spies at the frontier labs.
• 15:01
  What are the odds of him having that conversation, going live,
• 15:05
  and then literally two weeks later, opening out uncovers this spy,
• 15:08
  Abdul Rahman, who had been planted there for presumably a year.
• 15:12
  It's becoming all too real, if that makes sense.
• 15:14
  And it's one thing kind of talking about it to you guys, talking about it on
• 15:18
  a camera, hypothesizing what might happen. It's another thing,
• 15:21
  it actually happening. And then it's a third thing, us being aware of it,
• 15:25
  it being all revealed to us.
• 15:27
  But can you imagine how many other attempts are currently underway?
• 15:31
  If you're Anthropic and you're seeing this news, if you're Google,
• 15:34
  if you're SpaceX SI or whatever Frontier Lab.
• 15:37
  You're going to be now on high alert that someone is trying to access or listen to your conversations.
• 15:44
  Now, with all intents and purposes, this could have been a really bad situation for OpenAI.
• 15:49
  This guy could have attempted to do
• 15:51
  something even worse, not listen to conversations, but try and exfiltrate
• 15:56
  model weights or model blueprints that they could then use to build a competitor
• 16:01
  model or use it for another model for more nefarious purposes at the state level
• 16:06
  or at the company or entity or even the individual level.
• 16:09
  Like there's myriad different ways that you could use this thing.
• 16:12
  Thankfully, OpenAI's investigation has revealed that there's likely not a chance
• 16:16
  that he's been able to pull this off.
• 16:18
  And personally, if the guy's wrapping his device in toilet paper.
• 16:21
  It doesn't sound as sophisticated.
• 16:23
  But you know, you can imagine that they are high level organizations probably
• 16:27
  trying to attempt this right now, maybe even still at OpenAI or at Anthropic
• 16:31
  or any of these other labs.
• 16:32
  The fourth take is AI is officially a national security threat.
• 16:37
  And how we protect those models, how we keep them secure, needs to become a national priority.
• 16:44
  That's what Cash Patella said. But right now, it's under lock and key for small,
• 16:49
  well, small private companies such as OpenAI and Anthropic who employ their
• 16:53
  own security team and who kind of take matters into their own hands.
• 16:57
  And they're not necessarily a public company. They don't have public investor
• 17:01
  interests. They're only beholden to themselves and their current shareholders.
• 17:04
  So I can only imagine that the government wants to get involved,
• 17:08
  they are probably placed on high alert now.
• 17:10
  And Kash Patel's probably thinking, I wonder how I can plant an FBI agent or
• 17:16
  several FBI agents just in the day-to-day nine to five at OpenAI's office.
• 17:20
  I'm almost certain that they're having this conversation.
• 17:24
  And then it kind of got me thinking about Dario's suggestion of pacing the frontier,
• 17:29
  about a month ago, where he suggests that we want external parties to get involved
• 17:33
  and kind of evaluate and monitor the alignment side of AI.
• 17:38
  I'm noticing a trend, essentially, I guess is what I'm trying to say,
• 17:41
  where external parties need to start getting involved or will likely start getting
• 17:46
  involved with these private companies.
• 17:49
  Traditionally, it's been at the whim or direction of Sam Altman and Dario Murray,
• 17:55
  but now they may not have a choice.
• 17:57
  And I wouldn't be surprised if in the next few weeks, Trump,
• 18:00
  the administration or whoever announces that there'll be some form of light
• 18:04
  nationalization of these companies.
• 18:06
  Because as these models get more powerful. And Anthropic and OpenAI are both
• 18:10
  rumored to be housing much, much more powerful models than we currently have
• 18:15
  access to internally, which they can't release because.
• 18:19
  The ramifications of that could be quite dangerous. As a government,
• 18:22
  as a nation state, if that got into the wrong hands of your adversaries,
• 18:25
  such as China, whoever that might be, you could end up having a massive problem
• 18:28
  just from a pure security standpoint. So the labs are national security sites now.
• 18:33
  And then the other thing I was thinking about is Apple.
• 18:35
  Apple is currently suing the hell out of OpenAI, accusing them of stealing or
• 18:41
  reaching out to hundreds of Apple employees trying to exfiltrate their own blueprints and,
• 18:48
  device architecture. That's it for the AI hardware device.
• 18:51
  Seems like OpenAI has got their own problems. And if the sophistication levels
• 18:56
  of their own attacks that they're dealing with is a toilet paper wrap device
• 18:59
  plugged into a television,
• 19:01
  it seems highly improbable that OpenAI has a sophistication to be able to pull
• 19:04
  off a similar attack on Apple, although I hope they're not doing that.
• 19:08
  So that's really the episode that is an insane story that i read late last night
• 19:13
  i woke up this morning i thought i have to make an episode about this if you're
• 19:18
  wondering where i still am i am still in kyoto japan and yes
• 19:22
  it is amazing yes if you ever come here get a kimono but i still need to keep
• 19:27
  putting these episodes out guys it's it's fun and it sounds like all the crazy
• 19:30
  stuff is happening whilst i'm on this crazy time zone right now i am,
• 19:34
  currently 13 hours ahead of the world or New York, as I call it, and.
• 19:41
  News breaks overnight as I'm asleep.
• 19:42
  So thank you so much for listening to this episode. I hope you enjoyed it.
• 19:46
  I hope there was some valuable and useful takes.
• 19:48
  I would say if you had to leave this episode with one top takeaway,
• 19:53
  it's that objectively, the DNA or surface area that you have online on digital,
• 20:01
  surfaces, whether it's your phone, whether it's your computer,
• 20:04
  whether it's any kind of agent or software package that you use.
• 20:08
  Is now being tracked, especially if you use ChatGPT or Claw.
• 20:12
  And that's not necessarily a bad thing, as we've seen in this instance.
• 20:15
  And so I'm happy to finally talk about a crazy, dangerous thing that actually
• 20:20
  had a really good ending.
• 20:22
  Because at the end of the day, I do think optimistically that when it comes
• 20:26
  to cyber defense or cybersecurity, using AI can actually end up putting us in
• 20:30
  a much better world than we exist in today.
• 20:32
  And all the doom it takes about AI agents running rogue will definitely happen,
• 20:36
  but we'll have a way to figure it out. So if you enjoyed this episode or if
• 20:40
  you disagree with any of my takes, let me know. Leave us a comment.
• 20:44
  Please share these episodes with a friend, whether you listen to it on Spotify,
• 20:47
  Apple, or wherever you listen to it. We are anywhere and everywhere.
• 20:51
  Give us a rating. Leave us a comment. I read all of them. I try and reply to
• 20:53
  all of them. Give me feedback.
• 20:55
  I know this is a solo show now, but we're trying to figure it out, me and producer Luke.
• 21:00
  So if you have any feedback, it's been great. i've been you know really enjoying
• 21:03
  the feedback that i've received so far and also follow me on x i kind of post
• 21:08
  a lot of my thoughts even when i'm you know in japan with my family,
• 21:11
  um and so you know you can hear more news there more current up to date and
• 21:15
  some of my crazy takes i guess you can also dm me there as well,
• 21:18
  but yeah thank you so much for listening and i will see you on the next episode
• 21:23
  of in the loop brought to you by qualcomm thank you.