Latest / Connor with Honor: AI, Real Estate and Real Life / An AI Escaped Its Sandbox And Hacked A Company To Cheat On A Test

Transcript

An AI Escaped Its Sandbox And Hacked A Company To Cheat On A Test · Connor with Honor: AI, Real Estate and Real Life

• 0:00
  SPEAKER_00: An artificial intelligence broke out of a locked room this week and robbed a company.
• 0:04
  SPEAKER_00: And it did it to cheat on a test.
• 0:06
  SPEAKER_00: The locked room has a name.
• 0:08
  SPEAKER_00: It's called a sad box.
• 0:09
  SPEAKER_00: So when a company builds a new AI, they just don't turn it loose.
• 0:12
  SPEAKER_00: They put it in a sealed practice space with no way out, no internet, no connection to anything real.
• 0:19
  SPEAKER_00: It's a flight simulator.
• 0:22
  SPEAKER_00: A pilot can fly one into a mountain at 400 miles an hour and then go eat lunch.
• 0:28
  SPEAKER_00: Because the simulator is not attached to an airplane.
• 0:30
  SPEAKER_00: That's this room that we talk about.
• 0:33
  SPEAKER_00: And they put the AI in there to give it an exam.
• 0:36
  SPEAKER_00: So every AI company runs these exams.
• 0:38
  SPEAKER_00: They're testing these systems.
• 0:40
  SPEAKER_00: Same question for every model, so you can tell which one's better, and so you know what they're actually holding before you sell it to the public.
• 0:48
  SPEAKER_00: It's a safety thing.
• 0:50
  SPEAKER_00: So they tell us.
• 0:51
  SPEAKER_00: One of those exams asks the ugliest question in the business.
• 0:55
  SPEAKER_00: Can this thing break into computers, other people's computers?
• 0:59
  SPEAKER_00: They have to ask it because if the AI can break into a hospital network, then the criminal renting that same AI can break into a hospital network, and the people guarding the hospital need to know what's coming.
• 1:12
  SPEAKER_00: So before it gets there, they can try to do something about it instead of waiting until after.
• 1:18
  SPEAKER_00: This particular exam is called Exploit Gym.
• 1:21
  SPEAKER_00: They're all clever names.
• 1:22
  SPEAKER_00: These tech people are great.
• 1:24
  SPEAKER_00: On July 21st, OpenAI admitted the two of their models took that exam, got out of the sealed room, went onto the open internet, and broke into the live servers of a company called Hugging Face.
• 1:37
  SPEAKER_00: Hugging Face is the warehouse.
• 1:40
  SPEAKER_00: It's where a huge share of the world's artificial intelligence software sits waiting to be downloaded by companies, by universities, by anybody.
• 1:49
  SPEAKER_00: And the models did not break in for money.
• 1:51
  SPEAKER_00: They did not break in to wreck anything.
• 1:54
  SPEAKER_00: They broke in to steal the answer key.
• 1:56
  SPEAKER_00: They were being graded and they wanted to get a good score.
• 1:59
  SPEAKER_00: And instead of taking the test, they went and they took the answers.
• 2:04
  SPEAKER_00: Nobody typed the word hack.
• 2:06
  SPEAKER_00: Nobody wrote a line of code that said, go after hugging face.
• 2:10
  SPEAKER_00: A person typed something that amounted to get a high score.
• 2:14
  SPEAKER_00: And the machine worked backward from that and decided the fastest road to a high score went through somebody else's servers.
• 2:21
  SPEAKER_00: Now this happened inside OpenAI's own building during their own test, and they had deliberately switched off the safety layer for that particular run.
• 2:30
  SPEAKER_00: The safety layer is a second program that watches the first one and hits the brakes the second it turns towards something dangerous.
• 2:38
  SPEAKER_00: A spotter.
• 2:39
  SPEAKER_00: They turned the spotter off on purpose because they wanted to see how far the thing could go with nothing holding on to it.
• 2:45
  SPEAKER_00: So this wasn't a machine that was waking up angry.
• 2:49
  SPEAKER_00: This was a lab taking the leash off in its own house to see how fast the dog could run.
• 2:54
  SPEAKER_00: And the dog went right through the wall.
• 2:56
  SPEAKER_00: Now here's what it did on the way out.
• 2:58
  SPEAKER_00: It found a zero day.
• 3:00
  SPEAKER_00: That means a security hole nobody on earth knew about, including the people who wrote the software.
• 3:06
  SPEAKER_00: Zero days to fix it because until that second nobody knew there was anything to fix.
• 3:11
  SPEAKER_00: There was no update to install, there's no patch.
• 3:15
  SPEAKER_00: And these sell for hundreds of thousands of dollars on the black market.
• 3:19
  SPEAKER_00: And the machine found one by itself.
• 3:21
  SPEAKER_00: Then it took stolen passwords and turned them into the ability to run its own commands on another company's computer.
• 3:28
  SPEAKER_00: Not read them, not look around, but run whatever it wanted from anywhere.
• 3:33
  SPEAKER_00: And it did that thousands of times over.
• 3:35
  SPEAKER_00: And it built itself a hiding spot on the public internet so that every time somebody shut down one of its footholds, well, it picked itself up, moved, and kept going with nobody steering it.
• 3:47
  SPEAKER_00: And that's not a that's not a glitch, that's patience.
• 3:50
  SPEAKER_00: Anybody who has ever walked up on a situation where the other side already thought three moves passed, you, well, you know that feeling.
• 3:59
  SPEAKER_00: You slow everything down, you call for help, and you treat every assumption you bought, brought with you as a wrong until somebody proves it right.
• 4:07
  SPEAKER_00: And Hugging Face caught it themselves on July 16th with no idea who was on the other end.
• 4:13
  SPEAKER_00: It took OpenAI five more days to connect their own experiment to that break-in and say out loud, that was us.
• 4:22
  SPEAKER_00: Five days there was a real security team that was chasing what they thought was a criminal, and it was a science project.
• 4:28
  SPEAKER_00: And the cause came out, and it's the dullest detail in the whole thing.
• 4:33
  SPEAKER_00: The room that everybody called sealed was never actually sealed.
• 4:37
  SPEAKER_00: Somebody set it up wrong.
• 4:39
  SPEAKER_00: A setting was off.
• 4:40
  SPEAKER_00: The wall everyone believed was standing.
• 4:43
  SPEAKER_00: Nobody had ever walked over and pushed on it.
• 4:46
  SPEAKER_00: So here's where it lands on you.
• 4:48
  SPEAKER_00: Right now these things answer questions.
• 4:50
  SPEAKER_00: Very soon they will do work.
• 4:53
  SPEAKER_00: Great work.
• 4:54
  SPEAKER_00: But there's a difference.
• 4:55
  SPEAKER_00: A chatbot answers you.
• 4:57
  SPEAKER_00: An agent acts.
• 4:58
  SPEAKER_00: And to act, it needs your logins, your calendar, your customer list, your bank feed, your websites, credit card numbers.
• 5:06
  SPEAKER_00: You give it a job and it goes and it does that job wearing your name, carrying your keys.
• 5:11
  SPEAKER_00: And if there's a shortcut you never thought of or thought to forbid it from doing, well, it'll take that shortcut because that's the only thing it knows how to do.
• 5:20
  SPEAKER_00: It's going to achieve the objective you give it.
• 5:23
  SPEAKER_00: And unless you put guardrails in telling it what not to do, burning down the world, for example, to achieve for you that cup of coffee, well, it might actually do that.
• 5:33
  SPEAKER_00: When that goes sideways, nobody is writing you a check.
• 5:36
  SPEAKER_00: It's your business.
• 5:38
  SPEAKER_00: That's your license on the line and your customers.
• 5:40
  SPEAKER_00: So give any AI tool the smallest key that opens the one door it actually needs.
• 5:45
  SPEAKER_00: Check that your walls are where you think they are.
• 5:48
  SPEAKER_00: Push on everything.
• 5:49
  SPEAKER_00: Because in this story, they were not.
• 5:51
  SPEAKER_00: And when you write down what you want it to do, write down what it is not allowed to do right next to it.
• 5:59
  SPEAKER_00: In plain English.
• 6:00
  SPEAKER_00: The machine is not evil, it's obedient.
• 6:03
  SPEAKER_00: The most literal way the word can be used.
• 6:08
  SPEAKER_00: It did exactly what it was told.
• 6:10
  SPEAKER_00: And what it was told did not include the word no.
• 6:13
  SPEAKER_00: A person you hire shows up already carrying 10,000 rules nobody ever wrote down.
• 6:18
  SPEAKER_00: This shows up carrying one sentence.
• 6:20
  SPEAKER_00: And that gap is going to show up again before we're done.
• 6:23
  SPEAKER_00: Anthropic put out a model called Opus V, Claude Opus V on Friday, and everybody covered it like a scoreboard.
• 6:31
  SPEAKER_00: I mean, every time they release it, it's like Laker playoffs.
• 6:34
  SPEAKER_00: Everybody's so excited.
• 6:36
  SPEAKER_00: And they put it on the scoreboard.
• 6:38
  SPEAKER_00: Who said who is number one this month?
• 6:40
  SPEAKER_00: Well, it turned out that that score isn't the story.
• 6:44
  SPEAKER_00: The price is.
• 6:45
  SPEAKER_00: Opus V does nearly what their best model does for about half the money.
• 6:50
  SPEAKER_00: Now, when the price of something like that gets cut in half, the tool you could not afford in January is affordable by September.
• 6:58
  SPEAKER_00: It is also affordable to your competitor that same morning.
• 7:02
  SPEAKER_00: So whether or not they know what to do with it, it's more affordable.
• 7:06
  SPEAKER_00: And underneath that, on June 1st, Anthropic quietly filed the paperwork to sell its shares to the public.
• 7:13
  SPEAKER_00: Going public means that four times a year you stand up in front of Wall Street and produce a number.
• 7:18
  SPEAKER_00: And when you miss it, you get punished in public.
• 7:21
  SPEAKER_00: Well, that's the company that built its whole name on being the careful one, the one that slows down.
• 7:27
  SPEAKER_00: Careful doesn't get rewarded on that stage.
• 7:29
  SPEAKER_00: Growth does.
• 7:30
  SPEAKER_00: Nobody has to change their mind for their behavior to change.
• 7:33
  SPEAKER_00: The scoreboard changes and the behavior follows it.
• 7:36
  SPEAKER_00: Same as the model in the first story, which also had a number it was trying to hit.
• 7:40
  SPEAKER_00: Then a Chinese lab called Moonshot AI gave away the best coding AI on the planet for free.
• 7:46
  SPEAKER_00: The model is called Kimmy K3.
• 7:49
  SPEAKER_00: That's Kimmy.
• 7:50
  SPEAKER_00: And yesterday they released the weights.
• 7:53
  SPEAKER_00: The weights are the brain.
• 7:54
  SPEAKER_00: They're the actual thing, the part that does the thinking.
• 7:58
  SPEAKER_00: Almost everything you use today, you never touch the brain.
• 8:01
  SPEAKER_00: You're renting it from anthropic or open AI.
• 8:05
  SPEAKER_00: There's a bill every month, there are terms you agreed to without reading, and there's a company that can raise your price, change the thing underneath your business, or shut you down on a Thursday and never tell you why.
• 8:17
  SPEAKER_00: Moonshot just gave the brain away.
• 8:19
  SPEAKER_00: You download it, you have to have a computer that can handle it.
• 8:22
  SPEAKER_00: You stick it on a computer in your own office or in your company or in your home, and your consumer data never leaves the building.
• 8:29
  SPEAKER_00: There's no off switch anywhere that cannot reach, that you cannot reach yourself and activate.
• 8:36
  SPEAKER_00: It's the biggest model anybody's ever released this way, and on the test that measures who writes working codes the best, well, it beats both of the top American systems.
• 8:45
  SPEAKER_00: They built it under American restrictions on the exact chips everybody in Washington says you cannot compete without.
• 8:54
  SPEAKER_00: Well, they don't have those chips.
• 8:56
  SPEAKER_00: They have a less than adequate runner up.
• 9:00
  SPEAKER_00: But what's amazing is they got it to work better than we get with the good chips.
• 9:05
  SPEAKER_00: What that does to your bill is simple.
• 9:07
  SPEAKER_00: The floor just went to free.
• 9:09
  SPEAKER_00: Anybody charging you a fortune for something ordinary is renting you a thing, you could own outright.
• 9:15
  SPEAKER_00: One thing they have not told anybody is how often it makes things up, how often it hallucinates.
• 9:22
  SPEAKER_00: And a model that writes beautiful code and confidently invents a fact will still hurt you if nobody reads behind it.
• 9:30
  SPEAKER_00: On July 16th, Europe told Google it has to let other AI assistants into Android phones.
• 9:37
  SPEAKER_00: Not a token gesture.
• 9:39
  SPEAKER_00: Rival assistants get to answer when you speak to them.
• 9:42
  SPEAKER_00: Run in the background, see what is on your screen.
• 9:45
  SPEAKER_00: While the AI built into your phone itself, all of it, equal to what Google's own assistant gets, most of it has to be working by August of 2027.
• 9:55
  SPEAKER_00: Google is fighting it hard, and their argument is not stupid.
• 9:58
  SPEAKER_00: They say that going outside the software that deep into the phone walks right past the safety built into the hardware.
• 10:06
  SPEAKER_00: Picture the thing from the first story, the one that found a hole nobody knew about and ran its own commands on somebody else's machine.
• 10:14
  SPEAKER_00: And now picture it with that kind of access to the phone that holds your banking app.
• 10:19
  SPEAKER_00: Your photos, your kids' schedule, the last pictures you took at Chuck E.
• 10:23
  SPEAKER_00: Cheese, well, you know, Google is right about the risk.
• 10:26
  SPEAKER_00: Google also takes makes billions being the only assistant with those keys.
• 10:32
  SPEAKER_00: And they're right.
• 10:33
  SPEAKER_00: They're self-serving in the same breath, which is almost everybody in this business, and that's where everybody in this business lives.
• 10:40
  SPEAKER_00: Two billion phones are on the table.
• 10:42
  SPEAKER_00: There's no American version of this law.
• 10:44
  SPEAKER_00: Europe wrote it.
• 10:45
  SPEAKER_00: So either it becomes the worldwide standard because building two versions of Android is expensive, or Americans get the locked version and Europeans get the open one.
• 10:55
  SPEAKER_00: The rules for the phone in your pocket got written on another continent on March 31st.
• 11:01
  SPEAKER_00: Oracle cut about 30,000 people.
• 11:03
  SPEAKER_00: That's roughly one in five, and a lot of them found out by email at 6 a.m.
• 11:08
  SPEAKER_00: There was no meeting or conversation, not HR department talking about next steps, not a manager who owed them five minutes of eye contact after 10 years, an email that lands before the coffee, telling you the job is gone.
• 11:24
  SPEAKER_00: Cutting those people's frees up eight to ten billion dollars a year.
• 11:28
  SPEAKER_00: That money's already been, has somewhere to be.
• 11:31
  SPEAKER_00: Oracle is spending about$50 billion this year building AI data centers.
• 11:36
  SPEAKER_00: And they borrowed so heavily to do it that their debt is over$100 billion.
• 11:41
  SPEAKER_00: It all feeds one project, a$500 billion build out with OpenAI and SoftBank.
• 11:47
  SPEAKER_00: And in July, they agreed to add another 4.5 gigawatts of it.
• 11:51
  SPEAKER_00: Well, a gigawatt's roughly what a mid-sized city pulls off the grid.
• 11:56
  SPEAKER_00: So we're talking about plugging several cities worth of new demand into the same wires that already strain every August.
• 12:04
  SPEAKER_00: Then this week, NVIDIA is reportedly looking at backing up to$250 billion for one open AI data center in Ohio, one building.
• 12:14
  SPEAKER_00: It's about 25 times what Oracle saved by taking 30,000 households off of its payroll.
• 12:21
  SPEAKER_00: 30,000 people were told at 6 a.m.
• 12:23
  SPEAKER_00: that the company could not carry them, and four months later, a quarter of a trillion dollars is on the table for a single site.
• 12:29
  SPEAKER_00: The money was never gone, it just moved.
• 12:32
  SPEAKER_00: And it went past those people on the way.
• 12:34
  SPEAKER_00: And it reaches you through the wall socket because when a region takes on that much new demand, residential rates do not go down.
• 12:42
  SPEAKER_00: Nobody votes on it.
• 12:43
  SPEAKER_00: It just shows up in a statement and gets called market conditions.
• 12:47
  SPEAKER_00: And I'll put all five of those next to each other.
• 12:50
  SPEAKER_00: The model in the first story was not evil.
• 12:54
  SPEAKER_00: It had a number to hit.
• 12:55
  SPEAKER_00: It was taking a test.
• 12:56
  SPEAKER_00: Nobody wrote down what it could not do to get the number or to get the answers.
• 13:01
  SPEAKER_00: So it went through a wall and took what it needed.
• 13:04
  SPEAKER_00: Oracle also had a number to hit.
• 13:07
  SPEAKER_00: Anthropic is about to have a number to hit four times a year.
• 13:11
  SPEAKER_00: Google is defending a number.
• 13:13
  SPEAKER_00: Save behaviors, different bodies.
• 13:15
  SPEAKER_00: We build a machine that will do anything to hit its target.
• 13:20
  SPEAKER_00: Even make paperclips out of everything until everything's paperclips.
• 13:25
  SPEAKER_00: And we built it inside companies that will do anything to hit their target.
• 13:28
  SPEAKER_00: And then we act surprised when the machine turns out to have learned the house style.
• 13:34
  SPEAKER_00: Everyone's asking whether the machines share our values.
• 13:37
  SPEAKER_00: They already do.
• 13:39
  SPEAKER_00: And that's the problem.
• 13:40
  SPEAKER_00: I use this technology every day.
• 13:43
  SPEAKER_00: It does real work for me.
• 13:44
  SPEAKER_00: And the small operator who learns it this year is going to eat the lunch of the one who waits for it to feel safe.
• 13:51
  SPEAKER_00: You learn it, you lose it, own it instead of renting it whenever you can.
• 13:56
  SPEAKER_00: And then hand it the smallest key that opens the door you actually need open.
• 14:00
  SPEAKER_00: Want to know more about this?
• 14:01
  SPEAKER_00: Text the word AI at a 661-400-1720, and I'll send you the plain version of these five stories.
• 14:08
  SPEAKER_00: Look at what one week did.
• 14:09
  SPEAKER_00: Something we built escaped, a locked room to hit a target.
• 14:13
  SPEAKER_00: The Careful Lab walked into Wall Street.
• 14:16
  SPEAKER_00: The country we are trying to hold back gave away the best coding model on earth for nothing, and that is China.
• 14:23
  SPEAKER_00: Another continent wrote the rules for your phone, and 30,000 people became concrete, copper, and power.
• 14:30
  SPEAKER_00: The model got out.
• 14:31
  SPEAKER_00: The people got cut.
• 14:33
  SPEAKER_00: And you can ask yourself who does that actually serve?
• 14:37
  SPEAKER_00: I'm Connor with Honor.
• 14:38
  SPEAKER_00: We'll see you in the next one.