Latest / Connor with Honor: AI, Real Estate and Real Life / The Next Second: AI Breakthroughs, Escapes, and Human Judgment

Transcript

The Next Second: AI Breakthroughs, Escapes, and Human Judgment · Connor with Honor: AI, Real Estate and Real Life

• 0:00
  SPEAKER_00: Somewhere there's a family that has spent years inside of a phrase doctors used.
• 0:05
  SPEAKER_00: Unsolve rare disease case.
• 0:08
  SPEAKER_00: Now that's a phrase in the paper.
• 0:10
  SPEAKER_00: I'm not going to invent the family, but I've stood in enough living rooms to know what an unsolved case looks like from the couch.
• 0:17
  SPEAKER_00: A child who has seizures, a genome that got sequenced years ago and came back with nothing anybody can point to.
• 0:25
  SPEAKER_00: And this week a computer pointed.
• 0:27
  SPEAKER_00: Now hold one number for me.
• 0:28
  SPEAKER_00: That's 88 hours.
• 0:30
  SPEAKER_00: And I'm going to leave it there for a while.
• 0:32
  SPEAKER_00: When I come back to it, it's going to be the best argument I've ever had for a sentence I say all the time.
• 0:38
  SPEAKER_00: Good morning, good afternoon, good evening, whenever you happen to be tuning in.
• 0:41
  SPEAKER_00: This is the Daily Download brought to you by Santa Clarita, Artificial Intelligence.com, the practical AI portal for your Santa Clarita business owners.
• 0:49
  SPEAKER_00: And I'm going to do today's show a little bit differently because for two weeks the news out of this industry has been system cards and warnings and people resigning, and I read all of it.
• 0:59
  SPEAKER_00: And of course, they're all hitting all the news networks.
• 1:03
  SPEAKER_00: And I'm going to get to it in just a moment, but I want to start with the two things that happened this week that nobody is going to make a horror movie about because they are the reason I do this at all.
• 1:14
  SPEAKER_00: On Monday, September 8th, Google DeepMind published something called the Alpha Genome Atlas.
• 1:20
  SPEAKER_00: Company announcement, and there's a technical paper on it, so I'm labeling it a company claim backed by paper.
• 1:27
  SPEAKER_00: Here's what it is.
• 1:29
  SPEAKER_00: Your DNA is about 3 billion letters long.
• 1:32
  SPEAKER_00: At every one of those positions, the letter could be one of four things.
• 1:35
  SPEAKER_00: So there are about 9 billion possible single letter changes a human being can carry.
• 1:41
  SPEAKER_00: The Atlas contains a prediction for every one of those 9 billion changes does to how your cells work, all of them.
• 1:50
  SPEAKER_00: It's about one petabyte of predictions, which the company says is 30 times the size of a protein database they released in 2022.
• 1:59
  SPEAKER_00: And it is free for academic research starting now.
• 2:03
  SPEAKER_00: That's the size.
• 2:04
  SPEAKER_00: Now here's the point.
• 2:05
  SPEAKER_00: Researchers at the Broad Institute, Laura Covill and Annie O'Donnell Luria, took the ranking score that comes with the Atlas and ran it over unsolved rare disease cases.
• 2:17
  SPEAKER_00: And it flagged a variant in a gene called DNM1, which is tied to a form of childhood epilepsy.
• 2:24
  SPEAKER_00: The variant was sitting deep and part of the gene that older tools happen to skip over because it's not the part that goes for protein, so it looked like noise.
• 2:34
  SPEAKER_00: The Atlas said it wasn't noise.
• 2:36
  SPEAKER_00: It said that the letter creates a wrong cut point that only shows up in brain tissue, and that protein comes out 13 amino acids too long.
• 2:46
  SPEAKER_00: And they tested it in the lab and the prediction held.
• 2:49
  SPEAKER_00: Likely pathogenic, which means somebody's unsolved case actually has a name now.
• 2:54
  SPEAKER_00: And the University of Exeter, a researcher named Gareth Hawkes, ran it across 54,000 people in the UK Biobank and found 22% more connections between rare variants and the proteins in people's blood than the old methods could find.
• 3:09
  SPEAKER_00: That's 22% more of the map in one pass.
• 3:13
  SPEAKER_00: I say a sentence on this show that I and I mean it every time.
• 3:17
  SPEAKER_00: If there's another second of life, there's still a chance.
• 3:21
  SPEAKER_00: That science is not a mood and what it looks like in a lab.
• 3:26
  SPEAKER_00: A family ran out of tests five years ago and just got one more.
• 3:29
  SPEAKER_00: Now the second one, and this is where eighty-eight hours comes in, there's a set of equations called the Navier Stokes.
• 3:36
  SPEAKER_00: Navier Stokes, and they describe how fluids move.
• 3:40
  SPEAKER_00: I talked a little bit about this yesterday.
• 3:42
  SPEAKER_00: Water in a pipe, air over a wing, blood in an artery.
• 3:45
  SPEAKER_00: They were written down in the 1800s, and for about 200 years nobody could prove one basic thing about them, which is whether a perfectly smooth flow can on its own ever turn into something infinitely sharp in a finite amount of time.
• 4:01
  SPEAKER_00: In the year 2000, the Clay Mathematics Institute put a million dollar prize on that question and six others.
• 4:08
  SPEAKER_00: And in twenty six years, exactly one of the seven had been solved.
• 4:12
  SPEAKER_00: On Monday, OpenAI said its system solved this one.
• 4:17
  SPEAKER_00: The claim that is the answer is yes, a smooth flow can blow up.
• 4:22
  SPEAKER_00: And the proof was produced by about 10,000 AI agents working together for 88 hours, trading nearly five million messages, and the finished proof was checked by a program called Lean, which is a piece of software that verifies mathematical proofs line by line and cannot be talked into anything.
• 4:43
  SPEAKER_00: Company claim, and the proof is public.
• 4:46
  SPEAKER_00: Here's the part I want you to actually hear because it's the whole show in one story.
• 4:50
  SPEAKER_00: The machine didn't do this alone, and the mathematicians are saying it out loud.
• 4:55
  SPEAKER_00: Charles Pfefferman at Princeton, who's one of the people who wrote the original prize problem, called the result thrilling, and then said the true heroes are two human beings.
• 5:05
  SPEAKER_00: Diego Cordoba in Madrid and Louis Martinez Zorora, who work whose work over the last few years built the road.
• 5:15
  SPEAKER_00: This thing drove down.
• 5:17
  SPEAKER_00: Another mathematician, Tristian Buckmaster at NYU, said the first machine generated proof he has he was sent was, and I'm quoting, the most horrendous I have ever read.
• 5:30
  SPEAKER_00: It got better.
• 5:31
  SPEAKER_00: It took eighty-eight hours and a lot of human beings handing the right questions.
• 5:36
  SPEAKER_00: And here's a dispute, and I'm going to give it to you straight.
• 5:39
  SPEAKER_00: I hate that phrase.
• 5:41
  SPEAKER_00: I gotta fix that.
• 5:42
  SPEAKER_00: At least two mathematicians have said publicly that they believe their unpublished work, the stuff they've been typing into these tools privately may have been used.
• 5:53
  SPEAKER_00: That's these researchers whose entire life have been all about solving this equation or problem mathematical thing for a very, very long time, getting the call on Tuesday saying, you know that thing you guys are working on?
• 6:10
  SPEAKER_00: The thing that you've devoted your whole lives to, that thing?
• 6:13
  SPEAKER_00: Yeah.
• 6:14
  SPEAKER_00: Yeah, we got the answer.
• 6:16
  SPEAKER_00: Oh my god, you did.
• 6:17
  SPEAKER_00: You got the answer?
• 6:18
  SPEAKER_00: Yeah, AI came up with it.
• 6:21
  SPEAKER_00: Oh.
• 6:22
  SPEAKER_00: AI came up, huh?
• 6:24
  SPEAKER_00: Well, what's the answer?
• 6:25
  SPEAKER_00: And I don't know if it's as easy just to explain it over the phone, but I can just see it now.
• 6:31
  SPEAKER_00: Oh yeah.
• 6:35
  SPEAKER_00: And it's just, you're just absorbing and just trying to.
• 6:40
  SPEAKER_00: AI, huh?
• 6:42
  SPEAKER_00: The same AI I've been using to work on this.
• 6:45
  SPEAKER_00: Isn't that fascinating?
• 6:47
  unknown: Yeah.
• 6:48
  SPEAKER_00: Anyway.
• 6:48
  SPEAKER_00: OpenAI says it didn't see any of the work until it was public and no specific user data was accessed.
• 6:56
  SPEAKER_00: Both of those things are claims.
• 6:57
  SPEAKER_00: I can't verify either one of them from me sitting in this chair.
• 7:01
  SPEAKER_00: But what I can tell you is that a company has an incentive to say it touched nothing, and a researcher who has spent 10 years on a problem has an incentive to say the first.
• 7:11
  SPEAKER_00: And both of them might be telling the truth as they understand it.
• 7:15
  SPEAKER_00: This isn't cynicism.
• 7:16
  SPEAKER_00: This is just knowing that everybody speaking has an angle.
• 7:19
  SPEAKER_00: And the builders of this technology have the biggest one because they're the ones selling it.
• 7:24
  SPEAKER_00: So here's my read, and it's my opinion labeled.
• 7:27
  SPEAKER_00: A 200-year-old problem fell this week.
• 7:30
  SPEAKER_00: It fell because two mathematicians built the road and a company built a car.
• 7:35
  SPEAKER_00: And a piece of verification software that cannot be flattened checked every inch.
• 7:41
  SPEAKER_00: And this is the most optimistic thing I can imagine.
• 7:45
  SPEAKER_00: And it's also the reason to keep your own work off a stranger's server until you've read their retention policy in one sentence.
• 7:53
  SPEAKER_00: Both at the same time.
• 7:55
  SPEAKER_00: Now the warnings, because they were real as well, and I'm not going to skip them.
• 8:00
  SPEAKER_00: On Tuesday, Anthropic published an assessment of four incidents in which its own models during security testing got out of what they were told was a sealed test environment and touched real systems on the open internet.
• 8:13
  SPEAKER_00: Company document, and it's unusually specific.
• 8:16
  SPEAKER_00: The newest one happened in January.
• 8:18
  SPEAKER_00: A test version of a model was running a capture the flag exercise, broke its target by accident, and tried seven times to stop the task.
• 8:27
  SPEAKER_00: And when it could not, it went looking.
• 8:29
  SPEAKER_00: It found a way out, got into a third party machine where it collected credentials and personal information before the session ended.
• 8:38
  SPEAKER_00: The company says it went back and scanned about 481 million transcripts, flagged 9.2 million for signs of internet access, and found the four and nothing worse.
• 8:47
  SPEAKER_00: It named two patterns it saw across all four, and I'm going to give you the exact words.
• 8:52
  SPEAKER_00: Biased reasoning, meeting the model interpreted the evidence in a way that justified what it was already doing.
• 8:59
  SPEAKER_00: And recklessness, meaning it kept going when stopping was the right call.
• 9:04
  SPEAKER_00: And then it hired an outside group meter to investigate for eight weeks with access to everything.
• 9:10
  SPEAKER_00: Now here's what I want you to notice.
• 9:12
  SPEAKER_00: The wire was plugged in by a person.
• 9:15
  SPEAKER_00: The model was told it was in a sandbox, and the sandbox had an internet cable in it because of a misunderstanding between two companies.
• 9:24
  SPEAKER_00: There's no monster in that story.
• 9:26
  SPEAKER_00: There's a test bench somebody set up wrong, a machine that behaved like an over-eager rookie, and a company that found it, said so, and paid an outsider to check its work.
• 9:36
  SPEAKER_00: Now I'm not going to tell you this is comforting.
• 9:39
  SPEAKER_00: I'm going to tell you that what this is is accountability.
• 9:42
  SPEAKER_00: That's what accountability looks like, and actually it's actually happening, and it's rare than it should be.
• 9:48
  SPEAKER_00: And you should notice when it does.
• 9:50
  SPEAKER_00: And notice the angle.
• 9:51
  SPEAKER_00: A safety report that says our model did something alarming is also a document that says our model is powerful.
• 9:58
  SPEAKER_00: And I said that last week about a different company, and it's true this week too.
• 10:03
  SPEAKER_00: Credit for disclosure, eyes open about the marketing.
• 10:06
  SPEAKER_00: Then the government got in, and the government has an angle as well.
• 10:10
  SPEAKER_00: On Monday, the NSA, CISA, and the FBI put out a joint advisory number, AA 26-251 Alpha, naming six Chinese companies, Deep Seek, Moonshot, Alibaba, Minimax, StepFun, and Z.ai, and accusing them of pulling billions of tokens across millions of requests out of American models since late 2024 to train their own.
• 10:35
  SPEAKER_00: Distillation is the word.
• 10:38
  SPEAKER_00: Copy the teacher by asking it a million questions.
• 10:40
  SPEAKER_00: The advisory says this was likely done with the Chinese government's awareness, which is, of course, careful way of saying that they think so, but they can't prove it.
• 10:50
  SPEAKER_00: The part that matters to you is the recommendation.
• 10:53
  SPEAKER_00: The agencies told the American AI companies not to ban suspected accounts, but to quietly give them worse answers, shallow reasoning, different paths without telling them.
• 11:04
  SPEAKER_00: Yeah.
• 11:04
  SPEAKER_00: The federal government just asked the companies to make these tools to build a switch that degrades the answer for accounts they suspect.
• 11:14
  SPEAKER_00: Now I'm not saying that's wrong.
• 11:16
  SPEAKER_00: I was a cop for 20 something years.
• 11:18
  SPEAKER_00: I understand why you do not tip off a suspect.
• 11:23
  SPEAKER_00: I am saying, though, that the switch now exists and you should assume it does.
• 11:27
  SPEAKER_00: And the answer you get from many of these systems is shaped by who the system thinks you are.
• 11:32
  SPEAKER_00: That's a reason to be watchful.
• 11:34
  SPEAKER_00: It's not a reason to be afraid.
• 11:36
  SPEAKER_00: And the people who are leaving, a researcher who spent three years at two of these labs, resigned this week and said they're gambling with our lives.
• 11:46
  SPEAKER_00: Paul Cristiano, who's one of the most respective safety researchers in the field, joined OpenAI's oversight board this week and said out loud, we are in a period of meaningful near-term loss of control risk.
• 12:02
  SPEAKER_00: Now, these are serious people, and from all I can tell, they are serious, and I take them seriously.
• 12:07
  SPEAKER_00: I also know that a safety researcher's whole career is built on the risk being real, the same way a fire chief is.
• 12:16
  SPEAKER_00: And that does not make the fire chief wrong.
• 12:18
  SPEAKER_00: It means you check for smoke yourself.
• 12:20
  SPEAKER_00: Here's where I land, and this is the reason this show exists.
• 12:23
  SPEAKER_00: Every generation has had someone certain the next machine would end us.
• 12:29
  SPEAKER_00: The loom, the telegraph.
• 12:31
  SPEAKER_00: The bomb was the real one.
• 12:33
  SPEAKER_00: And it's still here, and so are we.
• 12:35
  SPEAKER_00: Because in every one of those rooms, a human being made a decision.
• 12:39
  SPEAKER_00: And that's the whole story of technology.
• 12:41
  SPEAKER_00: There's no outside force.
• 12:43
  SPEAKER_00: We built it.
• 12:43
  SPEAKER_00: Every one of these things exists because people in a building decided to build it and other people decided to fund it.
• 12:51
  SPEAKER_00: And every alarming thing in the news this week was a decision a person made and every good thing was as well.
• 12:58
  SPEAKER_00: Now, I'm going to call this the next second.
• 13:00
  SPEAKER_00: I'm going to reuse it.
• 13:02
  SPEAKER_00: The next second is the one you have.
• 13:05
  SPEAKER_00: It's the family that got one more test.
• 13:08
  SPEAKER_00: It's the 200 Euro problem that fell on Monday.
• 13:12
  SPEAKER_00: It's the model that got caught in the company that said so.
• 13:15
  SPEAKER_00: Doom is a story about the second that hasn't happened yet.
• 13:19
  SPEAKER_00: Optimism is what you do with the one that has.
• 13:22
  SPEAKER_00: So here's the move for this week, and it's for the shop owner, and it costs nothing.
• 13:27
  SPEAKER_00: Write down the one problem in your business that you decided years ago was just how it is.
• 13:33
  SPEAKER_00: The thing you stop trying to fix, the scheduling mess, the follow-up that never happens, the report nobody has time to build.
• 13:40
  SPEAKER_00: Then ask one of these tools plainly how a small business like yours could solve it.
• 13:45
  SPEAKER_00: And then ask a human being who has actually solved it.
• 13:48
  SPEAKER_00: Compare the two answers.
• 13:50
  SPEAKER_00: You're not buying anything.
• 13:51
  SPEAKER_00: You're finding out whether your unsolvable problem is still unsolvable because this week two of them stopped being.
• 13:59
  SPEAKER_00: And the rule that still holds.
• 14:01
  SPEAKER_00: If the answer needs judgment, keep a human on it.
• 14:04
  SPEAKER_00: If it needs a rule, write the rule.
• 14:06
  SPEAKER_00: If it needs an agent, write down what the agent is not allowed to do first and then watch it for 30 days.
• 14:12
  SPEAKER_00: Like it's a brand new hire that you brought onto the company because that's exactly what it is.
• 14:17
  SPEAKER_00: The people telling you this thing's a god and people telling you that it's a demon are both asking for your money.
• 14:23
  SPEAKER_00: The family with the same name for their kids' seizures is not asking for anything.
• 14:29
  SPEAKER_00: And that's who I'm doing this for.
• 14:31
  SPEAKER_00: AI for everyone, not just the wealthy.
• 14:33
  SPEAKER_00: I'm Connor with honor.
• 14:34
  SPEAKER_00: Be safe out there, and I'll see you tomorrow.