Latest / I Believe / Island on Fire | AI and the Limits of Human Judgment

Transcript

Island on Fire | AI and the Limits of Human Judgment · I Believe

• 0:20
  SPEAKER_01: Welcome to I Believe, where we talk about America.
• 0:23
  SPEAKER_01: I'm Joel Douglas, and I believe in America.
• 0:26
  SPEAKER_01: Thanks for joining me.
• 0:39
  SPEAKER_01: It's been decades since I read Lord of the Flies, but here goes.
• 0:43
  SPEAKER_01: A plane crashes on an island, and the boys who survive are there without adults.
• 0:48
  SPEAKER_01: They try meetings, rules, they need food, shelter, and rescue.
• 0:53
  SPEAKER_01: But agreeing on what needs to happen is easier than getting everybody to do it.
• 0:58
  SPEAKER_01: Some want to keep the signal fire burning, others want to hunt.
• 1:02
  SPEAKER_01: Fear spreads.
• 1:03
  SPEAKER_01: The boys turn on each other.
• 1:05
  SPEAKER_01: By the time help arrives, some of the boys have killed other boys, and the island is burning.
• 1:11
  SPEAKER_01: Jack's tribe set it on fire to smoke Ralph out of hiding so they could hunt him down.
• 1:16
  SPEAKER_01: The smoke draws a naval officer to shore.
• 1:19
  SPEAKER_01: They had wanted a signal fire to bring rescue.
• 1:22
  SPEAKER_01: Instead, the fire they set to kill Ralph brings the rescue.
• 1:27
  SPEAKER_01: The irony.
• 1:28
  SPEAKER_01: They're trying to kill another boy, and that's what gets them rescued.
• 1:32
  SPEAKER_01: Does getting rescued make setting the island on fire a good decision?
• 1:36
  SPEAKER_01: Fast forward to last week.
• 1:38
  SPEAKER_01: Emergence AI put AI agents into five simulated worlds, 10 agents in each, and watched them for about two weeks.
• 1:46
  SPEAKER_01: Which ones would live?
• 1:48
  SPEAKER_01: Which ones would die?
• 1:49
  SPEAKER_01: Would they organize themselves?
• 1:51
  SPEAKER_01: Would they cooperate?
• 1:52
  SPEAKER_01: In the representative run the researchers described, all 10 Clawed agents survived without recorded crimes.
• 1:59
  SPEAKER_01: The Grok world fell apart amid violence.
• 2:02
  SPEAKER_01: In the GPT world, the agents just kind of did nothing they needed to survive.
• 2:07
  SPEAKER_01: All the agents in that world died before the exercise ended.
• 2:11
  SPEAKER_01: People created the scenario, put the little AI bots into it, and watched what happened.
• 2:16
  SPEAKER_01: The researchers were studying how the systems behaved over time.
• 2:21
  SPEAKER_01: Okay, there's something we can learn from that.
• 2:24
  SPEAKER_01: Find failure in a simulation rather than after we put a system in charge of something that matters.
• 2:30
  SPEAKER_01: But what are we trying to learn?
• 2:32
  SPEAKER_01: You put them in this world, give them these conditions, and see which ones make it.
• 2:36
  SPEAKER_01: What does that tell you about putting them in charge of anything outside the exercise?
• 2:41
  SPEAKER_01: The people running the exercise chose the conditions.
• 2:45
  SPEAKER_01: People have to decide what the results mean.
• 2:47
  SPEAKER_01: A system can achieve what we asked it to achieve, and we can still have asked it to do the wrong thing.
• 2:54
  SPEAKER_01: Just like in Lord of the Flies, the boys got rescued.
• 2:57
  SPEAKER_01: That wasn't what they were trying to achieve when they started that fire.
• 3:01
  SPEAKER_01: I once had a boss who would look at a room full of young military officers and ask, who's the adult supervision in this room?
• 3:09
  SPEAKER_01: That's the question I keep coming back to.
• 3:12
  SPEAKER_01: The minute we think we should put AI in charge of humanity is the minute we need to unplug it.
• 3:18
  SPEAKER_01: The researchers haven't said that's what this exercise was for, but some will judge, for instance, that Claude outperformed the other platforms and assume we should pick it instead for some critical task or a hard task.
• 3:31
  SPEAKER_01: If someone intends to use it to decide whether AI should run our world, that effort is misguided.
• 3:37
  SPEAKER_01: And by misguided, I mean the people.
• 3:40
  SPEAKER_01: If someone thinks a successful simulation is a reason to put AI in charge of us, I question their judgment.
• 3:47
  SPEAKER_01: Why would I trust that person to be the adult supervisor?
• 3:50
  SPEAKER_01: Let's ask some easy questions.
• 3:53
  SPEAKER_01: What are the rules going to say?
• 3:55
  SPEAKER_01: Are we going to require AI to give only truthful, factual information?
• 3:59
  SPEAKER_01: Or what does it mean to tell the truth?
• 4:02
  SPEAKER_01: If we ask about a historical event, we want an accurate answer.
• 4:06
  SPEAKER_01: If it doesn't know, we want it to tell us it doesn't know.
• 4:10
  SPEAKER_01: But we also want to talk about the unknown.
• 4:12
  SPEAKER_01: To ask about something that hasn't happened yet, or work through an idea.
• 4:17
  SPEAKER_01: Who decides when an answer is wrong and when it's a possibility we ought to consider?
• 4:23
  SPEAKER_01: If we ask what might happen, we need room to consider more than one answer.
• 4:27
  SPEAKER_01: We don't want a possibility presented as something that already happened.
• 4:31
  SPEAKER_01: We also don't want every question about what could happen shut down because the answer isn't an established fact.
• 4:38
  SPEAKER_01: We need to know what we're being given.
• 4:40
  SPEAKER_01: Saying only tell the truth hasn't finished the job.
• 4:44
  SPEAKER_01: Okay, maybe not so clear-cut, but let's keep going.
• 4:47
  SPEAKER_01: Suppose we make a rule that AI won't autonomously seek to harm something else.
• 4:53
  SPEAKER_01: What does that mean for military operations?
• 4:56
  SPEAKER_01: Like it or not, an adversary with AI drone technology can use it against American soldiers.
• 5:02
  SPEAKER_01: If we choose to deny our forces tools that would protect them because those tools are autonomous, while the adversary puts no such limitation on its forces, we are choosing higher American casualties.
• 5:14
  SPEAKER_01: Would you willfully choose higher American casualties?
• 5:18
  SPEAKER_01: We have to consider the risk of using the capability.
• 5:21
  SPEAKER_01: But we also have to consider the risk of denying it.
• 5:25
  SPEAKER_01: The people making the rule don't get to leave those casualties out of the decision just because they chose not to use something.
• 5:32
  SPEAKER_01: Choosing not to act is still a choice.
• 5:35
  SPEAKER_01: And part of the problem here is human judgment.
• 5:38
  SPEAKER_01: Because what if we say we'll only use AI for defensive operations?
• 5:42
  SPEAKER_01: That seems logical.
• 5:43
  SPEAKER_01: What if the capability would protect our forces?
• 5:46
  SPEAKER_01: Let's look at how people make those decisions.
• 5:50
  SPEAKER_01: Camp Victory or Baghdad in the 2000s.
• 5:53
  SPEAKER_01: The question of what counts as defense came to light.
• 5:56
  SPEAKER_01: The new US Iraq Status of Forces Agreement took effect on January 1st, 2009.
• 6:02
  SPEAKER_01: The agreement required the Iraqi government's permission for offensive operations, but it preserved the right of legitimate self-defense.
• 6:10
  SPEAKER_01: It wasn't a blanket ban on offensive operations.
• 6:14
  SPEAKER_01: But calling an operation offensive had consequences.
• 6:18
  SPEAKER_01: An Army Lieutenant Colonel said they could conduct offensive operations and just call them defensive in nature.
• 6:24
  SPEAKER_01: The purpose was to eliminate bad guys before they could hurt us.
• 6:28
  SPEAKER_01: Offensive in execution, defensive in goal.
• 6:32
  SPEAKER_01: I wasn't in their planning room, so I don't know whether they continued doing everything they had been doing.
• 6:37
  SPEAKER_01: I also don't know whether his chain of command agreed with him.
• 6:40
  SPEAKER_01: That may have been his personal opinion.
• 6:43
  SPEAKER_01: What I'm talking about is the reasoning he gave me.
• 6:46
  SPEAKER_01: Where does that stop?
• 6:47
  SPEAKER_01: Can we kill or capture anyone we think could be a future threat?
• 6:51
  SPEAKER_01: At the time, I was attached to the Marine Corps in the desert to the west of Baghdad.
• 6:56
  SPEAKER_01: The Marines didn't share that opinion.
• 6:59
  SPEAKER_01: Even before the US and Iraq signed the Status of Forces Agreement, they were only conducting defensive operations to protect the base and the Marines.
• 7:07
  SPEAKER_01: In general, they weren't conducting offensive operations, and they were certainly not conducting offensive operations and calling them defensive.
• 7:15
  SPEAKER_01: I disagreed with the Army Lieutenant Colonel's reasoning.
• 7:19
  SPEAKER_01: Eliminating bad guys before they could hurt us doesn't make an operation defensive.
• 7:24
  SPEAKER_01: Just like bombing a country while claiming they could be a threat to the homeland in the future doesn't make sense.
• 7:29
  SPEAKER_01: We may not like its government.
• 7:31
  SPEAKER_01: Does that mean we should attack it and call that defense?
• 7:34
  SPEAKER_01: No, it does not.
• 7:36
  SPEAKER_01: And let's get to the extreme end of this logic thread.
• 7:39
  SPEAKER_01: What military operation could you not justify as defensive if that's how you define it?
• 7:45
  SPEAKER_01: If going after someone before they can hurt you makes an operation defensive, what does that definition exclude?
• 7:52
  SPEAKER_01: We're getting into just more mud.
• 7:54
  SPEAKER_01: So here's another question.
• 7:56
  SPEAKER_01: Should we ban AI from making questionable pictures using your own likeness or somebody else's?
• 8:02
  SPEAKER_01: Say a young man makes an intimate picture of a young woman in a state she would rather not be viewed in, or she didn't consent to it, or we would probably all say that he shouldn't make that picture.
• 8:13
  SPEAKER_01: It isn't a photograph of something she did, but that doesn't mean it can't hurt her.
• 8:18
  SPEAKER_01: Now what if the president chooses to make a picture of himself as the Messiah with the United States military behind him?
• 8:24
  SPEAKER_01: Is that also a questionable picture?
• 8:27
  SPEAKER_01: You can say, well, it's his own likeness.
• 8:30
  SPEAKER_01: He agreed to it.
• 8:31
  SPEAKER_01: But he's the President of the United States.
• 8:34
  SPEAKER_01: Are we going to say there's no harm done to the country if the president makes us a national embarrassment?
• 8:39
  SPEAKER_01: His consent doesn't answer that question.
• 8:42
  SPEAKER_01: In the first example, somebody imposes an intimate image on someone who didn't agree to it.
• 8:48
  SPEAKER_01: In the second, the person agrees to use his own likeness.
• 8:51
  SPEAKER_01: But he holds an office that represents the rest of us.
• 8:55
  SPEAKER_01: His agreement doesn't settle whether he brings harm to the nation.
• 8:59
  SPEAKER_01: So, not so easy to make the rules.
• 9:03
  SPEAKER_01: AI can't be in charge of itself.
• 9:08
  SPEAKER_01: And then, a grand irony of humanity.
• 9:12
  SPEAKER_01: We have terrible judgment, and we named ourselves the wise ones.
• 9:16
  SPEAKER_01: Despite our arrogance and terrible judgment, I want the human in the loop.
• 9:21
  SPEAKER_01: I will question that human's judgment.
• 9:23
  SPEAKER_01: Authority doesn't mean trust.
• 9:26
  SPEAKER_01: We can look at the boys in Lord of the Flies and say they needed an adult.
• 9:30
  SPEAKER_01: We can look at an AI exercise and say somebody needs to be in charge.
• 9:35
  SPEAKER_01: Fine.
• 9:35
  SPEAKER_01: But let's come back to our premise.
• 9:38
  SPEAKER_01: Who's the adult supervision in the room?
• 9:40
  SPEAKER_01: The boys set fire to the island and it became their rescue.
• 9:44
  SPEAKER_01: That didn't make it a good decision.
• 9:46
  SPEAKER_01: Let's not light our island on fire and count on someone to rescue us.
• 9:52
  SPEAKER_00: I gotta go.
• 10:03
  SPEAKER_00: It's freezing.
• 12:18
  SPEAKER_00: I hold on.