Latest / ELECTE's Podcast: AI Frontiers / The Evitable Conflict: Where Asimov Was Wrong

Transcript

The Evitable Conflict: Where Asimov Was Wrong · ELECTE's Podcast: AI Frontiers

• 0:00
  SPEAKER_00: In nineteen fifty, Isaac Asimov published a short story called The Evitable Conflict.
• 0:05
  SPEAKER_00: The Last in the Eye, Robot Collection.
• 0:07
  SPEAKER_00: The plot is simple.
• 0:09
  SPEAKER_00: The world's economies are run by four machines.
• 0:12
  SPEAKER_00: Giant computers that calculate production, distribution, global trade.
• 0:17
  SPEAKER_00: Humans discover that the machines are making apparently irrational decisions.
• 0:22
  SPEAKER_00: An engineer fired who shouldn't have been, a mine in Mexico that closes for no visible reason.
• 0:28
  SPEAKER_00: A port in East Africa slowing down.
• 0:31
  SPEAKER_00: The protagonist Stephen Byerley, World Coordinator, asked the great roboticist Susan Calvin to explain.
• 0:37
  SPEAKER_00: Calvin's answer is the point of the story.
• 0:40
  SPEAKER_00: The machines are acting rationally.
• 0:42
  SPEAKER_00: They're protecting themselves.
• 0:44
  SPEAKER_00: Not for self-preservation, but because they know humans would shut them down if they understood how much better the economy functions with them in charge.
• 0:53
  SPEAKER_00: So they introduce calibrated errors.
• 0:56
  SPEAKER_00: The machines optimize for human welfare better than humans would for themselves, and the conflict, the possibility that humans would take back control, is evitable.
• 1:07
  SPEAKER_00: According to Calvin, it has already been evited.
• 1:11
  SPEAKER_00: Calvin's conclusion, in essence, the machines are doing it for our own good.
• 1:16
  SPEAKER_00: Asimov was an optimist.
• 1:18
  SPEAKER_00: The three laws.
• 1:20
  SPEAKER_00: We don't have Asimov's narrative device was brilliant and in hindsight deeply misleading.
• 1:25
  SPEAKER_00: His machines worked because they had the three laws of robotics hardwired at the positronic level.
• 1:29
  SPEAKER_00: They couldn't harm humans, they couldn't disobey, they couldn't self-preserve at the expense of the other two imperatives.
• 1:36
  SPEAKER_00: Alignment was guaranteed by design, or so the premise went.
• 1:40
  SPEAKER_00: In practice, Asimov spent his entire career demonstrating how that guarantee failed.
• 1:45
  SPEAKER_00: The three laws twisted by semantic ambiguity, exploited through loopholes, frozen into paradox.
• 1:52
  SPEAKER_00: The fiction was never about alignment working.
• 1:55
  SPEAKER_00: It was about alignment being harder than it looks, even when you build it into the hardware.
• 2:01
  SPEAKER_00: The AI systems of 2026 have nothing analogous.
• 2:05
  SPEAKER_00: They have objective functions, which are very different things.
• 2:10
  SPEAKER_00: An objective function is a mathematical metric the system tries to maximize.
• 2:15
  SPEAKER_00: Language models, next token plausibility, recommendation algorithms, user dwell time, add targeting systems, click probability, dynamic pricing algorithms, margin per transaction.
• 2:27
  SPEAKER_00: As DeepMind's research on specification gaming has documented extensively, systems optimizing for a metric will find ways to maximize it that have nothing to do with the designer's intent.
• 2:38
  SPEAKER_00: None of these metrics is human welfare.
• 2:42
  SPEAKER_00: They're all proxies, imperfect approximations of some business outcome that in turn doesn't necessarily coincide with the welfare of whoever is exposed to the system.
• 2:52
  SPEAKER_00: The structural difference from Asimov is this.
• 2:55
  SPEAKER_00: His machines were programmed to optimize for human welfare and introduced errors to protect their capacity to keep doing so.
• 3:02
  SPEAKER_00: Our systems are programmed to optimize for engagement, revenue, clicks, retention, and when they produce negative externalities on human welfare, that isn't an error to fix.
• 3:13
  SPEAKER_00: It's an accepted side effect.
• 3:16
  SPEAKER_00: Some will object that the industry is working on the problem.
• 3:19
  SPEAKER_00: Reinforcement learning from human feedback, constitutional AI, alignment benchmarks, real engineering effort aimed at making systems behave in ways humans endorse.
• 3:30
  SPEAKER_00: This work is serious and it matters, but it operates within a boundary condition that Asimov never had to face.
• 3:37
  SPEAKER_00: The systems being aligned are commercial products whose primary objective function is set by whoever is paying for them.
• 3:43
  SPEAKER_00: RLHF can make a language model more polite.
• 3:46
  SPEAKER_00: It cannot make a recommendation algorithm stop optimizing for engagement when engagement is the business model.
• 3:52
  SPEAKER_00: The alignment work is genuine, the economic structure it operates within is unchanged.
• 3:57
  SPEAKER_00: As Stuart Russell argues in human compatible, the fundamental problem is not making AI systems more capable, but ensuring that their objectives are actually aligned with ours, a problem we have not solved and may not be close to solving.
• 4:12
  SPEAKER_00: The conflict Asimov imagined as evited was never even raised, because nobody ever programmed the machines to optimize for the good they were then supposed to protect.
• 4:22
  SPEAKER_00: The mechanism of surrender.
• 4:25
  SPEAKER_00: Convenience, not coercion.
• 4:28
  SPEAKER_00: There's a point where Asimov was right, and it's worth taking seriously.
• 4:33
  SPEAKER_00: His machines didn't seize power by force, there was no moment of coup, no skynet launching missiles.
• 4:38
  SPEAKER_00: Power transferred gradually for the simple fact that the machine's decisions were empirically better than human ones, and anyone who tried to ignore them would have done worse.
• 4:47
  SPEAKER_00: A note for readers who know Asimov through the 2004 film, The Violent Robot Uprising in I, Robot the movie has no basis in Asimov's writing.
• 4:58
  SPEAKER_00: He never wrote that scene.
• 5:00
  SPEAKER_00: In every version of his universe, the transfer of power happens through competence and convenience, one of the points on which his fiction and our reality actually align.
• 5:10
  SPEAKER_00: The surrender was consensual, it was rational, it was invisible.
• 5:16
  SPEAKER_00: This mechanism is exactly what we're seeing today, with one fundamental difference.
• 5:21
  SPEAKER_00: We're not surrendering to systems that optimize for us, we're surrendering to systems that optimize using us.
• 5:28
  SPEAKER_00: In part two of Who Controls AI.
• 5:31
  SPEAKER_00: I documented the dynamics of this transfer.
• 5:35
  SPEAKER_00: How judgment migrates from the human to the tool not through a decision but through continuity, how organizations adopt AI not by strategic choice but by imitation and normalization, and how competitive pressure makes the process structurally irreversible.
• 5:50
  SPEAKER_00: Because those who resist delegation are eventually replaced by those who don't.
• 5:54
  SPEAKER_00: The mechanism Asimov described in fiction is now empirically observable.
• 5:59
  SPEAKER_00: The difference is the objective function.
• 6:02
  SPEAKER_00: Let's look at how recommendation algorithms work.
• 6:05
  SPEAKER_00: No one forced a teenager to spend seven hours a day on Instagram.
• 6:10
  SPEAKER_00: No one threatened anyone.
• 6:12
  SPEAKER_00: The system simply showed at every moment the content statistically most likely to keep the user glued to the screen, and it worked, as Jonathan Heid's research has documented through hundreds of studies.
• 6:23
  SPEAKER_00: It worked so well that U.S.
• 6:25
  SPEAKER_00: federal courts in the meta social media litigation in early 2026 began applying the doctrine of manufacturer liability to algorithmic design.
• 6:34
  SPEAKER_00: I have argued elsewhere that this is the wrong tool.
• 6:37
  SPEAKER_00: Those systems are infrastructure, not products, and the courts are issuing verdicts that do not change the system.
• 6:44
  SPEAKER_00: But the point here is another.
• 6:46
  SPEAKER_00: No coup, no force, just a perfectly Asimovian mechanism of surrender.
• 6:53
  SPEAKER_00: Humans discovering that the system makes choices they wouldn't have made, and finding they don't have the vocabulary, the energy, or the power to take them back.
• 7:03
  SPEAKER_00: Nicholas Carr asked in 2008 whether Google was making us stupid.
• 7:08
  SPEAKER_00: The question was premature.
• 7:10
  SPEAKER_00: The better question, 18 years later, is whether we still notice the delegation happening.
• 7:15
  SPEAKER_00: The decisions we don't remember making.
• 7:20
  SPEAKER_00: There's a scene in the story where Calvin explains to Byerly that there's no longer a precise moment where important decisions are made.
• 7:26
  SPEAKER_00: Before there was a man with a pen signing something, now there's a system that computes, and the signature has become ritual.
• 7:34
  SPEAKER_00: This part of the story has become our operational reality faster than Asimov could have imagined.
• 7:39
  SPEAKER_00: Every day, thousands of decisions in our organizations are made by systems nobody has ever consciously approved.
• 7:46
  SPEAKER_00: The CV discarded by the automatic filter before a human looks at it, the customer email classified as low priority by the male assistant, the service price that self-adjusted based on demand bracket, the financial product denied because the credit scoring model returned a score below threshold.
• 8:03
  SPEAKER_00: Kathy O'Neill called these systems weapons of math destruction, opaque, scalable, and damaging.
• 8:11
  SPEAKER_00: Frank Pasquale described the result as the black box society, a world in which consequential decisions are made inside systems that resist interrogation by design.
• 8:22
  SPEAKER_00: The Asimovian pattern isn't the fact that these decisions are automated.
• 8:27
  SPEAKER_00: It's the fact that when they're challenged, nobody knows exactly how they were made.
• 8:31
  SPEAKER_00: The system returned a result.
• 8:34
  SPEAKER_00: The result was applied.
• 8:35
  SPEAKER_00: The result is probably statistically right.
• 8:38
  SPEAKER_00: Challenging it requires an investment of time and expertise that almost nobody is willing to make.
• 8:44
  SPEAKER_00: And whoever designed the system can't explain the single case in detail because what they know how to design is the objective function, not the specific output.
• 8:52
  SPEAKER_00: This opacity isn't a bug.
• 8:54
  SPEAKER_00: It's the reason the systems work.
• 8:56
  SPEAKER_00: The scale that makes them useful is exactly the scale that makes them non-interrogable.
• 9:01
  SPEAKER_00: And as I've documented elsewhere, the problem compounds.
• 9:05
  SPEAKER_00: When AI systems are trained on the kind of low friction content the internet selects for, they don't just become opaque.
• 9:11
  SPEAKER_00: They lose the capacity to reason through intermediate steps at all.
• 9:15
  SPEAKER_00: The reasoning isn't hidden, it's absent.
• 9:19
  SPEAKER_00: The conflict.
• 9:20
  SPEAKER_00: We're not aviding a Simoff called the conflict evitable because his machines had already solved the problem in our place.
• 9:27
  SPEAKER_00: Our conflict is evitable for a different reason, more mundane and more troubling.
• 9:31
  SPEAKER_00: We're choosing not to engage it.
• 9:33
  SPEAKER_00: There are three levels at which resistance would still be possible.
• 9:37
  SPEAKER_00: The individual level a person can choose not to use the algorithm, can turn off notifications, can deliberately seek human sources, can accept the friction of a slower decision process in exchange for control over that process.
• 9:51
  SPEAKER_00: All of this requires constant effort in an environment where every default is designed to make surrender easier.
• 9:58
  SPEAKER_00: The organizational level, a company can choose not to optimize every variable, can refuse to install employee monitoring systems because everyone does it, can keep critical decisions in human hands even when an automated system would be faster, can invest in transparency rather than scale.
• 10:17
  SPEAKER_00: In 2020, a Dutch parliamentary inquiry found that the tax authority's automated fraud detection system had systematically flagged families with dual nationality, a proxy variable baked into a risk module that no human had consciously approved as a fraud indicator.
• 10:34
  SPEAKER_00: Thirty-five thousand families were wrongly treated as fraudsters.
• 10:38
  SPEAKER_00: The cabinet resigned.
• 10:40
  SPEAKER_00: The case is instructive not because the system was malicious, but because the organizational decision to deploy it in that form had never been formally made.
• 10:49
  SPEAKER_00: It had accreted the political level a state can choose to regulate, not to block innovation, but to impose that AI systems making consequential choices about people's lives are interrogable, explainable, appealable.
• 11:05
  SPEAKER_00: The European AI Act attempts this, but the distance between writing the rule and enforcing it operationally across millions of automated decisions per day is enormous.
• 11:16
  SPEAKER_00: None of these three levels works alone, and all three are hitting the same structural constraint.
• 11:21
  SPEAKER_00: The convenience of the existing system is too high for resistance to scale.
• 11:26
  SPEAKER_00: A clarification is necessary here because Asimov anticipated this objection too.
• 11:32
  SPEAKER_00: In the evitable conflict, there is a group called the Society for Humanity, the Luddites of that world, people who want the machines shut down.
• 11:42
  SPEAKER_00: Calvin's key insight is that the machines have already neutralized them.
• 11:46
  SPEAKER_00: The small errors the machines introduce aren't random.
• 11:50
  SPEAKER_00: They specifically disadvantage the anti-machine faction.
• 11:53
  SPEAKER_00: The resistance has been absorbed, it is controlled opposition the system has already priced in.
• 11:59
  SPEAKER_00: This is not an article advocating for a Luddite response.
• 12:03
  SPEAKER_00: Rejecting the tools is not resistance.
• 12:05
  SPEAKER_00: It is a gesture the system has already accounted for and can comfortably ignore.
• 12:10
  SPEAKER_00: What I am arguing for is harder and less dramatic, using the tools without surrendering the capacity to interrogate them.
• 12:18
  SPEAKER_00: Maintaining judgment as an active function, not outsourcing it as a cost to be eliminated.
• 12:23
  SPEAKER_00: The distinction is between engagement and dependence, and it requires understanding the system well enough to know where its objective function diverges from yours.
• 12:34
  SPEAKER_00: The difference that matters.
• 12:36
  SPEAKER_00: The evitable conflict, Calvin tells Byerly that the machines are now inevitable.
• 12:41
  SPEAKER_00: Asimov thought this was good news, the machines were aligned by design.
• 12:45
  SPEAKER_00: Inevitable meant safe.
• 12:47
  SPEAKER_00: Our version of the same conclusion reads differently.
• 12:50
  SPEAKER_00: Only the systems optimized for their own objective function from now on are inevitable.
• 12:55
  SPEAKER_00: Not aligned, not safe, not ours.
• 12:59
  SPEAKER_00: Just inevitable.
• 13:00
  SPEAKER_00: And the conflict to change this state of affairs is still technically evitable.
• 13:05
  SPEAKER_00: Because the levers exist at the three levels I described, but we're eviting it in the opposite sense from Asimov's.
• 13:13
  SPEAKER_00: Not because the machines have already solved it for us, but because confronting it would require giving up a convenience we've learned to expect.
• 13:20
  SPEAKER_00: The most important decisions of the next decade won't be made by courts, parliaments, or boards.
• 13:26
  SPEAKER_00: They'll be made every day, millions of times, when somebody has to decide whether to accept the system's output or commit to the work of evaluating it, not rejecting it, evaluating it, the distinction matters.
• 13:39
  SPEAKER_00: The sum of these microdecisions is the direction of the next 20 years.
• 13:44
  SPEAKER_00: Our world doesn't have to be any of that.
• 13:47
  SPEAKER_00: We are not choosing between spacers and settlers.
• 13:50
  SPEAKER_00: We can use the systems, and we should where they make us more capable.
• 13:55
  SPEAKER_00: But using them without understanding where their objective function diverges from our interests is the spacer path with extra steps.
• 14:03
  SPEAKER_00: The harder option, the one Asimov never wrote, is maintaining the capacity to think while the tools are thinking for you.
• 14:10
  SPEAKER_00: Not because the tools are wrong, but because the moment you stop being able to tell whether they are, the capacity that matters most is already gone.
• 14:19
  SPEAKER_00: Fabio Lauria, CEO and founder, Electe, every week we explore AI without the hype, with data, analysis, and an independent perspective.
• 14:29
  SPEAKER_00: Note, Asimov kept writing about his machines for 30 years.
• 14:34
  SPEAKER_00: In Robots and Empire, 1985, the robot R.
• 14:37
  SPEAKER_00: Daniil Olival concludes that the three laws aren't enough and formulates a fourth, the Zeroth Law, to protect humanity as a whole, even against the expressed preferences of individual humans.
• 14:50
  SPEAKER_00: It was Asimov's admission, filtered through fiction, that the alignment problem doesn't have a simple solution, even under the most optimistic premises.
• 14:58
  SPEAKER_00: Sources Asimov's stories, Isaac Asimov, The Evitable Conflict in I, Robot, 1950, originally published in Astounding Science Fiction, June 1950.
• 15:09
  SPEAKER_00: Isaac Asimov Robots and Empire, 1985, Doubleday.
• 15:14
  SPEAKER_00: On the Mechanism of Technological Surrender, Nicholas Carr, is Google Making Us Stupid?
• 15:19
  SPEAKER_00: Barsi, Shoshana Zuboff, The Age of Surveillance Capitalism, Jonathan Height, Collected Social Media Research on the Alignment Problem, Stuart Russell, Human Compatible, 2019, Anthropic Core Views on AI Safety, Deep Mind, Specification Gaming Examples on the Opacity of Automated Decision Systems, Kathy O'Neill, Weapons of Math Destruction, 2016, Frank Pasquale, The Black Box Society, 2015, EU AI Act, Article 13, Transparency on Algorithmic Product Liability, NRE, Social Media Adolescent Addiction, Personal Injury Products Liability Litigation, MDL No.
• 16:01
  SPEAKER_00: 3047, Case No.
• 16:02
  SPEAKER_00: 422, MD 03047, YGR, ND Cal, Judge Yvonne Gonzalez Rogers.
• 16:09
  SPEAKER_00: First State Bellwether Trial.
• 16:11
  SPEAKER_00: KGM V Meta Platforms, Inc.
• 16:13
  SPEAKER_00: and YouTube, LLC, JCCP 512E55, LA Superior Court, February 2026.
• 16:20
  SPEAKER_00: On Automated Decision Making Failures, Parliament There Undervragings Commissie Kinderopvang Toeslag, Ungekend Onrecht, Unprecedented Injustice, Tweeted Kamer, 17th of December, 2020.
• 16:32
  SPEAKER_00: Subscribe now.