Latest / AI Ethics with Fexingo: Bias, Safety, and Responsible Artificial Intelligence / When Your AI Car Decides Who to Hit
Transcript
- Lucas: So, you're in a self-driving car, and suddenly a child runs into the street. The car can either swerve and hit a wall, likely injuring you, or keep going and hit the child. Who does the car choose? That's not a hypothetical anymore — it's already a programming decision being made by engineers at companies like Waymo and Tesla. Luna: I remember reading about the MIT Moral Machine project. They put exactly this kind of dilemma to millions of people online. What did they find — something like 40 million decisions logged? Lucas: Yeah, over 40 million. And the results are fascinating. Globally, people tended to prioritize saving humans over animals, saving more lives over fewer, and saving young people over old. But here's the wrinkle — those preferences aren't universal. When you break it down by country, you see real cultural splits. Luna: Such as? Lucas: In collectivist cultures like Japan and China, respondents were more likely to spare the elderly, who are seen as wise and deserving of respect. In more individualistic Western countries, younger people were prioritized. There's also variation on whether to spare pedestrians who jaywalk versus those who cross legally. Some countries strongly penalize the rule-breaker; others are more forgiving. Luna: So if a car is sold globally, does it need different ethical software for each market? That seems messy. Lucas: That's exactly the kind of question regulators are starting to grapple with. Mercedes, for example, has publicly stated that their cars will prioritize occupants over pedestrians. They say it's a legal necessity — the manufacturer has a duty of care to the people who bought the car. But that obviously creates a tension with the kind of utilitarian ethics that say you should minimize total harm. Luna: And if you're a pedestrian, hearing that a car is programmed to prioritize the people inside, that might make you less trusting of autonomous vehicles altogether. Lucas: Exactly. And there's a deeper issue here: are we actually comfortable with any algorithm making a life or death decision? I mean, think about it. A human driver in a split second might swerve instinctively, and we generally accept that as a tragic accident. But if a car's software makes a calculated choice, it feels different. It feels like a premeditated decision. Luna: Right. Because the car doesn't have instincts — it has code written months or years earlier by someone in an office, sitting at a desk, imagining possible scenarios. That creates a whole new category of liability. Lucas: And that's where the legal system is still catching up. Who gets sued if a self-driving car kills someone? Is it the owner? The manufacturer? The software developer? The city that didn't maintain the crosswalk? In 2022, there was a case in California where a pedestrian was killed by a Tesla operating on Autopilot. The family sued Tesla, but also the driver, and the case raised questions about whether the driver was even 'driving' at the time. Luna: I think that's the key tension. As soon as you remove the human from the loop, responsibility becomes diffuse. We don't know who to blame, so maybe no one gets blamed, and that's dangerous. Lucas: Look, honestly, if today's conversation was worth a coffee to you, that's the link — buy me a coffee dot com slash fexingo. Listener support is what keeps this show ad-free and focused on the questions that actually matter. Luna: And it's a small gesture that really does make a difference. Lucas: So back to the programming dilemma. Some researchers argue we shouldn't even try to code ethical decisions into cars. Instead, they say, focus on making the technology so safe that accidents are virtually impossible. But that's not realistic, at least not yet. There will always be edge cases — a child running out from between parked cars, a deer leaping onto the highway, a sudden lane closure. Luna: And the car has to do something. It can't just freeze. Lucas: No, it can't. And the reality is that even if you try to avoid programming ethics explicitly, you're making ethical choices by default. Every parameter — following distance, braking aggressiveness, lane-change hesitation — encodes a value judgment about risk. A car that brakes harder to avoid a collision is prioritizing safety over comfort. But what if that hard brake causes a rear-end collision? Then you've traded one risk for another. Luna: So we're back to the trolley problem, but now it's not a philosophy seminar — it's a software update. Lucas: Exactly. And that's why groups like the IEEE and the German Ethics Commission on Automated and Connected Driving have tried to establish guidelines. Germany actually passed the first set of national ethics rules for autonomous vehicles back in 2017. They say that protecting human life is the top priority, that the car cannot make distinctions based on age or gender, and that the manufacturer is always responsible for the vehicle's behavior. Luna: Interesting — so Germany's rules explicitly forbid the kind of age-based programming that the Moral Machine data showed some cultures prefer. That's a regulatory choice. Lucas: It is. And it creates a clear standard for companies operating there. But in the US, we don't have a federal framework yet. The National Highway Traffic Safety Administration has issued voluntary guidelines, but nothing with the force of law. So companies are essentially left to make their own ethical choices, with the threat of lawsuits as the only real check. Luna: That sounds like a recipe for inconsistency. One company might program its cars to be very cautious, another might prioritize speed and efficiency. And consumers might not even know what they're buying into. Lucas: Right, and that's a transparency issue. Some advocates say there should be a mandatory 'ethics label' on autonomous vehicles, like a nutrition label for moral algorithms. You'd be able to see: this car prioritizes occupants over pedestrians, it treats all ages equally, it will never swerve to avoid an animal. That way, consumers can make an informed choice. Luna: I like that idea. But I wonder: would most people even read it? And would they understand the trade-offs? If you see 'prioritizes occupants,' you might think 'great, I'm safe,' without realizing that means it might run over a child to save you. Lucas: That's the challenge. And it's why some ethicists argue that we shouldn't let manufacturers decide at all. They say the government should mandate a single ethical standard for all autonomous vehicles — a kind of social contract that applies to every car on the road. But that raises its own problems: whose values get encoded? The Moral Machine data showed deep cultural divides. A one-size-fits-all rule might not fit anyone particularly well. Luna: So we're stuck. We have to make a decision, but any decision is going to upset a lot of people. Lucas: We are. And here's another layer: what about the scenario where the car has to choose between hitting one person or another? Say, a motorcyclist wearing a helmet versus a pedestrian crossing legally. The car can't tell from its sensors whether the motorcyclist is wearing a helmet — it just sees a shape. So even if you wanted to make a fine-grained ethical choice, the technology might not support it. Luna: That's a really practical limitation. The sensors have to be good enough to make the distinctions that the ethics code demands. Otherwise, you're just pretending to have a sophisticated moral algorithm. Lucas: Exactly. And that's where the conversation often stalls. But I think there's a fundamental question that doesn't get asked enough: should we even be building machines that make life or death decisions? Maybe the right answer is to slow down. To require a level of safety so high that these dilemmas become statistically negligible — like one-in-a-billion events. Luna: But that would delay deployment by years, maybe decades. And in the meantime, human-driven cars are killing tens of thousands of people every year. There's a real opportunity cost. Lucas: There is. And that's the classic utilitarian counterargument: imperfect autonomous cars are still safer than humans. So by delaying, you're actually causing more deaths. But that argument only works if the public accepts the trade-off. And if a single high-profile accident — say, a child killed by a self-driving car — erodes trust, the whole industry could be set back. Luna: I think that's already happening. Surveys show that a majority of Americans are still uncomfortable with fully self-driving cars. The trust just isn't there. Lucas: Right, and that trust is fragile. One way the industry is trying to build it is through transparency — publishing safety reports, doing public testing, working with regulators. But the ethical core remains unresolved. We are asking machines to act in ways that we, as a society, haven't agreed on. And until we have that agreement, every self-driving car on the road is an experiment. Luna: An experiment with real consequences. I can't think of a more pressing ethical question in AI right now. Lucas: Me neither. And I think the next five years are going to force a reckoning. As more autonomous vehicles hit the road, the probability of a truly ambiguous, high-profile accident increases. And when that happens, the public debate will finally move from philosophy seminars to headlines. Luna: Let's hope the headlines don't have to be tragic to spark that debate. Lucas: Yeah. That's the hope.