Latest / AI Ethics with Fexingo: Bias, Safety, and Responsible Artificial Intelligence / When Your AI Insurance Adjuster Lowballs Your Claim Based on Your Photo
Transcript
- Lucas: You take a picture of your dented bumper, upload it to your insurance app, and within seconds an AI spits out a repair estimate. Feels efficient, maybe even fair — no human bias, just a machine looking at the damage. Except that machine might be systematically lowballing you based on what your car looks like and where you live. Luna: Wait — the AI is looking at the car, not the owner. How can it discriminate on things like age or neighborhood? Lucas: It's not reading your address or your date of birth. It's looking at the car's visible condition, the model year implied by design cues, maybe the background of the photo — a driveway versus a parking lot versus a street with certain housing. And if its training data was mostly photos of newer, luxury cars in well-maintained areas, it learns to associate certain visual patterns with higher repair costs. Luna: So an older sedan photographed in front of a modest home gets a lower estimate because the AI hasn't seen enough examples of that combination. Lucas: Exactly. And we have a concrete case from just a few months ago. In February 2026, a California driver named Maria Torres filed a claim with one of the top three auto insurers — I won't name them because the lawsuit is ongoing. She had a 2012 Honda Civic with a cracked taillight. Uploaded a photo from her driveway. The AI estimated the repair at $340. She took it to a body shop — actual quote was $620. Luna: That's nearly double. Did she challenge it? Lucas: She did, and after a human review the insurer approved the higher amount. But here's the thing — a consumer advocacy group, the Center for Digital Integrity, did a study last month. They submitted 2,000 photos of damaged vehicles to five major insurers' AI claims tools. For cars older than ten years, the initial AI estimate was on average 23 percent lower than what a human adjuster later approved. For cars in zip codes with median income below $50,000, the gap was 18 percent. Luna: Those numbers are hard to ignore. What's the technical reason? Is it the training data itself or the model architecture? Lucas: Mostly the training data. These models are trained on thousands of past claims photos paired with the final payout. But insurers naturally have far more claims on newer vehicles — they're more expensive to insure, so they get filed more often. A 2023 model BMW gets claimed maybe ten times as often as a 2012 Honda. So the model sees lots of BMWs with accurate high payouts, and few Hondas. It learns that when a car looks like a BMW, the payout should be high; when it looks like an older economy car, the payout should be lower. Even if the damage is identical. Luna: So it's not malicious. It's a statistical artifact of an unbalanced dataset. But the effect is systemic bias. Lucas: Right. And because it's a deep neural network processing pixel data, it's very hard to trace exactly which features drove the decision. Did it see the model year badge? The shape of the headlights? The condition of the paint? The tree in the background? It's a black box. That's part of why regulators are taking notice. Luna: What are they doing about it? Lucas: California's Department of Insurance proposed a rule in March 2026 requiring any insurer that uses AI for claims estimation to run an annual bias audit. They have to submit results showing that estimates don't systematically vary by vehicle age, location, or any proxy for race or income. The rule hasn't been finalized yet — comment period ends in August. But it's a signal. New York and Illinois are considering similar measures. Luna: Is there a technical fix? Can you just add more training data of older cars? Lucas: That's part of it. But it's not just more photos — it's having accurate payout data for those photos. Insurers need to collect a balanced dataset where each vehicle age bracket has hundreds of examples with verified human-adjusted payouts. Then they can train a model that learns the true relationship between visible damage and cost, not just the statistical echo of claims frequency. Luna: Some companies are already doing this. I read about a startup called FairClaim that offers an audit tool specifically for this problem. They claim their model reduces the gap to under 5 percent. Lucas: I've seen their work. They focus on what they call 'damage-equivalent pairs' — two photos of different vehicles with the same type and severity of damage, then force the model to assign the same estimate. It's a clever training technique. But adoption is slow because insurers' current models are already deployed and making money. Replacing them costs time and disrupts workflows. Luna: And there's the consumer's dilemma. Most people accept the AI estimate because it's fast and they trust the technology. They don't realize they might be leaving money on the table. Lucas: That's why transparency matters. If you know the AI might be lowballing you, you're more likely to get a second opinion. But right now, most insurers don't disclose that an AI generated the initial estimate. They just present it as 'your claim value.' Some states are pushing for mandatory disclosure labels — like 'This estimate was generated by an automated system and may not reflect actual repair costs.' Luna: It reminds me of the early days of credit scoring. People didn't know their score was computed by an algorithm, and it took regulation to force transparency. Lucas: Exactly. And we're in that same moment for insurance AI. The technology is already embedded, but the accountability mechanisms are still being built. If today's episode gave you something useful — maybe a way to think about your own claim process or just a clearer picture of how these systems work — that's exactly what we're here for. Luna: Yeah, it's that kind of practical insight that makes this show valuable. And the reason we can keep doing it without ads is listener support. Lucas: Right. If you want to help keep these conversations going, you can head to buy me a coffee dot com slash fexingo. It's a simple way to say this matters to you. Luna: And it really does make a difference. Every contribution goes straight into keeping the show independent. Lucas: Back to the insurance bias issue — there's another dimension I want to touch on. The same computer vision models are now being used for property claims, not just auto. Roof damage after a storm, for example. And early evidence suggests similar biases: older homes, rural properties, lower-value structures get systematically undervalued. Luna: Because the training data is dominated by suburban homes with newer roofs. Lucas: Exactly. So this isn't a narrow auto-claims problem. It's a pattern that recurs wherever AI is trained on historical data that reflects existing inequalities. The fix isn't just technical — it's about governance, auditing, and giving consumers the right to challenge a machine's judgment. Luna: I want to ask about the human side. When a person does get a human adjuster, does that actually fix the bias? Or is the human adjuster influenced by the AI's initial number? Lucas: That's a really good question. There's research from the Journal of Risk and Insurance last year showing that when human adjusters see an AI estimate first, they tend to anchor on it. Even if they're supposed to do an independent assessment, the final payout ends up closer to the AI's number than if they started from scratch. So the bias can propagate even with human oversight. Luna: So the solution needs to include removing the AI number from the human adjuster's screen until they've done their own evaluation. Lucas: Some insurers are testing that — blinding the human to the AI estimate. But it's not common. And it adds friction to a process designed for speed. The tension between efficiency and fairness is at the heart of this whole debate. Luna: It feels like we're in a phase where the technology is outpacing the ethics. But the regulatory push is real, and cases like Maria Torres's are forcing the conversation. Lucas: That's the story of AI ethics in 2026. The algorithms are already in production, and we're scrambling to build the guardrails after the fact. But episodes like this — where we can point to specific numbers, specific cases, specific proposed rules — give me some hope that we're moving from awareness to action. Luna: Next time you upload a photo for an insurance claim, you'll know to ask: was that estimate made by a machine, and does it see me fairly?