Latest / AI Ethics with Fexingo: Bias, Safety, and Responsible Artificial Intelligence / Your AI Insurance Adjuster Is Judging Your Photos
Transcript
- Lucas: Luna, I want to talk about something that probably happened to someone listening to this show. You file an insurance claim, you snap a photo of the damage with your phone, upload it to the app, and within seconds an AI decides whether to pay you or not. Luna: That is already happening. I know Lemonade has been doing this for years with their AI Jim, but I think it's gone way beyond just chatbots. Lucas: Right. Lemonade is the poster child, but the big incumbents — Allstate, State Farm, Progressive — they're all rolling out computer vision models that review photos submitted with claims. And they're also using photos from the underwriting side now. You apply for a policy, you snap a few pics of your living room, your front door, the roof. The AI evaluates risk in real time. Luna: So it's not just claims. It's deciding whether to insure you at all, and at what price, based on the visual state of your home. Lucas: Exactly. And here's the specific case I want to drill into. A few months ago, there was a widely shared story — I think it was on Reddit, then picked up by consumer advocates — about a homeowner in Florida who filed a claim for wind damage after a minor storm. The policyholder uploaded photos of a few missing shingles. The AI flagged the claim as 'high risk of fraud' and denied it. Luna: What was the basis? The AI saw something in the photo? Lucas: Turns out the AI had been trained on images of well-maintained suburban homes. The policyholder's house had peeling paint on the fascia board — not related to the storm damage at all. But the model associated 'peeling paint' with 'property neglect' and therefore with 'likely fraudulent claim.' It was a textbook spurious correlation. Luna: So the AI punished the homeowner for having a house that didn't look like the training data. That is exactly the kind of bias we've talked about with hiring algorithms and facial recognition. Lucas: It's the same pattern. The training data is not representative of the real world. Most property insurance training sets are dominated by photos from suburban and affluent areas — better lighting, newer homes, well-trimmed landscaping. Homes in older neighborhoods, rural areas, or low-income urban areas look different. And the AI learns that 'different' means 'risky.' Luna: I remember a study from MIT a couple years ago — they tested a computer vision model on photos of kitchens from Airbnb listings and found it could predict household income with surprising accuracy just from cabinet style and countertop material. Lucas: Right, and insurance companies are absolutely doing that, even if they don't admit it. If the AI sees laminate countertops and old appliances, it might infer lower income, which correlates with higher claims risk in their models, and then your premium goes up or you get denied. But the correlation is not causation — and it's also a proxy for race and class. Luna: And the homeowner has no idea why. You just get a letter saying 'your property does not meet our underwriting standards based on visual assessment.' No explanation of what the AI saw. Lucas: That's the black box problem again. Some states have started to regulate this. California, for example, has a regulation — I think it's from the Department of Insurance — that requires insurers to disclose when a decision was made or influenced by an algorithm. But enforcement is weak, and the companies argue it's proprietary. Luna: And even when they disclose, it's usually buried in fine print. The real issue is whether these models are accurate and fair. What does the data say about denial rates? Lucas: The data is sparse because insurers don't publish it. But a 2025 report from the Consumer Federation of America looked at complaint data from several states and found that claims denials based on photo review were disproportionately concentrated in majority-Black and Hispanic neighborhoods — even controlling for property value and claim type. Luna: So the algorithm is replicating historical redlining, but now through pixels instead of maps. Lucas: Exactly. And it's harder to fight because you can't point to a map and say 'this line is discriminatory.' The AI says it saw a 'high-risk roof profile' or 'inadequate property maintenance indicators.' How do you contest that? You'd need to audit the model, which costs tens of thousands of dollars. Luna: I want to talk about the privacy angle too. When you snap a photo for your insurance app, you're not just sending a picture of the damage. You're sending a picture of your living room — maybe with family photos on the wall, a visible prescription bottle on the counter, a laptop open on the table. Lucas: The AI can extract all of that. There's no guarantee that data is only used for the claim. The privacy policies for these apps are broad — they say things like 'we may use images to improve our models.' That means your photo of a broken window could end up training a model that evaluates your future applications for credit or even rental housing, if the insurer shares data with affiliates. Luna: And there's no opt-out. If you want insurance, you have to submit photos. Some companies are even requiring a video walkthrough of your entire home now. Lucas: That's becoming more common for new policies. Lemonade, Hippo, Kin — the so-called insurtechs — they all use some form of visual inspection. The pitch is convenience: you don't need an agent to come out, you can get a quote in five minutes. But the trade-off is that you're handing over a detailed visual record of your most private space, and the AI is making judgments about you based on aesthetics that correlate with socioeconomic status. Luna: I wonder about the long-term effect. If these models become widespread, will people start staging their homes for the insurance AI the way they stage for real estate photos? Clean up, buy new throw pillows, hide the clutter? Lucas: They already are. There are TikTok influencers giving tips on how to pass the AI underwriting photo review. 'Make sure your lawn is mowed, remove any visible trash, paint your front door a neutral color, and don't have a trampoline in the backyard.' That's literally advice I saw last week. Luna: So the AI is effectively shaping behavior — making people conform to a middle-class aesthetic standard to avoid being penalized. That's a form of social control. Lucas: It's a subtle but powerful one. And it disproportionately affects renters and lower-income homeowners who may not have the time or money to paint their fascia board or replace an aging roof. They're being priced out of insurance, which in some cases means they can't get a mortgage or they lose their coverage. Luna: There's a documented case in Florida after Hurricane Ian — some homeowners whose roofs were visibly damaged were denied because the AI said the roof was 'pre-existing wear and tear.' They had before and after photos proving the damage was from the storm, but the AI overrode the human adjuster. Lucas: That's a nightmare scenario. And it points to a deeper problem: these models are often deployed without adequate validation against real-world outcomes. The insurer saves money by denying claims, so there's a financial incentive to set the threshold low. The human adjuster is supposed to be the check, but in many companies, the AI's decision is final unless you appeal, and most people don't appeal. Luna: What can a listener do if they think their claim was unfairly denied by an AI? Lucas: First, demand a human review. Some states have laws that guarantee that right. Second, ask for a specific explanation of what the AI found — request the 'algorithmic impact statement' if your state requires it. Third, file a complaint with your state insurance commissioner. And if enough people do that, regulators start paying attention. Luna: And on the policy side, is there any legislation in the works? Lucas: A few bills. The Algorithmic Accountability Act at the federal level has been proposed but hasn't passed. New York and Colorado have introduced state-level bills requiring bias audits for AI used in insurance. But the insurance lobby is strong, and the industry argues that AI reduces costs and speeds up claims for everyone. They're not wrong that it can be faster — but fairness matters too. Luna: I think what's interesting is how this intersects with the broader conversation about AI and surveillance. Insurance is essentially an information business — the more you know about the risk, the better you can price it. But when that information is extracted from people without their meaningful consent, it crosses a line. Lucas: Absolutely. And it's not just photos. Some insurers are using social media data, credit scores, even your grocery purchases if you share a loyalty card. The photo is just the most visual example. But it's the one where the bias is easiest to see if you look. Luna: Which brings us back to the ethics. If we accept that these models have measurable bias, and companies deploy them anyway because they're profitable, what does that say about our regulatory system? Lucas: It says the system is broken. But there's also a role for consumers — and listeners of this show — to push back. Ask your insurer about their AI. If they won't tell you, consider switching. There are still mutual insurers and smaller companies that use traditional adjusters. It might take longer, but you get a human. Luna: And on that note — quick honest thing. This kind of reporting, digging into these cases, it takes time. And we're able to do it because a handful of listeners chip in monthly through buy me a coffee dot com slash fexingo. That's literally what funds making this many of these episodes, ad-free. Lucas: Yeah, it's a small group, but it makes a real difference. We don't have sponsors, we don't run ads. It's just listeners who believe in this work. So if you've gotten something out of today's episode, that's one way to keep it going. Luna: And we're not asking for a lot — a few dollars a month goes a long way when you add up the community. So if you're able, you know where to find us. Lucas: Alright, back to the AI insurance photo problem. One more thing I want to mention: there's a promising technology called 'algorithmic recourse' — it gives people a way to change the AI's decision by taking specific actions. For example, if the AI denied your claim because of a visual cue, the system could tell you 'trim the tree overhanging your roof and resubmit the photo.' That's being piloted in some European insurers. Luna: That's actually a good idea. It turns the black box into a transparent checklist. But it only works if the AI's reasons are valid and not biased. Lucas: Right. And that requires auditing. So the accountability piece is still essential. But at least it gives the consumer a path forward. Luna: So to wrap up: if you take one thing from this episode, it's that the next time you snap a photo for an insurance app, think about what else that image might be saying about you — and ask questions if something feels off. Lucas: And if you get denied, don't assume the AI was right. Appeal. That's your right.