Latest / AI Ethics with Fexingo: Bias, Safety, and Responsible Artificial Intelligence / How AI Models Are Learning Racism From Historical Redlining Maps
Transcript
- Lucas: So there's a 1937 map of Philadelphia, color-coded into four grades. Green for 'best,' blue for 'still desirable,' yellow for 'definitely declining,' and red for 'hazardous.' That red grade — that's where the term 'redlining' comes from. Luna: And those maps were drawn based on race, not just economics. Black neighborhoods got the red grade regardless of income or property condition. Lucas: Exactly. The Home Owners' Loan Corporation — HOLC — explicitly instructed surveyors to note the 'racial composition' of each area. A neighborhood with any non-white population was automatically marked down. Now fast-forward to 2026. Banks and insurers are feeding property-level data into AI models that estimate home values, set premiums, and approve loans. Luna: And somewhere in that training data, those old HOLC grades are still hanging around? Lucas: Not always the grades themselves, but the patterns they created. A home in a formerly redlined part of Detroit might have lower sale prices, less renovation history, and higher insurance claims — all because of decades of disinvestment that started with that map. The model doesn't see 'redlined.' It sees 'low transaction volume' and 'higher claim frequency' and learns to penalize those properties. Luna: So the bias is baked into the real estate data itself. The model just mirrors the past. Lucas: Right. And a 2025 study from the Federal Reserve Bank of Philadelphia tried to quantify this. They took a standard automated valuation model used by several large lenders and tested it on properties in Philadelphia and Baltimore. Even after removing race as an explicit variable, the model's error rates were higher in historically redlined neighborhoods. It systematically undervalued homes in those areas by an average of 4.3 percent compared to similar homes in green-graded zones. Luna: That's not a small gap. Over thirty years, a four percent undervaluation compounds into tens of thousands of dollars of lost equity for a homeowner. Lucas: And it affects insurance too. A separate paper from the Consumer Federation of America this January found that ai based homeowners insurance models were charging 12 to 18 percent higher premiums in neighborhoods that had been redlined in the 1930s, even after controlling for crime rates, weather risk, and claim history. The models had picked up on proxy variables like average lot size, distance to the nearest fire station, and the age of the housing stock — all correlated with the HOLC grade. Luna: Which is exactly what fair lending advocates have been warning about. Proxies can be just as discriminatory as the original protected class. Lucas: The thing is, the industry response has mostly been 'we'll add fairness constraints to the model.' But that Philadelphia Fed study showed that standard fairness metrics — like demographic parity or equal opportunity — didn't catch the redlining effect. The model passed those tests because the proxy variables created a statistically defensible rationale for the disparities. The model wasn't 'unfair' by the math; it was just reflecting historical fact. Luna: So the math itself is blind to history. That's a deeper problem than just tweaking the algorithm. Lucas: It is. And it's why some regulators are pushing for more than just fairness audits. The Consumer Financial Protection Bureau issued a guidance update in March 2026 that explicitly says lenders must check whether their AI models produce outcomes that correlate with historical redlining maps, even if the maps weren't used as input. They want a 'redlining impact analysis' alongside the standard disparate impact test. Luna: That's a pretty concrete regulatory demand. How are lenders reacting? Lucas: Quietly, mostly. A few large banks have started running those checks internally. JPMorgan Chase published a white paper in April describing how they layer a geospatial model on top of their valuation AI that flags any prediction that falls within a historically redlined tract. If the flag goes up, a human appraiser has to sign off on the value. But smaller lenders say they don't have the data or the staff to do that. Luna: And the insurers? I imagine they're even less eager to admit their models have a redlining problem. Lucas: You'd think so, but actually a couple of the bigger property insurers have started collaborating with academic researchers. There's a project at the University of California, Berkeley right now that's working with State Farm and Allstate to reverse-engineer their pricing models and test for redlining correlations. Early results from a preprint last month showed that zip code alone is a surprisingly strong proxy for the HOLC grade. Models that use zip code as a feature are effectively importing the redlining map. Luna: Zip code — the most basic geographic variable. You can't exactly run a mortgage or insurance model without location data. Lucas: You can't. And that's the crux of it. The problem isn't that a few bad models have a bug. The problem is that the entire data ecosystem — property values, claim histories, sale prices, renovation permits — is a palimpsest of redlining. Every new model trained on that data inherits the bias, even if the original maps have been digitized and forgotten. Luna: So what's the real fix? More regulation? Different model architectures? Lucas: I think the honest answer is that there's no purely technical fix. You can't de-bias data that is biased at the source. The Philadelphia Fed researchers suggested something more radical: train models on synthetic data that simulates what property values and claims would look like in a counterfactual world without redlining. Then compare the model's real-world predictions to that synthetic baseline. Luna: That's fascinating — and also really hard to do well. You'd have to build a whole alternative economic history. Lucas: Exactly. It's speculative. But it at least acknowledges that the bias isn't in the algorithm — it's in the world the algorithm is trying to describe. Luna: And speaking of acknowledging reality — a quick honest thing. A handful of listeners chip in monthly through buy me a coffee dot com slash fexingo, and that's literally what funds making episodes like this. No ads, no sponsors, just people who find these conversations useful. Lucas: Yeah, it's a small group, but it keeps the show going. And it means we can spend time on a topic like this without worrying about whether it fits a sponsor's agenda. Luna: Exactly. So back to the maps — you mentioned that even zip codes carry the redlining signal. Are there any models that have successfully avoided it? Lucas: A few experimental ones. There's a startup called FairValue that built a valuation model using only property characteristics — square footage, number of bedrooms, lot size, roof condition — and deliberately excluded any location-based feature. They claim their error rate is uniform across neighborhoods. But their model also has a higher overall error rate because it's missing so much context. You trade bias for accuracy. Luna: So it's a trade-off, not a solution. Lucas: Right. And in practice, lenders won't use a less accurate model if it hurts their bottom line. So the regulatory push is probably the stronger lever. The CFPB guidance is a start, but it's not a rule — it's just guidance. Actual rulemaking would take years. Luna: And in the meantime, the models are getting more powerful. Training on more data. Amplifying the pattern. Lucas: That's the concern. There's a feedback loop here: the more these models are used, the more they reinforce the pricing and valuation disparities that redlining created. A home in a historically redlined area gets undervalued, so it sells for less, so the next model sees a lower sale price, and the cycle continues. Luna: It's like the model is learning discrimination from the past and then re-teaching it to the future. Lucas: Exactly. And that's why just auditing for fairness isn't enough. You have to intervene at the data level, or at the regulatory level. Otherwise, the algorithm becomes a perfectly efficient engine for perpetuating a century-old injustice. Luna: And that's the kind of thing that keeps me up at night — not that AI is going to become sentient, but that it's going to become a perfect record of our worst historical patterns. Lucas: Yeah. The redlining maps are a concrete example of that. They're not just a history lesson. They're still shaping who gets a loan, who gets insurance, and at what price. And now they're shaping it through code that nobody looks at.