Latest / AI Ethics with Fexingo: Bias, Safety, and Responsible Artificial Intelligence / When Your AI Landlord Refuses to Renew Your Lease
Transcript
- Lucas: Luna, I want to start with a story that landed in my inbox last week. A listener in Austin named Priya told me she got a notice from her apartment complex that her lease would not be renewed. No reason given. Just a letter saying 'based on our risk assessment.' Luna: She asked what risk assessment? And they said it was generated by their software platform, and they couldn't share the details because the algorithm is proprietary. Lucas: Exactly. Now, Priya has never missed a rent payment in four years. She has no complaints on file. But her apartment management uses a tenant screening system that ingests data well beyond credit scores. Social media activity, public court records, even the sentiment of posts. And the model flagged her as high risk for eviction. Luna: So she's being judged by an algorithm that she can't see, can't challenge, and that might be factoring in things like the fact that she posted about a rent strike in another city five years ago. Lucas: That's the part that makes this hard to fix. Priya never got a hearing. She never saw the inputs. She just got a form letter. And this is not an outlier. A 2025 report from the National Fair Housing Alliance found that more than 40 percent of large property management companies now use some form of AI for tenant screening or lease renewal decisions. Luna: And a lot of those systems are built by companies like RealPage, which has been in the news for other reasons, or Yardi, or even Zillow's rental manager tools. Lucas: Right. And the fundamental problem is that these systems optimize for one thing: minimizing the landlord's risk. They are not designed to be fair to tenants. So if the model learns that tenants from a certain ZIP code have a slightly higher eviction rate historically, it will penalize anyone from that ZIP code — regardless of their individual payment history. Luna: That sounds like redlining, just automated. Lucas: It is algorithmic redlining. And because it's a machine learning model, the bias can be subtler. For example, the model might weigh the number of times you've changed your address in the last five years, or whether you've ever had a utility bill sent to collections. But those proxies can correlate with race or income, and the landlord can say 'we're not using race, we're using address changes.' Luna: Let me jump in with a data point. The Federal Trade Commission did a study in 2024 where they looked at 12 popular tenant screening algorithms. They found that 9 of them produced a statistically significant difference in denial rates between white applicants and applicants of color, even when controlling for income and credit history. Lucas: And that's 9 out of 12. That's not a few bad apples. That's a systemic issue. Now, the Fair Housing Act was written in 1968, and it has been updated to include disparate impact theory. But applying that to an opaque algorithm is really hard. How do you prove disparate impact when you can't see the inputs? Luna: You have to reverse-engineer the model. And that's expensive, and usually requires a class-action lawsuit. Priya isn't going to sue. She's just going to move. Lucas: And that's the quiet harm. People don't fight it. They just relocate, and the algorithm's bias becomes self-reinforcing, because the landlord never sees negative feedback. The model says 'don't renew,' the tenant leaves, the landlord thinks the model is working perfectly. Luna: It's a closed loop. And I think that's something our listeners can really relate to, because housing is so fundamental. So before we go deeper into how these models are built and what can be done, I want to mention something behind the scenes. Lucas: Sure, go ahead. Luna: This show is entirely listener-supported. We don't run ads, we don't have sponsors. And that's possible because a small group of our listeners chip in a few bucks a month through buy me a coffee dot com slash fexingo. That's literally what covers the research, the hosting, the transcription — everything. Lucas: Yeah, it's a small community, but it makes a big difference. If you get value out of episodes like this — and you want to see us keep digging into stories like Priya's — that link is buy me a coffee dot com slash fexingo. No pressure, but every contribution helps us stay independent. Luna: Alright, back to Priya's case. So let's talk about what's actually inside these models. Lucas, you've looked into how one of the biggest tenant screening platforms works. What did you find? Lucas: I looked at a platform called SafeRent, which is used by about 2 million rental units across the US. Their algorithm uses over 50 variables. Some are traditional: credit score, rental history, income. But others are surprising: the number of times you've moved in the last three years, whether you've ever had a public records filing — not eviction, just any filing — and even the number of social media connections you have in the area. Luna: Social media connections? Why would that matter? Lucas: The idea is that tenants with fewer local connections are more likely to break a lease or move out early. But it's a huge proxy for socioeconomic status and mobility. If you're a young professional who just moved to a new city for a job, you have fewer connections. The model penalizes you for being new. Luna: So the algorithm essentially punishes people for being transient, which disproportionately affects lower-income renters, students, and people in the gig economy. Lucas: Exactly. And here's the kicker: when Priya asked for a copy of her screening report, the property manager said the algorithm's output was 'proprietary trade secret' and they could not share it. Now, under the Fair Credit Reporting Act, she is entitled to see the data that went into the decision. But the model's logic — the weights, the interactions — that's protected as intellectual property. Luna: So there's a legal tension between a tenant's right to know and a company's right to protect its algorithm. And so far, the courts have been split. Lucas: There was a case in New York in 2024 where a judge ruled that a tenant could not compel the landlord to disclose the algorithm's code, because it was a trade secret. But the judge did order the landlord to provide a list of all factors that were considered. And that list was 47 items long. Luna: That seems like a step forward, but 47 factors is still opaque. How do you know which ones mattered most? Lucas: You don't. Because many of these models use gradient-boosted trees or neural networks, which are inherently non-interpretable. You can't just look at the coefficients like a linear regression. So even with the list, you can't reconstruct the decision. Luna: And that's a core problem for AI ethics in housing: interpretability. If a model makes a life-altering decision about where someone lives, the person affected deserves to understand why. Lucas: I agree. And some cities are starting to act. In 2025, San Francisco passed an ordinance requiring any automated tenant screening tool to undergo an annual bias audit, with results made public. And Seattle is considering a 'right to explanation' law that would require landlords to provide a plain-language summary of how the algorithm reached its decision. Luna: Do any of those laws actually help Priya? She got her notice in Austin, which has no such ordinance. Lucas: Not directly. But they set a precedent. And there is a growing movement among tenant advocacy groups to push for federal regulation. The Algorithmic Accountability Act, which has been introduced in various forms since 2019, would require impact assessments for high-risk AI systems, including tenant screening. Luna: But it hasn't passed. And even if it did, enforcement would be slow. In the meantime, what can a tenant do if they suspect they're being screened by an AI? Lucas: First, ask for your screening report. Under the FCRA, you have the right to a copy of any consumer report used in an adverse action. Second, look for patterns. If the denial or non-renewal seems out of step with your history, request a manual review. Some property managers have a human override process, even if they don't advertise it. Luna: And third, contact a fair housing organization. Groups like the National Fair Housing Alliance have started building databases of ai related complaints, and they can help you determine if there's a pattern of discrimination. Lucas: But here's the uncomfortable truth: even with all that, the system is stacked against the tenant. The model is proprietary, the landlord has limited incentive to investigate, and the tenant has a limited time to find a new place. So most people just move on. Luna: And that's why regulation matters. Because individual action can only do so much when the system itself is opaque. Lucas: I think we'll look back at this era and be shocked that we let algorithms decide who gets to rent an apartment without any real oversight. The technology is moving faster than the law, and tenants are paying the price. Luna: Lucas, what's one thing you think our listeners should take away from this episode? Lucas: I'd say: if you're renting, ask your landlord what screening tools they use. And if you get a denial or non-renewal that doesn't make sense, don't accept 'proprietary algorithm' as an answer. You have rights. They may be limited, but they exist. Luna: And if you want to support more episodes like this one, you know where to find us. Thanks for listening.