Latest / AI Ethics with Fexingo: Bias, Safety, and Responsible Artificial Intelligence / When Your AI Landlord Set Eviction Without Human Review
Transcript
- Lucas: So back in April, a property management firm in Cleveland called — let's call it 'Premier Property Group' — rolled out an AI system to automate its lease-violation enforcement. Luna: And by 'automate', you mean auto-generate eviction notices? Lucas: Exactly. The AI would scan camera footage and resident complaints to flag things like unauthorized pets, excessive noise, or unregistered guests. Then it would automatically generate a notice to vacate — no human reviewed it before it hit the tenant's door. Luna: How many people got those notices? Lucas: Over 200 households over a three-month period. Local tenant advocates got wind of it, and the city council started asking questions. When they actually audited the process, 87 percent of the contested notices were withdrawn once a human property manager reviewed the evidence. Luna: Eighty-seven percent. So the AI was wrong — or at least seriously overzealous — in nearly nine out of ten cases. Lucas: Right. And the errors weren't trivial. The AI flagged a tenant for having a dog — turned out it was a service animal. Another tenant got cited for 'excessive noise' that was actually a construction crew working on the building next door. The system had no context. Luna: Did Premier Property Group design the AI themselves, or did they buy it off the shelf? Lucas: They licensed it from a proptech startup called 'SafeStay Analytics'. SafeStay's pitch is that their AI reduces 'nuisance tenancies' and cuts legal costs for landlords. They claim a 30 percent drop in eviction-related legal fees for their clients. Luna: But at the cost of what? Tenants losing their homes over false positives. Lucas: Exactly. And there's no federal standard for how much human review is required before an AI can initiate an eviction. Some states have laws about landlord-tenant procedures, but they were written long before automated systems existed. Luna: So what happened in Cleveland? Lucas: After the audit, Premier Property Group paused the automated notice generation. They now require a human property manager to sign off on any eviction notice the AI recommends. But the AI still flags violations and drafts the notice — the human just has to click 'approve'. Luna: So the human is basically a rubber stamp. Lucas: That's the concern. Researchers at the AI Now Institute have studied this — they call it 'human-in-the-loop washing'. The company can say 'a human reviews every decision', but if the human reviews dozens of cases per hour, they're likely to defer to the machine. Luna: And the tenant doesn't even know an AI made the initial decision. The notice just says 'you have violated your lease terms'. Lucas: Right. No disclosure that an algorithm flagged the violation. So the tenant has no grounds to challenge the AI's reasoning. They can only argue against the evidence itself — which is often a camera image or a complaint log the AI interpreted. Luna: This feels like a microcosm of a bigger problem. AI systems making high-stakes decisions about housing, employment, credit — all with minimal oversight. Lucas: It absolutely is. And what's interesting is that some cities are starting to legislate. New York City's Local Law 144 requires bias audits for AI hiring tools. There's a bill in California — AB 302 — that would require landlords to disclose if they use automated systems for tenant screening. Luna: But nothing that mandates a meaningful human review before an eviction. Lucas: Not yet. And the housing advocacy groups argue that the burden should be on the landlord to prove the AI is accurate and fair, not on the tenant to disprove it after the fact. Luna: What about SafeStay? Have they changed their system? Lucas: They issued a statement saying they've updated their model to flag false positives better. But they didn't release any third-party audit. And they're still selling to property managers across the country. Luna: So the Cleveland situation might be happening in other cities right now, we just don't know. Lucas: Right. And the industry is growing fast. The proptech sector attracted over $30 billion in venture capital in 2025. A lot of that goes into 'efficiency' tools that automate landlord functions. Luna: It's one of those cases where efficiency and fairness are in direct tension. You can move fast and evict people quickly, or you can be careful and slow. Lucas: Yeah. And the companies argue that the AI is actually more consistent than a human manager — it applies the same rules to everyone. But that only works if the rules themselves are fair and the AI interprets them correctly. Luna: And it doesn't help that the training data might reflect historic biases in evictions. If the data over-represents certain neighborhoods or demographics, the AI will replicate that. Lucas: Exactly. There's a 2024 study from Princeton that found AI eviction models trained on historical data were 40 percent more likely to flag units in predominantly Black and Hispanic neighborhoods — even after controlling for income and lease-violation rates. Luna: So you get a feedback loop: biased data leads to biased decisions, which generates more biased data for the next model. Lucas: That's the core problem. And without regulation, the companies have little incentive to break the cycle. Luna: What about the tenants in Cleveland? Did any of them actually get evicted from the AI notices that weren't contested? Lucas: According to the city's report, 13 households didn't contest the notice and were evicted. The city is now trying to reach those tenants and offer legal aid, but it's unclear how many can be helped retroactively. Luna: Thirteen families lost their homes because an algorithm made a mistake. Lucas: That's the human cost. And it's why the AI ethics community is pushing for a 'right to explanation' in housing — you should be able to know exactly why a decision was made about your home. Luna: It seems like a basic transparency measure. If a human manager evicts you, you can ask them why. If an AI does it, you should be able to see the inputs and the logic. Lucas: Right. And some European countries already have that under GDPR. If an automated decision has legal effect on you, you have the right to human intervention. But in the US, there's no equivalent. Luna: So what's the realistic path forward? More local laws? Federal action? Industry self-regulation? Lucas: I think it'll be a mix. Cities like Cleveland and New York are acting. There's a bill in Congress — the Algorithmic Accountability Act — that's been proposed a few times but hasn't passed. And some proptech companies are starting to offer bias audits proactively, partly to avoid regulation. Luna: But if the Cleveland case tells us anything, it's that self-regulation has limits. Lucas: Absolutely. The company only paused the system after public pressure and a city council inquiry. Without that spotlight, those 13 evictions might still be happening. Luna: Honestly, if this episode made you think about how AI is shaping something as fundamental as where people live, and if it felt worth your time, you know, those are the kinds of conversations that keep this show going ad-free. Lucas: Yeah, listener support is what lets us dig into these cases without chasing clicks. If today's conversation gave you something usable, the link is buy me a coffee dot com slash fexingo. Really small thing, but it adds up. Luna: And it keeps us independent. So, back to the question of what a fairer system looks like — Lucas, you mentioned the right to explanation. Could that technically be built into these AI systems? Lucas: It can. Explainable AI is a whole research area. You can train models that output not just a decision but a set of contributing factors. For example, 'this violation was flagged because camera X captured a dog in unit Y at time Z.' But most companies don't implement it because it adds complexity and cost. Luna: And maybe because transparency would make it easier for tenants to spot errors and challenge decisions. Lucas: Right. If you know exactly what evidence the AI used, you can say 'that's not my dog, it's my neighbor's' or 'that construction noise was from the city, not me.' Without that, you're just arguing against a black box. Luna: So the Cleveland case is a warning. But also a blueprint for what needs to change. Lucas: Exactly. It shows the harm of automating without accountability, and it shows the fix — human review that's more than a formality, transparency in how decisions are made, and a regulatory framework that catches up to the technology. Luna: Let's hope more cities and states take note.