Latest / AI Ethics with Fexingo: Bias, Safety, and Responsible Artificial Intelligence / When Your AI Loan Officer Denies You for Your Social Media
Transcript
- Lucas: So you apply for a loan — maybe a mortgage, maybe a small business line of credit. Your credit score is solid, your debt to income ratio is fine, you've got a steady job. And you get denied. Luna: Happens more than you'd think, but usually there's a reason. Late payments, high utilization, something in the file. Lucas: Right. But what if the reason is a photo you posted on Instagram two years ago? A tweet where you joked about gambling? A Facebook check-in at a bar? Luna: Wait — banks are actually looking at my social media when I apply for a loan? Lucas: Some are. And the practice is growing fast. There's a 2025 study from the Federal Reserve Bank of Philadelphia that looked at this directly. They found that AI models trained on social media data denied loans to about 12 percent more applicants compared to traditional underwriting — even when those applicants had identical financial profiles. Luna: Twelve percent more denials — purely based on what someone posts online. That's huge. Lucas: It is. The researchers fed an AI model social media profiles of real loan applicants — with permission, this was a controlled study — and had it predict creditworthiness. The model picked up patterns like: people who post about travel are higher risk, people who use certain slang are lower income, people who follow certain influencers are more likely to default. Luna: That sounds like digital profiling, not credit analysis. Are there any rules against this? Lucas: That's the scary part. The Equal Credit Opportunity Act, or ECOA, was passed in 1976. It prohibits discrimination based on race, color, religion, national origin, sex, marital status, age — but it says nothing about social media activity. The law was written for a world where your credit file was a piece of paper in a filing cabinet. Luna: So the AI is essentially inventing new categories of risk that have no legal oversight. Lucas: Exactly. And the models aren't just looking at obvious red flags like posts about gambling. They're analyzing tone, sentiment, network connections. One startup I looked at — let's call them CrediScan — uses natural language processing to score applicants on 'reputational risk.' They claim it predicts default better than FICO. Luna: Better than FICO — but FICO is based on actual financial behavior. This is based on... what, exactly? Grammar? Lucas: Partly. The Philadelphia Fed study found that the model placed heavy weight on things like: does the applicant use more than three exclamation points per post? That correlated with higher default rates in their training data. Also: do they follow accounts that post about get rich quick schemes? Negative signal. Luna: That's absurd. I use exclamation points all the time. I'm just enthusiastic. Lucas: And statistically, you might be slightly more likely to miss a payment — according to the model. But here's the problem: correlation isn't causation. Maybe people who use lots of exclamation points are younger, and younger people have thinner credit files. The model isn't adjusting for that — it's just picking up a proxy. Luna: Which means it could be discriminating by age, even if it's not explicitly looking at age. That's exactly the kind of proxy discrimination the ECOA was meant to prevent. Lucas: Right. And because the models are black boxes, lenders can say 'we don't know why it denied them, the AI just flagged them.' That's not a legal defense, but it's hard to prove discrimination when the decision-making process is opaque. Luna: Let me ask this — is this widespread? Are big banks doing this, or is it just fintech startups? Lucas: A 2024 survey by the Consumer Bankers Association found that about 15 percent of large banks are either using or piloting social media data in credit decisions. Another 30 percent said they're exploring it. The fintechs are further ahead — companies like Upstart and LendingClub have experimented with alternative data, though they focus more on utility bills and bank transaction history, not social posts. Luna: So it's growing. And regulators are starting to notice, right? Lucas: The CFPB — the Consumer Financial Protection Bureau — issued a guidance in early 2026 saying that using AI to analyze social media for credit decisions could violate ECOA if it results in disparate impact on protected groups. They haven't fined anyone yet, but they've opened investigations into at least two lenders. Luna: One of which, I think I read about — a fintech called SocialCredit, based in San Francisco. They were using Instagram photos to assess 'lifestyle stability.' Lucas: That's the one. Their model flagged applicants who posted photos with luxury brand logos as higher risk — the theory being that they're living beyond their means. But also flagged people who posted no photos at all, because 'lack of social presence indicates instability.' Luna: So you can't win. Post too much, you're reckless. Post too little, you're unstable. Lucas: Exactly. And the CFPB investigation is looking at whether that disproportionately impacts low-income applicants who might not have the time or resources to curate a 'stable' online image. The early findings suggest the model denied loans to Black and Hispanic applicants at a rate 18 percent higher than white applicants with similar financial profiles. Luna: That's a clear disparate impact. So what's the argument for using this data at all? Lucas: Proponents say it expands access to credit for people with thin credit files — the unbanked and underbanked. If you're a recent immigrant or a young person with no credit history, your social media profile might actually help you get a loan when traditional scoring can't evaluate you. But the evidence so far suggests it's doing the opposite — it's denying people who would otherwise qualify. Luna: And it's not transparent. Consumers don't know their Instagram feed is being judged. Lucas: That's the other big issue. Most lenders don't disclose that they're using social media data. The Fair Credit Reporting Act requires that if you're denied credit based on a 'consumer report,' you have to be told. But social media profiles aren't considered consumer reports in the traditional sense, so lenders argue they don't have to disclose. Luna: So you could be denied and never know why. That feels like a due process problem. Lucas: It is. And the EU is actually ahead on this — the AI Act, which came into full effect in 2025, classifies credit scoring as 'high-risk' AI, which means any model using social media data has to undergo a conformity assessment, be transparent about its inputs, and allow consumers to contest decisions. The US has nothing equivalent. Luna: Do you see that changing? Any federal bills in the works? Lucas: There's the Algorithmic Accountability Act, which has been introduced in various forms since 2019. The latest version, introduced in late 2025, would require companies to audit their AI models for bias, including in credit. But it hasn't passed yet. And even if it does, enforcement is always the question. Luna: So what can a consumer do right now to protect themselves? Lucas: First, assume your public social media is being watched. Lock down your accounts — set them to private. Second, be careful about what you post, but also realize that even private posts might be accessible if a lender uses a third-party data broker that scrapes data through app permissions or friend networks. Luna: It's almost like you have to treat your social media like a credit report — monitor it, know what's there, and correct errors. Lucas: Exactly. And there's a growing movement for 'digital credit reports' — a right to see what data is being used about you. Some states, like California under the CCPA, already give you the right to request what data companies have collected. But using that for credit denials is still untested. Luna: Let's zoom out for a second. The Fed study found a 12 percent higher denial rate. That's a huge number. If this becomes standard practice, how many people could be affected? Lucas: Millions. About 45 million Americans are either unbanked or underbanked — they have thin or no credit files. If lenders start using social media as a proxy, those are exactly the people who could be most harmed. The Fed study simulated a scenario where social media scoring was applied to all applicants, and it found that the overall denial rate would increase by about 8 percent — meaning roughly 10 million additional denials per year. Luna: Ten million people denied credit because of something they posted. That's a systemic problem. Lucas: And it's not just loans. The same models are being used for insurance premiums, rental applications, even job screening. The AI doesn't care if you're a good person — it cares about patterns in your data. Luna: It's like we're building a digital caste system based on online behavior. Lucas: That's a strong way to put it, but not inaccurate. And the really insidious part is that people don't know they're being judged, so they can't change their behavior. It's a hidden tax on anyone who doesn't fit the model's narrow view of 'responsible.' Luna: Speaking of hidden — we should be transparent about something ourselves. This show doesn't run on ads, and it never will. It runs because a small group of listeners chip in through buy me a coffee dot com slash fexingo. That's literally what pays for the research and production of episodes like this one. Lucas: Yeah, and it's not a big ask — it's just the truth. If you get value from these deep dives, that's the only way they keep coming. No strings, no perks, just keeping the thing ad-free. Luna: Exactly. And back to the AI loan officer — there is one potential bright spot. Some researchers are working on 'explainable AI' models that can tell you exactly why you were denied and what you could change. But those are still experimental. Lucas: And even explainable AI doesn't solve the underlying fairness question. If the model says 'you were denied because you follow 47 accounts that post about cryptocurrency' — that's an explanation, but it doesn't make the decision just. It just makes it transparently unjust. Luna: So what's the path forward? Regulation? Better models? Consumer education? Lucas: All of the above. But I think the most important thing is updating the ECOA to explicitly cover algorithmic decision-making and alternative data. The law was written for a different era, and until Congress catches up, we're in a Wild West where your loan application can be sunk by a tweet. Luna: A tweet from three years ago that you don't even remember writing. Lucas: Exactly. And that's the real risk — not that AI is inherently biased, but that we're deploying it in a legal vacuum where the people being judged have no recourse. I think that's the story that doesn't get told enough. Luna: Well, we just told it. Thanks for digging into this one, Lucas. Lucas: Thanks, Luna. If this made you think — share it. And if you want more, you know where to find us.