Latest / AI Ethics with Fexingo: Bias, Safety, and Responsible Artificial Intelligence / When Your AI Credit Score Is Actually a Social Score
Transcript
- Lucas: So there's this story out of Ohio — a woman in her late twenties, stable job, no debt, never missed a rent payment. She applies for a car loan in March of this year, and gets denied. Not because of any traditional credit issue — she doesn't really have a credit history. But the lender used an AI model that scored her based on non-traditional data. Luna: Let me guess — they looked at her social media. Lucas: Exactly. Specifically, the algorithm flagged her for following certain TikTok creators. The model had correlated that behavior with higher default risk. She had no idea that her online activity was being used to judge her financial trustworthiness. Luna: That's the kind of thing that makes people's heads spin. I mean, we've talked about AI in recruiting, AI in insurance — but credit scoring is the foundation of so much financial access. If that's biased or opaque, it cascades. Lucas: That's the core of it. And it's not just social media. Alternative credit scoring models now look at everything from your phone's battery level — I'm not kidding — to your shopping habits, your browsing history, even how quickly you scroll through terms of service agreements. Luna: And this is supposed to help people who are 'credit invisible', right? The Consumer Financial Protection Bureau estimated that about 45 million Americans either have no credit file or one that's too thin to generate a score. Lucas: Right. The pitch is financial inclusion. Companies like Nova Credit and Petal have built their models around non-traditional data to give people a shot. Nova Credit lets immigrants use their foreign credit history. Petal uses cash flow data and — they claim — your potential, not just your past. That's genuinely useful for people locked out of the system. Luna: But the Ohio case shows the flip side. When the data points aren't transparent, and the model is a black box, you don't know why you were denied. And you have no real recourse. Lucas: And that's the tension. The same tools that can include can also discriminate in new, subtle ways. The CFPB has been looking into this — they put out a report in 2025 that specifically warned about using social media data for credit decisions. But it's not banned. It's still happening. Luna: Which brings up a question we've danced around before: who regulates these models? The CFPB has some authority over fair lending, but it's not like they're auditing every fintech startup's algorithm. Lucas: That's why conversations like this matter. And if today's episode gave you something useful — a new angle on credit scoring, or something to think about — a couple of dollars a month genuinely helps keep these conversations going. You can find us at buy me a coffee dot com slash fexingo. Even small contributions add up and let us stay independent and ad-free. Luna: Yeah, it's a small way to support something you value. And we really appreciate it. Lucas: Alright, let's get back to the data. So the Ohio woman's case isn't isolated. There's a broader pattern of these alternative models using proxies for race and class. For example, if the model sees that you shop at discount stores or use certain types of prepaid phone plans, it might correlate that with lower repayment likelihood. But those behaviors are often themselves correlated with low income or minority status. Luna: That's pretty much textbook disparate impact. Even if the model doesn't explicitly use race or income, if the proxies end up disproportionately excluding certain groups, it violates the spirit of fair lending laws. Lucas: Exactly. And the thing is, these models are often proprietary. The companies that build them argue that revealing the algorithm would let people game the system. But that secrecy makes it impossible for consumers to understand — let alone challenge — the decision. Luna: So what's the path forward? More regulation? More transparency? Or do we accept that some people will be unfairly denied in exchange for broader inclusion? Lucas: I think most experts would say regulation needs to catch up. The Equal Credit Opportunity Act was passed in 1974. It says lenders can't discriminate based on race, color, religion, national origin, sex, marital status, age, or public assistance income. But it doesn't say anything about TikTok follows or battery levels. Luna: Right. The law hasn't kept pace with the data points. And the CFPB is trying, but they're limited. They issued guidance in 2025 saying that using social media data could violate ECOA if it results in discrimination. But enforcement is reactive, not proactive. Lucas: Meanwhile, the industry is moving fast. Nova Credit has partnerships with major lenders and credit bureaus. Petal has issued hundreds of thousands of credit cards. The alternative data genie is out of the bottle. The question is whether we can build guardrails now, or whether we wait for a scandal big enough to force action. Luna: And there's also the question of consent. Most people have no idea their social media or phone usage is being used this way. The disclosures are buried in terms of service agreements that nobody reads. Lucas: Exactly. And that's where the ethics get really murky. Because on one hand, you have someone who can't get a traditional credit card because they've never borrowed money. On the other hand, you have someone denied a loan because they liked the wrong Instagram post. The same technology can help one and hurt another. Luna: And the person being denied might never know why. They just get a letter saying 'based on your application, we were unable to approve you.' No explanation about the algorithm. Lucas: That's the part that bothers me most. In the Ohio case, it took a reporter digging into it for the woman to find out that her TikTok activity was a factor. The lender itself might not have even fully understood how the model reached its conclusion. Luna: That gets to the black box problem. Even the companies using these models sometimes don't know exactly why a score came out the way it did. Lucas: Right. And that's a huge risk for lenders too. If they can't explain a denial, they could be vulnerable to regulatory action or lawsuits. But right now, the incentive to expand credit access — and profit — seems to outweigh those risks. Luna: So what would meaningful reform look like? Do we need a new federal law specifically about algorithmic credit scoring? Lucas: Some advocates are calling for exactly that. A few proposals floating around include: mandatory bias testing before a model can be used, a requirement that consumers be told which data points were used in their score, and the right to opt out of alternative data entirely. Luna: And there's also the possibility of a centralized registry of credit scoring models, so regulators and researchers can study them without revealing trade secrets. Lucas: That's a clever idea. Something like the model registry some cities have for predictive policing algorithms. You don't have to open-source the code, but you have to document the inputs, the performance, and the disparate impact analysis. Luna: Would that be enough? I mean, even with documentation, the models can change over time. And if they learn from new data, old tests might not apply. Lucas: That's a real concern. Model drift is a thing. A model that's fair today could become biased tomorrow if the underlying data shifts. So you'd need ongoing monitoring, not just a one-time test. Luna: That's a lot of regulatory infrastructure to build. And it's not clear we have the will or the resources. Lucas: But we're already seeing some movement. In late 2025, the CFPB proposed a rule that would require lenders to explain adverse actions based on complex algorithms. It's not finalized yet, but it's a step. Luna: And what about the companies themselves? Are any of them proactively trying to be more transparent? Lucas: Some are. Petal, for instance, has been fairly open about their use of cash flow data and they don't use social media. But they're the exception. Most are still opaque, because they see the algorithm as a competitive advantage. Luna: So we're in this weird place where the technology is evolving faster than the rules, and consumers are caught in the middle. Lucas: That's the tension, exactly. The promise of alternative credit scoring is real — millions of people could get access to credit who otherwise wouldn't. But without safeguards, the same systems can entrench inequality in new, harder to see ways. Luna: It's a classic AI ethics dilemma. The tool isn't inherently good or bad, but the context and the rules around it matter enormously. Lucas: And the stakes are high. Credit scores affect your ability to rent an apartment, buy a car, even get a job in some states. If the score is based on opaque data, your financial life is at the mercy of a black box. Luna: I think the takeaway for listeners is: be aware. If you're applying for credit from a fintech company, look at what data they collect. Ask questions. And if possible, check your credit report regularly anyway. Lucas: And for the broader conversation: we need to keep pushing for transparency and accountability. Because the technology isn't going away, but the rules can catch up. The question is whether they will before more harm is done. Luna: Yeah. And that's a question for all of us — not just regulators, but consumers, advocates, and people building these systems. Lucas: Alright, that's where we'll leave it. Thanks for listening, and we'll be back with another episode soon.