Latest / AI Ethics with Fexingo: Bias, Safety, and Responsible Artificial Intelligence / How AI Models Learn Bias From Your Selfies
Transcript
- Lucas: You know that moment when you upload a selfie to a job application and you think, 'Eh, it's just a photo, what's the harm?' Luna: Yeah, I've done that. Figure it's just for the hiring manager to put a face to the name. Lucas: Right. Well, that photo might be doing a lot more than you think. A 2025 study from Stanford's Human-Centered AI group found that a standard facial-recognition model could predict someone's political affiliation with 72 percent accuracy from a single, untagged photo. Luna: Seventy-two percent — that's better than random. And they're not even looking at your LinkedIn posts. Lucas: Exactly. The model isn't reading bumper stickers. It's picking up on subtle visual cues — grooming style, facial hair, glasses shape, even the background of the photo. Things the human eye might not consciously register. Luna: So we're talking about a system that can infer something deeply personal — political leanings, sexual orientation, maybe even health conditions — just from how you present yourself in a photo. Lucas: That's the worry. And it's not just academic. Companies are already using image analysis in hiring, insurance underwriting, and credit decisions. There's a term for it — 'visual profiling.' Luna: Visual profiling. Like racial profiling, but with algorithms and selfies. Lucas: Pretty much. And the data pipeline is opaque. You upload a photo for what you think is a legitimate purpose — verifying your identity, submitting a job application — and the company runs it through a model that extracts dozens of inferred attributes. Luna: Do they tell you they're doing that? Usually buried in the privacy policy, I'm guessing. Lucas: If at all. A lot of these models are sold by third-party vendors who claim they only analyze 'demographic characteristics' or 'personality traits.' But the line between what's useful for a job and what's discriminatory is blurry. Luna: Give me a concrete example. What's the case that's in the news right now? Lucas: There's a property-insurance startup called — I'll just call it 'HomeVue' — that was sued last month in federal court. The allegation is that they used a selfie-analysis tool to deny policies in majority-Black neighborhoods. The tool supposedly assessed 'risk tolerance' based on facial expressions and background clutter. Luna: Risk tolerance from a selfie. That's pseudoscience wearing a lab coat. Lucas: It absolutely is. The lawsuit says the model flagged Black applicants as having 'higher impulsivity' at a rate 3.4 times higher than white applicants. The company says the tool is just one of many factors, but the plaintiffs argue it's a proxy for race. Luna: And this is happening under what legal framework? I know Illinois has the Biometric Information Privacy Act — BIPA — but does that cover inference from photos? Lucas: It's a gray area. BIPA regulates the collection of 'biometric identifiers' like face geometry. But the law was written in 2008, before deep learning could infer abstract traits from a photo. Some courts have ruled that simply scanning a photo to create a face template is covered. But inferring 'conscientiousness' or 'political affiliation' — that's new territory. Luna: So we have a technology moving faster than the law. The classic story. Lucas: True. But there are efforts to catch up. The proposed federal 'Algorithmic Accountability Act' would require companies to audit their models for bias, including visual profiling tools. And the FTC has started to signal that using AI to make 'unsupported inferences' about consumers could be an unfair trade practice. Luna: What about the companies building these models? Are they doing anything to self-regulate? Lucas: Some are. A few of the big cloud providers — AWS, Google Cloud, Azure — have bans on using their facial-recognition APIs for certain inferential purposes. But enforcement is weak. And there are dozens of smaller vendors offering 'personality analysis' or 'honesty detection' based on facial cues, with little oversight. Luna: It feels like we're in a Wild West phase. And the average person doesn't know they're being profiled every time they upload a photo. Lucas: That's the core problem. The Stanford study I mentioned — they also surveyed 2,000 Americans. Only 12 percent knew that AI could infer political affiliation from a photo. And when told, 87 percent said they wanted the practice banned. Luna: So there's a massive awareness gap. What can someone listening today actually do? Lucas: First, think twice before uploading any photo to a site or app that isn't purely social. For job applications, many companies don't actually need your photo — and in some states, it's illegal to require one. You can often skip it. Luna: But sometimes they ask for a headshot for the company directory or your profile. That feels normal. Lucas: Right. Ask yourself: is this an employer known for respecting privacy? If they have a third-party AI vendor, you probably won't know. But you can ask in the interview process — 'How is my photo data used? Is it analyzed by any automated system?' Luna: And if they dodge the question, that's a red flag. Lucas: Exactly. Second, use tools that strip metadata from your photos before uploading. Some apps let you remove location data, but the visual content itself is harder to strip. You can use image filters or cropping to reduce the amount of background information. Luna: So if you're in a messy room, maybe don't use that as your profile pic for a loan application. Lucas: Right. The model might infer 'low conscientiousness' from clutter. That sounds ridiculous, but that's the kind of thing these tools are trained to pick up on. Luna: It's almost like we need a 'visual privacy' literacy campaign. Lucas: We do. And I think that's where episodes like this one fit in. If today's conversation gave you something usable — a reason to pause before uploading a selfie, or a question to ask your employer — that's the point. Luna: Absolutely. And keeping conversations like this going is why listener support matters. We don't run ads, and that's intentional. The way we stay ad-free is through people who find value in what we do and want to keep it that way. Lucas: If that's you, you can find us at buy me a coffee dot com slash fexingo. It's a simple way to say 'this is worth keeping around.' No pressure, just an option. Luna: And it genuinely helps. We've been listener-supported since day one, and it keeps us free to dig into topics like this without worrying about sponsor constraints. Lucas: Exactly. So, back to the regulatory piece — I think the most interesting development is a recent court ruling in California that could set a precedent. A judge allowed a class-action suit to proceed against a ride-sharing company that used driver selfies for 'safety checks' but allegedly also ran them through an undisclosed personality model. Luna: So the court said that's a valid claim, even without a specific law covering that exact practice. Lucas: Yes. The judge relied on California's Unfair Competition Law and the state's constitutional right to privacy. The argument was that the company's conduct was 'immoral, unethical, or oppressive' — even if no statute directly prohibited it. Luna: That's a big deal. It means companies can't just hide behind 'well, it's not illegal yet.' Lucas: Right. And it sends a signal to insurers, lenders, and employers that visual profiling carries legal risk, even if the tech is new. Luna: So where does this leave us? The technology is already here, the awareness is low, the law is catching up but slowly. What's the one thing you'd want listeners to remember? Lucas: That your selfie is data. It's not just a picture — it's a vector of inference. And until we have stronger guardrails, treat it with the same caution you'd treat your Social Security number. Don't hand it out casually. Luna: And if you're a company using this tech, maybe think twice — or at least be transparent. Because the courts and the public are starting to pay attention. Lucas: Exactly. On that note, we'll be back next week with another angle on the ethics of AI. Until then, keep your selfies safe. Luna: See you next time.