Latest / AI Ethics with Fexingo: Bias, Safety, and Responsible Artificial Intelligence / When Your AI Hiring Tool Asks About Your Hobbies
Transcript
- Lucas: So there's this hiring tool that a startup rolled out in early 2025 — it claims to predict 'cultural fit' by scraping a candidate's public social media profiles, even their hobby lists on LinkedIn and photos on Instagram. Luna: Wait — they're analyzing your vacation photos to decide if you belong at their company? Lucas: Exactly. The tool, built by a third-party vendor called FitScore AI, was pitched to HR departments as a way to reduce turnover. The idea was that if they could infer your personality traits — openness, conscientiousness, extroversion — from your digital footprint, they could match you to teams with similar profiles. Luna: And I'm guessing that went about as well as you'd expect. Lucas: It did. By mid-2025, a reporter at The Markup got hold of internal data from a company using FitScore — call them TechNova — and found that candidates over 45 were being flagged as 'low cultural fit' at more than double the rate of younger applicants. Why? Because older candidates posted fewer photos of themselves at concerts or hiking trips — activities the AI associated with 'energy' and 'collaboration'. Luna: So instead of judging their actual work experience, the AI was judging their lack of youthful-looking hobbies. Lucas: Right. And it gets worse. The tool also penalized candidates who listed niche hobbies — say, competitive chess or birdwatching — because those didn't match the 'team-oriented' profile the AI had learned from the existing workforce, which was mostly young software engineers who posted about rock climbing and craft beer. Luna: That's basically encoding the current team's bias into an automated gatekeeper. If your team is already homogeneous, the AI will filter for more of the same. Lucas: Precisely. And this isn't just a one-off startup problem. A 2024 study from Northeastern University tested three commercial AI hiring tools that claimed to predict personality from text — including resumes and social media posts — and found that all three showed statistically significant age and gender bias. For example, one tool consistently rated female candidates as lower in 'leadership potential' because their language was more collaborative. Luna: So the same collaborative language that might be praised in a performance review is penalized by an AI looking for 'assertive' keywords. Lucas: Exactly. The legal landscape here is murky. The Equal Employment Opportunity Commission, or EEOC, has guidelines saying that hiring practices that disproportionately impact protected groups — like age, race, gender — can be illegal even without intent. But personality tests have historically been in a gray zone, because they're not directly asking about protected characteristics. Luna: But if the AI infers those characteristics from your data — like inferring age from your graduation year or the style of your photos — then it's effectively using them as proxies. Lucas: Right. And the EEOC has started paying attention. In late 2025, they launched a new initiative specifically targeting ai driven hiring tools that use 'inference-based' methods. But enforcement is slow. The companies argue that the tools are trade secrets, so they won't reveal the exact algorithms. Luna: Which means the burden of proof falls on the candidate, who has no idea why they were rejected. Lucas: Exactly. There's also a deeper issue here around consent. Candidates don't know that their Instagram photos are being scraped. Even if the posts are public, the context changes when an AI is using them to make employment decisions. Luna: So what can someone do if they suspect this is happening? I mean, short of scrubbing their entire online presence. Lucas: Some privacy advocates recommend using separate professional and personal accounts, and locking down the personal ones. But that's not always feasible. There's also a growing movement for what's called 'algorithmic transparency' in hiring — some states like Illinois and Maryland have passed laws requiring companies to disclose when they use AI in hiring, and to allow candidates to request an alternative assessment. Luna: But those laws don't cover the inference from social media angle explicitly. Lucas: Not yet. That's the next frontier. In fact, the EU's AI Act, which started phasing in this year, classifies hiring AI as 'high risk', which means it has to undergo conformity assessments. But even there, the specifics on using inferred personality data are still being debated. Luna: So we're in this gap where the technology is already deployed, but the regulation hasn't caught up. Lucas: Exactly. And that's the space where a lot of these ethical questions live. Luna: You know, it's funny — we talk about these tools as if they're inevitable, but they're only here because companies are buying them. And that's actually what keeps this show going too, in a different way. A small group of listeners chip in monthly through buy me a coffee dot com slash fexingo, and that's literally what funds us making episodes like this, ad-free. Lucas: Yeah, it's a good reminder that things like this — independent journalism — only happen when people decide to support them directly. We don't have advertisers to answer to, which means we can dig into stories like this without pulling punches. Luna: Exactly. So if today's conversation gave you something useful, that's the way to keep it going. Lucas: Back to the biases — let's talk about what companies can do instead. Some are moving to 'skills-based' hiring, where they use AI only to evaluate work samples or structured interviews, not personality proxies. Luna: Right. And there's research showing that skills-based assessments are actually better predictors of job performance than personality tests anyway. Lucas: A 2023 meta-analysis from the Journal of Applied Psychology found that work-sample tests have a validity coefficient of about 0.54, compared to around 0.20 for most personality inventories. So companies might actually be getting worse candidates by using these inference tools. Luna: So it's both ethically problematic and potentially ineffective. That's a bad combination. Lucas: It is. And yet, because these tools are marketed as 'innovative' and 'data-driven', they get adopted without scrutiny. The real cost is borne by the candidates who never know why they were passed over. Luna: What's the one thing you'd want our listeners to take away from this? Lucas: That 'cultural fit' is often a mask for unconscious bias, and when you automate it, you just scale that bias. If you're a job seeker, be aware that your online hobby list might be data for an AI you never agreed to. And if you're in HR, ask your vendors exactly what data they're using and how they validate it for bias. Luna: Sound advice. Thanks, Lucas. Lucas: Thanks, Luna. That's it for this episode of AI Ethics with Fexingo.