Latest / Hiring & Firing with Fexingo: Recruitment, Performance, and Workforce Decisions / The Bias Hidden in Your AI Resume Screener
Transcript
- Lucas: So there's this assumption floating around that ai powered resume screeners are more objective than human recruiters — that they strip out bias. And a study published last year by the National Bureau of Economic Research really calls that into question. Luna: I've heard that argument a lot. The idea that machines don't care about your name or where you went to school — they just match keywords. But the NBER found otherwise? Lucas: Exactly. The researchers ran a controlled experiment with one of the leading AI screening tools — I won't name the vendor, but it's widely used by Fortune 500 companies. They submitted thousands of identical resumes, varying only the candidate's name, gender markers, and educational background. Luna: And what happened? Lucas: The AI systematically scored candidates with non-traditional educational backgrounds lower — even when their skills and experience were identical. For instance, a candidate listing a bootcamp certification instead of a four-year degree was ranked an average of 18 percent lower. The same pattern held for candidates with non-linear career paths, like gaps or industry switches. Luna: So the AI learned bias from the training data — the historical hires that were predominantly from traditional backgrounds. Lucas: That's exactly it. The tool was trained on past hiring decisions, which themselves reflected human bias. So it encoded that bias and then applied it at scale. The researchers estimated that for a company hiring 500 people a year, this screener would filter out roughly 60 qualified candidates from non-traditional backgrounds annually. Luna: That's a real business cost — you're losing talent that could be an edge. I remember a case from 2024: a mid-sized tech company called Vertex Analytics — they hired about 200 engineers a year. They implemented an AI screener to reduce recruiter workload, and within six months they saw their diverse hire rate drop by 12 percent. Lucas: Right — that's the kind of real-world consequence the NBER study flagged. Vertex Analytics eventually did a bias audit and found that the AI was penalizing resumes with the word 'freelance' or 'self-taught' even when the skills listed were identical. Their fix? They retrained the model using only skills-based parsing — ignoring education, ignoring job titles, just matching the specific technical skills required. Luna: And did that recover the diverse hire rate? Lucas: It did — within the next two quarters, diverse hires went back up to pre ai levels. But here's the thing: the company also had to add a human review step for any resume that the AI rejected but that contained at least 70 percent of the required skills. That caught a lot of false negatives. Luna: So the key takeaway isn't 'don't use AI' — it's 'audit your AI constantly and build in human overrides.' Lucas: Exactly. And that's where I think a lot of companies are dropping the ball. They deploy these tools and assume they're objective because they're algorithmic. But algorithms can be biased in ways that are hard to detect without regular testing. The NBER study recommends what they call 'adversarial fairness testing' — essentially, submitting fake resumes that vary only the protected characteristics and seeing if the scores change. Luna: That sounds like something every company using an AI screener should do quarterly. But how many actually do it? I'd guess fewer than 10 percent. Lucas: Probably less. And here's another angle: some of these vendors are starting to offer fairness dashboards, but the metrics they report aren't standardized. You might see 'demographic parity' reported one way by vendor A and a completely different way by vendor B. Without a common framework, comparison is meaningless. Luna: So regulation might be needed. I know the EU's AI Act classifies hiring tools as high-risk, requiring bias audits. But in the US, it's still a patchwork. Lucas: That's right. New York City's law on automated hiring tools went into effect in 2023, and a few other states have followed. But most companies are still self-regulating, which means many aren't regulating at all. The NBER study suggests that even when companies do audit, they often test on their own historical data — which has the same biases they're trying to correct. Luna: You need an external test set, essentially. That's what Vertex Analytics did — they used a synthetic resume pool from a third-party vendor. It cost them about 15 grand for the audit, but it probably saved millions in lost productivity from bad hires and missed talent. Lucas: And that 15 grand is a rounding error compared to the cost of a single bad hire, which we've talked about before — can be hundreds of thousands when you factor in recruiting, training, and lost team productivity. So the ROI on bias auditing is actually pretty clear. Luna: That reminds me of something I've been thinking about. If a company is listening to this and wondering where to start, what's the single most impactful change they can make this quarter? Is it the bias audit, or something else? Lucas: I'd say: turn off education and years of experience filters in your AI screener. Just for a month. See what happens to your candidate pool. A lot of companies find that when they remove those filters, the quality of applicants doesn't drop — it actually broadens. And then you can run a side-by-side comparison to see how many good candidates you were previously filtering out. Luna: That's a low-risk experiment. And it doesn't require a big budget. Honestly, if one company does that and finds a great candidate they would have missed, that's worth a coffee to the show, right? Lucas: Look, we keep this show ad-free because we want to talk about exactly this kind of practical, actionable stuff. If these conversations have moved your work forward in some small way, the link is buy me a coffee dot com slash fexingo. No pressure — but listener support is what keeps us independent. Luna: Exactly. And we appreciate every contribution. Now, back to the bias audit — Lucas, you mentioned adversarial testing. Can you walk through what that actually looks like in practice? Lucas: Sure. You create a set of matched resumes. For each real resume, you generate a version where you change the name to something stereotypically male or female, change the educational institution to a less traditional one, maybe adjust the timeline to show career gaps. Then you run both versions through the screener. If the scores differ by more than, say, 5 percent, you've got a bias flag. Then you investigate which features are driving the difference. Luna: And you can do that internally if you have a data science team. Otherwise, there are third-party auditors like Parity AI or the Algorithmic Justice League that offer this as a service. Lucas: Right. And the NBER study also found something interesting: the bias wasn't just about demographics. It also penalized candidates who used unconventional formatting — like a functional resume instead of a chronological one. The AI was essentially trained on a narrow definition of 'professional' formatting. Luna: So even if you're a white male from an Ivy League school, if you use a creative resume template, you might get downgraded. The bias is broader than we think. Lucas: Exactly. It's not just about race or gender — it's about conformity to a norm. And that norm is shaped by the training data, which in many cases comes from companies that have historically hired a certain type of person. So the AI ends up perpetuating that homogeneity. Luna: That's a powerful point. And it suggests that the solution isn't just fairness metrics — it's retraining the model on a more diverse set of resumes, including those from nontraditional backgrounds. Some companies are starting to do that. Lucas: Yeah, I've seen a few: a company called HireVue, for example, now offers a 'bias-mitigated' version of their screener that's trained on a balanced dataset. But the question is how many of their clients actually use that version versus the default one. Luna: Probably a minority, unless it's mandated. So we come back to the need for either regulation or strong internal advocacy. Lucas: And that's where hiring managers and recruiters in the audience can make a difference. Ask your vendors: 'What fairness testing have you done? Can I see the results?' If they can't answer, that's a red flag. Luna: Alright, so to wrap up: AI screeners aren't inherently fair. They can encode and amplify existing bias if not carefully designed and audited. The fix is regular adversarial testing, skills-based parsing, and a human review step for borderline cases. And starting small — like turning off education filters — can reveal a lot. Lucas: That's the takeaway. And as AI tools become more sophisticated — multimodal, video interviews, voice analysis — the potential for new kinds of bias grows. So staying vigilant is going to be an ongoing challenge for every hiring team. Luna: Thanks, Lucas. That was a great deep dive. Lucas: Thanks, Luna. Talk to everyone next time.