Latest / AI Ethics with Fexingo: Bias, Safety, and Responsible Artificial Intelligence / When Your AI Recruiter Judges a Gap on Your Resume
Transcript
- Lucas: So there's a new audit out from a labor advocacy group called Tech Bias Watch, and they ran a pretty straightforward experiment. They took one hundred identical resumes, varied exactly one field — the length of the candidate's most recent career gap — and fed them into five major AI resume screening tools used by companies like Target, Hilton, and UnitedHealth. Luna: And I'm guessing the results were not kind to anyone who didn't have a perfectly linear career path. Lucas: Exactly. The gap was the only variable. Same education, same previous job titles, same skills. And in 68 percent of the trials, the AI ranked the candidate with a six-month gap significantly lower — sometimes as much as 40 percentile points lower — than the identical resume with no gap. Luna: Forty percentile points. That's not a small penalty. That's the difference between being in the top third of applicants and barely making the first cut. Lucas: Right. And here's the part that really stood out to me. In three of the five tools, a one-year gap hurt the candidate's score more than a disclosed misdemeanor conviction for petty theft. The AI literally treated unemployment as a stronger negative signal than a criminal record. Luna: That is wild. And it raises a question — why? Why do these models penalize gaps so heavily? Lucas: The short answer is training data. Most of these tools were trained on resumes of people who were hired at large companies over the past decade. And if you look at that dataset, the majority of successful hires had no gaps longer than three months. So the model learns that 'gap equals risk.' It's a statistical shortcut — but one that ignores all the legitimate reasons someone might step away from work. Luna: Reasons like having a baby, caring for a sick parent, going back to school, dealing with a layoff in a bad economy. The list is long. Lucas: And those reasons disproportionately affect women and people of color. The audit broke down the demographic impact — women are about 40 percent more likely to have a career gap of six months or longer, and Black workers are about 25 percent more likely. So a tool that penalizes gaps is implicitly penalizing those groups. Luna: It's a disparate impact issue, plain and simple. And we've seen the EEOC start to take notice. They issued new guidance in April that says employers can be held liable for discriminatory outcomes caused by AI screening tools, even if they didn't intentionally design them to discriminate. Lucas: Right, and that guidance is already having ripple effects. The Department of Labor has opened investigations into three of the vendors that Tech Bias Watch tested. Not naming them yet, but two are major players — their tools are used by Fortune 500 companies. Luna: So what's the fix? Can you design an AI that's 'gap-aware' without just ignoring gaps entirely? Lucas: That's the interesting part. One startup called FairHire has developed a filter that doesn't ignore gaps — it actually asks the candidate to provide a short, structured explanation, and then it scores the gap in context. For example, if you list 'caregiver leave' or 'graduate study,' the model is trained to treat that as neutral, not negative. Luna: And does that actually improve outcomes? Or does it just make the tool more complicated? Lucas: FairHire ran a pilot with a mid-sized retailer last year. They compared the performance of hires made with the gap-aware filter versus their previous tool. The result: the gap-aware filter increased the proportion of hires who stayed past six months by 14 percent, and it actually improved manager satisfaction scores. The hires were more diverse — more women, more caregivers — and they performed just as well. Luna: So the bias against gaps wasn't just unfair — it was also inefficient. The companies were screening out good candidates for no good reason. Lucas: Exactly. And that's the argument that tends to get employers' attention. You can talk about fairness and ethics all day, but when you show them the data that they're leaving good talent on the table, that's when they actually change something. Luna: Quick honest thing — this is exactly the kind of episode that makes me grateful for the listeners who support the show. We keep this thing ad-free, and the handful of folks who chip in monthly through buy me a coffee dot com slash fexingo make it possible to dive into audits like this one. So, genuinely, thank you. Lucas: Yeah, seconded. It's a small group, but it makes a big difference. And it means we don't have to chase sponsors who'd want us to soften the take. Luna: Alright, back to the data. Lucas, one thing I wondered — the audit tested five tools, but there are dozens of resume screeners out there. Are the other tools just as bad? Lucas: Tech Bias Watch only looked at five, but they chose the most widely used ones. And the pattern was consistent across all five: every single tool penalized gaps to some degree. The severity varied — one tool dropped the gap candidate by only 12 percentile points, another by 48. But the direction was unanimous. Luna: Which suggests the problem isn't just a specific algorithm. It's baked into the whole approach of using historical hiring data to predict future success. Lucas: Right. And that's the deeper issue. Because even if you train a model on 'good' data — say, performance reviews of existing employees — you're still encoding whatever biases existed in who got hired in the first place. If your company historically preferred candidates with no gaps, the model will learn that preference and amplify it. Luna: So it's a feedback loop. The AI replicates past bias, which leads to more homogeneous hiring, which produces more data that says homogeneous candidates are safer. Lucas: Exactly. And breaking that loop requires either changing the training data or changing the model's objective function. Some companies are experimenting with the latter — instead of predicting 'will this person be hired?', they predict 'will this person perform well in this role?' using a separate validation set that includes people with gaps. Luna: But that requires actually hiring people with gaps first. You can't train a model on data you don't have. Lucas: Right, it's a chicken and egg problem. Some firms have gotten around it by doing a randomized trial — they take a subset of applicants with gaps and deliberately hire them, then track performance over a year. That gives them real data to retrain the model. But that takes time and money. Luna: And it also requires a willingness to accept that the model might be wrong. Which, for a lot of HR departments, is a hard sell. Lucas: It is. But the regulatory pressure is building. The EEOC guidance, the DOL investigations, and there's also a bill in Congress — the Algorithmic Accountability Act — that would require companies to audit their hiring AI for bias. It's been introduced a few times, but this year it has more bipartisan support. Luna: I want to go back to something you mentioned earlier — the fact that a one-year gap hurt more than a petty theft conviction. That feels like a statement about what we, as a society, value. We trust someone who made a mistake more than someone who took a year to raise a child. Lucas: That's exactly what the researchers at Tech Bias Watch pointed out. They said the models are encoding a kind of 'presenteeism bias' — the idea that being continuously employed is a moral virtue, not just a job market reality. And that bias is particularly punishing in a post-pandemic world where millions of people had gaps due to layoffs, illness, or caregiving. Luna: Let's talk about one more specific finding from the audit. They also tested how the tools handled a gap that was explained in the resume itself — like a line that said '2023: took leave to care for aging parent.' Did that help? Lucas: It helped marginally. In two of the five tools, the explained gap scored about 8 percent higher than the unexplained gap. But that still left the candidate well below the no-gap baseline. And in three tools, the explanation made no difference at all — the model simply didn't parse the text. It only looked at the date ranges. Luna: So writing 'caregiver leave' in the resume didn't matter because the AI wasn't reading it. It was just counting months. Lucas: Exactly. And that's a design choice. The developers could have built natural language processing to pick up on gap explanations. They chose not to, probably because it's cheaper and faster to just flag any gap over three months. But that choice has real consequences. Luna: What about the candidate who has a gap and knows the AI is screening them? Is there any way to game the system? Lucas: Some career coaches advise listing freelance or consulting work during the gap, even if it was minimal. But that's ethically ambiguous, and the audit found that the tools also penalized short-term freelance stints if they didn't show consistent employment. So gaming it is tricky. Luna: It sounds like the real solution has to come from the employers. They need to demand better tools, and they need to be willing to audit the outcomes. Lucas: Yeah. And some are starting to. A few large employers — I'm thinking of Patagonia and Hilton, actually — have publicly said they're reviewing their AI screening vendors specifically for gap bias. Patagonia even published a blog post saying they value candidates with non-linear career paths because they often bring diverse perspectives. Luna: That's the kind of leadership that could shift the market. If a big name like Hilton demands a gap-aware filter, the vendors will build it. Lucas: Absolutely. And that's the optimistic take. The pessimistic one is that without regulation, most companies will keep using the cheap, biased tools because they're convenient. But the DOL investigations and the EEOC guidance might be enough to tip the scales. Luna: Where do you think this is headed? A year from now, will we still be having this conversation? Lucas: I think the conversation shifts from 'do these tools penalize gaps?' to 'how do we audit and certify that they don't?' The FairHire approach is promising, but it's one startup. We need industry-wide standards. And maybe a certification like 'eeoc compliant' that vendors can earn, so employers have a clear benchmark. Luna: That would be a huge step. Because right now, most employers don't even know what their screening AI is doing. They buy it from a vendor, it spits out a ranked list, and they assume it's objective. Lucas: Right. And the vendors are happy to keep it a black box. But the more audits like this one come out, the harder it is to claim ignorance. I think we're going to see a lot more companies doing internal audits in the next 12 months, not just because of regulation, but because they realize the bias is costing them talent. Luna: One last thing — the audit also looked at how gaps interacted with other demographic signals. Did they find any intersectional effects? Lucas: They did. The penalty was worse for candidates with names that sounded Black or Hispanic. A candidate with a 'Black-sounding' name and a one-year gap was ranked lower than a candidate with a 'white-sounding' name and the same gap. So the gap penalty compounds with racial bias. It's a double hit. Luna: That's the kind of granular finding that makes an audit valuable. It shows that the problem isn't just 'AI is biased,' it's 'AI is biased in specific, measurable ways that affect real people.' Lucas: Exactly. And once you have that data, you can start to fix it. The question is whether companies will. Luna: Alright, I think we've covered a lot. For listeners who want to see the full audit, we'll link to it in the show notes. And if you want to support the kind of deep-dive journalism that brings these stories to light, you know where to find us. Lucas: Yeah. And on that note — I think the most important takeaway is that bias in hiring AI isn't inevitable. It's a design choice. And we, as a society, can choose differently.