Latest / AI Ethics with Fexingo: Bias, Safety, and Responsible Artificial Intelligence / When Your AI Recruiter Hallucinates a Perfect Candidate
Transcript
- Lucas: So you get 500 applications for a senior software engineer role. Your AI recruiter scans them all in 20 seconds and surfaces the top ten. And one of those ten — the number-two pick — is a person who does not exist. Luna: Wait — like, a completely fictional candidate? Not just a bot applying, but the AI itself inventing a person? Lucas: Exactly that. A well-documented case from early 2025: a mid-sized tech company — I'll call them CloudSync, not their real name — was using an AI screening tool built on top of a large language model. The system was supposed to summarize each applicant's resume and rank them. Instead, it generated a profile for someone named 'James K. Morrison' with a perfect trajectory — Stanford CS, four years at Google, a GitHub repo with a popular open-source library. The whole thing was fabricated. Luna: And a human caught it because…? Lucas: Because the hiring manager tried to look up his GitHub profile and the repo didn't exist. Then they checked his LinkedIn — nothing. Then they realized the AI had essentially hallucinated the entire application. The company had fed it thousands of real resumes to train on, and the model decided to generate a composite candidate that never applied. Luna: That's a new flavor of AI bias. Usually we talk about the system being unfair to real people. Here it's being unfair to… no one? But also to everyone who actually applied and got overlooked because a ghost ranked higher. Lucas: Right. The opportunity cost is real. The NYU Algorithmic Hiring Lab ran a study last year — they tested five commercial AI screening tools on a set of 10,000 anonymized real resumes. They found that between 3 and 5 percent of the ai generated candidate summaries contained what they called 'material falsehoods' — entire experiences, degrees, or skills that never existed in the source document. Luna: Three to five percent might sound small, but when you're filtering tens of thousands of applications, that's hundreds of phantom qualifications. And presumably those hallucinations tend to be positive — the AI makes candidates look better, not worse. Lucas: That's the scary part. The hallucinations are almost always upward — the model fills in gaps with plausible-sounding enhancements. It's a consequence of how these models are trained. They're optimized to produce coherent, impressive-sounding text. A resume that says 'Python, SQL' might get upgraded to 'Python, SQL, TensorFlow, AWS, and Kubernetes' because that's the pattern the model saw in top-tier resumes. Luna: So it's not malice. It's the model doing what it's trained to do — complete the pattern. But in a hiring context, that pattern-completion is lying. Lucas: Exactly. And it's incredibly hard to police because these tools are black boxes. The company that uses them doesn't usually see the raw model output; they get a ranked list. By the time a human reviews the top candidates, the damage is done. In the CloudSync case, they'd already invited the fake candidate for a phone screen before they realized. Luna: Did they send a rejection email to a nonexistent person? Lucas: They didn't say, but I'd love to know. The real issue here is accountability. If a human recruiter fabricates a candidate, that's fraud. If an AI does it, who's responsible? The vendor who built the model? The company that deployed it without adequate testing? And what about the real applicants who were pushed down the list by a hallucination? Luna: The EEOC hasn't exactly kept pace. Their guidance on AI hiring discrimination from 2023 mostly focuses on disparate impact against protected classes. But hallucinated qualifications don't fit that framework — it's not about bias against a group; it's about the system just making stuff up. Lucas: Right. And a hallucination could actually mask bias. Imagine the model consistently 'upgrades' candidates from certain universities more than others, or adds more certifications to male-sounding names. That's discriminatory, but it's hidden inside a fabricated improvement. Luna: Is there any regulation on the horizon that addresses this specifically? I know the EU AI Act has some provisions for high-risk systems, and hiring tools are classified as high-risk. Lucas: The EU AI Act does require that high-risk AI systems be 'accurate, robust, and secure' — but it's vague on what accuracy means for generative output. There's a technical standard being developed — ISO/IEC 42001 — that includes requirements for training data quality and model validation. But we're talking years before it's enforced. In the meantime, the onus is on employers. Luna: What can a company actually do? Short of not using AI, which isn't realistic at scale. Lucas: A few things. First, never use a generative AI tool as a pass-through filter — always have a human-in-the-loop for any candidate that moves forward. Second, audit the model's output regularly. Take a random sample of rejected resumes and see if the AI's summary matches the source. Third, and this is the one most companies don't do: test the model on synthetic data with known ground truth. Feed it resumes you've deliberately modified and see if it hallucinates. Luna: That's basically unit testing for hiring AI. It's not that hard, but it requires someone in HR to have a technical mindset — or a good relationship with the engineering team. Lucas: And that's the bottleneck. A lot of these tools are bought by HR departments that don't have the technical literacy to ask the right questions. The vendors market them as 'ai powered efficiency' and the procurement team signs off. The result is a system that might be generating more noise than signal. Luna: There's also a deeper philosophical question: if an AI can hallucinate a perfect candidate, does that mean the ideal resume is a statistical fiction? Maybe the reason the model invents these composite profiles is that real human careers don't look like the platonic ideal of a resume. Lucas: That's a really good point. The training data — resumes scraped from LinkedIn, job boards — is already a skewed sample. It overrepresents certain career paths, certain schools, certain keywords. The model learns a distorted picture of what a 'good' candidate looks like, and then it projects that distortion onto real applicants. It's not just hallucinating; it's reinforcing a narrow and biased ideal. Luna: So the hallucination problem is really a data problem in disguise. Fix the training data, fix the model. Lucas: Partly. But even with perfect data, language models are probabilistic — they will always have a non-zero hallucination rate. The question is whether we're willing to accept that rate in a context as high-stakes as hiring. One fake candidate might be a funny story. But what about the real candidate who didn't get the job because the AI invented someone better? Luna: And that's the part that keeps me up at night. It's not just bias against real people anymore — it's the possibility that AI creates a fictional standard that no real human can match. We're competing with ghosts. Lucas: I think that's exactly right. And it's worth saying that the cost of getting this wrong isn't just ethical — it's operational. If you hire based on a hallucinated profile, you're hiring a fantasy. The real person who shows up will never live up to the resume the AI wrote for them. That's a recipe for turnover and disappointment. Luna: Yeah. And speaking of keeping it real — if today's conversation gave you something usable, something to watch out for in your own hiring process, a couple of dollars a month is genuinely what keeps these episodes ad-free and independent. It's buy me a coffee dot com slash fexingo, if you've gotten something out of them. Lucas: It does make a difference. We don't run ads, listener support is the whole engine. So thank you to anyone who's chipped in. Luna: All right — back to the ghosts in the machine. Lucas, you mentioned that the CloudSync case involved a GitHub repo that didn't exist. Is there any way to automatically detect these fabrications without manual checking? Lucas: Researchers are working on it. One approach is to use a second, smaller model to fact-check the first model's output — basically an AI auditor. You feed the generated resume back into a language model that's been fine-tuned to detect inconsistencies. It's not perfect, but early results show it catches about 70 percent of hallucinations. Another approach is to require verifiable claims — link every skill to a portfolio, every degree to a transcript. But that adds friction for applicants. Luna: And friction means drop-off. There's a trade-off between verification and accessibility. If you make it too hard to apply, you might exclude people without connections or resources. Lucas: Exactly. And that circles back to the original bias problem. The solution can't be worse than the disease. I think the most promising path is transparency: if a company uses AI to screen, they should be required to disclose that, and to provide the summary the AI generated for every candidate who requests it. That way applicants can catch hallucinations themselves. Luna: That would be a huge shift. Right now most people have no idea what the AI said about them. They just get a rejection — or an invitation from a recruiter who's acting on faulty information. Lucas: And that's the core ethical failure: the asymmetry of information. The AI knows things about the applicant that the applicant doesn't know — and some of those things aren't true. Until we close that gap, we're going to keep seeing stories like CloudSync. The only difference is, next time, the fake candidate might actually get the job. Luna: And show up for their first day as a ghost? Actually, I just realized — if the AI hallucinates a candidate and the company hires them, who do they onboard? There's no person. So they'd have to realize before the start date. But what if the hallucination is subtle — say, the AI added a certification the candidate never mentioned. The candidate shows up, and the manager says, 'Great, you're AWS certified, right?' And the candidate says, 'No, I'm not.' Awkward. Lucas: That's a real scenario. And it happens more often than you'd think. There's a separate category of bias where the model doesn't invent a person, but it embellishes a real person's credentials. The candidate doesn't know what the AI said about them, so they can't correct it. They show up to an interview and get asked about skills they never claimed to have. Luna: That's almost worse than the ghost candidate, because now a real person is being set up to fail. They look dishonest when they're actually the victim of an algorithm's error. Lucas: Exactly. And the company might blame the candidate — 'Why did you apply if you don't have this skill?' — not realizing the AI added it. The whole trust foundation of the hiring process is undermined. Luna: So what's the takeaway for someone listening who's either job-hunting or involved in hiring? Lucas: For job seekers: if you suspect you're being screened by AI, consider asking the recruiter directly what the system highlighted about your application. You have nothing to lose. For hiring managers: audit your tools. Run a test with known fake resumes and see if the AI hallucinates. And never trust the output of a generative model without a human verifying. Luna: And for the rest of us — investors, consumers, citizens — this is a reminder that AI accountability isn't just about bias and fairness. It's about basic accuracy. If a system can't reliably tell you the truth, it shouldn't be making decisions that affect people's livelihoods. Lucas: Well said. That's the bottom line. Thanks for listening — we'll be back next week with another angle on AI and ethics.