Latest / Hiring & Firing with Fexingo: Recruitment, Performance, and Workforce Decisions / Why Requiring References Backfires on Your Hiring
Transcript
- Luna: So Lucas, I just went through the process of hiring an assistant producer for our team. And the most time-consuming part wasn't the interviews or the skill test — it was the reference calls. Lucas: Right. And I bet you learned almost nothing from those calls that you didn't already know. Luna: Exactly. Everyone says the candidate was great, hardworking, a team player. It's like reading a script. Lucas: You're not alone. There's a 2025 study in the Journal of Applied Psychology that looked at over 2,000 reference letters across industries. The correlation between what those letters said and the person's actual job performance six months later? About 0.12. For context, a correlation of 0.3 is considered weak. Luna: 0.12 is basically noise. So why do we still do it? Lucas: Tradition, mostly. And a belief that references reveal something about character. But the data says they mostly reveal who the candidate is friends with or who they buttered up. Luna: Before we go deeper — I want to mention something. We keep these episodes ad-free because we think that makes the conversation more honest. If you've gotten value from these, you can support that choice at buy me a coffee dot com slash fexingo. Lucas: Yeah, that's the only reason we can dig into stuff like this without worrying about sponsor constraints. So back to references — companies like Automattic, the folks behind WordPress.com, dropped reference checks entirely about five years ago. Their reasoning was exactly this: the signal to noise ratio is terrible. Luna: And did their hiring quality suffer? Lucas: Actually, it improved. They replaced references with a paid work-sample trial that lasts a few weeks. Candidates build something real, and the team evaluates the output. That's a correlation of 0.54 with job performance, according to a meta-analysis from Schmidt and Hunter back in 1998 that's still cited today. Luna: Okay, but what about the edge case where a reference reveals a serious red flag, like harassment or fraud? Lucas: Good question. The problem is that most employers won't give that negative reference anyway. A 2023 survey by the Society for Human Resource Management found that 78 percent of companies have a policy to only confirm dates of employment and job titles. They're terrified of defamation lawsuits. Luna: So references are essentially useless for negative information too. Lucas: Exactly. The only time you get an honest negative reference is if the candidate listed someone who is either very naive or very angry. And that's rare. Luna: But what about positive references from past supervisors who actually give specific examples? Lucas: That's the best-case scenario, and even then it's biased. The candidate chooses who to list. You're not getting a random sample of their past colleagues. You're getting their fan club. Luna: So if references don't work, what's the alternative for verifying a candidate's past contributions? Lucas: I'd say two things. First, structured work samples — like Automattic's paid trial, or a case study, or a code review. Anything that simulates the actual job. Second, ask for a portfolio of specific, reviewed past projects. Not a list of duties, but a narrative of a problem they solved, with measurable outcomes. Luna: But doesn't that favor candidates who have time to prepare a portfolio? Lucas: It can, which is why you should also accept alternatives like a live conversation about a past project. The key is that you're evaluating the work itself, not a letter from someone who might be their cousin. Luna: So you're saying we should just drop references entirely? Lucas: I'm saying the burden of proof should be on the reference to show it adds value. Given the data, I'd tell most hiring managers to cut the reference call time by 80 percent and reinvest that hour into a better work sample. Luna: I think I'm going to try that on my next hire. The assistant producer we just hired — I spent three hours on reference calls. That's three hours I could have used to review their editing portfolio. Lucas: And you probably would have learned more. Here's the thing: references feel like due diligence. They give us a sense of comfort. But comfort isn't the same as accuracy. Luna: That's a good line. Comfort isn't accuracy. Lucas: The best recruiters I know treat reference calls as a check on the candidate's storytelling ability. They ask things like 'Tell me about a time you failed' and see if the candidate's version matches the reference's version. But even then, it's weak. Luna: So if I'm a hiring manager listening, what's the one action I should take? Lucas: Next time you hire, run a quick experiment. For half your candidates, do references as usual. For the other half, replace the reference call with a structured work sample — even just a one-hour simulation. Then compare the outcomes after six months. I'd bet the work sample group has lower regret hires. Luna: I like that. A/B test your hiring process. Lucas: Exactly. And if you do, let us know what you find. We might do a follow-up episode on listener experiments. Luna: Alright, so to wrap: references are weak predictors, work samples are stronger, and comfort isn't accuracy. Lucas: That's the takeaway. And one more thing — if you're a candidate listening, don't stress about references. The system is broken. Focus on building something you can show.