Latest / Hiring & Firing with Fexingo: Recruitment, Performance, and Workforce Decisions / How One Firm Cut Interview Bias by Scoring for Work Sample Tests
Transcript
- Lucas: So there's this mid-sized logistics firm in the Midwest, about eight hundred employees, and they had a classic problem. They were hiring frontline warehouse managers, and roughly one in three of them washed out within the first year. That's not just a recruiting headache—that's real money in rehiring, retraining, and lost productivity. Luna: Right, and I bet the interviews felt great. These managers probably came in, talked about leadership, hit all the buzzwords. Lucas: Exactly. The interviews were long, behavioral, very polished. And yet the hires were failing on the actual job—things like scheduling shifts under a tight budget, resolving a conflict between two workers, or prioritizing tasks when a truck arrives late. None of that came through in the interview. Luna: So what did they do? Did they just toss the interviews entirely? Lucas: Not entirely. They kept a short screen, but they replaced the two-hour behavioral marathon with a work sample test. Every candidate for a manager role got the same set of scenarios pulled from real incidents at their own warehouses. They had ninety minutes to write out how they'd handle each one, and a panel scored the responses against a rubric. Luna: I love that—real scenarios, not hypotheticals. But how do you score something like that objectively? I mean, there's a lot of room for 'I'd talk to the employee' vagueness. Lucas: That's the key. They forced specificity. For the conflict scenario, the rubric awarded points for concrete steps—like 'I'd ask each person what they observed, then bring them together to set a norm.' It penalized generic answers like 'I'd listen and mediate.' And for the scheduling task, they had to produce an actual schedule that met cost constraints. If they didn't hit the numbers, they didn't pass. Luna: So it's almost like a mini simulation, not just a writing test. Lucas: Exactly. And the results were striking. After a year, first-year turnover among managers hired this way dropped from about one in three to under one in five. That's a thirty percent relative improvement in retention, just by changing how they evaluated candidates. Luna: That's huge. And I'm guessing it also changed who they hired—maybe fewer people who are just good at interviews. Lucas: That's the part that got interesting. The HR team did a quick analysis of the hires before and after. They found that candidates who had gone to certain prestigious schools, or who came from a banking background, were less likely to score high on the work sample. But they weren't necessarily worse employees—they just hadn't done this kind of hands-on operational work. So the test actually filtered for people who could do the job, not just talk a good game. Luna: But I have to push back a little. That sounds great for a blue-collar role like warehouse manager. But does that scale to, say, a marketing manager or a software engineer? I mean, you can't just give a work sample test for every role. Lucas: You can, actually, if you're thoughtful. For a marketing manager, you could give them a mock campaign brief and ask them to outline a strategy with a budget. For a software engineer, you'd give them a coding challenge that mirrors a real bug you've dealt with. The idea is to make the task resemble the actual job, not just abstract skills. Luna: Okay, but there's still the scoring problem. With something like marketing, it's more subjective. How do you make that rubric work? Lucas: You need to define what 'good' looks like before you even see the candidates. For the marketing example, you might score on structure, audience insight, budget realism, and a clear call to action. You can even have a panel score independently and then compare. The logistics firm did that—two scorers, and if they diverged by more than a point, they discussed it and calibrated. Luna: That calibration step is smart. It probably also trained the interviewers to think more about evidence and less about gut feel. Lucas: That's exactly what happened. In fact, the internal recruiters—who were initially skeptical—told me they actually felt more confident in their decisions. They had concrete evidence to point to, and it made the final hiring conversation with the ops VP a lot easier. Luna: So what was the resistance? You said they were initially skeptical. What were they worried about? Lucas: Mostly that they'd miss out on 'hidden gems'—people who have the potential but don't have direct experience. The work sample actually surfaced those people, because it's about how you approach a problem, not where you've worked. But the recruiters worried it would feel too mechanical, that it would turn off candidates who expected a traditional interview. Luna: Did it? Did candidates complain? Lucas: Surprisingly, no. In fact, their post-interview survey scores went up. Candidates said the test felt more relevant and fairer than the old interview. One person even said, 'I feel like I actually showed you what I can do.' Luna: That's the kind of feedback you want. But there's a legal angle here too. I mean, isn't there a risk that a work sample could inadvertently screen out some protected groups? Lucas: That's a real concern, and the firm handled it carefully. They validated the test against job performance data, and they made sure it measured the critical tasks, not something tangential. When you do that, work sample tests are actually more defensible than traditional interviews, because they're directly tied to the job. Courts have looked favorably on them for that reason. Luna: Good to know. So what's the one practical takeaway for our listeners who might want to try this? I mean, not everyone can overhaul their whole hiring process tomorrow. Lucas: Start small. Pick one role that you hire for a lot—like a customer service rep or a sales associate—and design a fifteen-minute work sample. Maybe it's a mock email from an angry customer, and you ask them to respond. Use a simple rubric: tone, accuracy, and whether they solved the problem. Test it on your current top performers to see what a good response looks like, then roll it out. Luna: That's a low-risk experiment. And if it works, you've got a better hire and a story to tell. Lucas: And if today's episode gave you a concrete idea you want to try, that's exactly what keeps us going. These shows stay ad-free because listeners like you support us directly. If that's you, buy me a coffee dot com slash fexingo. It's a small way to keep the conversations coming. Luna: Yeah, and we genuinely appreciate it. It keeps the lights on and the microphones warm. Lucas: Right. So back to that practical step—the firm also found that the work sample improved their diversity of hires. Because they were measuring skill, not pedigree, they ended up with a more diverse manager team than before, and that was a side benefit they hadn't fully expected. Luna: That's a powerful side effect. And it makes sense—interviews often favor people who are comfortable in that setting, which can correlate with privilege. A work sample levels the playing field. Lucas: Yeah, and it's not just about fairness. It's about getting the best person for the job. In the end, that's what hiring should be about. Luna: Agreed. So next time you're hiring, maybe skip the 'tell me about a time you...' and hand them a real problem. Lucas: That's the takeaway. And if you do try it, we'd love to hear how it goes. We're on LinkedIn—tag us with your results.