Latest / Hiring & Firing with Fexingo: Recruitment, Performance, and Workforce Decisions / Why Unstructured Interviews Predict Almost Nothing
Transcript
- Lucas: Luna, here's a number that should bother every hiring manager in the country: the classic unstructured job interview — you know, the forty-five-minute chat where you ask 'tell me about yourself' and 'what's your biggest weakness' — predicts future job performance with a correlation of about 0.20. Luna: 0.20 on a scale where 1.0 is perfect prediction? That's barely above random. Lucas: Barely. And that's from a 2024 meta-analysis published in the Journal of Applied Psychology — 117 studies, over 30,000 candidates. The corrected correlation was 0.20. Compare that to a structured interview — same setting, same interviewer, but every candidate gets the same job-relevant questions scored on a fixed rubric — and the correlation jumps to 0.57. Luna: So we've known this for decades. Why do companies still default to the unstructured chat? Lucas: Because it feels right. The unstructured interview gives the interviewer a sense of rapport, a gut feeling. And that gut feeling is incredibly persuasive, even though it's almost useless for predicting who will actually perform. The meta-analysis also found that interviewers' confidence in their own unstructured judgments was high — but accuracy was low. Human intuition is a terrible hiring tool. Luna: And if today's episode moved your thinking on this even a little, honestly, if it was worth a coffee to you, that link is buy me a coffee dot com slash fexingo. It's the smallest possible ask, and it keeps this show ad-free. Lucas: Yeah, exactly. We don't run sponsors — listener support is what keeps this going. So if you find value in these conversations, that's the place. Now, back to the science: there's a really concrete example from a mid-sized tech company — let's call them NexusTech — that made the switch. Luna: I love a good case study. What did they do? Lucas: They had been doing the typical free-form interview for every engineering hire. Hiring managers asked whatever came to mind. One candidate might get five behavioral questions, another might spend thirty minutes talking about a side project. After a year of that, they analyzed their own data and found zero correlation between interview ratings and performance ratings six months in. Luna: Ouch. So they were basically flipping a coin. Lucas: Worse — a coin flip would at least be unbiased. Their unstructured process was also introducing systematic bias. Candidates who were extroverted, or who shared a hobby with the interviewer, got higher scores. So they redesigned the interview around a structured scorecard: five competencies — problem-solving, communication, collaboration, technical skill, and learning agility — each with a four-point rubric with behavioral anchors. Luna: And the same questions for every candidate? Lucas: Same core questions, yes. Each competency had two must-ask questions, plus follow-ups allowed for clarification but not for introducing new topics. Every candidate got the same first impression because the first question was always the same. After one year, NexusTech tracked new-hire performance ratings against the previous year's cohort. The structured-interview hires scored 0.8 standard deviations higher on average. Luna: 0.8 standard deviations is enormous. That's like moving from the 50th percentile to the 79th. Lucas: Right. It's a massive lift. And they also saw a drop in early attrition — fewer people leaving in the first six months. The structured interview wasn't just predicting performance better, it was also helping candidates self-select. When you ask concrete, job-relevant questions, candidates get a clearer picture of the role and culture. Some self-disqualify, and that's fine. Luna: But I hear this pushback a lot: structured interviews feel stiff, they kill rapport, they might scare off creative candidates. What's the data on that? Lucas: Great question. There's a 2022 study in the International Journal of Selection and Assessment that looked at candidate reactions. They compared structured and unstructured interviews in a field setting with 400 applicants. Candidates in the structured interviews actually rated the process as fairer, more job-relevant, and more respectful of their time. Rapport scores were similar between the two formats — interviewers can still be warm and engaged within a structured format. Luna: So the fear of structure harming candidate experience is overblown. Lucas: Completely. And the idea that structure filters out creativity is not supported. If creativity is a relevant competency, you build it into the rubric. Ask a situational question: 'Here's a problem we face. Walk me through how you'd approach it.' That's structured, and it lets creative thinkers shine. The structure is about consistency, not rigidity. Luna: What about the argument that unstructured interviews reveal personality fit, or culture fit? Lucas: That's a dangerous phrase — 'culture fit' is often a cover for similarity bias. The data shows that unstructured interviews are worse at predicting culture fit than structured ones, because in an unstructured format, 'fit' just means 'I like you.' A structured interview can include questions designed to assess alignment with values: 'Tell me about a time you disagreed with a team decision. How did you handle it?' That gives you real information about values, not just whether the interviewer enjoys the conversation. Luna: So the real barrier isn't effectiveness — it's habit. And maybe fear of change. Lucas: Exactly. The meta-analysis I mentioned also found that organizations that trained their interviewers on structured techniques saw an average validity increase of 0.15 points. Just training. And yet, a 2025 survey from the Society for Human Resource Management found that only 28 percent of organizations use fully structured interviews for all roles. The other 72 percent are leaving money on the table. Luna: What's the single biggest lever for a company that wants to move from unstructured to structured? Lucas: Start with the scorecard. Define the three to five competencies that actually predict success in that role — not the laundry list from the job description, but the ones that separate top performers from average. Then write behavioral or situational questions that target each competency. And train every interviewer to ask those questions and rate responses against the rubric. No ad-libbing. Luna: And what about the panel? Do structured interviews work better with one interviewer or a panel? Lucas: Panel interviews can be structured too, but they add complexity. The research shows that a panel of two to three trained interviewers using the same rubric can achieve inter-rater reliability of 0.80 or higher — that's excellent. But if each panel member is asking their own questions, you lose structure. The key is that all panelists use the same scorecard and are trained to rate independently before discussing. Luna: So the process is standardized, but the conversation can still feel natural. Lucas: Right. The best structured interviews feel like a conversation, not an interrogation. The structure is invisible to the candidate — they don't know that every candidate is asked the same first question. They just know the questions are relevant and the interviewer is engaged. That's the sweet spot. Luna: One more thing — does structure help reduce bias across gender and race? Lucas: Yes, significantly. A 2023 meta-analysis by the Journal of Organizational Behavior found that structured interviews reduce demographic subgroup differences by about 40 percent compared to unstructured interviews. The standardized questions and scoring minimize the influence of stereotypes and interpersonal biases. It's not a panacea — bias can still leak in through the rubric design itself — but it's a massive improvement. Luna: So the case is overwhelming. Why aren't more companies doing it? Lucas: Two reasons. First, inertia — the unstructured interview is the default, and changing a process that 'feels' fine requires a champion. Second, managers overestimate their own interviewing skill. In that same SHRM survey, 76 percent of hiring managers said they were above average at interviewing. Math alone says that's impossible. The overconfidence gap is real. Luna: And overconfidence is hard to argue with data because the feedback loop is broken — you never know about the candidate you didn't hire who would have been great. Lucas: Exactly. The cost of a bad unstructured interview isn't visible. You don't see the false negatives — the great candidates you passed over because your gut said no. And you don't always connect the false positives — the bad hires you made — back to the interviewing process. So the status quo persists. Luna: Alright, so for any hiring manager listening today: what's the one change you can make by next week? Lucas: Write down the three most important competencies for the role you're hiring for right now. Then rewrite your first interview question to be a behavioral question tied to competency one. That's it. Start there. Test it with three candidates. Compare your notes to how you felt after your old unstructured first question. I suspect you'll see the difference immediately. Luna: Small steps, big evidence. I'm sold. Lucas: The data's been clear for twenty years. The question is whether we're willing to trust the numbers over our own intuition. Thanks for listening — next episode, we'll tackle reference checks. Do they add anything useful?