Latest / Hiring & Firing with Fexingo: Recruitment, Performance, and Workforce Decisions / How a One-Question Screener Cut Bad Hires by 40 Percent
Transcript
- Lucas: So I came across something this week that feels almost too simple to work, but the numbers are pretty striking. A mid-size SaaS company — about 400 employees — added a single question to their application process and cut early-stage mis-hires by roughly 40 percent. Luna: One question? Not a test, not a work sample — just a question. Lucas: Just one. And it wasn't even a trick question. It was basically: 'Describe a situation where you had to solve a problem using.' They made it required — you couldn't submit the application without answering. Luna: So it's a mini behavioral question, but at the screening stage, not the interview. Lucas: Exactly. And the effect was that a huge chunk of applicants — people who would have sailed through the resume screen — either self-selected out or revealed that they didn't actually have the practical experience they claimed on paper. Luna: I can see that. Someone says 'five years Python' on their resume, but when asked to describe a real debugging challenge, they either skip the question or give a vague answer. That tells you a lot. Lucas: Right. And the company tracked it over six months. They compared the cohort before they added the question to the cohort after. The drop in mis-hires — people they let go within 90 days — went from about 25 percent down to 15 percent. Luna: That's a massive improvement for zero cost. Did they change anything else in the process? Lucas: Nothing. Same job posts, same interview panel, same decision criteria. The only variable was that question. And the hiring managers reported that the candidates who made it to interviews were just better prepared and more aligned with the role. Luna: It makes sense when you think about it. Most of our screening filters — years of experience, degrees, keywords — are proxies. This is a direct sample of how the person thinks about their work. Lucas: It's actually a version of what some researchers call 'minimalist screening.' The idea is that traditional filters are noisy. A degree from a good school might correlate with ability, but it's not the same as seeing someone reason through a problem. Luna: And the noise is expensive. Every bad hire you make costs, what, 30 percent of their annual salary by some estimates? Lucas: Easily. So even a modest improvement in early-stage selection has huge ROI. And the beauty of this approach is that it scales. You're not asking your recruiters to read every answer carefully — you're using the question as a gate. People who can't articulate a concrete example don't move forward. Luna: But isn't there a risk that good candidates who are bad at writing on the spot get filtered out? Some people are much better talking through a problem than typing it. Lucas: That's a fair concern. The company I looked at said they allowed short answers — two to three sentences. They weren't looking for eloquence. They were looking for specificity. If someone wrote something like 'I fixed a bug by checking the logs' — that's too vague. But if they wrote 'I noticed the API was returning a 503 error under load, so I added retry logic with exponential backoff' — that tells you they've actually done it. Luna: So the bar is basically 'can you recall a real instance with enough detail that it's clearly not made up.' Lucas: That's it. And the interesting thing is that this also reduces bias. When you're screening by resume, you're unconsciously influenced by school names, previous companies, personal connections. A standardized question levels the playing field a bit. Luna: Yeah, because the answer is evaluated on content, not on pedigree. Did you catch any data on how it affected diversity? Lucas: They didn't publish that specifically, but there's research from Harvard Business Review showing that structured screening questions — especially when they're job-relevant — tend to reduce demographic disparities compared to unstructured resume review. So it's plausible. Luna: So what's the catch? Why isn't every company doing this? Lucas: I think two things. First, most companies are stuck in the habit of asking for cover letters, which are essentially the same idea but executed poorly. Cover letters are generic, people copy-paste them, and they're often about motivation, not competence. Luna: Right, 'I've always been passionate about X.' That doesn't tell you if they can do the job. Lucas: Exactly. So the question has to be skill-specific. The second barrier is that hiring managers worry about scaring off applicants. They think, 'If we require a written answer, people will drop out.' And they're right — some will. But that's the point. You want people who are serious enough to invest two minutes. Luna: And the ones who drop out were probably not your best candidates anyway. It's a feature, not a bug. Lucas: Speaking of features, we should mention that the reason we can dig into stuff like this without a paywall or ads is that some listeners chip in a few dollars to keep the show going. If these conversations have ever made you think differently about a hiring decision or saved you from a bad hire, you can support us at buy me a coffee dot com slash fexingo. Luna: Yeah, it genuinely makes a difference. Even a couple of bucks a month helps us stay independent and focused on the tactics that actually work. Lucas: Alright, back to the screener. So the company I mentioned — they actually refined their question over time. The first version was too broad: 'Tell us about a time you solved a problem.' That got too many generic answers. Luna: Right, because 'problem' could mean anything from a spreadsheet error to a client meltdown. You need to anchor it in the specific domain. Lucas: They changed it to something like: 'Describe a specific technical challenge you faced while building or maintaining a software feature, and how you resolved it.' That focused the answers. Luna: So the lesson is: the question needs to be narrow enough that a good answer is obviously good, and a bad answer is obviously bad. Lucas: And you want to avoid yes or no questions. 'Do you have experience with SQL?' — everyone says yes. But 'Give an example of a complex SQL query you wrote and what it accomplished' — that separates people. Luna: I could see this working for non-technical roles too. For a project manager, you could ask about a time they brought a delayed project back on schedule. Lucas: Absolutely. The principle is universal: ask for a concrete instance that requires the core skill. You don't need multiple questions. One well-designed question can do more than a dozen resume bullet points. Luna: What about roles where the core skill is hard to define? Like, say, a sales role that requires relationship-building. Lucas: Good question. You can still do it. 'Describe a time you turned a skeptical prospect into a long-term customer.' That's specific enough. If they can't give a real example, they probably haven't done it. Luna: So the key is that the question must be about a past behavior, not a hypothetical. 'What would you do if...' doesn't work as well. Lucas: Exactly. Hypotheticals measure imagination, not experience. Past behavior is a better predictor of future behavior. That's the whole premise of behavioral interviewing, but we usually deploy that in the interview, not at the application stage. Luna: Which is a waste, because by the time you interview someone, you've already invested time in screening them. If you can filter earlier, you save everyone time. Lucas: That's the real efficiency gain. Recruiters spend hours reviewing resumes that are full of fluff. A good screener question can cut that workload by half. And candidates appreciate it too — they know they're being evaluated on something real. Luna: So if someone wants to try this tomorrow, what's the playbook? Lucas: Step one: pick the single most important skill for the role. Not 'communication' or 'teamwork' — pick something specific to the daily work. Step two: write a question that asks for a concrete example of using that skill. Step three: make it required in the application form, with a short character limit — maybe 500 characters. Step four: have someone on the team read the answers and flag anyone who gives a vague or generic response. Luna: And you don't need any fancy software. Just add a text field to your ATS or even your Google Form. Lucas: Right. And track the results over a few months. Compare the quality of hires before and after. I suspect most teams will see an improvement. The ones that don't probably have a poorly designed question. Luna: It's such a low-risk experiment. Worst case, you waste a few hours writing the question and reviewing answers. Best case, you cut your mis-hire rate by 40 percent. Lucas: Exactly. And the beauty is that it forces you as a hiring manager to be clear about what you actually need. If you can't articulate one skill that's essential, you probably haven't thought enough about the role. Luna: That alone might be the most valuable takeaway. Lucas: Yeah, the question is as much for the company as it is for the candidate. So next time you post a job, try adding one good question. See what happens. Luna: I might try that for our next hire. I'll let you know how it goes.