Latest / Hiring & Firing with Fexingo: Recruitment, Performance, and Workforce Decisions / How a Blind Audition Doubled the Diversity of New Hires
Transcript
- Lucas: Let me tell you about the quarter that a 50-person SaaS company called KiteSync turned their hiring pipeline upside down. Q4 of last year, they were getting maybe a hundred resumes per open role. By their own count, 18 percent of candidates who made it to a first interview were from underrepresented backgrounds. Not great. Luna: Eighteen percent — that's basically one in five. For a company that prides itself on inclusive culture, that's a problem. Lucas: Exactly. So their head of people — a woman named Dana Chen — decided to try something that's been debated in hiring circles for years: blind screening. Remove names, educational institutions, graduation dates, and any photo from the initial application review. She didn't announce it to the team. She just quietly changed the applicant tracking system to hide those fields from the first reviewer. Luna: Wait — she didn't tell the hiring managers? That seems risky. Lucas: It was intentional. She wanted to see the raw effect before managers started second-guessing the process. And the result? In the first quarter of this year, the diversity of their interview slate jumped from 18 percent to 41 percent. More than doubled. The quality score assigned by the hiring team on those blind-screened candidates was actually slightly higher than their historical average. They made three hires from that cohort, and two of them came from that previously underrepresented pool. Luna: Okay, that's a compelling story. But I've read that a lot of companies try blind hiring and then abandon it within a few months. What did Dana do differently? Lucas: Great question. The biggest failure mode is that companies remove names and schools but then hand the same unstructured resume to a hiring manager who's used to scanning for keywords. They don't change the evaluation criteria. Dana paired blind screening with a structured knockout quiz — ten job-relevant scenarios, multiple-choice, timed at fifteen minutes. Candidates who scored above a threshold moved to a phone screen that also used a standardized rubric. So the process wasn't just blind, it was also consistent. Luna: A skills-based gate before the human even looks at anything. That makes sense — you're not just removing bias, you're adding signal. Lucas: Exactly. And that's backed by real research. There's a famous natural experiment from the world of symphony orchestras. In the 1970s and 80s, major orchestras started using blind auditions — musicians performed behind a screen so the jury couldn't see gender or race. A 2000 paper by Goldin and Rouse from the National Bureau of Economic Research found that blind auditions increased the probability that a woman would advance out of preliminary rounds by 25 percent. And by the 1990s, the proportion of women in the top five orchestras had gone from under 5 percent to about 25 percent. The screen was a huge part of that shift. Luna: So the NBER study is basically the gold standard proof that blind evaluation works when the evaluation itself is structured around merit. But in corporate hiring, the challenge is that 'merit' is harder to define than a perfect rendition of a Mozart concerto. Lucas: Right. And that's why many blind-hiring efforts fail — they remove demographic markers but don't replace them with objective criteria. You end up with a hiring manager looking at a stripped-down resume and still making gut calls. Dana's insight was to front-load the objectivity with that skills quiz. And she also built a simple rubric for the phone screen: four dimensions — problem-solving, communication, role-specific knowledge, and cultural contribution — each scored one to five, with behavioral anchors written out. Luna: Cultural contribution versus cultural fit — that's a subtle but important distinction. Contribution implies the candidate adds something new, rather than just blending in. Lucas: Exactly. And that rubric was shared with every interviewer beforehand. They calibrated once on a mock candidate. No one was making it up as they went along. Luna: That's the part that most companies skip. They adopt the tool — the blind resume review — but not the discipline. The rubric, the calibration, the structured quiz. Lucas: You've hit the core issue. A 2023 survey from the Society for Human Resource Management found that about 30 percent of large employers had tried some form of blind hiring. But fewer than 10 percent reported sustained use after one year. The ones that stuck with it were the ones that also changed how they evaluated candidates, not just how they filtered them. Luna: If today was actually useful to you — especially if you're thinking about overhauling your screening process — the way these conversations stay ad-free is through listener support. People who find value here sometimes drop a coffee at buy me a coffee dot com slash fexingo. It's a small gesture that keeps us independent and focused on what works, not what sells. Lucas: Yeah, that's the thing — we don't run ads, so we rely on that direct connection. If this episode gave you one idea you can use, that's what we're here for. And now back to the data — because what's interesting is that KiteSync's results aren't an outlier. Luna: They're not. There's a similar experiment from a large accounting firm that anonymized resumes for entry-level roles and saw the proportion of Black and Hispanic candidates invited to interview increase by about 30 percent. But the twist was that the hiring managers initially rated those candidates lower on 'professionalism' in the interview — a classic bias that moves from resume to face to face. Lucas: Which tells you that blind screening is necessary but not sufficient. You have to also structure the interview. Dana at KiteSync actually trained her interviewers on the rubric, ran mock interviews, and then gathered feedback every week for the first month. She found that interviewers were inadvertently asking different questions to different candidates — one would get a behavioral question, another a hypothetical. So she scripted the core questions. Luna: That's a lot of work for a 50-person company. But the payoff was a 23 percentage point increase in diversity of the slate. And they made three hires — two from underrepresented groups — with above-average performance ratings at 90 days. Lucas: Right. And the cost? Dana estimated it took her about 15 hours to build the quiz and rubric, plus three hours for interviewer training. That's 18 hours total. For a company that was already spending dozens of hours on unstructured screening that wasn't producing diverse slates. So the ROI is pretty clear. Luna: What about pushback? I imagine some hiring managers felt like their autonomy was being taken away. Lucas: There was initial resistance. One engineering manager said, 'I can tell a lot from a university name.' Dana's response was to share the data — after the quarter was over, she revealed the blind screening results and the historical bias. She showed that candidates from non-target schools were actually performing just as well in the quiz and phone screen. The manager's biggest concern — quality — was addressed with data. Luna: So she didn't just impose the process; she used the results to sell it internally. That's smart change management. Lucas: Exactly. And the key metric she tracked wasn't just diversity percentage — it was offer acceptance rate. Because sometimes a blind process can lead to candidates feeling like the company is impersonal. But here, acceptance rates stayed the same, and time to hire actually dropped by about eight days because the quiz weeded out mismatches early. Luna: I want to go back to the orchestra example for a second. The blind audition screen eliminated gender and race, but it also standardized what was being evaluated — a live performance of the same piece. In corporate hiring, the equivalent would be a work sample test. Did Dana incorporate that? Lucas: She did, but only for technical roles. For a backend engineer role, she added a 90-minute coding task that was reviewed blind — the reviewer didn't see the candidate's name or background. That task was the single best predictor of on-the-job performance, according to their three-month review. For non-technical roles, the quiz and structured phone screen were enough. Luna: So the pattern is clear: remove demographic information, add objective assessment, and structure the human interaction. That's the recipe. Lucas: That's the recipe. And the companies that do all three see results. A meta-analysis by the Harvard Business Review in 2024 looked at 30 studies on blind hiring and found that, on average, it increased the diversity of shortlisted candidates by 35 percent — but only when paired with structured interviews. Without structure, the effect dropped to single digits. Luna: Thirty-five percent is huge. And yet most companies that try it stop before they add the structure. They get discouraged when the first batch of blind-screened candidates doesn't magically fix everything. Lucas: Right. They treat it as a quick fix instead of a system change. Dana's approach was the opposite — she treated it as an experiment. She set a three-month trial, collected data, and then made adjustments. For example, she found that the quiz cutoff was too high initially, filtering out some good candidates. So she lowered the threshold and added a second, more nuanced assessment for those who barely missed the cutoff. Luna: Iteration on top of structure. I think that's the real lesson here. It's not about removing names and calling it a day. It's about building a system that you refine based on what the data tells you. Lucas: Exactly. And the beauty is that this system doesn't require a huge budget. A simple ATS that can hide fields, a Google Form for the quiz, a shared document for the rubric — that's it. The investment is in time and discipline, not software. Luna: Which brings us back to the core question: why don't more companies do this? If the evidence is so strong and the cost is low, what's the barrier? Lucas: I think it's a combination of inertia and ego. Managers believe they're good at reading people. And changing a process that feels like it's worked — even if the data says otherwise — is uncomfortable. Plus, blind hiring can feel 'cold' to some leaders. They worry about candidate experience. Luna: But Dana found that candidates actually reported a more positive experience because they felt the process was fair. She surveyed every candidate who went through the process, and the Net Promoter Score was 62 — 10 points higher than before. Lucas: That's the part that often gets overlooked. Fairness is a feature. Candidates know when they're being judged on arbitrary signals. A blind process, when done right, signals that you care about what they can do, not where they came from. Luna: And that's exactly the kind of signal that attracts top talent. People who have options tend to choose employers who seem objective. Lucas: So if you're listening and thinking about trying this, start with one role. Run a small experiment. Block out the fields, design a skills-based gate, and write a structured interview guide. Measure your slate diversity before and after. I'd bet you'll see a shift. And if you do, drop us a note — we'd love to feature your story on a future episode. Luna: I'd love to hear about a company that tried it and saw no effect, too. That would be just as instructive. Lucas: Absolutely. Negative results teach us a lot. But based on the evidence — from orchestras to SaaS companies — the odds are good that a thoughtful blind screening process will improve both fairness and quality. And in a tight labor market, that's a competitive advantage. Luna: It's one of those rare things where doing the right thing also happens to be the smart thing. Lucas: It is. And with that, I think we've covered the blind hiring playbook. Thanks for listening, and we'll see you next time.