Latest / The Science of Leading / AI Sourcing: Promise, Bias, and the Human-in-the-Loop

Transcript

AI Sourcing: Promise, Bias, and the Human-in-the-Loop · The Science of Leading

• 0:00
  Welcome to the show-Edwin, if a recruiter types "account executive,
• 0:04
  SaaS, Chicago" into a search bar,
• 0:06
  they might get 200 profiles.
• 0:08
  But if an AI sourcing tool looks for adjacent signals instead-pipeline
• 0:13
  ownership, vertical experience,
• 0:15
  even patterns in career moves-it suddenly surfaces people no one explicitly
• 0:19
  searched for.
• 0:20
  The quiet candidates.
• 0:20
  Yes, and that idea-quiet candidates-is really the
• 0:26
  center of this.
• 0:26
  The strongest prospects are often not actively looking.
• 0:30
  They're doing good work somewhere else.
• 0:32
  And historically, they were invisible unless their profile matched the
• 0:36
  exact language a recruiter happened to search for.
• 0:36
  Right,
• 0:40
  and language is messy.
• 0:41
  One person writes "customer expansion," another writes "account growth,"
• 0:46
  and a traditional search treats those as completely different-even though
• 0:49
  they're basically the same capability.
• 0:49
  Exactly.
• 0:52
  So now you're not just automating search-you're defining competence.
• 0:57
  And that's where this gets more serious.
• 0:59
  A well-tuned system expands your understanding of talent.
• 1:03
  A poorly tuned one just automates a narrow definition of it.
• 1:03
  So when vendors
• 1:08
  talk about "faster sourcing," you're basically saying... speed isn't the
• 1:12
  interesting part?
• 1:12
  Speed matters,
• 1:15
  but it's not the differentiator.
• 1:16
  Consistency is.
• 1:18
  A human recruiter on Monday morning and that same recruiter late Friday
• 1:22
  are not evaluating candidates the same way.
• 1:25
  An AI system applies the same logic every time.
• 1:28
  That can be useful-but also dangerous.
• 1:28
  Because "same logic every time"
• 1:33
  sounds fair... until you realize the logic itself might be flawed.
• 1:33
  Precisely.
• 1:38
  Consistency is not the same as accuracy.
• 1:41
  If the model overvalues certain employers,
• 1:44
  certain career paths, or uninterrupted trajectories,
• 1:47
  it will apply that preference perfectly-at scale.
• 1:50
  It will be wrong very efficiently.
• 1:50
  And it looks clean while doing it.
• 1:55
  That's the unsettling part.
• 1:56
  You get a dashboard, neat scores-82,
• 1:59
  61-and suddenly people treat those numbers like they were discovered,
• 2:03
  not designed.
• 2:03
  That distinction matters.
• 2:06
  These systems don't discover truth.
• 2:08
  They reflect decisions.
• 2:10
  What data was included, what success was defined as,
• 2:14
  what patterns were rewarded.
• 2:16
  If past hiring favored linear careers,
• 2:19
  the model may quietly penalize anything that deviates from that.
• 2:19
  So AI
• 2:24
  sourcing is helpful when it finds patterns humans miss-but risky when
• 2:28
  it just repackages old habits as data?
• 2:28
  That's close,
• 2:31
  but I'd sharpen it.
• 2:32
  It's not just old preferences-it's accumulated organizational habits.
• 2:37
  Things that were never formally decided,
• 2:40
  just repeated.
• 2:41
  The model treats those as signals,
• 2:43
  even if they were never intentional.
• 2:43
  And a lot of those habits come down
• 2:47
  to... comfort.
• 2:48
  Same background, same industry,
• 2:50
  same type of profile.
• 2:51
  It feels safe.
• 2:51
  Yes, and the system can reinforce that without anyone noticing.
• 2:56
  If every shortlist keeps surfacing the same kind of candidate,
• 3:00
  it can look like quality-when it's actually repetition.
• 3:00
  There's also the
• 3:04
  outreach side.
• 3:05
  "Personalized messaging" sounds great-until it turns into very polished,
• 3:10
  very irrelevant emails.
• 3:10
  Exactly.
• 3:13
  Good personalization reflects actual relevance.
• 3:16
  Something like, "I noticed you've led post-merger integrations"-that's
• 3:21
  meaningful.
• 3:22
  Referencing a university or a generic title is just noise.
• 3:26
  Bad sourcing doesn't improve connection.
• 3:28
  It just scales interruption.
• 3:28
  So if someone wants the upside without turning
• 3:34
  hiring into a black box... where do they start?
• 3:34
  Start small.
• 3:37
  One use case.
• 3:39
  For example, passive candidate discovery for a difficult role.
• 3:43
  And define success clearly-better response rates,
• 3:47
  less manual sourcing time, broader candidate pools without lowering quality.
• 3:47
  That
• 3:52
  constraint matters.
• 3:53
  Because otherwise teams roll this into everything at once,
• 3:56
  and when something goes wrong,
• 3:58
  nobody knows where it started.
• 3:58
  Exactly.
• 4:00
  Narrow scope allows for real evaluation.
• 4:03
  And anything that influences decisions-especially shortlists-needs
• 4:03
  human
• 4:08
  review.
• 4:08
  Not symbolic review.
• 4:10
  Actual scrutiny."Not symbolic review" is important.
• 4:14
  Because sometimes the human is technically there-but they're just
• 4:14
  approving
• 4:17
  whatever the system suggests.
• 4:17
  And that's not oversight.
• 4:21
  That's delegation of judgment.
• 4:23
  A shortlist determines who gets seen,
• 4:25
  and in hiring, being unseen is often the deciding factor.
• 4:25
  Let's make this
• 4:30
  practical.
• 4:31
  If I'm evaluating a tool through something like the OAD lens-what do
• 4:34
  I need to understand upfront?
• 4:34
  Three things.
• 4:37
  First, transparency-what data is being used.
• 4:41
  Second, exclusions-what's missing.
• 4:44
  And third, auditability-can you track outcomes over time and see if certain
• 4:49
  groups are consistently ranked lower.
• 4:49
  Because bias doesn't always show
• 4:54
  up as rejection.
• 4:55
  Sometimes it's quieter.
• 4:56
  A candidate just keeps landing lower in the list,
• 4:59
  so no one ever reaches them.
• 4:59
  Exactly.
• 5:01
  Bias often shows up as ranking patterns,
• 5:05
  not outright exclusion.
• 5:06
  That's why you need to compare input and output.
• 5:09
  Who entered the pool, who rose to the top,
• 5:12
  who received outreach, who moved forward.
• 5:12
  And then there's data quality.
• 5:17
  If profiles are uneven, outdated,
• 5:19
  or overly polished, the system might just reward whoever sounds the best
• 5:23
  on paper.
• 5:23
  Correct.
• 5:25
  Poor data doesn't improve through processing.
• 5:28
  It becomes consistently misleading.
• 5:30
  So the real question isn't whether the AI is sophisticated-it's whether
• 5:35
  the signal it's using is reliable.
• 5:35
  Let's test a scenario.
• 5:39
  A team uses AI sourcing for engineering managers.
• 5:42
  The tool finds strong passive candidates quickly.
• 5:45
  Recruiters spend less time searching and more time talking to people.
• 5:48
  That's the ideal.
• 5:48
  That's the productive version.
• 5:51
  The system handles breadth and initial ranking,
• 5:54
  and the recruiter focuses on relationships and fit.
• 5:57
  In that case, AI removes repetitive work and increases meaningful interaction.
• 5:57
  But
• 6:03
  the failure version is... the same speed,
• 6:06
  just aimed in the wrong direction.
• 6:08
  The shortlist leans too heavily on certain companies or titles,
• 6:11
  and everyone moves faster toward the wrong candidates.
• 6:11
  Exactly.
• 6:15
  Faster toward the wrong candidates.
• 6:17
  That's the risk.
• 6:19
  Velocity feels like progress because activity increases,
• 6:23
  but if no one questions the assumptions behind the ranking,
• 6:26
  speed becomes acceleration without direction.
• 6:26
  So the real question isn't
• 6:31
  whether AI can move faster.
• 6:32
  It's whether a team can stay deliberate when speed is no longer a constraint.
• 6:32
  Yes.
• 6:37
  And the organizations that handle this well will make a conscious choice
• 6:41
  to keep judgment slower than the software.
• 6:43
  If you're looking to operationalize that kind of structured,
• 6:47
  evidence-based hiring, you can test OAD's tools-like behavioral assessments-for
• 6:53
  free at o-a-d-dot-a-i.
• 6:55
  It's a practical way to introduce structure without losing oversight.
• 6:55
  Which
• 7:00
  is probably the balance everyone's actually trying to get to-speed where
• 7:03
  it helps, and thinking where it matters.