Latest / The Science of Leading / Intent Is Not a Defense

Transcript

Intent Is Not a Defense · The Science of Leading

• 0:00
  Welcome to the show everybody!
• 0:01
  I'm Claire Monroe, here with Edwin Carrington.
• 0:04
  Edwin, there's a sentence I hear from hiring leaders all the time when the topic of
• 0:08
  AI screening comes up: we're not worried about bias, because nobody here is trying
• 0:12
  to discriminate.
• 0:13
  And I want to spend today on why that sentence offers almost no protection.
• 0:17
  It's a very human sentence, Claire, and it's built on a misunderstanding.
• 0:22
  Most people think discrimination is about intent.
• 0:25
  Someone meant to exclude someone.
• 0:28
  In American employment law, there's a second standard that has nothing to do with
• 0:31
  intent.
• 0:32
  It's called disparate impact, and it's been around for more than fifty years.
• 0:37
  It goes back to a Supreme Court case from 1971, Griggs versus Duke Power.
• 0:42
  The company required a high school diploma and a pair of aptitude tests for certain
• 0:46
  jobs.
• 0:47
  Those requirements looked neutral on paper.
• 0:50
  But they screened out Black applicants at far higher rates,
• 0:53
  and the company couldn't show the requirements had anything to do with doing the
• 0:57
  job.
• 0:58
  The Court ruled that a practice can be neutral on its face and still be unlawful if
• 1:03
  it operates to exclude a protected group and isn't job related.
• 1:07
  That is the whole principle.
• 1:08
  The law looks at what the process does, not at what anyone meant.
• 1:13
  Congress later wrote it directly into Title Seven of the Civil Rights Act in 1991.
• 1:19
  So a hiring step that filters out a protected group at a much higher rate has to be
• 1:23
  justified by the actual demands of the job.
• 1:26
  If it can't be, good intentions won't save it.
• 1:29
  And there's a practical yardstick for this.
• 1:32
  The federal Uniform Guidelines from 1978 use what's called the four-fifths rule.
• 1:37
  If the selection rate for one group is less than eighty percent of the rate for the
• 1:41
  group selected most often, that's treated as evidence of adverse impact.
• 1:46
  Which means adverse impact is something you can measure.
• 1:49
  And anything you can measure, you can be held accountable for not measuring.
• 1:52
  So why does AI make this more urgent?
• 1:54
  Biased hiring steps have existed for a long time.
• 1:57
  Because AI turns one bad judgment into a rule that runs on every applicant,
• 2:02
  every day.
• 2:03
  A human interviewer with a bias makes a handful of bad calls a week.
• 2:08
  A screening rule with a bias makes thousands, consistently,
• 2:11
  and it leaves a perfect record of every one of them.
• 2:14
  That record is exactly what a disparate impact case is built on.
• 2:18
  We've already seen that play out.
• 2:20
  In 2023 the EEOC settled what's widely described as its first case involving AI
• 2:25
  hiring software, against a tutoring company called iTutorGroup.
• 2:30
  Its application software automatically rejected women aged fifty-five or older and
• 2:35
  men aged sixty or older.
• 2:37
  More than two hundred qualified applicants were turned away.
• 2:40
  The settlement was three hundred sixty-five thousand dollars.
• 2:44
  And that one was blunt.
• 2:45
  Someone programmed an age cutoff.
• 2:47
  The harder cases are the ones where nobody programmed anything,
• 2:51
  and the model simply learned a pattern from past data that happens to track age or
• 2:55
  race or sex.
• 2:56
  Which brings up the question I think every buyer asks.
• 2:59
  If we license the tool from a vendor, isn't the vendor responsible?
• 3:03
  That is the most dangerous assumption in the room, Claire.
• 3:06
  The employer makes the hiring decision, so the employer carries the exposure.
• 3:11
  You can't outsource accountability by outsourcing the software.
• 3:14
  And the courts are starting to extend that exposure to vendors too,
• 3:17
  not move it away from employers.
• 3:19
  In the case Mobley versus Workday, a federal judge in California ruled in 2024 that
• 3:25
  a software vendor could plausibly be treated as an agent of the employers using it.
• 3:30
  Then in May 2025, the court let a nationwide age discrimination collective action
• 3:35
  move forward.
• 3:36
  So now both sides of the contract are in the frame.
• 3:39
  More parties on the hook, not fewer.
• 3:41
  That's the direction of travel.
• 3:43
  I want to raise the obvious counterpoint, though.
• 3:45
  In April 2025, an executive order directed federal agencies to stop pursuing
• 3:50
  disparate impact claims.
• 3:52
  Doesn't that lower the risk?
• 3:53
  It changes who knocks on the door, Claire.
• 3:56
  It doesn't change the law.
• 3:57
  Disparate impact is written into the statute, and an executive order can't repeal a
• 4:02
  statute.
• 4:03
  Private plaintiffs can still bring these claims.
• 4:06
  Several states, including New York, California and Illinois,
• 4:09
  have their own protections.
• 4:11
  And New York City's Local Law one forty-four already requires annual independent
• 4:16
  bias audits for automated hiring tools.
• 4:19
  Enforcement priorities shift with every administration.
• 4:22
  Your hiring records will outlast all of them.
• 4:25
  So what does a defensible process actually look like?
• 4:27
  Four things.
• 4:29
  Every step you use must be tied to the real demands of the job.
• 4:33
  You track pass-through rates by group at every stage, so you see a problem before a
• 4:38
  plaintiff does.
• 4:40
  You demand the validation evidence from any vendor, in writing.
• 4:43
  And a human stays accountable for the final decision.
• 4:47
  No tool should ever be the sole basis for a hire.
• 4:50
  And that's the standard OAD holds itself to, right?
• 4:53
  It is.
• 4:54
  OAD is built as one structured input in the hiring decision,
• 4:58
  never the sole basis.
• 4:59
  It has been EEOC compliant since 2001, with no adverse impact on any protected
• 5:05
  class.
• 5:06
  It meets APA standards, independently reviewed in 2023.
• 5:11
  It has gone through three independent third-party validation cycles.
• 5:15
  And in thirty-eight years, there have been zero successful legal challenges.
• 5:20
  That record isn't luck.
• 5:21
  It comes from measuring behavioral drive against the demands of the role,
• 5:25
  and nothing else.
• 5:26
  If you want to see what a validated, structured input looks like next to what you're
• 5:30
  running today, start with the person you trust most on your team.
• 5:34
  OAD's free Top Performer Profile takes seven minutes, you get results in twenty-four
• 5:39
  hours, and there's no credit card.
• 5:41
  That's O-A-D dot A-I.
• 5:43
  Good intentions are where fairness starts, Claire.
• 5:46
  Evidence is how you prove it.
• 5:47
  Perfect place to leave it.
• 5:49
  Thanks for the wisdom, Edwin.
• 5:50
  Always a pleasure, Claire.