Latest / The Science of Leading / Beyond Gut Feel: Building Fair, Defensible Leadership Evaluation Systems

Transcript

Beyond Gut Feel: Building Fair, Defensible Leadership Evaluation Systems · The Science of Leading

• 0:00
  Welcome back to The Science of Leading.
• 0:01
  I'm Claire, and today we're talking about something that drives a lot
• 0:05
  of HR folks quietly up the wall: leadership evaluations.
• 0:05
  Mm-hm.
• 0:09
  The annual ritual where everyone pretends we're assessing leadership...
• 0:14
  and instead we reward confidence,
• 0:16
  charisma, and good storytelling.
• 0:16
  Exactly.
• 0:19
  I've sat in so many calibration meetings where the "strong leader" is
• 0:23
  just the person who talks a good game.
• 0:24
  Or the system is so complicated nobody even uses it.
• 0:28
  So Edwin, can we start with the biggest ways these evaluations fail?
• 0:28
  Let's
• 0:33
  name three.
• 0:33
  First, gut-feel reviews.
• 0:35
  A manager thinks, "I like this person,
• 0:38
  they seem strong in meetings," and that vibe turns into a rating.
• 0:41
  No structure, no shared standard.
• 0:44
  That's charisma bias in action.
• 0:46
  Second, hero worship of results.
• 0:49
  "They hit their numbers, so they must be a great leader," even if their
• 0:53
  team's burned out and updating LinkedIn.
• 0:56
  Third, over-engineered frameworks.
• 0:58
  Fifteen competencies, six-point scales,
• 1:02
  a 40-page guide... and then in practice people ignore it and go right back
• 1:06
  to gut feel.
• 1:06
  Yeah, the beautiful framework that lives in a slide deck,
• 1:10
  not in actual decisions.
• 1:12
  When you say "charisma bias," what does that look like on the ground?
• 1:12
  It's
• 1:16
  the leader who sounds brilliant in presentations,
• 1:18
  so we assume they're strong at coaching,
• 1:20
  conflict resolution, all of it.
• 1:22
  But if you ask their direct reports,
• 1:24
  you hear, "We never get clear priorities," or "They avoid tough
• 1:24
  conversations."
• 1:30
  The show is great.
• 1:31
  The day-to-day leadership behaviors are not.
• 1:31
  So let's flip this.
• 1:35
  For an HR leader who's tired of that pattern,
• 1:37
  how would you define a fair, defensible leadership evaluation system?
• 1:37
  I'd
• 1:42
  use four words.
• 1:43
  Role-specific, behavior-based,
• 1:45
  multi-method, and decision-linked.
• 1:48
  Role-specific means you're clear about what "good" looks like for a team
• 1:52
  lead versus a senior VP.
• 1:54
  Behavior-based means you score what people actually do-how they run one
• 1:59
  on ones, how they make decisions-rather than labels like "is a visionary."
• 2:04
  Multi-method means you never rely on one input.
• 2:07
  You combine 360 feedback, a validated survey-something practical like
• 2:13
  OAD-performance data, maybe a simulation.
• 2:16
  And decision-linked means you know exactly what this system is informing:
• 2:21
  who's ready for promotion, who needs coaching,
• 2:24
  who's a succession candidate, where you have leadership risk.
• 2:24
  I like that
• 2:29
  you put decisions last, but also... it sounds like that's where most people
• 2:33
  should start and don't.
• 2:34
  They pick tools first and only later ask,
• 2:37
  "What are we going to do with all this data?"That's right.
• 2:40
  Before you buy a tool or design a form,
• 2:43
  you should answer, very plainly: what decisions must this evaluation
• 2:47
  support in the next year or two?
• 2:49
  Promotion and succession are obvious.
• 2:52
  Development is another: what will guide coaching plans and training investments?
• 2:57
  And risk management: do we have leaders whose behaviors are damaging
• 3:02
  culture, increasing attrition, or creating ethical risk?
• 3:02
  So if I'm building
• 3:07
  or fixing a system, I might literally write at the top of the doc,
• 3:11
  "This process informs: promotion,
• 3:13
  succession, development planning,
• 3:15
  and leadership risk calls." And then sanity-check every question or tool
• 3:20
  against that list.
• 3:20
  Exactly.
• 3:22
  If a piece of data won't change a decision,
• 3:25
  it's probably noise.
• 3:27
  Fair and defensible does not mean exhaustive.
• 3:29
  It means you can explain, calmly,
• 3:32
  how you evaluated a leader, why you trust the data,
• 3:35
  and how it tied to an actual decision.
• 3:35
  Okay,
• 3:39
  so we've named the failure modes and the north star.
• 3:42
  Next we'll get into the mechanics: how you actually design that behavior-based,
• 3:46
  role-specific framework without turning it into a bureaucratic side quest.
• 3:46
  Alright,
• 3:51
  Edwin, let's get practical.
• 3:53
  I'm an HR leader.
• 3:54
  I want something my managers will actually use.
• 3:56
  Where do I start on this competency and behavior piece without drowning
• 4:00
  in complexity?
• 4:00
  Start small and start with your own reality.
• 4:04
  Three questions: What does this role really own?
• 4:08
  Where do leaders here typically fail?
• 4:11
  And what do your best leaders do differently?
• 4:13
  From that, build a short competency model by level.
• 4:17
  For example: team leads, mid-level leaders,
• 4:21
  senior leaders.
• 4:21
  Can you walk through a simple version of that?
• 4:21
  Sure.
• 4:26
  Team leads: you might focus on coaching and feedback,
• 4:30
  basic prioritization, and managing team conflict.
• 4:33
  Mid-level leaders: cross-functional influence,
• 4:37
  managing managers, and decision quality across a broader scope.
• 4:41
  Senior leaders: strategic thinking,
• 4:44
  building other leaders, and culture shaping.
• 4:46
  That's your skeleton.
• 4:48
  Then you translate each competency into observable behaviors.
• 4:48
  This is
• 4:54
  where people tend to stay vague,
• 4:55
  right?
• 4:55
  "Great communicator," "strategic," all that.
• 4:55
  Yes,
• 4:59
  which is useless for scoring.
• 5:01
  Take "communication" for a team lead.
• 5:03
  Behavior examples: "Explains the top three priorities and why they matter,"
• 5:09
  "Checks understanding and adjusts when people look confused," "Addresses
• 5:14
  conflict directly and respectfully instead of avoiding it." Those are
• 5:18
  things a manager, a peer, or a direct report can actually observe.
• 5:18
  And
• 5:24
  then we need some kind of rating scale that anchors those behaviors.
• 5:27
  How would you build that?
• 5:27
  That's where behaviorally anchored rating scales-BARS-come
• 5:32
  in.
• 5:32
  You keep the scale simple, say one to five,
• 5:35
  and define what each point looks like.
• 5:38
  For example, for "clarity of priorities" for a team lead: One: "Rarely
• 5:44
  communicates priorities; team often unsure what matters." Three: "Usually
• 5:50
  communicates priorities clearly; confusion gets corrected when raised."
• 5:56
  Five: "Communicates priorities and trade-offs clearly and repeatedly;
• 6:01
  team stays aligned even when things change." Now different raters can
• 6:06
  score more consistently, because they're reacting to descriptions,
• 6:10
  not to their mood.
• 6:10
  That already feels more fair.
• 6:14
  I can picture an HRBP putting those anchors right into the 360.
• 6:18
  Speaking of, how do you combine all these methods without making a noisy
• 6:21
  mess?
• 6:21
  You assign each method a job.
• 6:24
  Direct reports are your best source for day-to-day leadership behaviors:
• 6:29
  coaching, clarity, psychological safety.
• 6:32
  Peers see collaboration and influence.
• 6:35
  The manager sees scope and outcomes.
• 6:37
  A fast, validated survey like OAD adds a view into underlying traits
• 6:42
  and style-how this person is likely to behave under pressure.
• 6:45
  Simulations or behavioral interviews show how they think in real time.
• 6:50
  You don't average everything blindly.
• 6:52
  You weight each source based on what it sees best.
• 6:52
  Can you give an example
• 6:58
  of that weighting in practice without turning it into a math project?
• 6:58
  Sure.
• 7:02
  Imagine you're assessing "conflict resolution" for a mid-level leader.
• 7:06
  You might treat direct report and peer 360 scores as primary data.
• 7:10
  The manager's view is secondary.
• 7:13
  A simulation where they handle a tough stakeholder conversation is also
• 7:18
  primary.
• 7:18
  Their self-assessment and OAD style profile are supporting data-useful
• 7:24
  to explain patterns, but not the main evidence.
• 7:26
  If self and others disagree, that gap tells you about self-awareness.
• 7:26
  So
• 7:33
  the trick is: be explicit.
• 7:35
  "For this competency, here's what we're listening to most,
• 7:38
  here's what we're using as a cross-check." That alone would be a big
• 7:42
  upgrade from the "everything counts equally... or maybe nothing does" approach.
• 7:42
  Exactly.
• 7:48
  And document it in plain language so any leader can understand it.
• 7:52
  The goal is a framework people actually trust and can use in real
• 7:52
  conversations-not
• 7:58
  a theoretical model that only lives with HR.
• 7:58
  So now we've run the 360,
• 8:03
  maybe an OAD survey, looked at performance data,
• 8:05
  maybe a simulation.
• 8:06
  I've got this thick packet on a leader.
• 8:08
  The question I always hear is,
• 8:10
  "Now what?" How do we turn that into growth instead of just an interesting
• 8:13
  PDF?
• 8:13
  You start by ruthlessly prioritizing.
• 8:17
  One or two development themes,
• 8:19
  not ten.
• 8:20
  For each theme, define a specific behavior change,
• 8:24
  and then a simple action plan with coaching support.
• 8:27
  For example, theme: "Avoids difficult conversations." Behavior change:
• 8:33
  "Addresses performance issues within 72 hours,
• 8:36
  using a structured script." Actions: weekly review of open issues,
• 8:41
  scheduled conversations, plus coaching on language.
• 8:41
  So instead of,
• 8:45
  "Work on communication," it's,
• 8:47
  "In your weekly team meeting, state the top three priorities and ask
• 8:51
  each person to repeat what they heard." That's tangible.
• 8:51
  Exactly.
• 8:56
  Then you add measurement.
• 8:58
  Maybe a short pulse survey to direct reports: "My leader clearly communicates
• 9:03
  our top priorities" on a simple scale.
• 9:05
  You re-check after, say, three months.
• 9:08
  That's how you connect evaluation to real behavior change.
• 9:08
  What about
• 9:13
  trust?
• 9:14
  A lot of leaders get nervous when they hear "360" or "assessment." How
• 9:19
  do we protect psychological safety while still being honest?
• 9:19
  You need clear
• 9:23
  rules.
• 9:24
  Anonymity thresholds for direct report feedback-no reporting a separate
• 9:29
  score if there are only one or two people.
• 9:31
  Very careful handling of comments to remove identifying details.
• 9:36
  And you're transparent about boundaries: this data is primarily for development
• 9:41
  and succession, not as a surprise weapon in performance management.
• 9:45
  If you will use it in high-stakes decisions,
• 9:48
  say exactly how.
• 9:48
  And there's a DEI angle here too.
• 9:52
  If the process is vague, bias creeps in.
• 9:55
  If it's behavior-based and multi-method,
• 9:57
  you have more to stand on when someone asks,
• 9:59
  "Why was this person promoted and not that one?"That's right.
• 10:03
  Behavior anchors, multiple perspectives,
• 10:06
  and validated tools help reduce the impact of "people like me" bias.
• 10:11
  You can say, "Here's the competency model,
• 10:14
  here's the data from direct reports,
• 10:17
  peers, manager, and a survey like OAD,
• 10:20
  here's how we interpreted it, and here's the development plan." It doesn't
• 10:25
  make bias disappear, but it gives you a much fairer,
• 10:29
  more defensible system.
• 10:29
  Last piece I want to hit is scaling.
• 10:33
  HR teams are busy.
• 10:35
  How do you make this something we can actually run every year,
• 10:38
  not just for a one-time pilot?
• 10:38
  Keep the governance light but real.
• 10:43
  Standard templates: one-page competency model per level,
• 10:47
  a common 360 questionnaire with behavior anchors,
• 10:51
  a simple summary report for leaders and for sponsors.
• 10:54
  A basic calibration process so ratings mean the same thing across functions.
• 11:00
  And a clear yearly cycle: when you collect data,
• 11:03
  when you run debriefs, when you review progress.
• 11:03
  So almost like an operating
• 11:09
  rhythm: evaluate, debrief, coach,
• 11:11
  pulse-check, repeat.
• 11:11
  Exactly.
• 11:13
  And build feedback loops.
• 11:15
  Quarterly, ask: Are we learning the right things from this process?
• 11:20
  Are development plans actually changing behavior?
• 11:23
  Are there groups consistently under- or over-rated?
• 11:26
  Then refine.
• 11:28
  The system should get simpler and sharper over time,
• 11:31
  not heavier.
• 11:31
  If someone listening is thinking,
• 11:34
  "Our current leadership reviews are mostly opinion and politics," what's
• 11:38
  a concrete next step they could take after this episode?
• 11:38
  Pilot a better
• 11:42
  way with a small group of leaders.
• 11:44
  Define a tight competency set,
• 11:47
  run a structured 360 with behavior anchors,
• 11:50
  add a quick, validated survey like OAD to get a style and trait perspective,
• 11:55
  and then build one-page development plans.
• 11:58
  Compare that to your usual process.
• 12:00
  If the conversations feel clearer and less political,
• 12:03
  you're on the right track.
• 12:03
  And if you want to experiment without a huge
• 12:08
  investment, you can actually test OAD for free at OAD.
• 12:08
  ai.
• 12:12
  Use it alongside your current leadership evaluations and see if it gives
• 12:15
  you cleaner, behavior-focused data you can defend in a promotion or succession
• 12:20
  discussion.
• 12:20
  Used that way, it's not a magic answer.
• 12:23
  It's one more reliable lens in a fairer system.
• 12:23
  Edwin,
• 12:27
  thanks as always.
• 12:28
  This gives HR and People leaders a really practical path to move from
• 12:32
  "vibes-based" leadership reviews to something they can actually stand
• 12:35
  behind.
• 12:35
  My pleasure, Claire.
• 12:35
  And thanks to all of you for listening.
• 12:40
  We'll be back with more on designing systems that make leadership better,
• 12:44
  not just louder.
• 12:45
  For now, take care, and we'll talk to you next time.