Latest / Safe As Podcast / E64: How can we move forward from notions of 'human error'?

Transcript

E64: How can we move forward from notions of 'human error'? · Safe As Podcast

• 0:05
  Declaring that we found and fixed human error is tempting,
• 0:09
  and masquerades is a suitable explanation.
• 0:12
  But what can we learn from complex system failures about
• 0:15
  how people contribute to safety rather than mend the source of
• 0:19
  failure? Good, everyone.
• 0:21
  I'm Ben Hutchinson and this is safe as I proved the analysis of
• 0:25
  safety and risk research. visitsafetyinsights.org for more
• 0:29
  research. Today's paper is from Woods and
• 0:33
  Cook 2002, titled 9 Steps to Move Forward from Error
• 0:38
  Publishing, Cognition, Technology and Work.
• 0:41
  This is a discussion paper and provides several maxims and
• 0:45
  insights on how we can move beyond simple labels of human
• 0:49
  error. Unfortunately, because this is
• 0:52
  quite dense, I've had to skip heap, so I recommend checking
• 0:55
  out the original paper and in classic style I've mixed up some
• 0:59
  terms like steps and accents. If this offends you then write
• 1:04
  me an I'll tempered letter on how I've let you down.
• 1:07
  One, they say to pursue second stories beneath the surface
• 1:12
  after failure. Explanations often default to
• 1:15
  first stories, which are overly simplified accounts of the
• 1:19
  apparent causes framed by hindsight knowledge.
• 1:23
  This hindsight bias narrows and distorts our view of practise
• 1:26
  after the fact and hides dilemmas, complexities and
• 1:30
  difficulties that the practitioners face at the time.
• 1:34
  This narrows learning by prematurely closing
• 1:37
  investigations and inquiry, masking pressures that shape
• 1:41
  performance and obscuring how organisations actually function
• 1:45
  beneath the surface. First stories feel satisfying
• 1:49
  but lead to sterile responses that limit learning and
• 1:52
  improvement. Progress and safety requires
• 1:55
  uncovering and, in a way, constructing these second
• 1:58
  stories and some of the next steps show how two try to escape
• 2:03
  hindsight bias as much as possible.
• 2:06
  First, stories told after the accidents really tell us very
• 2:10
  little or nothing about the factors that influence human
• 2:13
  performance before the fact. They simplify complexity and
• 2:16
  distort how practitioners normally succeed.
• 2:20
  The result can be reduced flow of safety, critical information
• 2:24
  and new complexities. At plague practise, several
• 2:28
  techniques and even structures in the organisation must be
• 2:31
  leveraged to try to escape the hindsight bias.
• 2:34
  This is essential for learning. Three, understand work is
• 2:38
  performed at the sharp end. Second stories examine
• 2:42
  interactions of organisational, economic, human, and
• 2:46
  technological factors. Practitioners constantly manage
• 2:49
  trade offs and conflicting goals using the tools and resources
• 2:52
  they have available. According to paper, improving
• 2:56
  safety depends on investing in resources that support
• 2:58
  practitioners in meeting the demands and overcoming the
• 3:01
  inherent hazards in that setting.
• 3:04
  Progress depends on understanding actual work, not
• 3:07
  the work that's imagined in procedures or prescribed in the
• 3:09
  procedures or believed to happen from managers and whatnot.
• 3:14
  Human performance is, in a sense, a type of adaptation to
• 3:17
  the complexity of the environment and whatever demands
• 3:20
  and constraints are placed on them.
• 3:22
  Re propositions result from this idea.
• 3:25
  Look for sources of success. Failure occurs in situations
• 3:29
  that usually succeed. Study as systems deliver success
• 3:32
  that's find opportunities to fail.
• 3:35
  Two, look for difficult problems 2.
• 3:39
  Understand failure. Look at what makes problems
• 3:41
  difficult. Studying challenging problems
• 3:43
  reveals brittleness and expert strategies to adapt. 3 Be
• 3:48
  practise centred. Understand work from the
• 3:51
  practitioner's perspective. They talk about the
• 3:54
  psychologist's fallacy, which assumes distant observers.
• 3:58
  Grasp the true nature of practise.
• 4:01
  Descriptions of work and actual work both differ.
• 4:05
  Both must be examined down to the 4th maxim.
• 4:09
  Search for systemic vulnerabilities.
• 4:12
  They say that safety is created and sometimes broken in systems,
• 4:15
  not individuals. The goal is finding
• 4:18
  vulnerabilities, not blaming people. 3 supporting principles
• 4:22
  come out of this. One is a systems maximum.
• 4:26
  Safety is an emergent property of systems.
• 4:29
  Work practises evolved to cope with hazards and constraints.
• 4:32
  These adaptations become brutal under change, and precursor
• 4:36
  events reveal vulnerabilities, but acting on them often
• 4:40
  requires sacrificing productivity, creating internal
• 4:43
  conflict 2 Detection and recovery are critical.
• 4:47
  Systems must continually learn about threats and respond
• 4:50
  adaptively 3 Safe organisations deliberately search for and
• 4:55
  learn about systemic vulnerabilities, kind of what we
• 4:57
  discussed. Four Study how practise creates
• 5:01
  safety. A frequent fallacy assumes
• 5:04
  systems are inherently safe and values reflect unreliable
• 5:07
  people. Second, stories show how
• 5:09
  practitioners continually create success and create a paper in
• 5:14
  the face of these threats. Teams adapt tools and procedures
• 5:17
  to guard against hazards. Safety is constructed, not
• 5:21
  given, not an inherent feature of most complex systems, and
• 5:26
  search for underlying patterns. After failure, people focus on
• 5:30
  surface traits, but progress comes from identifying deeper
• 5:34
  patterns of coordination and adaptation.
• 5:36
  On to the next maxim Seven. Examine how change produces new
• 5:41
  vulnerabilities. Vulnerabilities shift with
• 5:43
  changes in capacities and capabilities, technology and
• 5:47
  pressure. Some principles here is safety
• 5:50
  is a dynamic process. Systems operate in changing
• 5:53
  worlds. Planned defence is a road under
• 5:56
  production pressure, creating drift towards failure.
• 5:59
  Every change can introduce new hazards, even while providing
• 6:02
  benefits. 2 It's a law of stretch systems under resource
• 6:07
  pressure. The benefits of change are taken
• 6:10
  in increased productivity rather than enhanced resilience.
• 6:15
  Change often increases coupling to boost efficiency, raising
• 6:18
  complexity and vulnerabilities. 3 Increasing coupling increases
• 6:24
  complexity. More coupling, as in more these
• 6:27
  processes are connected, creates new cognitive and collaborative
• 6:31
  demands and new value modes. And four, they talk of a window
• 6:36
  of opportunity principle, periods of change, and moments
• 6:40
  to identify and treat emerging vulnerabilities when it's fresh
• 6:43
  in people's minds. Last two maxims surrender 8 Use
• 6:48
  new technology to support and enhance expertise.
• 6:52
  Technology is often assumed to reduce human variability and
• 6:55
  breakdown. In practise, they say, pesky
• 6:58
  details determine whether computerization introduces new
• 7:01
  value paths. So technology can both help and
• 7:06
  harm depending on how it fits actual work.
• 7:09
  They talk about a joint systems principle.
• 7:12
  People in automation form a distributed cognitive system.
• 7:15
  Effective technology design requires understanding expert
• 7:18
  performance and what challenges is it and another principle.
• 7:23
  There is no neutral in design. Design either supports expertise
• 7:27
  or hinders. Finally, 9 Tame complexity
• 7:31
  through new forms of feedback. Failure arises from breakdowns
• 7:35
  in adaptation to complexity. Resilience requires sensing
• 7:39
  emerging effects of decisions and policies.
• 7:42
  In other words, feedback, especially feedback about the
• 7:45
  future. Better feedback helps counter
• 7:47
  drift towards failure. Feedback should reveal
• 7:51
  relationships and patterns, show change in sequence, be future
• 7:55
  oriented, and reflect context. Finally, feedback across all
• 8:01
  levels is critical because production pressure continually
• 8:04
  pushes systems towards reduced margins of safety.
• 8:08
  So what can we make of the findings?
• 8:10
  At a high level, we could say always dig past the first story.
• 8:15
  Design investigations to minimise hindsight bias.
• 8:18
  You can never fully eliminate it, but try to understand local
• 8:22
  rationality, why things made sense to people in the moment.
• 8:26
  Study work at the sharp end. Observe how practitioners
• 8:29
  usually accomplish tasks in context.
• 8:32
  Build solutions that fit real practise rather than based on
• 8:35
  outsider assumptions. Look for systemic
• 8:38
  vulnerabilities, not scapegoats. Study how practise creates
• 8:42
  safety. Identify these adaptive
• 8:45
  behaviours. Work around in everyday
• 8:47
  recoveries that keep the system functioning despite all these
• 8:50
  systemic issues. Treat safety as dynamic.
• 8:54
  Anticipate new vulnerabilities created by chain.
• 8:57
  Apply technology to augment human expertise and not to
• 8:59
  replace judgement. That's in feedback.
• 9:02
  That's integrated, event based, future oriented and context
• 9:06
  sensitive and support detection and recovery capabilities, which
• 9:11
  means design the system to reveal precursors and enable
• 9:14
  early intervention. That's it on Safe.
• 9:16
  As I'm Ben Hutchinson, please help share, rate and review and
• 9:22
  checkout Safety insights.org for more research.
• 9:25
  Finally, feel free to support Safe As by shouting a coffee
• 9:29
  link in the show notes.