Latest / Safe As Podcast / Ep 7: How investigations blind us to control effectiveness

Transcript

Ep 7: How investigations blind us to control effectiveness · Safe As Podcast

• 0:07
  Have you ever looked at a recurring workplace incident and
• 0:10
  thought, didn't we investigate this already?
• 0:13
  It's a frustrating sense of deja vu, isn't it?
• 0:16
  We pour effort into understanding what went wrong,
• 0:19
  yet the same kinds of problems keep resurfacing.
• 0:22
  This week we're uncovering how if our tools and mental models
• 0:26
  are only designed to find specific kinds of problems, like
• 0:30
  human error, then that's all we're ever going to find.
• 0:34
  This has been called what you look for is what you find.
• 0:38
  Therefore, what if our investigations are defined less
• 0:41
  by what they find or socially construct is causal, but more by
• 0:46
  what they leave in the dark? Good day, everyone.
• 0:50
  I'm Ben Hutchinson. This is Safe as a podcast
• 0:52
  dedicated to the thrifty analysis of safety, risk and
• 0:56
  performance research. Visit safetyinsights.org for
• 0:59
  more research. So today's study explored how
• 1:03
  incident investigations approach and failed to learn from
• 1:06
  incidents, including the types of controls implemented after
• 1:10
  investigations. It's titled Practitioners
• 1:13
  perspectives on incident investigations, Production and
• 1:16
  hassle 2017 in safety science. So what were their methods?
• 1:21
  222 survey respondents were included in this data across
• 1:25
  mining, construction, transport, postal warehousing and other
• 1:30
  industries. They also reviewed incident
• 1:33
  investigation manuals and procedures.
• 1:35
  Oh hey, future Ben here speaking after this episode has been
• 1:39
  recorded. Now, despite knowing better,
• 1:43
  I've used the terms barrier and control synonymously in this
• 1:47
  episode, Even though, to be precise, barriers and safeguards
• 1:51
  are both types of controls. So controls can be considered a
• 1:55
  higher order category. I'll cover more precise
• 1:58
  definitions of controls, barriers and safeguards in a
• 2:01
  future pod, but using this paper's definition, a control is
• 2:06
  an act, object, or system intended to arrest or mitigate
• 2:11
  an unwanted event, and that a control must be specifiable,
• 2:14
  measurable, and auditable. So what did they find regarding
• 2:19
  the identification of controls present at the time of incident?
• 2:23
  Just over 50%, around 57% of all industry survey respondents
• 2:27
  indicated that the extent of controls being present at the
• 2:30
  time of the incident was considered in their
• 2:32
  investigation process. So what this means is that
• 2:36
  around 57% of the respondents typically look at the barriers
• 2:41
  or controls during the investigation.
• 2:43
  So in mining, it was around 50% of the investigations looked at
• 2:46
  the barriers, whereas in construction up to 73% of
• 2:51
  investigations always considered whether the controls were in
• 2:54
  place. In transport, postal,
• 2:57
  warehousing, it was around 64 1/2%.
• 3:00
  This suggests that most organisations are at least
• 3:03
  asking the basic question, where are safeguards in place When
• 3:07
  this incident happened for assessment of the effectiveness
• 3:10
  of those controls, around 57% of the respondents said the control
• 3:14
  effectiveness wasn't always considered as part of the
• 3:17
  investigation process. Similarly, 60% of the mining and
• 3:21
  close to 60% of construction indicated the control
• 3:24
  effectiveness wasn't considered in their investigation process.
• 3:28
  In contrast, 58% of transport indicated that control
• 3:32
  effectiveness was always considered.
• 3:34
  So while many check if controls were present, a majority are
• 3:39
  failing to ask the crucial follow up question.
• 3:41
  Did that control actually work as intended?
• 3:44
  Next they looked at absent controls that if they had been
• 3:47
  present they might have prevented the incident.
• 3:50
  Data across the whole sample indicated that absent controls
• 3:54
  wasn't always considered part of the current analysis.
• 3:56
  It was just over 50%. It was around 57% in mining and
• 4:01
  other groups around 59%. So the consideration of control
• 4:05
  has been absent at the time of incident wasn't always
• 4:07
  considered in construction. It was around 55% indicate that
• 4:11
  their investigation always identified absent controls that
• 4:15
  if present may have been effective to help prevent or
• 4:18
  mitigate the event. This highlights a significant
• 4:21
  missed opportunity for learning. Many investigations aren't
• 4:25
  asking what else could we have done that wasn't there at the
• 4:28
  time. Regarding opportunity to learn,
• 4:32
  only around 61% of the investigations focus on
• 4:34
  controls, less than 42% focus on evaluating both present and
• 4:39
  absolute controls, and just 34% of incidents assess or make
• 4:44
  recommendations about enhancing the effectiveness of controls.
• 4:48
  This means that while incidents happen, a surprisingly low
• 4:51
  percentage of investigations are actually getting to the core of
• 4:55
  control effectiveness, which is vital for real learning.
• 4:58
  Next, they evaluated the incident manuals.
• 5:01
  One finding was that the most frequently used tools by
• 5:04
  investigators don't routinely require arresting and mitigation
• 5:08
  controls to be identified as part of the incident
• 5:10
  investigation process. In practical terms, the very
• 5:14
  forms and checklist investigations use often don't
• 5:17
  prompt them to think about how to stop or lessen the impact of
• 5:20
  an incident. Another interesting finding was
• 5:23
  that only 14 of the 24 investigation documents that the
• 5:27
  researchers evaluated actually required a description of the
• 5:30
  incident or required any human factors considerations or the
• 5:34
  sequence of events be part of the investigation.
• 5:37
  I think this absence of human facts integration, at least in
• 5:40
  the documentation, can be quite telling.
• 5:42
  This suggests that foundational elements for understanding the
• 5:46
  how and whys of incident and performance, routine and
• 5:49
  unroutine performance, and particularly the human element
• 5:52
  are often missing from the procedural blueprints of
• 5:55
  investigations. They've found very little
• 5:58
  reference to the role of mitigation controls, although
• 6:01
  prevention controls were more readily recognised.
• 6:04
  So investigators were comfortable looking for
• 6:06
  prevention controls but very rarely the mitigation controls.
• 6:11
  Mitigation is quite a big area of research within the
• 6:14
  resilience engineering world. It's also part of failing
• 6:17
  safely, that when disaster occurs, how well are we set up
• 6:22
  to succeed. They also know that the
• 6:25
  initiating events or the initiating hazards was neglected
• 6:28
  by over 60% of the organisation. Incident investigation
• 6:31
  documentation. This means organisations are
• 6:34
  frequently documenting what happened and the contributing
• 6:37
  factors, although very narrowly defined, but are often
• 6:41
  overlooking how it started or how to contain it after it
• 6:45
  begins. So what are some limitations?
• 6:49
  For one, it's a survey, so the death of findings are somewhat
• 6:52
  limited. It's also unclear how
• 6:54
  representative the findings are. They note that while the mining
• 6:58
  sample was really large, the other sectors weren't safe.
• 7:02
  So what do we make of the findings?
• 7:04
  Well, as a study found, some existing investigation
• 7:07
  approaches and tools weren't that good at directing people to
• 7:10
  understand the context of incidents, or likely by
• 7:14
  extension, understanding hazardous work.
• 7:17
  Normal daily hazardous work. A focus on barrier performance
• 7:22
  also wasn't a big focus in investigations.
• 7:25
  This is problematic if your worldview and your approaches
• 7:28
  are based around barrier performance, for instance in the
• 7:31
  critical control management frameworks or similar
• 7:33
  frameworks. As a way to improve the
• 7:36
  evaluation of real daily work and the effectiveness of our
• 7:40
  barriers, why not try these sorts of questions?
• 7:44
  What barriers were expected to be in place and what barriers
• 7:48
  were actually in place? Did the barriers perform as
• 7:51
  expected? If not, how with the barriers
• 7:55
  and their performance specifications even known by
• 7:58
  stakeholders? What happened last time this
• 8:00
  task was done? With the barriers used well?
• 8:04
  How is this task normally done? Or what does good look like?
• 8:08
  What's good recognised industry practise?
• 8:11
  Or what about some questions from hop or safety tool
• 8:13
  approaches? What makes it difficult to apply
• 8:16
  these controls? When was the last time you had
• 8:19
  to work around the barriers? What's critical for ensuring
• 8:23
  safe and reliable work of this kind?
• 8:26
  If I gave you $1000 to improve the work, how would you spend
• 8:30
  it? That's it on Safe As I'm Ben
• 8:33
  Hutchinson. If you've found this useful then
• 8:36
  please share, write and review and check out safetyinsights.org
• 8:42
  for more research.