Latest / Safe As Podcast / Ep 12: Human performance in barrier/critical control systems

Transcript

Ep 12: Human performance in barrier/critical control systems · Safe As Podcast

• 0:07
  The unsung heroes of safety aren't always in a rule book.
• 0:11
  Often it's the continuous performance of people plugging
• 0:15
  the gaps when our systems fall short.
• 0:18
  What are some better ways to consider the roles of people
• 0:21
  within various systems? Grab a coffee, this is a longer
• 0:25
  pod. Good day everyone.
• 0:27
  I'm Ben Hutchinson and This is Safe as a podcast dedicated to
• 0:31
  the thrifty analysis of safety, risk, and performance research.
• 0:35
  Visit safetyinsights.org for more research.
• 0:39
  Today's paper is from McLeod 2017, titled Human Factors in
• 0:44
  Barrier Management, Hard Truths and Challenges in Process Safety
• 0:49
  and Environmental Protection. This paper discussed some hard
• 0:53
  truths in the assurance of human performance.
• 0:56
  It said human performance continues to be relied upon as a
• 0:59
  control. IT organisations may have
• 1:02
  miscalibrated ideas on how human performance can be relied upon
• 1:06
  when needed or how. And normally, human performance
• 1:10
  is the overwhelming reason for successful outcomes even in the
• 1:14
  face of poor systems and resources.
• 1:16
  Therefore, organisations may struggle to ensure that the
• 1:19
  performance of people that they rely on can reasonably be
• 1:23
  expected to happen when and where as needed, and that their
• 1:26
  expected controls are as robust as expected in the face of human
• 1:30
  performance variability. First, a common claim following
• 1:33
  investigations is that if only people had followed the rules,
• 1:37
  then the incident wouldn't have happened.
• 1:40
  But this assertion relies on some implicit assumptions that
• 1:44
  the organisation actually had all of the required procedures
• 1:47
  it needs. The procedures are specific,
• 1:51
  accurate, clear up to date or even valued by people.
• 1:56
  The people have the knowledge, skills and training to know what
• 1:59
  procedures to leverage and when, that they'll accurately
• 2:03
  recognise the situations that call for that procedure and then
• 2:07
  carry them out under the conditions that exist at the
• 2:09
  time. Where procedures are almost
• 2:11
  always written for a perfect world, they operate often in a
• 2:15
  vacuum that doesn't exist in reality.
• 2:17
  It's said that some hard truths about how people see and
• 2:20
  interpret the world and respond are hard because they can be
• 2:24
  difficult and inconvenient to design and manage for.
• 2:28
  But being hard doesn't make them less valid or less important to
• 2:31
  address. So some of the hard truths
• 2:34
  discussed by the author are human.
• 2:36
  Emotion, thought, performers, and attitudes are all highly
• 2:39
  situated. They're influenced by the
• 2:42
  situational context at the time. Design and layout of work
• 2:46
  systems, the equipment, interfaces and the environment
• 2:49
  influence how people sense and respond in the world.
• 2:52
  People optimise their performance, even if it may be
• 2:55
  riskier in hindsight. And people are not necessarily
• 2:58
  rational in a classical sense. For instance, type 1 and type 2
• 3:02
  thinking and pattern matching and heuristics, or how sensitive
• 3:06
  people are to loss aversion. This isn't a bug though, it's
• 3:09
  just a feature of humans. So let's cover some of the
• 3:12
  definitions that we're using in this paper.
• 3:15
  In an early episode I said I'd cover more precise definitions
• 3:18
  of controls, etcetera. Well, here we are.
• 3:21
  So control means measures that are expected to be in place to
• 3:24
  prevent incidents. Controls are comprised of
• 3:27
  barriers and safeguards. Barriers are types of controls
• 3:31
  that are assessed as being sufficiently robust and
• 3:33
  reliable, and they can be relied on as a primary control measure
• 3:37
  against incidents. They can be passive or active
• 3:41
  and be a combination of human elements and technology and
• 3:44
  other engineering elements. In contrast to a barrier,
• 3:49
  safeguards are controls that support and underpin the
• 3:53
  availability and performance of barriers, but cannot meet the
• 3:56
  standards of robustness or reliability to rely on as a full
• 4:00
  barrier. So in other words, barriers are
• 4:03
  at the highest tier, whereas safeguards are often things that
• 4:07
  help support barriers or things that really don't meet the same
• 4:10
  definition. They're not as reliable.
• 4:13
  Another distinction can be made around human barrier elements or
• 4:17
  organisational barriers. This is when the company
• 4:19
  explicitly prescribes how decisions, how to be taken, what
• 4:23
  is to be done by means of rules or instructions and procedures.
• 4:28
  There's also operational barriers.
• 4:30
  These are when there's no specifically prescribed manner
• 4:33
  of deciding or acting where the individual is given quite a bit
• 4:37
  of discretion to take appropriate action.
• 4:39
  This relies more on operating skills and capabilities.
• 4:42
  Right this second there's some people furiously typing an angry
• 4:45
  letter to me about the ICM Ms definition of critical controls.
• 4:50
  I like that approach too. But remember, town is big enough
• 4:54
  for more than one typology, and the approach presented today has
• 4:59
  decades of support from offshore oil and gas.
• 5:02
  But whatever flats you wait next, the order covers some of
• 5:06
  the criteria for robust controls, particularly barriers.
• 5:09
  Remember several factors should be meant for controls to be
• 5:13
  classed as full barriers. These are must be specific to a
• 5:18
  single potentially hazardous event, so specific it must be
• 5:22
  independent of other protection layers, independent if you
• 5:26
  recount it on to do what it was designed to do, dependability.
• 5:30
  It's capable of being audited or observed auditability.
• 5:35
  Now this is a proviso there. These make a lot of sense for
• 5:38
  engineering systems, but are a little bit more challenging when
• 5:42
  we're coming to human performance.
• 5:44
  For instance, assuring true independence with human
• 5:47
  performance is a major challenge.
• 5:49
  Factors like workload and fatigue and distraction can
• 5:52
  defeat multiple controls. Organisational factors can
• 5:56
  influence control performance, like incentivizing certain
• 5:59
  outcomes or contractual arrangements.
• 6:01
  Independence achieved by having double cheques by another person
• 6:05
  may also not be truly independent since it's also
• 6:08
  affected by a range of factors. This has also been called the
• 6:11
  fallacy of social redundancy. So there's a range of these
• 6:15
  factors and are said to often get overlooked when deciding how
• 6:18
  to assure human performance. According to the author,
• 6:22
  judgments about the likely effectiveness of controls that
• 6:25
  rely on human performance often aren't clear about what is
• 6:29
  actually expected or intended from people.
• 6:32
  So we often don't really clarify what we expect of human
• 6:35
  performance, if it's acting as some sort of control, and how we
• 6:39
  even would assess the effectiveness.
• 6:42
  Things like the design of the work environment or equipment
• 6:44
  interfaces. If the control relies on someone
• 6:47
  opening or closing a valve and a clear intention is necessary
• 6:51
  that people will know which valve to operate, how, when and
• 6:54
  why to operate it, and the valve needs to be designed and
• 6:57
  labelled in a way to minimise the chance of people not
• 7:00
  operating it out of sequence. The author draws on guidance
• 7:04
  from other human factors organisations and then it's
• 7:08
  argued that most organisational measures should be treated as
• 7:11
  safeguards rather than as barriers because they just don't
• 7:15
  meet that higher reliability and effectiveness.
• 7:18
  So safeguards would include local warnings and science or
• 7:22
  design and implementation of alarms, human machine
• 7:25
  interfaces, job designer. More organisational safeguards
• 7:29
  are more to ensure that the barriers that are expected to
• 7:32
  function are not degraded or defeated by other factors.
• 7:37
  If you're familiar with bow ties, then things like
• 7:39
  escalation or degradation factors would be more akin to
• 7:43
  safeguards. So, quoting McLeod, safeguards
• 7:46
  cannot and do not need to provide the same level of risk
• 7:50
  reduction as barriers. Nevertheless, safeguards should
• 7:54
  still have clear ownership, be capable of being audited and be
• 7:58
  traceable to some elements of the organisation management
• 8:00
  system. Importantly, our control may be
• 8:04
  a barrier in one situation but may be treated as a safeguard
• 8:08
  elsewhere in the organisation if the company is unable or
• 8:11
  unwilling to invest the resources to ensure that that
• 8:14
  barrier functions to the necessary specifications.
• 8:18
  Again, the author draws on some guidelines from a human factors
• 8:21
  organisation and draws out eight concerns with human
• 8:24
  organisational factors. So, top events in a bow tie.
• 8:29
  The top events are situated too far to the right, where the
• 8:32
  events that are sought to be avoided are too close to the
• 8:35
  consequences. Too many barriers are
• 8:37
  identified, many of which don't meet the accepted criteria for
• 8:40
  being a barrier. In fact, most of the controls
• 8:43
  that rely on human performance meet the definition of
• 8:46
  safeguards rather than as full barriers.
• 8:49
  Human and organisational factors are rarely incorporated into
• 8:52
  barrier models, and ideas of cognition and complexity are
• 8:56
  also rarely incorporated into the performance of barriers.
• 9:00
  Workers imagine verse work has done rarely way into
• 9:02
  considerations. Again, we often have a perfect
• 9:05
  world view around procedures and controls.
• 9:07
  They'll work first time every time.
• 9:10
  Human error is frequently identified as a threat in the
• 9:13
  bow tie, and then barriers are identified to block the error
• 9:16
  from leading to the top event. Remember, this is a myth that we
• 9:19
  should be trying to avoid, not an endorsement that we should be
• 9:22
  seeing human error. In that way.
• 9:24
  The implicit expectations of human performance is rarely made
• 9:27
  explicit and barrier models are often designed implemented to
• 9:33
  the workforce in a manner that doesn't really properly support
• 9:36
  their operational use. We do safety to them rather than
• 9:39
  with them. One interesting point from the
• 9:42
  paper is how human error shouldn't be identified as a
• 9:44
  threat in the bow tie analysis since it creates A misleading
• 9:49
  impression that the risk of human error is being adequately
• 9:52
  managed by barriers. Further, according to the paper,
• 9:56
  it can promote a focus on minimising human performance
• 9:59
  variability over recognising the real barriers and ensuring that
• 10:02
  they are as robust as can be. Focusing on human error in a
• 10:06
  threat line also removes human performance factors out of their
• 10:09
  context. Critically, treating people as a
• 10:13
  threat in the bow tie also misses the opportunity to
• 10:16
  develop a deeper understanding of the ways people provide
• 10:19
  flexibility and adaptability and therefore contribute to the
• 10:22
  system resilience. Therefore, seeing human and
• 10:26
  performance variability as threats reinforces a negative
• 10:29
  view of people as unreliable factors to be managed.
• 10:33
  Instead, more focus should be directed towards understanding
• 10:36
  the performance requirements of the interactions of people and
• 10:39
  technology and what's needed to ensure the robustness of
• 10:42
  barriers and their escalation factors, which I would argue
• 10:46
  heavily involves learning from normal work and work analysis
• 10:50
  methods, etcetera. Next, it's argued that barriers
• 10:53
  should consider the following factors.
• 10:55
  It should be considered the performance of the barrier and
• 10:57
  what it's expected to deliver, and that should be specific to
• 11:00
  the threat in the situation. Who's involved in delivering the
• 11:03
  performance? Who detects the barrier?
• 11:05
  Who decides what needs to be done?
• 11:07
  Who takes action? What info is needed for the
• 11:10
  successful performance of the situation?
• 11:13
  What decisions or judgments are likely to be involved?
• 11:16
  What actions need to be taken and how will the operators know
• 11:19
  whether the actions have been successfully completed?
• 11:22
  Do they ever receive feedback during the task?
• 11:25
  Are there any other technical or non technical guidance to be
• 11:27
  followed? Next, the paper also looks at
• 11:31
  the standards for successful forms of the barrier, arguing
• 11:35
  that this should include the maximum allowable time to detect
• 11:38
  an event to trigger the function, the accuracy of
• 11:41
  interpreting the events, the maximum allowable time to
• 11:44
  initiate a response, the acceptable reliability,
• 11:48
  tolerance limits for acceptable performance, and some other
• 11:50
  factors. So we're right at the end.
• 11:53
  In all, the paper argues that people are nearly always a
• 11:57
  positive element in complex sociotechnical systems.
• 12:00
  The objective should therefore be to strive to make people as
• 12:04
  reliable as possible by setting them up to succeed.
• 12:08
  Make it as easy as possible to do the right thing and as hard
• 12:11
  as possible to do the unsafe, hazardous thing.
• 12:14
  Organisations operating as complex socio technical systems
• 12:18
  should seek to ensure they have in place the necessary systems
• 12:21
  and support structures and should design and operate their
• 12:24
  activities in ways that allow people to be as productive and
• 12:28
  adaptable as they can be. That's it on Safe As.
• 12:32
  I'm Ben Hutchinson, please help share, rate and review and
• 12:36
  checkout Safety insights.org for more research.
• 12:39
  Finally, feel free to support Safe As by sharing a coffee link
• 12:44
  in the shy nights.