Latest / Safe As Podcast / E75: Better indicators for your risk control system

Transcript

E75: Better indicators for your risk control system · Safe As Podcast

• 0:00
  The absence of incidents doesn't prove the effectiveness of
• 0:03
  controls. Controls can degrade even while
• 0:06
  an organisation is handing out awards for two years.
• 0:09
  LTI Free Nancy Levinson describes the challenge.
• 0:13
  There are always warning signs before a major accident, and
• 0:16
  such weak signals are often perceived only as noise.
• 0:20
  So how can we identify and measure those signals before the
• 0:23
  fire and brimstone? Everyone welcome to SAFE as I'm
• 0:26
  Doctor Ben Hutchinson. Now the question I probably get
• 0:30
  the most is what indicators should we be using?
• 0:33
  I've covered dozens of studies on indicators in safety and
• 0:37
  broader organisational indicators.
• 0:39
  Check them out on my site. There's so much there to read.
• 0:42
  Now today I'm going to try and sidestep that whole debate and
• 0:45
  we're going to focus on one specific element of indicators,
• 0:48
  focusing on the effectiveness of our risk control system.
• 0:52
  So there's extensive debate regarding indicator typologies,
• 0:56
  lead verse lag, drive, verse monitoring, proactive verse
• 1:00
  reactive, and more. Andrew Hopkins provides a more
• 1:02
  focused thread. He notes that the primary
• 1:04
  purpose of performance indicators must be to evaluate
• 1:08
  the effectiveness of the specific controls upon which the
• 1:11
  risk control system relies. From this perspective, the
• 1:15
  concern isn't the volume of audits or inspections completed,
• 1:18
  but whether the specific barrier is designed to prevent a serious
• 1:21
  incident or unwanted event. A healthy step one.
• 1:25
  The risk control system. The U KS HSG 254 guide focuses
• 1:30
  on assurance and indicators for major hazards.
• 1:33
  It assumes management systems are already in place and shifts
• 1:36
  emphasis towards checking whether risk control systems are
• 1:40
  operating as intended. GC254's contribution is the
• 1:44
  system of dual assurance. For every risk control system,
• 1:48
  you require both a leading and a lagging indicator.
• 1:51
  The process involves 1. Identifying the risk control
• 1:54
  systems and their required outcomes 2.
• 1:56
  Setting lagging indicators for each system to monitor what can
• 2:00
  go wrong, effectively monitoring the failure of immediate causes
• 2:03
  3. Identifying critical elements of
• 2:05
  each system, the specific components that must function
• 2:09
  for the whole system to work 4. Setting leading indicators to
• 2:13
  monitor the health of these critical elements Step 2
• 2:16
  Defining performance and criticality.
• 2:18
  Moving really into the heart of this logic, the ICM Ms critical
• 2:22
  control management framework targets high consequence
• 2:25
  material unwanted events. The goal is to define the
• 2:28
  critical controls, objectives, performance requirements and how
• 2:32
  that performance is verified in practise.
• 2:34
  Critical control is distinct from a standard or regular
• 2:37
  control. It's crucial to preventing the
• 2:39
  event and its absence or failure would significantly increase
• 2:43
  risk, despite other existing controls.
• 2:45
  To ensure these work, the ICMM defines 3 specific performance
• 2:49
  lenses. 1 Availability is the control in place and accessible
• 2:54
  to reliability and the control performance function to the
• 2:58
  required standard. 3 Survivability will the control
• 3:01
  function during the stresses of the event.
• 3:04
  To bridge the gap between high level risk assessment and field
• 3:07
  verification, both the ICMM and QGN 35 utilise a critical
• 3:12
  control information summary. Here's a work example.
• 3:15
  Methane ignition on a long wall. So consider the material
• 3:18
  unwanted event of a methane ignition.
• 3:19
  Unidentified critical control is the automatic methane power
• 3:22
  trip. The objective is to
• 3:24
  automatically isolate electrical power to the long wall face when
• 3:28
  methane concentration exceed a certain threshold, preventing a
• 3:32
  potential ignition source performance requirements.
• 3:34
  Well, the trip must activate exactly that threshold and
• 3:37
  complete the power isolation in less than two seconds or
• 3:40
  whatever the criteria is. We have support activities and
• 3:43
  these are the background tasks ensuring the control stays
• 3:46
  healthy, such as regular sensor calibration and physical
• 3:49
  protection of the sensor heads, and then the verification.
• 3:52
  The indicators in part weekly in situ gas testing, which is a
• 3:56
  formal workplace monitoring and monthly software trip testing, a
• 4:00
  form of formal auditing, provide the leading indicators of the
• 4:04
  controls health. And there's probably a range of
• 4:06
  other things you could do. And here's a potential shortcut.
• 4:09
  If you've already produced bow ties as part of your risk
• 4:11
  management or critical control framework implementation, then
• 4:15
  you've done most of the heavy lifting already.
• 4:17
  Use your bow ties to guide your risk control system indicators.
• 4:21
  Identify the ways to monitor and verify the barriers or critical
• 4:24
  controls and the support structures and the degradation
• 4:27
  factors that either enable their success or affect their
• 4:31
  integrity. It doesn't need to be a gigantic
• 4:34
  list. Identify the few critical things
• 4:36
  that have to go right first time, every time, or just
• 4:40
  otherwise critical and particularly susceptible to
• 4:43
  reliability or erosion. Step three.
• 4:45
  Condition verse compliance. This example illustrates Hopkins
• 4:49
  point. We must distinguish between
• 4:51
  compliance and condition. Compliance could be like did we
• 4:54
  finish the weekly gas testing and condition what was the
• 4:57
  failure rate during those tests? Indicators must report the
• 5:01
  actual condition, the defect rate or trigger breaches, rather
• 5:05
  than just the activity of checking whether it was done or
• 5:07
  not. For triggers and tarps, a
• 5:10
  performance trigger is a point at which a control is considered
• 5:13
  not maintained. Trigger set the criteria for
• 5:15
  what action must be taken, by whom and when.
• 5:18
  So if that critical methane reading requires immediate
• 5:23
  action, let's say it's 5% requires immediate action, but
• 5:26
  the reading is 4.95%, what action should the operator take?
• 5:30
  This is where triggers and tarps come into it.
• 5:32
  So tarps are your if their mechanism if trigger level X,
• 5:36
  action set Y, escalation Z. This is a real example where
• 5:41
  let's say it was 5% and they measured it at 4.98%.
• 5:44
  Some incidents have shown that people just didn't take action
• 5:47
  because it's technically they only had to escalate the issue
• 5:50
  if it was 5%. Let's say.
• 5:51
  In summary, effective assurance requires moving past broad
• 5:55
  organisational metrics to monitor the specific
• 5:58
  performance, specifications, availability, reliability, and
• 6:02
  survivability of the barriers we rely on to prevent catastrophe.
• 6:06
  You can still set broader measures and indicators, but
• 6:10
  they shouldn't come at the expense of the effectiveness of
• 6:13
  your risk control system and being laser focused on
• 6:16
  understanding and monitoring the effectiveness of your systems.
• 6:20
  I highly recommend you check out these sources that I've linked
• 6:22
  today, including Nancy Levinson's work on indicators.
• 6:25
  Maybe I'll cover that paper alone on another episode.
• 6:29
  The short of it is identify the things that have to happen or
• 6:33
  cannot happen, implement the controls or constraints and then
• 6:36
  design indicators to measure the health of those constraints and
• 6:39
  assumptions and controls. That's targeted.
• 6:41
  It's focused on the things that are most important to your
• 6:44
  organisation and its risk control systems and it doesn't
• 6:47
  involve really lofty, pie in the sky broad organisational
• 6:52
  indicators. It's targeted on the things that
• 6:54
  have to happen or can't happen. That's it on safe as.
• 6:57
  Thanks everyone. Please like and subscribe and
• 7:00
  leave a comment directly on the video.
• 7:02
  It really helps and link in show notes to support the channel.