Latest / Safe As Podcast / E42: The blindspots of incident reporting approaches

Transcript

E42: The blindspots of incident reporting approaches · Safe As Podcast

• 0:06
  Are your safety numbers telling you the whole story or just what
• 0:10
  you want to hear? Let's explore how our safety
• 0:13
  reporting may be providing A skewed view of risk.
• 0:17
  Hi everyone, I'm Ben Hutchinson and this is Safe as a podcast
• 0:21
  dedicated to the thrifty analysis of safety, risk and
• 0:25
  performance research. Visit safetyinsights.org for
• 0:28
  more research. Today's paper is from Getart ET
• 0:32
  al 2021 titled Out of Selective reporting Distort Understanding
• 0:37
  of Workplace Injuries in the Journal of Safety explored a new
• 0:42
  method for studying under reporting of injuries in company
• 0:45
  data and for also revealing systematic biases for the
• 0:50
  methods. Company injury data from an
• 0:53
  Australian energy company was matched to workplace injury
• 0:56
  insurance provider data using a bespoke method called 1 to one
• 1:00
  Injury Matching. As noted by the authors, this
• 1:04
  one to one method allows 2 points of view, the company
• 1:08
  incident management system view which reflects measurement
• 1:12
  behaviours from the industry and a view of incidents from a
• 1:15
  government insuring body. The authors summarise some key
• 1:19
  issues with relying on reported injuries for measuring safety
• 1:22
  performance. One, a lack of statistical
• 1:25
  significance of changes in injuries due to the low number
• 1:28
  of injuries, injury rates driven by lower severity and more
• 1:32
  common injuries which may be misleading about high potential
• 1:36
  hazards. The number in nature of reported
• 1:39
  injuries being more sensitive to changes in reporting and the
• 1:42
  classification of the injuries than to real differences in
• 1:45
  safety performance. Indeed, a key goal in this paper
• 1:48
  was to explore whether the LTIFR are more indicative of changes
• 1:53
  in claiming behaviour and claims management and have changes in
• 1:57
  their safety performance. So what did they find?
• 2:01
  Using the one to one injury method, it was found that only
• 2:04
  19% of insurance claims were categorised as recordable
• 2:09
  injuries within the incident management system.
• 2:12
  This means that a vast majority of incidents that were
• 2:15
  considered work related and legitimate by the insurer were
• 2:19
  not formally recorded as such by the company.
• 2:22
  All of these were said to be accepted by the insurer with
• 2:25
  some level of medical treatment or lost time.
• 2:28
  The authors identified two ways that injuries were incorrectly
• 2:31
  classified in that studied company injuries being
• 2:34
  classified as first aid only despite the injuries needing
• 2:38
  more serious medical treatment, or injuries being classified as
• 2:42
  not work related despite the insurer decision that the injury
• 2:46
  was caused by work related factors.
• 2:49
  Essentially, injuries requiring more significant medical care or
• 2:53
  confirmed as work related by the insurer were frequently
• 2:56
  downgraded or dismissed within the company's internal system.
• 3:00
  Also of interest is that only 17% of the high severity
• 3:04
  injuries and 33% of the very high severity injuries were
• 3:08
  considered recordable injuries within the internal reporting
• 3:11
  system. This suggests that even some of
• 3:14
  the more serious incidents often went unrecorded in the company's
• 3:18
  official recordable data. Furthermore, it was demonstrated
• 3:23
  that if using a recordable classification scheme is to help
• 3:26
  identify the most severe injuries in this category, then
• 3:29
  this might not actually be happening in practise, where an
• 3:32
  injury marked as recordable is just as likely to be less than
• 3:36
  medium severity as it is to be above medium severity.
• 3:40
  So this highlights a potential blind spot that the recordable
• 3:44
  label often didn't align with the actual severity of the
• 3:48
  injury, with many of the severe cases being miscategorized.
• 3:52
  Although high and very high severity injuries are slightly
• 3:55
  more likely to be recorded as recordable injuries compared to
• 3:59
  with the less severe injuries, several cases involving
• 4:02
  significant medical costs and lost time were still recorded as
• 4:06
  not work related or as a first aid injury only rather than as
• 4:11
  recordables. Furthermore, the cost of not
• 4:15
  work related injuries seem to exceed total cost of recordable
• 4:19
  injuries for what types of injuries are classified as
• 4:22
  recordable. While overall 19% of injuries in
• 4:26
  this company were classified as recordable, 30% of hand injuries
• 4:30
  but only 8% of back injuries were recordable.
• 4:33
  Expanding that argument, it's said that several other body
• 4:37
  parts are also associated with exertion or chronic injury, but
• 4:42
  have a high likelihood of not being categorised as recordable
• 4:45
  injuries. This reveals a type of skew in
• 4:49
  their reporting and categorization where certain
• 4:52
  common, often chronic injury types are significantly
• 4:56
  underrepresented in the official recordable figures.
• 4:59
  So in discussing the findings, it's said that the definitions
• 5:02
  of work related or first aid and medical treatment within
• 5:06
  industry have significantly departed from the insurer
• 5:09
  definitions, and that these narrower definitions are evident
• 5:13
  in the patterns of injury types classified as recordable within
• 5:16
  organisations. So the company's internal
• 5:19
  criteria for classifying injuries and categorising them
• 5:22
  were found to be far stricter or selective than those used by an
• 5:25
  insurer, leading to fewer incidents meeting their
• 5:28
  recordable threshold. Consistency across research,
• 5:31
  including this study, suggests a more systematic problem exists
• 5:35
  across the industry for distorted injury reporting.
• 5:38
  And this likely isn't related to the company's hiding injuries,
• 5:41
  but more likely related to pressures to reduce the number
• 5:44
  of recordable injuries for tendering or insurance premiums
• 5:48
  and the like. So the findings suggest that
• 5:51
  under reporting isn't necessarily intentional
• 5:54
  concealment, a rather pervasive issue driven by external and
• 5:58
  internal pressures, pressures to maintain low recordable injury
• 6:02
  statistics. They state that their
• 6:04
  interpretation is that the pressure stems from the use of
• 6:07
  recordable injuries as a performance measure both within
• 6:11
  companies and along supply chains.
• 6:14
  So this underscores that the act of using recordable injuries is
• 6:18
  some sort of key performance indicator can create a powerful
• 6:22
  incentive for biassed reporting. Also of interest is that the
• 6:25
  risk assessment ratings generated in the injury
• 6:28
  classifications had little to no recognisable connection to the
• 6:32
  actual injury impacts. There was, in contrast, a close
• 6:36
  link between the risk rating and whether the injury was
• 6:40
  categorised as recordable. This suggests that internal risk
• 6:44
  assessments were influenced by the desired recordable outcome
• 6:48
  rather than by the actual severity of the injury.
• 6:52
  It's said that the selection biases in reporting have the
• 6:54
  potential to create an echo chamber in which some of the
• 6:58
  most serious threats to safety are misunderstood.
• 7:01
  This means that biassed reporting can lead to a skew A
• 7:04
  skewed perception of the actual exposure to risks, potentially
• 7:09
  causing organisations to overlook critical hazards.
• 7:12
  This is compounded by the fact that a consistent, measurable
• 7:16
  definition of actual instant severity doesn't exist.
• 7:19
  Therefore, the authors aptly state that industry has an
• 7:24
  altered perception of risk that is more aligned with recordable
• 7:27
  injury definitions than actual injury outcomes.
• 7:30
  This problem also impacts other ways of learning from incidents
• 7:34
  such that moderate or lower severity injuries may be treated
• 7:37
  as either statistically equivalent to a fatality or
• 7:40
  statistically irrelevant with no in between and pressure in the
• 7:45
  classification and justification of moderate severity injuries as
• 7:48
  none events or not work related. So what do we make of the
• 7:52
  findings? Well, I think nicely the authors
• 7:55
  argue that until additional large scale research across
• 7:58
  multiple companies is conducted, quoting the paper, the default
• 8:03
  assumption for any company should be that recordable
• 8:05
  injuries are not representative of the severity and nature of
• 8:08
  actual injuries. Limitations.
• 8:12
  The main limitation is that it's based on a single energy
• 8:15
  provider in Australia. So generalizability is really
• 8:18
  challenging. But adding to that, we probably
• 8:22
  have reason to believe that the findings will be found
• 8:25
  elsewhere, probably almost everywhere in some form.
• 8:29
  That's it. On Safe as I'm Ben Hutchinson,
• 8:32
  please help share, rate and review and
• 8:35
  checkoutsafetyinsights.org for more research.
• 8:39
  Finally, feel free to support Safe As by shouting a coffee
• 8:42
  link in the show notes.