Latest / Safe As Podcast / E20: SMS certification and its impacts on safety performance

Transcript

E20: SMS certification and its impacts on safety performance · Safe As Podcast

• 0:07
  When Chase certifications implement systems, but what
• 0:11
  impact are we really having on operational performance?
• 0:14
  Is achieving certification like ISO 18,000 and ONE or 45,000 and
• 0:19
  One really about improving the governance, assurance or
• 0:23
  performance of systems? Or is it more about window
• 0:27
  dressing and legitimacy being seen to be doing the right
• 0:30
  thing? Let's find out.
• 0:33
  Good everyone. I'm Ben Hutchinson and This is
• 0:35
  Safe as a podcast dedicated to the Thrifty analysis for Safety,
• 0:39
  Risk and performance research. Visit safetyinsights.org for
• 0:43
  more research. Today's study is from Dewborg ET
• 0:48
  al 2024 titled Effectiveness of OHSAS 18,000 and One in Reducing
• 0:55
  Accidents at Work, a follow up study of 13,102 workplaces from
• 1:01
  Safety Science. The study evaluated the
• 1:04
  effectiveness of adopting certified health and safety
• 1:07
  systems, particularly 18,000 and one among 805 adopters versus
• 1:13
  12,000 matched controls or non adopters using a longitudinal
• 1:17
  design over 2010 to 2018. They used government reportable
• 1:22
  incidents. Providing some background.
• 1:25
  Some research has highlighted a positive difference for
• 1:28
  organisations who adopt certification, noting adopters
• 1:33
  are more likely to enforce health and safety rules and
• 1:35
  procedures than non adopters, and the level of safety training
• 1:39
  was also high among the adopters.
• 1:42
  Another study found that the adopters fared better than non
• 1:45
  adopters in terms of the relative number of safety
• 1:47
  violations identified by the applicable health and safety
• 1:51
  regulator. Other work suggested that
• 1:53
  adopters seem to have a higher overall level of health and
• 1:56
  safety efforts. The cross sectional research
• 1:59
  designs really rule out any causal direction. 1 Spanish
• 2:03
  found that positive effects of adopting a certified system
• 2:07
  where accident rates seemed to decrease and another Spanish
• 2:10
  study found a weak relationship between certification and
• 2:14
  accident rates. 1 driver of seeking certification might be
• 2:19
  legitimation. For instance, implementation
• 2:23
  might be more focused on increasing legitimacy among
• 2:26
  important stakeholders and an improved public image as the
• 2:29
  primary driver for being certified.
• 2:32
  And in some certified organisations, this underlying
• 2:36
  driver for legitimacy being seen to be doing the right thing
• 2:40
  might actually counter or limit the effectiveness of these
• 2:43
  certified systems. In fact, other research has
• 2:47
  called this legitimacy logical motive.
• 2:49
  Window dressing that is being seen to care more about safety
• 2:54
  by becoming certified. Other work might actually call
• 2:57
  this symbolic safety or demonstrated safety.
• 3:00
  So what was found in this study? Key findings were that the risk
• 3:04
  of severe accidents at work was 14% lower among the adopters of
• 3:08
  certified systems compared to non adopters two years before
• 3:12
  the data certification. But no further decrease in the
• 3:15
  accident ratio could be established three years after
• 3:18
  the follow up of being certified.
• 3:21
  So what this means is that companies with an above average
• 3:24
  pre certification safety performance are the ones who
• 3:28
  largely become certified. Say there is some sort of pre
• 3:32
  certification selection effect. In contrast, they found no
• 3:37
  decrease in the accident rate for the reportable accidents
• 3:40
  three years after being certified.
• 3:42
  They say that these findings support some other research that
• 3:45
  for instance, US certified companies to 18,000 and one
• 3:49
  tended to be safer than comparable workplaces before
• 3:52
  their certification. So the companies that are
• 3:55
  already safer by that very narrow definition around
• 3:58
  incidents are the ones who tended to get certified rather
• 4:02
  than the ones that had worse reportable incident performance.
• 4:05
  They also know that this is in line with other research that
• 4:08
  adopters of certified systems perform better than non
• 4:11
  adopters. But regarding process related
• 4:14
  health and safety efforts and content related efforts, the
• 4:18
  results couldn't support these hypothesis.
• 4:20
  So what that suggests is even though there might not be a big
• 4:23
  difference in portable incidents, maybe certified
• 4:26
  companies tend to perform better with other sorts of activities,
• 4:29
  safety efforts and interventions and whatnot.
• 4:32
  They couldn't support that. They also observed some evidence
• 4:35
  that there was a tendency that the introduction of a certified
• 4:38
  system maybe does improve companies procedures relating to
• 4:41
  the reporting of less severe accidents, but this tendency
• 4:44
  wasn't statistically significant.
• 4:46
  In this research they also discussed practises in large
• 4:49
  companies who had pre certification audits with a
• 4:52
  certification agency. Essentially this provides
• 4:55
  feedback on what needs to be changed prior to certification.
• 4:58
  Essentially, it's seen as a type of gaming where they're pretty
• 5:02
  much setting up systems to be better at working through the
• 5:06
  certification audit rather than actually being more functional.
• 5:10
  Including, they highlight that this longitudinal study couldn't
• 5:13
  confirm that workplaces adopting a certified system report less
• 5:17
  severe accidents and work within a three year follow up period.
• 5:21
  And already before the date of certification, the adopting
• 5:24
  workplaces had a lower rate of severe accidents at work
• 5:27
  compared to non adopters. So this indicates that adopters
• 5:30
  have a higher safety performance by reportable severe incidents
• 5:34
  than the average workplace. In any case, adopters didn't
• 5:38
  reduce the risk of severe accidents at work during that
• 5:41
  follow up period. These results quite in the
• 5:44
  paper, call into question the effects of certified systems as
• 5:48
  an instrument for improving safety in the workplace.
• 5:51
  So what do we make of the findings?
• 5:53
  This one's another challenging 1.
• 5:54
  I think recognising that certification isn't just about
• 5:58
  coordinating or describing systems for functional goals.
• 6:01
  It's also about a bunch of other goals like legitimation, window
• 6:05
  dressing, tendering, you know, being seen to be doing the right
• 6:08
  thing. So recognising that we might not
• 6:10
  always be solving the most important issues when managing
• 6:14
  these certified systems via our paperwork and systems and
• 6:17
  processes. What I mean is certification can
• 6:21
  potentially have some unintended byproducts, shifting more
• 6:25
  efforts towards managing paperwork and documentation as
• 6:28
  purely part of certification and away from functional goals.
• 6:32
  And I think, drawing on Greg Smith's words, we need to
• 6:35
  connect process with purpose. What are we trying to achieve
• 6:38
  with the elements of the certified system and are they
• 6:41
  actually designed, configured, resourced and understood how
• 6:45
  these systems and elements can actually achieve those goals?
• 6:49
  And importantly, we have to be brave to shed the processes that
• 6:54
  and helping us achieve our goals.
• 6:56
  And I think there's a clear cautionary tale here, not to
• 6:59
  mistake certification with being a safer approach or a better,
• 7:03
  more functional system for limitations.
• 7:06
  Of course, there was several, the old chestnut that I bang on
• 7:09
  about every time. But incident data, whatever,
• 7:12
  it's statistically rare, randomly distributed, difficult
• 7:15
  to make sense of, rah rah rah, but the findings are still
• 7:18
  interesting. That's it on Safe.
• 7:20
  As I'm Ben Hutchinson, please help share, rate, and review and
• 7:26
  checkout Safety insights.org for more research.
• 7:29
  Finally, feel free to support Safe As by shouting a coffee
• 7:33
  link in the show notes.