Latest / Safe As Podcast / E78: The most effective safety tactics according to science

Transcript

E78: The most effective safety tactics according to science · Safe As Podcast

• 0:00
  If you want fewer accidents at work, the evidence seems to
• 0:03
  point in a pretty clear, but not too surprising direction.
• 0:06
  Change the work before you try to change the worker.
• 0:09
  This is a central message from today's systematic review of
• 0:12
  workplace safety interventions, a broad review that analysed
• 0:16
  over 100 studies from 120 safety interventions across randomised
• 0:20
  trials, controlled before and after studies and serial measure
• 0:24
  designs. Their underpinning finding was
• 0:26
  that interventions aimed at the group or the organisational
• 0:30
  level generally outperformed interventions aimed mainly at
• 0:33
  the individuals, and the most consistently strong changes
• 0:37
  resulted from engineering controls, especially when they
• 0:40
  worked without relying on workers constantly choosing to
• 0:43
  use them. The review also found that
• 0:45
  multifaceted interventions can work well, but not just because
• 0:48
  they were multifaceted. They tended to be more effective
• 0:51
  when they included stronger organisational or structural
• 0:54
  components, especially engineering controls.
• 0:57
  This pattern supports the hierarchy of control.
• 0:59
  The more effective approaches remove or reduce the hazard at
• 1:03
  its source instead of relying on people to compensate for it
• 1:06
  through vigilance, training, or effort alone.
• 1:09
  Good everyone. Welcome to Safe as I'm Doctor
• 1:11
  Ben Hutchinson. So let's unpack the
• 1:13
  effectiveness of accident prevention interventions at each
• 1:16
  level. Note I can only scratch the
• 1:18
  surface of this article. It's like 170 pages without even
• 1:21
  the references. The full reference is Durebog ET
• 1:25
  al 2022, titled Safety Interventions for the Prevention
• 1:28
  of Accidents at Work, a systematic review published in
• 1:32
  Campbell's Systematic Reviews. So we'll start with safety
• 1:34
  interventions directed at the individual level.
• 1:36
  When the review looked at interventions and mainly at
• 1:38
  individual workers, the findings were pretty unimpressive.
• 1:42
  Safety campaigns, teaching and education only approaches
• 1:46
  generally showed little or no clear effect on injuries when
• 1:50
  they were used on their own. They found that interventions
• 1:53
  aimed at attitudinal and belief modification had no or only
• 1:58
  modest effects on reducing injuries or improving behaviour.
• 2:01
  They say it's uncertain whether attitude or belief modifications
• 2:05
  as a safety intervention has any effects on work injuries, at
• 2:09
  least on their own. The same broad pattern held for
• 2:12
  many behaviour focused approaches such as general
• 2:15
  coaching, feedback or safety training.
• 2:19
  The authors are careful here. They say training on its own
• 2:22
  should not be expected to meaningfully reduce injury rates
• 2:25
  without stronger supporting measures around it, nor without
• 2:28
  being connected to specific hazards or specific gaps in
• 2:32
  knowledge or competencies. And that last point about being
• 2:35
  connected to specific hazards is worth dwelling on for a moment.
• 2:38
  The review found that roughly half of all studies evaluated
• 2:42
  interventions targeting all types of injuries, which
• 2:45
  implicitly assumes a common cause for all workplace
• 2:49
  accidents. A training programme designed to
• 2:51
  reduce needle stick injuries, for instance, can't reasonably
• 2:54
  be expected to reduce falls. When you lump all injury tyes
• 2:58
  together as your outcome, you're going to obscure real effects,
• 3:01
  generate null results. The stronger interventions in
• 3:04
  this review tended to be granted in firm epidemiological evidence
• 3:08
  of what actually causes the specific injury that were have
• 3:11
  been targeted. Weaker interventions often
• 3:13
  lacked that theoretical and causal foundation altogether.
• 3:17
  A related point the authors make, and I think this is a
• 3:19
  useful framing for practitioners, is that when an
• 3:22
  intervention appears to fail, there are really two different
• 3:25
  explanations, maybe more. Either the theory was wrong,
• 3:28
  meaning the mechanism couldn't work, or the delivery and
• 3:30
  adoption were poor, meaning that the mechanism might have been
• 3:34
  sound but it was never properly implemented.
• 3:36
  The review notes that most studies didn't collect the
• 3:38
  process evaluation data needed to distinguish between these two
• 3:42
  types of failure. The evidence specifically around
• 3:44
  behavioural safety interventions were said to point in different
• 3:48
  directions. However, specifically for
• 3:50
  individual feedback or coaching at medium term follow up, the
• 3:55
  evidence is moderate for little or no effect.
• 3:58
  So they're moderately confident there is little or no effect
• 4:00
  based on the evidence. There was an important
• 4:02
  exception. More interactive counselling
• 4:04
  approaches, especially group discussions, showed medium term
• 4:08
  promise. So the review does leave room
• 4:10
  for the idea that worker engagement can matter.
• 4:13
  And it does matter. But the stronger results came
• 4:15
  from approaches that involve discussion and participation,
• 4:18
  not just one way messaging or generic instruction.
• 4:21
  The review also found that more physiological geared approaches
• 4:24
  like physical training, you know, intended to make workers
• 4:26
  more resilient to injury were poorly substantiated.
• 4:29
  Overall, the evidence here was limited and tended towards no
• 4:33
  clear effect on reducing injuries at work.
• 4:35
  Next, the safety interventions directed at the group or
• 4:37
  organisational level. This is where the findings
• 4:39
  became more interesting. Interventions aimed at climate
• 4:42
  norms and leadership showed modest effects.
• 4:46
  Leadership based efforts to improve safety communication or
• 4:49
  strengthen safety climate did show some positive signals, but
• 4:52
  the evidence wasn't really strong or as consistent.
• 4:56
  As for the structural approaches, Unfortunately there
• 4:58
  were few high quality studies on things like leadership in
• 5:00
  climate and culture that met their inclusion criteria.
• 5:03
  They state that one there is no clear demonstration of the
• 5:06
  relationship between the intermediary measures like
• 5:09
  climate scores, safety index scores or behaviour and true
• 5:12
  limited evidence that leadership based safety interventions have
• 5:16
  little to moderate effect. That first point is worth on
• 5:18
  backing. Even in the higher quality
• 5:20
  climate studies, the review couldn't establish a clear link
• 5:23
  between improving climate scores or safety behaviours and
• 5:26
  actually reducing injuries. Most climate studies used proxy
• 5:30
  outcomes rather than injury data, and the one high quality
• 5:33
  study that did measure injury outcomes only looked at minor
• 5:36
  non serious injuries. So the mechanism the climate
• 5:39
  literature often assumes, as in better climate leads to safer
• 5:42
  behaviour which leads to fewer injuries, isn't clearly
• 5:45
  supported by the evidence reviewed here.
• 5:47
  The strongest findings in the whole review set under
• 5:50
  structural modification engineering controls were the
• 5:53
  most effective category. Overall, the review repeatedly
• 5:55
  found larger effects where hazards were designed out
• 5:59
  separate from workers, or controlled in ways that didn't
• 6:02
  depend heavily on worker decisions.
• 6:04
  Simply put, a safer machine, safer device or a safer system
• 6:08
  of work tends to be trying to persuade people to be more
• 6:11
  careful, or perhaps simply targeting behaviours in
• 6:15
  isolation. That is, engineering controls
• 6:17
  provided higher reductions in injuries and these effects were
• 6:20
  stronger where the intervention could demonstrate the effects
• 6:23
  independent of human behaviour. Overall, they say engineering
• 6:26
  controls provided strong evidence of strong effects at
• 6:29
  short term follow up, but limited evidence to show strong
• 6:33
  effects at the medium and long term.
• 6:35
  This is a limitation present throughout the evidence base.
• 6:38
  Now, within engineering controls, the review makes an
• 6:40
  important distinction between what we might call passive and
• 6:43
  active controls. Passive controls are those where
• 6:45
  the hazard is physically removed and no worker decision is
• 6:48
  required. The classic example being like
• 6:50
  safety buns. Active controls are those where
• 6:52
  safety equipment is available but workers still have to choose
• 6:55
  to use it. Like maybe patients, ceiling
• 6:57
  lifts and hospitals being a good example.
• 6:59
  The review found that passive controls had substantially
• 7:02
  stronger effects. In studies where the old hazard
• 7:04
  was physically removed, the pooled effect was strong and
• 7:07
  consistent. Where workers retained the
• 7:09
  decision to adopt the new equipment, effects were weaker
• 7:12
  and less reliable. So it's not just engineering
• 7:14
  controls broadly, it's engineering controls that
• 7:17
  eliminate the decision to use them at the source.
• 7:19
  And connected to that, the authors make what I think is
• 7:22
  really an important practical statement.
• 7:23
  Training workers to deal with dangerous tasks, including
• 7:26
  things like PPE use, should be a lush resort only when more
• 7:30
  effective measures aren't feasible.
• 7:32
  And critically, such efforts enacted alone are unlikely to
• 7:35
  effectively prevent injuries. The review also found that
• 7:38
  regulation enforcement can help prevent injuries, but their
• 7:41
  effects were generally smaller than those of engineering
• 7:44
  controls. Enforcement appeared to work
• 7:45
  more consistently than legislation alone, but again
• 7:48
  with modest effect sizes. Even so, the authors make an
• 7:52
  important practical point. Because reputation enforcement
• 7:55
  can reach very large populations, even modest effects
• 7:58
  can matter a lot when you aggregate it.
• 8:00
  Interestingly, the effects of enforcement seem more consistent
• 8:03
  than legislation, and further on legislation, the review found
• 8:06
  evidence that in some cases, legislation simply doesn't seem
• 8:10
  to work, particularly when the legislation requirements were
• 8:13
  passed with little scientific evidence base.
• 8:15
  One study provides a telling example.
• 8:17
  Legislation targeting tractor accidents among farm youth was
• 8:21
  passed after negotiations had essentially gutted the key
• 8:24
  provisions, and it mandated an educational approach that had
• 8:27
  already been shown to be ineffective for farm safety.
• 8:30
  No effect was found, and the authors say that if in effect
• 8:32
  had been found, it would have been hard to attribute it to the
• 8:35
  legislation anyway. In contrast, when legislation
• 8:38
  mandated an engineering solution, as in like the
• 8:40
  needlestick legislation, it tended to work.
• 8:42
  So the quality of the underlying evidence base for the regulation
• 8:46
  matters enormously. Probably like anything for
• 8:48
  economic incentives, very limited research suggests some
• 8:51
  medium and long term follow effects.
• 8:53
  It's worth noting that this finding is actually inconsistent
• 8:56
  with some earlier systematic reviews which found more
• 8:59
  positive results for financial incentives from insurers.
• 9:02
  So this is 1 area where the evidence picture is still a
• 9:04
  little bit unsettled. In all, they argue for moderate
• 9:07
  evidence for little or no effects of enforcement at medium
• 9:11
  term follow up and limited evidence of a small effect at
• 9:14
  short term follow up. The review also found some
• 9:16
  limited evidence for administrative controls.
• 9:18
  Things like policies and modified safety procedures and
• 9:21
  internal monitoring systems. Next on to multifaceted safety
• 9:24
  interventions. The review examine these
• 9:26
  interventions that combine multiple elements.
• 9:28
  Here, the key lesson was that simply adding more components
• 9:31
  doesn't automatically make an intervention more effective.
• 9:34
  Multifaceted interventions focused only at the individual
• 9:36
  level tended to show little effect overall, but multifaceted
• 9:40
  interventions at the group or organisational level across
• 9:43
  levels perform better, especially when they include
• 9:45
  engineering controls or other strong structural elements.
• 9:48
  Now further on these multifaceted interventions,
• 9:50
  including those at the individual level, they conclude
• 9:53
  that combining attitudinal and behavioural components seem to
• 9:56
  have no or a low effect on reported injuries with a
• 9:59
  moderate level of evidence. And for multifaceted
• 10:01
  interventions, again using interventions across different
• 10:04
  levels, then there seems to be some consistent findings.
• 10:07
  These interventions use multiple levels are more likely to be
• 10:11
  effective when they include at least an engineering control.
• 10:14
  Also they say that there's moderate evidence that these
• 10:16
  multifaceted interventions across levels have a moderate
• 10:19
  effect at short term follow up, and for the serial measures
• 10:23
  studies, strong evidence of a moderate effect at medium term
• 10:27
  follow up. So I think a fair interpretation
• 10:29
  is combining weak individual tactics doesn't reliably produce
• 10:33
  strong safety outcomes, but combining strong organisational
• 10:36
  and structural measures can. This contradicts big flashy
• 10:40
  overhauls that throw lots of stuff at a problem, like a big
• 10:43
  safety campaign hoping that something will stick.
• 10:45
  Now a contrast with the previous literature review comes from a
• 10:48
  slightly older study, 2013, from Christian and colleagues.
• 10:52
  They looked at the antecedents of safe behaviour and injury
• 10:55
  outcomes and if there's a link between them.
• 10:57
  I raised this because lots of people are interested in
• 10:59
  behavioural interventions. They found that safety knowledge
• 11:01
  and safety motivation with the strongest correlates of safety
• 11:05
  performance behaviours, while safety climate was also highly
• 11:08
  important. When the outcome shifted from
• 11:10
  behaviour to actual accidents and injuries, group safety
• 11:14
  climate had the strongest association.
• 11:16
  Their meta analysis supported the broader idea that more
• 11:20
  digital person and situational factors work through knowledge,
• 11:24
  motivation and safety behaviour. So this is debated in your
• 11:27
  research. So this is one way to interpret
• 11:29
  some of these behaviour and individual level approaches in
• 11:32
  Christians study. They do support the idea that
• 11:35
  work and knowledge and motivation matter for safe
• 11:37
  behaviour in some sense, but they also show that once you
• 11:40
  care about real injury outcomes, the shared context, especially
• 11:44
  the group safety climate, then the climate becomes more
• 11:47
  important, more central. So even in this older, more
• 11:50
  behaviour oriented framing, the narrative still seems to be a
• 11:53
  focus on the context rather than simply the worker and behaviour.
• 11:56
  That is, behaviour does seem to matter, but the surrounding
• 12:00
  system and climate shapes whether risky behaviour is
• 12:03
  likely and whether injuries are actually prevented.
• 12:06
  So what can we make of the findings?
• 12:07
  You know, the current review gives a fairly blunt answer to a
• 12:11
  practical question. What works best to prevent
• 12:13
  accidents at work? Not in the large but rather
• 12:16
  mythologically weak body of evidence, particularly for
• 12:18
  medium and long term effectiveness.
• 12:20
  An empirically supported response is not about more
• 12:23
  campaigns and messaging or more worker focused behaviour.
• 12:26
  The stronger evidence supports creating safer environments,
• 12:30
  safer equipment, safer tools and safer systems of work.
• 12:33
  Worker focused measures will still have a place, but mainly
• 12:37
  as support, not as the main line of defence.
• 12:39
  So if you're deciding where to put effort and money, the review
• 12:42
  points you towards upstream controls first.
• 12:45
  Redesign the hazard, change the equipment, change the system,
• 12:49
  strengthen organisational controls and use trainings to
• 12:52
  support those changes and gaps, not to substitute for them.
• 12:55
  Really. Pretty much the hierarchy of
• 12:57
  control in a nutshell.