Latest / This Week In Wellbeing Measurement / Measuring What Matters? Plus: SWEMWBS and Nature-Based Therapy

Transcript

Measuring What Matters? Plus: SWEMWBS and Nature-Based Therapy · This Week In Wellbeing Measurement

• 0:00
  Jenny: If someone says a program helped people, what would you want them to show you?
• 0:04
  Davis: I’d want proof, but I’d also want the human story, because some of the stuff that matters most doesn’t fit neatly in a chart.
• 0:11
  Jenny: Right, but if the story never has to face evidence, then every program becomes a success because someone found one moving quote.
• 0:19
  Davis: And if the evidence only counts what an organization can make measurable, the long relationships and slow changes disappear, which is exactly the problem we’re walking into this week...welcome to This Week In Wellbeing Measurement on paperboy.fm.
• 0:49
  Davis: Big reset this week: 95 qualified papers, meaning papers that cleared the wellbeing-measurement screen, versus 11 last episode, so that's 84 more. Behind them were about 540 authors and 28 countries, up from just 3 countries last time.
• 1:08
  Jenny: And the pool feeding that jump was huge: 704 search hits, meaning the rough pile before screening, versus 50 last time. That's not just a bigger week; it's a different net, so I'd ask whether the query caught more mental health and wellbeing papers, or whether journals released a backlog.
• 1:27
  Davis: The country spread also looks less lopsided than usual, even if the leaders are familiar. The USA and Britain each had 7 papers, China had 6, and India had 4, which makes the measurement question travel across health systems, universities, and workplaces.
• 1:44
  Jenny: Method-wise, this was survey-heavy: 23 survey papers, then 10 qualitative studies and 10 quantitative studies. I like seeing 5 longitudinal studies, which follow people over time, and 5 RCTs, randomized trials where people are assigned to different conditions, because those can test whether a wellbeing measure tracks real change.
• 2:07
  Davis: The author mix was pretty wide too: 130 first-time authors, and here first-time means their first-ever paper in the metadata, not just new to our feed. Then there were 212 emerging authors and 201 experienced ones, so the week wasn't only senior labs setting the agenda.
• 2:25
  Jenny: The theme sweep fits the episode: mental health showed up 7 times, wellbeing 6, and physical activity, adolescents, and burnout each hit 4. So the live question is whether these metrics are measuring what matters, what is easy to count, or what institutions are actually ready to act on.
• 2:47
  Jenny: Alright, let's get into the papers with Measuring What Matters - or What Can Be Measured? Social Impact as Sensemaking, by Heidi Myyryläinen, Kristiina Brusila-Meltovaara, and Ieva Žebrytė, a twenty twenty-six conference paper about social impact reports as stories organizations build, not neutral scorecards they simply fill in.
• 3:08
  Jenny: The plain point is that impact measurement is often meaning-making first and counting second, and in their close read of public impact communications from three hybrid companies, meaning companies that mix a social mission with market activity, the numbers tended to highlight discrete outcomes while the messier relational, process-based, and long-term impacts showed up more as language around the numbers.
• 3:31
  Davis: So how would we know whether an organization measured the real impact, rather than the impact that was easiest to report, especially if the evidence is the public-facing version they chose to publish?
• 3:41
  Jenny: That's exactly the pressure point, because the authors use qualitative analysis, meaning a structured close reading, across three phases: how the companies define social impact, how they argue for those definitions, and which parts become measurable or reportable, so this is strongest as a warning about measurement logic in three cases, not a universal map of every hybrid organization.
• 4:05
  Davis: The practical takeaway is almost brutally simple: if you're designing an impact metric, document what it leaves out as clearly as what it counts, because this lands right in the What Counts As Impact thread where the definition can tilt the result before the dashboard ever opens.
• 4:19
  Davis: That dashboard point is exactly where Nicholas Drake's 2026 paper in Ergo comes in, An Account of Wellbeing for Wellbeing Frameworks, because he asks what governments are even counting when they use wellbeing to track economic and social progress.
• 4:33
  Davis: His claim is pretty bold for a philosophy paper: no existing account is good enough for government wellbeing frameworks, and an account here just means the basic theory of what makes someone's life go well.
• 4:43
  Davis: So he proposes the Reflective Value account, where wellbeing is tied to what people would value on reflection, rather than only pleasure, desire satisfaction, or a fixed list of goods chosen in advance.
• 4:55
  Jenny: If this is philosophical rather than empirical, what would make a government confident enough to build policy around it?
• 5:02
  Davis: Drake's method is conceptual triage: he identifies the conditions a public wellbeing account has to meet, tests existing options against those conditions, and argues Reflective Value fits the policy job better, but the limitation is real because he doesn't test whether it improves actual policy decisions in a city, ministry, or national statistics office.
• 5:22
  Jenny: That keeps it in the What Counts As Impact thread for me, because before an agency picks indicators like life satisfaction, income, health, or social connection, it should say which picture of a good life those indicators are supposed to represent.
• 5:37
  Jenny: That line about picking indicators has a very practical cousin: before you trust a trend, ask whether the ruler stayed still. Hanzlová and Prokeš do that in Longitudinal measurement invariance of the Short Warwick-Edinburgh Mental Well-Being Scale among adolescents in the Czech Republic, with three annual waves from twenty twenty-three to twenty twenty-five.
• 5:58
  Jenny: The clean finding is that the Czech short Warwick-Edinburgh scale behaved consistently across time for thirteen thousand seven hundred thirty-nine secondary school students. In plain terms, the wellbeing questionnaire seemed to measure the same thing in twenty twenty-three, twenty twenty-four, and twenty twenty-five, so a change in scores is less likely to be a measurement glitch.
• 6:21
  Davis: So if the scale is stable, what did they actually see happen to adolescent wellbeing over time?
• 6:26
  Jenny: They saw a small but steady increase from twenty twenty-three to twenty twenty-five, and the pattern looked almost the same whether they used simple average scores or latent means, which are model-based estimates of the underlying wellbeing trait. Method-wise, they used confirmatory factor analysis, meaning they checked whether the answers fit the expected one-factor structure, then multiple-group CFA across years to test configural, metric, scalar, and strict invariance, which is the ladder from same basic shape to same item behavior and error patterns.
• 7:01
  Jenny: The strongest piece is that the scale reached full strict invariance, with changes in CFI, RMSEA, and SRMR staying under the usual cutoffs, so the statistical ruler really did look stable. The limit is also clear: this supports tracking Czech adolescents with this Czech version of the scale, not every age group, country, or translation.
• 7:23
  Davis: That makes this a perfect What Counts As Impact paper, but from the measurement shop floor. If a ministry says teen wellbeing rose a little over three years, this is the kind of check that tells you whether they found a real shift or just watched the survey instrument wobble.
• 7:39
  Paperboy.fm: This is the free version of the podcast. Subscribe at paperboy.fm to access a dozen different paper review podcasts for five dollars a month.