Latest / This Week In Wellbeing Measurement / Who Defines Success? Plus: REvIEW and Zimbabwe’s ZCS scale

Transcript

Who Defines Success? Plus: REvIEW and Zimbabwe’s ZCS scale · This Week In Wellbeing Measurement

• 0:00
  Jenny: Have you ever filled out a form and thought, this is not what actually matters here?
• 0:05
  Davis: Constantly, though I get why the form exists, because somebody has to count something before they can fund it.
• 0:11
  Jenny: I love a neat checkbox until it starts replacing reality, like when a climate project can report the things it delivered while the people living there are asking whether water, income, services, and control actually improved.
• 0:25
  Davis: Right, the count is not the enemy, but the count has to follow the people, or it turns into a receipt instead of a measure.
• 0:33
  Jenny: So if the metric can prove a project happened but miss whether a community got what it needed...welcome to This Week In Wellbeing Measurement on paperboy.fm.
• 0:57
  Davis: This week the funnel got loud: 1,090 query hits, 92 qualified papers, about 500 unique authors, and 49 countries. That matches the through-line: measurement isn't converging on one perfect score; it's widening who gets counted and where.
• 1:17
  Jenny: And the qualified set jumped from 57 to 92, up 61%. I don't want to call that a field-wide boom yet, because the mix is survey-heavy at 21 papers and qualitative at 19, so part of the rise may be broader ways of asking what wellbeing means.
• 1:36
  Davis: The search pool rose too, from 657 hits to 1,090, up about 66%. The topic sweep helps explain the pull: mental health shows up 12 times, wellbeing and psychological wellbeing 5 each, and biomarkers 5, so the feed is catching both lived experience and body-level measures.
• 1:56
  Jenny: The author spread is the bigger eyebrow-raiser: 236 unique authors last time, 507 this time, and countries went from 18 to 49. What's driving that: a broader database catch, more international publication, or just one week where China, the UK, India, and the U.S. all showed up at once?
• 2:19
  Davis: And the career mix is not just senior labs. Of 507 authors, 96 are first-time, meaning first-ever paper in the metadata, 240 are emerging, and 171 are experienced; that's almost half early-career, which can make the measurement debate less settled and more inventive.
• 2:41
  Jenny: Small caveat: we don't have city or institution tags this week, so the geography is country-level only. But the practical read is clear: if you're choosing a wellbeing metric, this week says ask whose context it can see, not just whether the score looks tidy.
• 3:00
  Jenny: Alright, let's get into the papers with Community versus donor and implementer-defined success metrics in climate mitigation and adaptation projects, a twenty twenty-six review by David Muyaloka, I. Milupi, and E. Mwanaumo about a very basic fight: who gets to say a climate project worked.
• 3:18
  Jenny: The plain finding is that funders often count the work done, while communities judge whether life actually changed. Donors and implementers tended to emphasize outputs, meaning countable deliverables like beneficiaries reached and activities completed, while communities emphasized outcomes and process, like access to resources, services, and income stability.
• 3:41
  Davis: So if the community and the funder disagree about what success means, whose metric should decide whether the project worked?
• 3:48
  Jenny: The authors used a systematic literature review following PRISMA, which is a checklist-style process for finding, screening, and reporting studies so the review isn't just cherry-picked. That gives the pattern some weight across existing climate mitigation and adaptation literature, but it's still a literature review, so it can show a recurring mismatch and can't prove how every local project should be evaluated.
• 4:13
  Davis: The practical takeaway is pretty sharp: build the monitoring framework with community-defined outcomes before the donor reporting template hardens into concrete. This is exactly the Who Defines Success thread, because a spreadsheet can say “activity completed” while the people living there are still asking whether the water, income, or service access got more reliable.
• 4:33
  Davis: That donor spreadsheet point lands differently here, because this paper is basically a workforce wellbeing dashboard that refused to turn green: Mental health of healthcare workers in England during the first three years of the COVID-19 pandemic.
• 4:47
  Davis: Lamb and colleagues followed twenty-two thousand ninety-two NHS workers across seventeen English Trusts, and the headline is bleakly steady: about half reported probable common mental disorders at every survey point, meaning their scores suggested anxiety, depression, or similar distress that would warrant concern.
• 5:06
  Jenny: How did they measure mental health strongly enough to say the problem persisted for three years, rather than just catching four bad survey days?
• 5:14
  Davis: They used a prospective cohort study, which means they followed the same broad group forward over time, with online surveys at four points from April twenty twenty to March twenty twenty-three, covering depression, anxiety, alcohol misuse, PTSD, burnout, wellbeing, resilience, moral injury, and post-traumatic growth.
• 5:34
  Davis: The strongest predictor was brutally simple: if someone met the baseline cut-off for a poor mental health outcome, they were more likely to keep meeting it later, and the study found no consistent mental health gap between clinical and non-clinical staff, though younger, female, lower-paid, unsupported, and morally injured workers were at higher risk.
• 5:54
  Jenny: The self-report piece matters, because surveys aren't the same as diagnoses, but with seventeen Trusts and three years of repeated measurement, you can't wave this away as pandemic noise; for the Care Systems Strain thread, the metric has to include porters, admin staff, pay bands, manager support, and moral injury, not just whether doctors and nurses showed up for another shift.
• 6:18
  Jenny: That self-report point is exactly where this next climate paper lives: Rating Evaluation - Impacts of extreme weather, or REvIEW, asks how you count what extreme weather did to someone's life when the damage isn't just a diagnosis or a flood-depth number.
• 6:33
  Jenny: The tool is a twenty-nine-item questionnaire about the lived aftershock of extreme weather, with each item rated on a six-point Likert scale, which means people mark degree of impact rather than just saying yes or no. In the Australian study, the measure held together around three domains: livelihood, wellbeing, and connection.
• 6:54
  Davis: So what does this tool capture that a rainfall map or a hospital record would miss, like lost work, stress at home, or the feeling that your usual people aren't reachable anymore?
• 7:03
  Jenny: That's the idea: the authors developed the items with an advisory group of healthcare practitioners, academics, and environmental scientists, then ran an Australian mixed-method, cross-sectional survey, meaning they combined development work with a one-time survey rather than following people over years. The evidence is promising but early, because validation in one Australian context doesn't make this a universal climate wellbeing measure.
• 7:30
  Davis: For the Metrics Under Pressure thread, that matters because a storm count can tell you where weather hit, but REvIEW is trying to show whose income, mental state, and social ties got bent afterward, which is the part recovery plans often find out too late.
• 7:45
  Paperboy.fm: This is the free version of the podcast. Subscribe at paperboy.fm to access a dozen different paper review podcasts for five dollars a month.