Latest / Truth in Motion with Dr. Jocelyn / The Fair Test: When Chance Decides

Transcript

The Fair Test: When Chance Decides · Truth in Motion with Dr. Jocelyn

• 0:02
  SPEAKER_00: Welcome to Truth in Motion with Dr.
• 0:05
  SPEAKER_00: Jocelyn, where insight, clarity, and wisdom help you thrive.
• 0:09
  SPEAKER_00: Thank you for joining me today.
• 0:12
  SPEAKER_00: Yesterday, we followed people forward.
• 0:16
  SPEAKER_00: 40,000 British doctors watched for 50 years.
• 0:20
  SPEAKER_00: A town in Massachusetts watched for three generations.
• 0:26
  SPEAKER_00: And we found the one weakness even the biggest of those studies can't escape.
• 0:31
  SPEAKER_00: People choose their own habits.
• 0:34
  SPEAKER_00: And people who choose one thing often differ in many other areas.
• 0:39
  SPEAKER_00: Today we meet the fix.
• 0:42
  SPEAKER_00: What if instead of letting people choose, we let chance decide?
• 0:47
  SPEAKER_00: That's the randomized control trial, often called an RCT.
• 0:53
  SPEAKER_00: We'll see where it came from.
• 0:55
  SPEAKER_00: We'll see how one trial overturned something doctors had been telling women for years.
• 1:01
  SPEAKER_00: And we'll look at what trials actually show about acupuncture, craniocral therapy, and nutrition.
• 1:10
  SPEAKER_00: Then we'll look honestly at what a trial can't tell us.
• 1:14
  SPEAKER_00: So as always, why does this matter?
• 1:18
  SPEAKER_00: Because the randomized trial is the test most of medicine trusts above all others.
• 1:24
  SPEAKER_00: When someone says, quote unquote, it's proven, or quote unquote, there's no evidence, they usually mean trials.
• 1:34
  SPEAKER_00: So we need to know what a trial really tests.
• 1:37
  SPEAKER_00: Sometimes a trial overturns advice that millions of people were given.
• 1:42
  SPEAKER_00: Sometimes a trial shows that care many of us already chose has real measured benefits.
• 1:49
  SPEAKER_00: And sometimes a trial answers a narrower question than the headline claims.
• 1:56
  SPEAKER_00: If we don't know how trials work, we can be talked into something or talked out of something by people who are counting on us not to ask.
• 2:07
  SPEAKER_00: When we do know, we can hold our own in the conversation.
• 2:11
  SPEAKER_00: And that's what today is for.
• 2:13
  SPEAKER_00: Envelopes and chants.
• 2:16
  SPEAKER_00: Let's go back to Britain in 1946.
• 2:20
  SPEAKER_00: A new drug called streptomycin had been discovered in America.
• 2:25
  SPEAKER_00: It looked like it might cure tuberculosis, which was killing people across Europe.
• 2:30
  SPEAKER_00: But it was very expensive, and post-war Britain had little money to buy it.
• 2:35
  SPEAKER_00: There was only enough for a small number of patients.
• 2:39
  SPEAKER_00: So who should get it?
• 2:42
  SPEAKER_00: The statistician Austin Bradford Hill, the same Bradford Hill we met on Tuesday, had an answer.
• 2:49
  SPEAKER_00: If there isn't enough for everyone, let chance decide and make it fair, so that no one could pick the patients most likely to recover.
• 2:59
  SPEAKER_00: So here's how they did it.
• 3:01
  SPEAKER_00: At each hospital there was a numbered series of envelopes.
• 3:05
  SPEAKER_00: Inside each one was a card marked either S for streptomycin or C for control.
• 3:13
  SPEAKER_00: The order of the cards was set by a list of random numbers.
• 3:17
  SPEAKER_00: When a patient joined the trial, the next envelope was opened.
• 3:23
  SPEAKER_00: No doctor could steer the sickest patients one way or the other.
• 3:28
  SPEAKER_00: And Hill thought about fairness to the patients too.
• 3:31
  SPEAKER_00: With so little of the drug, most people couldn't have it anyway.
• 3:36
  SPEAKER_00: Patients in the control group were given priority for a hospital bed and would be offered streptomycin later if it proved to work.
• 3:45
  SPEAKER_00: 55 patients got streptomycin plus bed rest.
• 3:50
  SPEAKER_00: Fifty-two got bed rest alone, which was the standard treatment at the time.
• 3:55
  SPEAKER_00: After six months, four of the streptomycin patients had died.
• 4:00
  SPEAKER_00: In the bed rest group, 15 had died.
• 4:04
  SPEAKER_00: The results were published in the British Medical Journal in 1948.
• 4:09
  SPEAKER_00: It's widely counted as the first modern randomized control trial, which means the randomized trial, the method much of medicine now calls the gold standard, is about 78 years old.
• 4:23
  SPEAKER_00: Hold on to that number.
• 4:26
  SPEAKER_00: 78.
• 4:27
  SPEAKER_00: We'll come back to it tomorrow.
• 4:30
  SPEAKER_00: So what makes a trial fair?
• 4:33
  SPEAKER_00: Three things.
• 4:35
  SPEAKER_00: First, randomization.
• 4:38
  SPEAKER_00: Chance decides who gets what.
• 4:41
  SPEAKER_00: That spreads all the other differences evenly between the groups.
• 4:46
  SPEAKER_00: The healthy habits, the money, the stress, the genes, even the differences nobody thought to measure.
• 4:53
  SPEAKER_00: That's what a cohort study can never do.
• 4:57
  SPEAKER_00: Second, a control group.
• 4:59
  SPEAKER_00: Something to compare against.
• 5:01
  SPEAKER_00: James Lynn's question from Monday, which is compared to what?
• 5:07
  SPEAKER_00: It might be no treatment, the usual treatment, or a placebo, which is a stand-in that looks and feels like the real thing, but leaves out the active part.
• 5:18
  SPEAKER_00: Third, we have blinding.
• 5:20
  SPEAKER_00: Where possible, the people in the trial don't know which group they're in, and the people measuring the results don't know either, because hope and expectation are real forces.
• 5:33
  SPEAKER_00: They change how people feel and how researchers see.
• 5:38
  SPEAKER_00: Let me show you why this matters so much.
• 5:41
  SPEAKER_00: In 1976, researchers began following about 129,700 married registered nurses aged 30 to 55 in 11 American states.
• 5:58
  SPEAKER_00: It's called the Nurses' Health Study, and it's one of the largest cohort studies ever done.
• 6:04
  SPEAKER_00: In 1991, the study reported on about 48,000 nurses past menopause.
• 6:12
  SPEAKER_00: The women who took estrogen had about 44% less major heart disease than the women who didn't.
• 6:20
  SPEAKER_00: It looked like hormones protected women's hearts.
• 6:24
  SPEAKER_00: And for years, many women were told that hormone therapy after menopause could help protect their hearts.
• 6:32
  SPEAKER_00: But remember Wednesday's big question?
• 6:36
  SPEAKER_00: Did people choose it?
• 6:38
  SPEAKER_00: They did, and women who chose hormone therapy were often different from women who didn't.
• 6:44
  SPEAKER_00: They tended to be leaner and more likely to take care of their health in other ways.
• 6:50
  SPEAKER_00: So the government funded a randomized trial, the Women's Health Initiative.
• 6:56
  SPEAKER_00: 16,608 women aged 50 to 79 were assigned by chance to take either a hormone pill or a placebo.
• 7:07
  SPEAKER_00: The pill was a specific combination of estrogen from horses plus a synthetic progestin.
• 7:15
  SPEAKER_00: The trial was planned to run eight and a half years.
• 7:19
  SPEAKER_00: It was stopped after about five.
• 7:22
  SPEAKER_00: Here's what they found.
• 7:24
  SPEAKER_00: The women taking the hormone pill had more heart disease, not less.
• 7:29
  SPEAKER_00: About 29% more.
• 7:32
  SPEAKER_00: They also had more strokes, more blood clots in their lungs, and more invasive breast cancer.
• 7:39
  SPEAKER_00: They did have fewer hip fractures and fewer colon cancers.
• 7:45
  SPEAKER_00: In real numbers, for every 10,000 women taking the pill for a year, there were about seven more heart events, eight more strokes, eight more blood clots in the lungs, and eight more breast cancers, and about six fewer colon cancers and five fewer hip fractures.
• 8:06
  SPEAKER_00: The cohort study had seen a real pattern.
• 8:10
  SPEAKER_00: Women who took hormones did have healthier hearts, but the hormones weren't the reason.
• 8:16
  SPEAKER_00: The women were.
• 8:17
  SPEAKER_00: That's the power of the FAIR test.
• 8:20
  SPEAKER_00: And here's the careful part.
• 8:22
  SPEAKER_00: The trial answered one question, one specific pill in women who were on average 63 years old.
• 8:31
  SPEAKER_00: It didn't test every form of hormone support in every woman at every age.
• 8:37
  SPEAKER_00: A trial tells us exactly what it tested and no more.
• 8:42
  SPEAKER_00: What trials show about this kind of care?
• 8:45
  SPEAKER_00: Now let's turn to the kind of care I practice.
• 8:49
  SPEAKER_00: What happens when acupuncture, craniosacral therapy, and nutrition are put to a fair test?
• 8:57
  SPEAKER_00: Let's look at acupuncture first.
• 8:59
  SPEAKER_00: In 2018, researchers pooled the individual patient data from 39 trials, covering 20,827 people with chronic pain, back and neck pain, osteoarthritis, headaches, and shoulder pain.
• 9:17
  SPEAKER_00: Acupuncture did better than no acupuncture, and it did better than sham acupuncture for every one of those conditions.
• 9:26
  SPEAKER_00: And a year later, about 85% of the benefit was still there.
• 9:31
  SPEAKER_00: In Germany, a trial of 1,162 people with chronic low back pain compared three groups: real acupuncture, sham acupuncture with shallow needles at points that were not chosen for treatment, and the usual care, which was medication, physical therapy, and exercise.
• 9:53
  SPEAKER_00: After six months, 47.6% of the acupuncture group had improved.
• 9:59
  SPEAKER_00: Usu had a 27.4% improvement, and the sham acupuncture group had 44.2% improvement.
• 10:10
  SPEAKER_00: Now a critic will point to the sham number and say, see, that's all placebo.
• 10:15
  SPEAKER_00: But notice both needle groups did far better than usual care.
• 10:20
  SPEAKER_00: And a sham needle still pierces the skin.
• 10:23
  SPEAKER_00: It still touches the body.
• 10:25
  SPEAKER_00: A sham that does something isn't a true blank.
• 10:28
  SPEAKER_00: That's one of the hardest problems in testing this kind of medicine.
• 10:32
  SPEAKER_00: And we're going to come back to that.
• 10:35
  SPEAKER_00: Cranial sacral therapy.
• 10:38
  SPEAKER_00: A 2020 review gathered 10 randomized trials with 681 people living with chronic pain.
• 10:46
  SPEAKER_00: Cranial sacral therapy reduced pain more than sham treatment.
• 10:51
  SPEAKER_00: And in the two trials that checked, the benefit was still there six months later.
• 10:56
  SPEAKER_00: The trials are small and the authors call for more.
• 11:00
  SPEAKER_00: But the early evidence is promising.
• 11:03
  SPEAKER_00: Let's look at nutrition.
• 11:05
  SPEAKER_00: Here, the trial evidence is some of the strongest in all of medicine.
• 11:09
  SPEAKER_00: In the diabetes prevention program, 3,234 adults at high risk of type 2 diabetes were assigned by chance to one of three groups: a lifestyle program with healthier eating, at least 150 minutes of activity a week, and a goal of losing about 7% of body weight.
• 11:32
  SPEAKER_00: The other group was taking the diabetes drug metformin or a placebo.
• 11:39
  SPEAKER_00: Over about three years, the lifestyle program cut new cases of diabetes by 58%.
• 11:46
  SPEAKER_00: The drug cut them by 31%.
• 11:49
  SPEAKER_00: Food and movement beat the pill in a fair test.
• 11:55
  SPEAKER_00: So should a trial be the final word on everything?
• 11:58
  SPEAKER_00: No, and here's why.
• 12:00
  SPEAKER_00: A trial tests parts.
• 12:03
  SPEAKER_00: To be fair, it has to hold everything steady.
• 12:06
  SPEAKER_00: One fixed treatment given the same way to everyone.
• 12:10
  SPEAKER_00: But the kind of care I practice is individualized.
• 12:14
  SPEAKER_00: Two people with the same back pain may need very different treatments.
• 12:19
  SPEAKER_00: A trial wants one recipe.
• 12:21
  SPEAKER_00: Good practice adjusts the recipe to the person.
• 12:25
  SPEAKER_00: A trial needs a blank to compare against.
• 12:28
  SPEAKER_00: For a pill, that's easy, a sugar pill, but there's no truly blank needle and no truly blank touch.
• 12:36
  SPEAKER_00: A trial usually tests one thing at a time, but many of us receive care that works together.
• 12:43
  SPEAKER_00: Needles, food, herbs, hands-on work, sleep and rest, all of it at the same time.
• 12:50
  SPEAKER_00: A trial of one part on its own can't show us what the parts do altogether.
• 12:57
  SPEAKER_00: And trials are usually short.
• 12:59
  SPEAKER_00: They're looking over weeks or months.
• 13:01
  SPEAKER_00: Many of the changes we care about most, the ones that build slowly as a body heals, take longer than most trials last.
• 13:10
  SPEAKER_00: A trial wants to remove the relationship between the practitioner and the patient so it won't affect the results.
• 13:18
  SPEAKER_00: But in healing, the relationship is often part of the medicine.
• 13:22
  SPEAKER_00: And most of all, a trial tells us what happened to the average person in the group.
• 13:28
  SPEAKER_00: None of us is the average person.
• 13:32
  SPEAKER_00: A treatment that helped most people in a trial may not help me or you.
• 13:39
  SPEAKER_00: And a treatment that didn't help the average person may be exactly what one particular person needs.
• 13:46
  SPEAKER_00: That's why the best evidence still has to meet a real person.
• 13:51
  SPEAKER_00: So what does this mean for us?
• 13:53
  SPEAKER_00: Here's what this gives us.
• 13:55
  SPEAKER_00: When we hear randomized trials showed, we can ask three questions.
• 14:00
  SPEAKER_00: First, compared to what?
• 14:03
  SPEAKER_00: Nothing?
• 14:04
  SPEAKER_00: A placebo, a sham?
• 14:06
  SPEAKER_00: The usual treatment?
• 14:08
  SPEAKER_00: The answer changes everything.
• 14:10
  SPEAKER_00: Second, who was in it and are they like me?
• 14:14
  SPEAKER_00: The hormone trial was about women in their 60s taking one particular pill.
• 14:20
  SPEAKER_00: Was it about somebody like me?
• 14:22
  SPEAKER_00: Third, how big was the difference in real numbers?
• 14:26
  SPEAKER_00: Not just 29% more, but how many people out of 10,000?
• 14:33
  SPEAKER_00: These questions make us wiser patients, and they let us see clearly, without overselling, where the kind of care we may already choose stands on solid ground.
• 14:45
  SPEAKER_00: Here's this week's practice.
• 14:47
  SPEAKER_00: This week we keep questioning one layer deeper.
• 14:51
  SPEAKER_00: Here's today's practice.
• 14:53
  SPEAKER_00: The next time you hear studies show, ask the first question.
• 14:59
  SPEAKER_00: Compared with what?
• 15:01
  SPEAKER_00: And if you want to go further, ask the other two.
• 15:04
  SPEAKER_00: Who was in it?
• 15:06
  SPEAKER_00: And how big was the difference in real numbers?
• 15:09
  SPEAKER_00: You'll start to notice how often the answer is left out.
• 15:13
  SPEAKER_00: And you'll start to hear health news in a new way.
• 15:16
  SPEAKER_00: Today we met the FAIR test.
• 15:18
  SPEAKER_00: We opened envelopes in post-war Britain where chance decided who got streptomycin and saw how that simple step made a test fair.
• 15:27
  SPEAKER_00: We saw how a huge study of nurses found a pattern and how a randomized trial showed the hormones weren't the reason.
• 15:35
  SPEAKER_00: We looked at what trials show about acupuncture, craniosacral therapy, and nutrition, and found real measured benefit.
• 15:43
  SPEAKER_00: Food and movement even beat a diabetes drug in a FAIR test.
• 15:48
  SPEAKER_00: And we saw what a trial can't do.
• 15:51
  SPEAKER_00: It tests parts on an average person.
• 15:54
  SPEAKER_00: And we saw why that matters.
• 15:56
  SPEAKER_00: The randomized trial is the test medicine trusts the most.
• 16:01
  SPEAKER_00: When we know how it works, we can't be talked into something or out of something so easily.
• 16:08
  SPEAKER_00: Tomorrow we end the week with a question the ladder of evidence leaves out.
• 16:13
  SPEAKER_00: The randomized trial is about 78 years old.
• 16:17
  SPEAKER_00: Acupuncture has more than 2,000 years of written practice.
• 16:22
  SPEAKER_00: What about knowledge built from watching patients carefully generation after generation?
• 16:29
  SPEAKER_00: And what about what the patient tells us?
• 16:33
  SPEAKER_00: Not all empiricism is equal.
• 16:37
  SPEAKER_00: I look forward to meeting you there.
• 16:40
  SPEAKER_00: Join me tomorrow.
• 16:44
  SPEAKER_00: Keep the conversation going.
• 16:46
  SPEAKER_00: Find your next steps in notes from a fellow traveler, my blog.