Latest / Common Sense Health / When Health Screening Goes Wrong
Transcript
- Stephen Thomas BSc. Hons: Hello again. Now population health screening tests can be a minefield and this video will tell you some uncomfortable truths. It will explain why a positive screening result often tells you more about mathematics than it does about diseases and medicine. We're all taught, well implicitly at least, that a medical test result is a verdict. Positive means you've got it, negative means you don't. Job done. But... Screening tests don't work like that in the real world because real world testing is a bit more messy than that. And if you take nothing else from this video, take this. A screening test can be good. but still produce a lot of false alarms when you use it to screen a large population. firstly, let's cover what a screening test is and also what it isn't. Now, a screening test is not a diagnostic test. Screening is a sorting hat. It tries to decide who should get looked at more closely. And that matters because the question most people ask after a positive result is, what are the chances I actually have the disease? that question has a name is called positive predictive value or PPV. And it's the number that gets quietly mugged in a dark alley whenever people talk about accuracy. So two actual screening tests with real world stats will be talked about. But before we get into that, we will talk about the maths so you can understand the jargon. And trust me, you will understand the jargon if I do my job right. So firstly, let me explain why 95 % accurate. is not as good as it sounds. So I'm going to run a simple example and even this trips people up including plenty of clinicians. So the first word is prevalence. This is how many people in a population get the disease. So to make maths easy we will use one in a thousand. Next is sensitivity. How precise the test is. Let's say it's a hundred percent. Well, now that's great. But we come to the next term specificity. Now that's 95 percent. So it makes mistakes. on 5 % okay so we have the setup now we screen a thousand people who actually has the disease now remember we said that one person in a thousand has it on average if one person has it then that does mean that 999 people don't So what happens to the one person who has it? Well, we said the sensitivity is 100%. So they do test positive and this is known as a true positive. So we have a true positive of TP of one. But what about the other 999? Now remember, we believe that 999 people don't have it. And this is where specificity comes into it. And we said the rate is 95%. Now what that means is that 5 % will test positive in error. And I think most we've heard the term false positives and we can calculate that false positives or FP will equal 5 % of 1000 so approximately 50 people will get a false positive. So with that in mind, let's recap. What a positive results look like. You have to add all the positives. So we have total positives. That's the one we got and the ones that we said were false positives. Well, that's 50. So the total plus the force is 51. Now using that it should be clear that out of a hundred people, roughly two in one hundred probably do actually have the disease. So to be clear this means if your test is positive your chance of truly having the disease is 2%. That's not a failure of the test, it's a consequence of screening a population. Remember 49 out of 50 don't have it. Now let's look at two genuine widely used population level screening tests. We'll add the sort of performance figures that you see in the literature as well. So these are all published documents. So screening for breast cancer. I lost a girlfriend to breast cancer. So it is classically deployed, but it is emotionally loaded, understandably, because many people do have this problem. The sensitivity is around 83 to 95 percent, and that varies by age and breast density and screening intervals. So the translation for that is it catches many cancers, but not all. But then we get to specificity which is commonly again around 90 to 95 percent in many contexts. So what does that mean? Most people without cancer they will get a reassuring negative test but When we look at the recall rate, like being called back after a screen, many programs sit around 5%, with some analysis highlighting roughly 4-5.5 % as a sweet spot, balancing detection and unnecessary follow-up. But then we get false positives. Depending on the program and definition, some sources report that up to 10 % mammograms can be false positives. So the important point isn't mammographs are bad or breast cancer screening is good. It's this. Even with decent specificity, false positives are baked into the process when you test a lot of people at low baseline risk. Or we could look at the fecal immunochemical test. Let's call it FIT because it is a bit of a mouthful. It's used for colorectal cancer screening. It's used widely because it's non-invasive and easy to roll out on scale. The sensitivity for it is often around 79 to 83 percent, but it varies with thresholds and populations. So the translation for that is it detects most cancers but not all. And then we get into specificity, which again is often around 90 to 95 percent. In short, this means that most people without cancer do test negative. Yay, that's good. But there's a lot of tradeoffs in the wild. So some fit approaches report very high sensitivity, like I say, 96 percent, but much lower specificity. So sometimes it's as low as 60 67%. So in simple terms, this means you potentially catch more, but you also alarm for far more people. And there's also stage nuance. Fit sensitivity can vary by stage. Broadly, it's more likely to pick up more advanced cancers than very early disease. So again, the test isn't lying. It's doing what it does. We just have to interpret it properly. And this is where the joined up thinking bit matters because we can look beyond the screening. And I do bang on about it, lovingly of course, and it is all about context when you're looking at bloods or scans or biomarkers. So a number in isolation is not a diagnosis. Always remember that it's a clue. And whether that clue is meaningful depends on the baseline risk, the pretest probability, how likely it was before you got tested, and the test's sensitivity and the specificity. what you do next. Well, do you repeat the test? Do you try different tests, clinical assessment, whatever? The principle applies everywhere, including when interpreting blood results. And yes, I'll say it again, because it's important. Interpreting bloods in the context of the way you eat, whether it's vegan, vegetarian, carnivore, low carb, ketogenic, is crucial. If you change the underlying physiology and you keep reading the numbers through the old lanes, you'll generate the same sort of false certainty or false panic that we see in bad screening interpreters. So should we screen or not? Screening can be genuinely life-saving in the right circumstances for the right groups with the right follow-up pathway. The problem is when we treat screening like a vending machine, put in a swab, take a sample, scan, whatever, get out a truth. Well, that's not medicine, that's mythology. Takeaways for this particular video is remember the following things. A positive screening test is not a diagnosis. It's a prompt for further assessment, the positive predictive value is the number that answers what are my chances and it depends heavily on the prevalence of the disease. So low prevalence and imperfect specificity you'll get lots of false positives even with a really good test. So context always wins. Risk, symptoms and follow-up matter more than a single headline result. I'm going to leave you with this closing thought. If you ever want to make a room go quiet ask someone proudly announcing It's 95 % accurate to tell you the PPV. It's the medical equivalent of asking a magician where the rabbit actually comes from. And on that note, may your screening tests be sensible, your follow-ups appropriately cautious, and your statistics not committed by an overconfident person with a printer. And hopefully this made some sense to you. ⁓ There are some images, obviously, accompanying the video to try and make the maths a bit more simple, but do bear in mind exactly what I said. We do need to understand positive results, false positives and the chances. That's what I'm talking about. I'm not telling anyone to not screen. Do listen to your doctor. This is just purely for education and information purposes. And I will tell you my own story. Like I say, I lost a girl for enter breast cancer by the time I was 23. I'd lost both my parents as well. So my dad died. of ⁓ cancer of the spine, my mother died of colorectal cancer and my girlfriend died of breast cancer. So it's not something I'm glibly talking about but I do think it's important because I can remember. the anxiety and the panic that my mother had every time she went for a scan, every time she had to go to hospital. She was absolutely petrified in the later stages of her life about every single result. And it was never told to her exactly what was being said, what the results actually meant, what the chances of her surviving were. And she was told things that just frankly weren't true at the time because she outlived the diagnosis. She was told that she didn't have long to live. And from that moment, she lived a further 11 years. It matters to me to try and get the facts out there and like I say it's in the literature These tests are not 100 % perfect. They never are there's always a chance for a false positive And of course if you get the false positive you will panic you definitely will panic Even if you know the numbers, but hopefully with the information that's out there And maybe this video will make you do a bit more research You'll go into those things with a bit more knowledge and therefore you'll feel well armed to take either result a positive or a negative with the information that you need so you'll able to be a bit calmer about what's going on and also be able to think about how to follow it up and make a big difference to your health. Anyway, thank you for listening this far. If you really enjoyed it, do like, share and subscribe. If you've got this far and you didn't enjoy it and it's made you tut a lot, I do apologize. I'll try and do better on the next video. Anyway, thanks for listening.