Latest / This Is Why with Dr. Busti / 95% Confidence Intervals Explained for Medical Literature

Transcript

95% Confidence Intervals Explained for Medical Literature · This Is Why with Dr. Busti

• 0:00
  SPEAKER_00: Alright, well, welcome to this lecture where I'm gonna be covering confidence intervals as part of our series on summary statistics specifically in the area of inferential statistics.
• 0:10
  SPEAKER_00: Now, the 95% confidence interval is one of the most common metrics that we pair with the p-value and effect size for the completeness and overall interpretation of the results.
• 0:23
  SPEAKER_00: Now, if that seems a little confusing or scary, don't worry, I'm gonna I'm gonna make it easy, but it is also part of a series that we're doing, okay, to make it easy.
• 0:32
  SPEAKER_00: Now, I'm Dr.
• 0:33
  SPEAKER_00: Bustai, your faculty and host on This Is Why, a show that helps healthcare professionals transform the medical knowledge from what to why.
• 0:40
  SPEAKER_00: Because when we know why we do what we do in the proper context of the evidence, it really changes the how, and that changes everything about those decisions that we make every day.
• 0:51
  SPEAKER_00: As I mentioned just a minute ago, if you need a full playlist or you need structure, which I'll cover here in a minute where we're at, we have a playlist on this area of evidence-based medicine for descriptive and inferential stomach statistics.
• 1:07
  SPEAKER_00: Okay?
• 1:08
  SPEAKER_00: You can see those on YouTube, but also at thisiswhy.
• 1:29
  SPEAKER_00: Okay?
• 1:31
  SPEAKER_00: And also this particular topic on confidence intervals typically follows the review on what a p-value is and what it's not.
• 1:40
  SPEAKER_00: Okay.
• 1:41
  SPEAKER_00: And again, if you need that, so you to put the two together is the best way to utilize this particular topic.
• 1:51
  SPEAKER_00: Alright, so now orienting us where we are in the series.
• 1:55
  SPEAKER_00: So part one before the study starts, by a researcher, they're supposed to do that.
• 2:00
  SPEAKER_00: Part two, at the completion of the study, this is the analysis, and that's where we're at.
• 2:04
  SPEAKER_00: Part three of our series is really geared towards diagnostic tests.
• 2:09
  SPEAKER_00: Now, going back to part one before the study starts, that's our statistical analysis plan.
• 2:14
  SPEAKER_00: Again, deciding what is our null and alternative hypothesis.
• 2:18
  SPEAKER_00: How many groups are we going to have?
• 2:20
  SPEAKER_00: Okay, what type of data am I dealing with?
• 2:22
  SPEAKER_00: Is it nominal, ordinal, continuous?
• 2:25
  SPEAKER_00: That reflects our endpoint.
• 2:26
  SPEAKER_00: What's the question?
• 2:27
  SPEAKER_00: It goes back to the null.
• 2:29
  SPEAKER_00: Again, we have lectures on all those topics if you need them.
• 2:32
  SPEAKER_00: Now, if you are considering analysis, right, then you have to decide is it parametric analysis or non-parametric analysis, though I anticipate.
• 2:40
  SPEAKER_00: If you're going to do inferential statistics, which is where we're at today, right, right now, you have to consider alpha values, one-sided, two-sided, power analysis to determine that sample size and where the level of significance is going to be.
• 2:55
  SPEAKER_00: Now we are in part two.
• 2:57
  SPEAKER_00: So the study was done by the researcher, right?
• 3:00
  SPEAKER_00: And now they're doing summary statistics of the results.
• 3:03
  SPEAKER_00: And that's broken down into our descriptive statistics and our inferential statistics.
• 3:07
  SPEAKER_00: And inferential statistics, we are right here.
• 3:10
  SPEAKER_00: We're covering the 95% confidence interval.
• 3:12
  SPEAKER_00: But remember, it's in the context of other metrics or two things that we uh report in inferential statistics, that p-value, the effect size, or that you are reading yourself and doing in real time, which is not really that hard to do.
• 3:28
  SPEAKER_00: Okay, so what is a confidence interval?
• 3:31
  SPEAKER_00: I mean, what it what does it really tell us?
• 3:34
  SPEAKER_00: Why do we re reveal that?
• 3:36
  SPEAKER_00: Okay, so think about repeating a study under these sort of, if you could do it under perfect ideal conditions, which unfortunately doesn't exist, the theoretical, okay, where no random error is happening, or you were to repeat the study indefinitely many times, you would be 95% confident that the true population finding or the result is somewhere in that interval.
• 4:08
  SPEAKER_00: Now you don't know exactly where it is, but it's somewhere in there.
• 4:10
  SPEAKER_00: And so what do we typically teach and want or hear?
• 4:15
  SPEAKER_00: We want tight confidence intervals.
• 4:17
  SPEAKER_00: We don't want wide confidence intervals because the true population result can be anywhere in between there.
• 4:23
  SPEAKER_00: So the tighter that we are, the more confident we are that the true population result parameter is somewhere in here.
• 4:33
  SPEAKER_00: Okay, so that tighter the smaller that is, the better.
• 4:37
  SPEAKER_00: Okay.
• 4:39
  SPEAKER_00: Now, so the closer to the point of your study that truly lies in that midpoint of that 95% confidence, the more likely it's probably leaning towards the true population result or effect.
• 4:52
  SPEAKER_00: Okay, set it another way, because sometimes we need to think about it more than one direction.
• 4:58
  SPEAKER_00: The point estimate, okay, or that midpoint, is our single best guess.
• 5:04
  SPEAKER_00: Values near that center are more compatible overall with the data than those that are on the periphery or the edges.
• 5:15
  SPEAKER_00: And again, if we have a wide confidence interval, if you're on the periphery, the real result, well, that that makes it, you know, it's it's off.
• 5:24
  SPEAKER_00: Something's off with the data.
• 5:25
  SPEAKER_00: We want to be more centered if we can, and we want tight intervals.
• 5:30
  SPEAKER_00: We don't want big, wide areas.
• 5:34
  SPEAKER_00: So, how what are some of the things that influence that interval?
• 5:38
  SPEAKER_00: Okay, so you have to think going back to what what are the components of making that calculation.
• 5:44
  SPEAKER_00: I don't want to dive deep into the calculation, okay?
• 5:48
  SPEAKER_00: Uh I'm not trying to you know cause a seizure or anybody with this stuff.
• 5:54
  SPEAKER_00: I know it gets to be a little bit frustrating and confusing, but remember there's it depends on the type of confidence interval we're doing dealing with, okay?
• 6:02
  SPEAKER_00: And it depends also on the the standard error of the mean.
• 6:06
  SPEAKER_00: Okay, are we doing 95% confidence intervals or are we doing 99% confidence intervals?
• 6:12
  SPEAKER_00: Okay, that that then influences how many standard deviations, which then influences our Z-score that we are going to use in that calculation.
• 6:22
  SPEAKER_00: Okay, so but if you look at the calculation of the confidence interval, right, not only do you need that mean and standard deviation, consideration of the standard area of the mean, but the sample size.
• 6:33
  SPEAKER_00: And I just want to make a comment here that the larger the sample size, the tighter you start to see the confidence interval.
• 6:42
  SPEAKER_00: Why?
• 6:42
  SPEAKER_00: Because you regress and look, start to mimic back to the population.
• 6:48
  SPEAKER_00: So if you could study the whole population, that would be ideal.
• 6:52
  SPEAKER_00: But it's not feasible, is it?
• 6:54
  SPEAKER_00: And so sample size becomes very important contributor to the overall confidence interval that we see.
• 7:01
  SPEAKER_00: And we see that that's true for data when you look at them, especially on forest plots, you can it's very easy to see that delineation.
• 7:08
  SPEAKER_00: Okay.
• 7:09
  SPEAKER_00: Okay, so how to determine if a 95% confidence interval is actually statistically significant?
• 7:18
  SPEAKER_00: Right?
• 7:19
  SPEAKER_00: That goes back again, thinking about the p-value.
• 7:21
  SPEAKER_00: Is the p-value less than 0.05?
• 7:24
  SPEAKER_00: That's what most people you utilize, that benchmark, right, for statistical significance.
• 7:30
  SPEAKER_00: So you're at that's one question that we're asking.
• 7:33
  SPEAKER_00: Okay, we talk about this in the p-value lecture as well.
• 7:37
  SPEAKER_00: That the p-value is simply talking about statistical difference.
• 7:42
  SPEAKER_00: Are we able, how how likely is that null hypothesis actually true?
• 7:47
  SPEAKER_00: And so what do we typically want?
• 7:49
  SPEAKER_00: We want p-values that are very, very, very, very small.
• 7:52
  SPEAKER_00: Because we want to be able to say, oh, the results, especially if they're different, they're believable.
• 7:57
  SPEAKER_00: That's real.
• 7:58
  SPEAKER_00: There's a real difference there.
• 7:59
  SPEAKER_00: Okay, so we can reject the null hypothesis.
• 8:02
  SPEAKER_00: All right, so how do we determine if it's statistically significant?
• 8:06
  SPEAKER_00: Well, the result is not going to be statistically significant.
• 8:09
  SPEAKER_00: I'm gonna underline statistical here.
• 8:13
  SPEAKER_00: That means you can't reject the null hypothesis if the 95% confidence interval contains the null value of 0, 0.0 for means when you're comparing them, or 1.0 for various ratios.
• 8:28
  SPEAKER_00: And when I talk about ratios, I'm talking about relative risk, risk ratio, hazards ratios, uh, those kind of things, okay?
• 8:36
  SPEAKER_00: Where the numerator and denominator, if they're the same number, i.e.
• 8:40
  SPEAKER_00: each group same, well, then they're one.
• 8:43
  SPEAKER_00: Well, it can't be different.
• 8:45
  SPEAKER_00: Then the null is true.
• 8:47
  SPEAKER_00: Remember, the null hypothesis is about saying there's no difference between what you normally do versus the intervention group, okay?
• 8:55
  SPEAKER_00: Or exposed group.
• 8:57
  SPEAKER_00: And so let's look at an example or application of that in the context of statistical significance.
• 9:03
  SPEAKER_00: Okay, that's all we're talking about here.
• 9:05
  SPEAKER_00: Statistical, not clinical relevance, not clinical significance.
• 9:09
  SPEAKER_00: Remember, p-values do not determine clinical relevance or significance.
• 9:15
  SPEAKER_00: So when you look at this, this is looking at cases of patients receiving certain types of drugs and their odds of achieving some endpoint.
• 9:24
  SPEAKER_00: Does what does the endpoint matter?
• 9:26
  SPEAKER_00: No, make it blood pressure goal.
• 9:28
  SPEAKER_00: It doesn't matter.
• 9:29
  SPEAKER_00: Right?
• 9:30
  SPEAKER_00: But it's an odds ratio.
• 9:31
  SPEAKER_00: So you have to always look at the confidence interval.
• 9:33
  SPEAKER_00: What type of data are we dealing with?
• 9:35
  SPEAKER_00: Because remember, if it's means, then it would be does that confidence interval include 0.00?
• 9:41
  SPEAKER_00: If it's a ratio, right, odds ratio, relative risk, risk ratio, hazards ratio, and it includes 1.0, it cannot be statistically significant.
• 9:53
  SPEAKER_00: So if you just go across here looking at the data of the different groups and combinations of antihypertensives in this basic diagram here, you can see here, does this confidence interval cross through 1.0?
• 10:07
  SPEAKER_00: Well, it does.
• 10:08
  SPEAKER_00: So that cannot be statistically significant.
• 10:11
  SPEAKER_00: The p-value is greater than 0.05.
• 10:13
  SPEAKER_00: I don't know what it is, but it doesn't matter.
• 10:16
  SPEAKER_00: It is not statistically significant.
• 10:19
  SPEAKER_00: What about beta blockers with a diuretic?
• 10:22
  SPEAKER_00: 0.62 to 1.52.
• 10:24
  SPEAKER_00: Again, cannot have statistical significance.
• 10:28
  SPEAKER_00: The p-value is greater than 0.05.
• 10:30
  SPEAKER_00: All right, so you go down the list and you see, oh, well, here's one.
• 10:34
  SPEAKER_00: This is above calcium channel blockers.
• 10:37
  SPEAKER_00: 1.04 to 2.39.
• 10:44
  SPEAKER_00: Does it include 1.00?
• 10:48
  SPEAKER_00: It does not.
• 10:50
  SPEAKER_00: So this one would be statistically significant.
• 10:54
  SPEAKER_00: Now you're talking about, well, what's the odds of what?
• 10:57
  SPEAKER_00: Maybe it's a side effect, which we know calcium channel blockers can do that.
• 11:01
  SPEAKER_00: Let's say it's low extreme edema.
• 11:04
  SPEAKER_00: Okay?
• 11:04
  SPEAKER_00: Well, we know that uh calcium channel blockers can do that.
• 11:08
  SPEAKER_00: Calcium channel blockers with a diuretic now become statistically insignificant because remember, if you look at that data, it does cross through 1.00.
• 11:21
  SPEAKER_00: Okay, so that diuretic is basically getting rid of the edema.
• 11:25
  SPEAKER_00: And that's what that's what that's really, the odds is really about in this particular set of data.
• 11:31
  SPEAKER_00: It's not, that's a distraction at the moment.
• 11:34
  SPEAKER_00: All you have to do is interpret confidence interval and answer the question.
• 11:37
  SPEAKER_00: And that's what board exams like to do many times.
• 11:41
  SPEAKER_00: They'll throw data at you, they'll throw all these things, they'll get you all wound up, maybe distracted, and really all they're asking you to do is interpret in numbers.
• 11:49
  SPEAKER_00: It almost is irrelevant what the context is.
• 11:53
  SPEAKER_00: Okay, but you need to look at some things to get you oriented.
• 11:58
  SPEAKER_00: All right, I hope that made sense.
• 12:00
  SPEAKER_00: So confidence intervals in the context, okay.
• 12:04
  SPEAKER_00: Remember, we should always do it with p-values and effect size.
• 12:07
  SPEAKER_00: What confidence intervals are showing us that magnitude, right, the direction, and the degree of precision of the effect.
• 12:16
  SPEAKER_00: Okay, the p-value alone tells you nothing about clinical importance or relevance.
• 12:22
  SPEAKER_00: Okay, I cannot drill that point home more.
• 12:26
  SPEAKER_00: And that is why these are taught typically back to back or in context of each other.
• 12:34
  SPEAKER_00: And so when you watch the p-value lecture, when I dive into more details about p-values, you hear me say some of these points.
• 12:40
  SPEAKER_00: Okay?
• 12:42
  SPEAKER_00: So remember, a uh a Y confidence interval, just because it has statistical significance, doesn't mean it's clinically relevant.
• 12:51
  SPEAKER_00: Okay?
• 12:52
  SPEAKER_00: And that's why we need to look at the range of it.
• 12:55
  SPEAKER_00: And so I like to use or incorporate real published information that's out there to drive on this point.
• 13:04
  SPEAKER_00: And that's this proactive trial.
• 13:06
  unknown: Okay.
• 13:07
  SPEAKER_00: Now, this was a study looking at the secondary prevention of macrovascular events in patients with type 2 diabetes, okay, who were being given pyoglitosone.
• 13:18
  SPEAKER_00: Okay, so it was a randomized controlled trial published in the Lancet 2005.
• 13:24
  SPEAKER_00: Okay, and so when you look at this diagram here, you see that the has you can look at it and it gives you there's a hazards ratio.
• 13:33
  SPEAKER_00: The red line represents, and I know that those of you on the podcast can't see this, but it's um basically two lines and they're starting to divert away in a graph.
• 13:43
  SPEAKER_00: Okay, and um the y-axis is the proportion of events, the x-axis is the time for randomization.
• 13:53
  SPEAKER_00: And when you look at the hazards ratio, so that means there's a numerator and a denominator, so it if it includes one point zero, okay, then it cannot be statistically significant.
• 14:04
  SPEAKER_00: So the hazards ratio is 0.90.
• 14:08
  SPEAKER_00: And the confidence interval goes from 0.8 all the way up to 1.02.
• 14:14
  SPEAKER_00: So does that 95% confidence interval include 1.00?
• 14:19
  SPEAKER_00: It does.
• 14:20
  SPEAKER_00: And because of that, the p-value, which you see below it, is not less than 0.05.
• 14:28
  SPEAKER_00: Again, we're talking about statistical significance here.
• 14:32
  SPEAKER_00: So is it a statistically significant result?
• 14:38
  SPEAKER_00: No.
• 14:39
  SPEAKER_00: Is there clinical relevance?
• 14:40
  SPEAKER_00: I mean, if you just look at the graph that you start to see the line splitting, it looks like pyogletosone is providing and conferring some benefit and protection.
• 14:50
  SPEAKER_00: When I look at the confidence interval and I look at the hazard ratio, it does look like it is, but man, that confidence interval just reaches beyond it.
• 15:00
  SPEAKER_00: Okay?
• 15:01
  SPEAKER_00: And so when they when you break down then death, okay, and the individual different components versus the total, okay, then you start to see, well, is there anywhere in here where the p-value is statistically significant?
• 15:18
  SPEAKER_00: And the answer is no, it's not.
• 15:21
  SPEAKER_00: So it's it feels like a like a largely negative study.
• 15:25
  SPEAKER_00: And so what why don't you ask yourself?
• 15:28
  SPEAKER_00: P-values not less than 0.05, the confidence intervals include 1.00, and every single one of their endpoints.
• 15:37
  SPEAKER_00: Well, what may be the problem, right?
• 15:39
  SPEAKER_00: It may be related to sample size, it may be due to duration.
• 15:44
  SPEAKER_00: Was there enough time to find that difference?
• 15:47
  SPEAKER_00: Right?
• 15:48
  SPEAKER_00: So some of that goes back to the way the study was designed and set up.
• 15:53
  SPEAKER_00: That's where step one, that statistical analysis plan, becomes so important.
• 15:57
  SPEAKER_00: How you define what is meant to be clinically relevant, maybe harder to achieve statistically, to see that.
• 16:09
  SPEAKER_00: And that's why you have to put it into context.
• 16:11
  SPEAKER_00: But when you look across some of this, you can see variations in benefits, some coronary revascularization, acute coronary syndrome, again, but they're all statistically insignificant.
• 16:24
  SPEAKER_00: Let's do another one and look at it from another perspective, from a meta-analysis or systematic review.
• 16:29
  SPEAKER_00: And this is looking at aspirin in the primary prevention of patients with cardiovascular events.
• 16:39
  SPEAKER_00: And it's papers like this and things you know that have evolved over the over the years that have made it this topic of do we use aspirin for primary or secondary prevention?
• 16:50
  SPEAKER_00: The data is pretty set overall in secondary prevention.
• 16:55
  SPEAKER_00: That was the ISIS II trial, especially in patients with uh presenting with acute myocardial infarction.
• 17:01
  SPEAKER_00: But primary prevention has been debated a lot, right?
• 17:06
  SPEAKER_00: And you'll see people in clinical practice who have never had a stent, never had a procedure, never had a heart attack or a stroke or a vascular event or equivalent thereof, and they're put on on aspirin just in case or for primary prevention.
• 17:20
  SPEAKER_00: And so where does some of that come from?
• 17:22
  SPEAKER_00: Well, this is a meta-analysis again published in American Journal of Cardiology 2006, and you might say, okay, that's old, who cares?
• 17:30
  SPEAKER_00: Well, it actually still creates the basis for some of the problem, right?
• 17:35
  SPEAKER_00: And the debate.
• 17:36
  SPEAKER_00: Because you'll see here in the data, when you look at the defined endpoints, and they looked at, they chose six trials, okay, primary prevention trials.
• 17:47
  SPEAKER_00: Why do you do a systematic review and meta-analysis?
• 17:49
  SPEAKER_00: Well, so you combine their data, and the idea is to increase the power because you have more sample, more patients to include in the analysis.
• 17:58
  SPEAKER_00: But obviously, the studies that you pick and choose through inclusion and exclusion criteria will drive that result, won't it?
• 18:07
  SPEAKER_00: So you can see here at these endpoints, uh, we have total uh corneal article disease, non-fatal MI, total cardiovascular events, stroke, cardiovascular-related mortality, and then all cause mortality.
• 18:18
  SPEAKER_00: So when we talk about results and why are we using a drug, you may have to put it in the context of the individual endpoint that matters to you or the patient.
• 18:29
  SPEAKER_00: Okay, because sometimes finding all-cause mortality reductions is very difficult.
• 18:34
  SPEAKER_00: Do you really truly have a sample size large enough to find that?
• 18:39
  SPEAKER_00: All right.
• 18:39
  SPEAKER_00: And so when you look here, they give you the odds ratios, they give you the 95% confidence intervals, and then they give you the p-values.
• 18:47
  SPEAKER_00: And you can see that several of the p-values, total coronary heart disease, uh, you have non-fatal MIs, total cardiovascular events, all of those are statistically significant.
• 18:58
  SPEAKER_00: And being an odds ratio, that means the numerator and denominator cannot be the same.
• 19:03
  SPEAKER_00: They cannot include 1.00.
• 19:06
  SPEAKER_00: So do any of those 95% confidence intervals include 1.00?
• 19:10
  SPEAKER_00: And they don't.
• 19:11
  SPEAKER_00: And so therefore, the p-values are accurate.
• 19:14
  SPEAKER_00: They're less than 0.05.
• 19:16
  SPEAKER_00: Now, they give us the p-value.
• 19:18
  SPEAKER_00: If they didn't, I could still look at the confidence interval.
• 19:20
  SPEAKER_00: If you removed this column from this diagram chart, I could still determine that what was statistically significant or not.
• 19:31
  SPEAKER_00: Now let's go back and ask clinical relevance.
• 19:34
  SPEAKER_00: Okay, so if these are your targeted kind of endpoints, and that's how you define them, you might say yeah.
• 19:42
  SPEAKER_00: But then if you look at the other ones, stroke, cardiovascular-related mortality, all-cause mortality, all of these be fall and extend beyond 1.00.
• 19:54
  SPEAKER_00: And so they are statistically insignificant.
• 19:57
  SPEAKER_00: And you see that that's true.
• 19:59
  SPEAKER_00: Even all the all-cause mortality, it gets close though, look at it.
• 20:04
  SPEAKER_00: P-value of 0.071.
• 20:08
  SPEAKER_00: That means there's a 7.1% chance that the results here that are found are due to chance or random error.
• 20:20
  SPEAKER_00: That's not that bad, actually, right?
• 20:22
  SPEAKER_00: So I mean, uh just a few more percentage points and we would have statistical significance.
• 20:29
  SPEAKER_00: So you have to stop and go, is that clinically relevant?
• 20:33
  SPEAKER_00: Well, that's why they put forest plots out there where you can kind of look at the combined effort of the summary or pooled analysis here, these larger diamonds, they give you that visual on the forest plot of which side is favoring what.
• 20:48
  SPEAKER_00: So these are on the left side favoring aspirin.
• 20:52
  SPEAKER_00: When you look at the individual trials, depending on the endpoint, right, up here, it gives you a different visual.
• 21:02
  SPEAKER_00: Like for stroke, you can see, my goodness, the studies are all over the place.
• 21:07
  SPEAKER_00: And if you look at the blobs over on the stroke, you have some really small blobs with very wide confidence intervals.
• 21:16
  SPEAKER_00: Well, why is that?
• 21:18
  SPEAKER_00: Well, sample size problem, right?
• 21:20
  SPEAKER_00: So, what's the main study influencing the stroke data?
• 21:25
  SPEAKER_00: This study right here has the largest blob.
• 21:28
  SPEAKER_00: But you have other studies over here that have decent sized blobs and decent sized confidence intervals, but they're crossing through the point of a line of no difference.
• 21:38
  SPEAKER_00: And so that's what these visuals are helping you to see.
• 21:41
  SPEAKER_00: But you can see with total uh uh coronary heart disease or cardiac heart disease, and then patients with cardiovascular related death, MI, stroke combined, you see these favoring the overall benefit towards aspirin.
• 21:56
  SPEAKER_00: And that's where this creates debate because remember, we're always trying to.
• 22:00
  SPEAKER_00: To infer back.
• 22:01
  SPEAKER_00: That's what inferential statistics is trying to do, right?
• 22:04
  SPEAKER_00: We want to be able to extrapolate back onto the population.
• 22:07
  SPEAKER_00: So what are the reasons why we don't always find statistical significance?
• 22:12
  SPEAKER_00: Well, it could be, you know, the way we design the study, the risk of bias, random error, we got the alpha values that we do, one-sided, one-tailed, or one-sided or two-tailed, one-sided or two-sided, sorry, the power analysis.
• 22:27
  SPEAKER_00: Do we have an appropriate sample size?
• 22:30
  SPEAKER_00: But also the controlling of various confounders.
• 22:33
  SPEAKER_00: It's hard to do that in certain studies.
• 22:35
  SPEAKER_00: That's why we do inclusion and exclusion criteria, even for a systematic review and meta-analysis.
• 22:40
  SPEAKER_00: But do confounders matter?
• 22:43
  SPEAKER_00: Yes, they do.
• 22:45
  SPEAKER_00: This is an example of looking at hormone replacement therapies, okay, and the adjustment for socioeconomic status.
• 22:55
  SPEAKER_00: When you don't, okay, when you don't adjust for it, you see that the data tends to fall on an odds ratio, in this case, a relative risk that's less than one, some of those data points being statistically significant, right?
• 23:18
  SPEAKER_00: Some that are not.
• 23:20
  SPEAKER_00: But then when you add in for adjustments for socioeconomic status, look what happens to the results.
• 23:30
  SPEAKER_00: Doesn't matter anymore, right?
• 23:32
  SPEAKER_00: It goes away.
• 23:33
  SPEAKER_00: And so I'm simply trying to help you to see that there is a lot that goes into understanding and interpreting the literature.
• 23:45
  SPEAKER_00: What does it mean?
• 23:46
  SPEAKER_00: Is it clinically relevant?
• 23:49
  SPEAKER_00: Did the researchers do their due diligence to reduce the risk and introduction of bias andor how much random error do we have being exposed?
• 24:00
  SPEAKER_00: So, core concepts for confidence intervals.
• 24:04
  SPEAKER_00: Study results should include a point estimate with a 95% confidence interval, typically a p-value with it, but though you should be able to interpret whether it's statistically significant or not.
• 24:16
  SPEAKER_00: Okay, remember, all p-values trying to help us understand its statistical significance.
• 24:20
  SPEAKER_00: Are you accepting or how likely is the null hypothesis true?
• 24:24
  SPEAKER_00: Remember, we want small p-values.
• 24:27
  SPEAKER_00: To be complete, you got to do that.
• 24:29
  SPEAKER_00: 95% confidence intervals reflect the range that include the true population result.
• 24:35
  SPEAKER_00: And that's why we desire small or very narrow interval confidence intervals.
• 24:42
  SPEAKER_00: Now, when you're assessing and interpreting those confidence intervals, if that includes a ratio, odds ratio, hazards ratio, relative risk, risk ratio, then remember if it includes 1.00 in that interval, it cannot be statistically significant.
• 24:58
  SPEAKER_00: It still might have clinical relevance and again open to debate, right, if you interpret p-values correctly and understand what they tell you, what they don't tell you.
• 25:09
  SPEAKER_00: But remember that small sample size, remember, it makes a wider confidence interval.
• 25:13
  SPEAKER_00: I showed you an example of that in one of the analysis where the blob is really small, the small blob reflects the sample size.
• 25:20
  SPEAKER_00: So large sample sides help us narrow in on the confidence interval, small sample sides make them wider.
• 25:26
  SPEAKER_00: Smaller sample sides also run the greater risk of not achieving sufficient power to find the result if in fact it's actually there.
• 25:35
  SPEAKER_00: All of those things, guys, go into our ability to properly interpret.
• 25:42
  SPEAKER_00: And that's why we don't all agree.
• 25:43
  SPEAKER_00: That's why there's debate.
• 25:44
  SPEAKER_00: That's why when guideline committees change out, sometimes recommendations change.
• 25:49
  SPEAKER_00: The way we use and interpret or what type of data is acceptable will shift.
• 25:54
  SPEAKER_00: So I hope you found this to be helpful.
• 25:57
  SPEAKER_00: I hope it cleared up a few things.
• 26:00
  SPEAKER_00: If you're still not fully clear, maybe watch it again.
• 26:03
  SPEAKER_00: Put it in the context of the other components of summary statistics and inferential statistics as it relates to this particular playlist because that together builds the foundation.
• 26:15
  SPEAKER_00: But I hope that you found it to be helpful.
• 26:16
  SPEAKER_00: I'm Dr.
• 26:17
  SPEAKER_00: Bussai and thank you so much for joining us.
• 26:19
  SPEAKER_00: If you're enjoying this content, please leave us a comment.
• 26:23
  SPEAKER_00: If there's something you want to contribute, please uh put that note in.
• 26:27
  SPEAKER_00: We'd love to uh all learn from each other, okay?
• 26:30
  SPEAKER_00: Um, and please give us a like and subscribe to the channel.
• 26:33
  SPEAKER_00: It helps to support uh us doing this and providing this kind of content for you.
• 26:38
  SPEAKER_00: Take care.