Latest / This Is Why with Dr. Busti / How to Interpret a Forest Plot in a Meta-Analysis

Transcript

How to Interpret a Forest Plot in a Meta-Analysis · This Is Why with Dr. Busti

• 0:00
  SPEAKER_00: You know, many clinicians can recognize a forest plot, but far fewer lack the confidence to interpret what it's actually telling them.
• 0:10
  SPEAKER_00: This is why forest plots matter, because they're often the single figure that summarizes the results of a meta-analysis.
• 0:19
  SPEAKER_00: But if you don't understand what the line of no effect, confidence intervals, pooled estimates, or heterogeneity actually mean, it's easy to draw the wrong conclusion from that evidence.
• 0:33
  SPEAKER_00: Therefore, a forest plot isn't just a collection of statistics, it's a visual summary of whether multiple studies consistently support or fail to support a clinical intervention.
• 0:49
  SPEAKER_00: Understanding how to interpret a forest plot allows you to critically evaluate the evidence rather than simply accepting the author's conclusions or even worse.
• 1:01
  SPEAKER_00: Practicing what I call abstract medicine, where clinicians think they understand the evidence simply because they read the abstract to the study.
• 1:11
  SPEAKER_00: On that note, I'm Dr.
• 1:12
  SPEAKER_00: Budsty, your host on This Is Why.
• 1:15
  SPEAKER_00: Welcome back to our series on systematic reviews meta-analysis, which is part of a larger overview of evidence-based medicine, biostatistics, and clinical literature evaluation.
• 1:27
  SPEAKER_00: And this is why my goal is to help healthcare professionals transform their medical knowledge from what to why, because when you know why you do what you do in the proper context of that evidence, it changes the how.
• 1:44
  SPEAKER_00: And that changes everything about the patient-oriented decisions that we make every day.
• 1:50
  SPEAKER_00: Now we're part of this series, I've covered a number of different topics depending on where you're coming in.
• 1:58
  SPEAKER_00: I recommend that if you are coming into this lecture first, that you stop and you go back and you watch the lecture that I have on what is a systematic review of meta-analysis.
• 2:12
  SPEAKER_00: You might assume that you know it, but do you know how it differs from a Cochrane review?
• 2:16
  SPEAKER_00: A review article, okay.
• 2:18
  SPEAKER_00: What are the pros and the cons?
• 2:22
  SPEAKER_00: The second topic that I cover is the steps.
• 2:25
  SPEAKER_00: How are you supposed to conduct one?
• 2:28
  SPEAKER_00: Okay, systematic review of meta-analysis, and what are the individual components of generating the statistical analysis to give us that point estimate that we're gonna look at today in the forest plot?
• 2:40
  SPEAKER_00: Do you understand the methods and the model?
• 2:43
  SPEAKER_00: Do you understand some of the basics of should you have combined the studies?
• 2:48
  SPEAKER_00: Tests of heterogeneity.
• 2:51
  SPEAKER_00: What about grade?
• 2:53
  SPEAKER_00: Okay, we're gonna focus on grade assessment a little bit more detail.
• 2:56
  SPEAKER_00: And then ultimately, did the publication report itself correctly?
• 3:02
  SPEAKER_00: Did they do this, the researchers do the study correctly?
• 3:04
  SPEAKER_00: And then we as the readers need to know how to critically appraise it.
• 3:08
  SPEAKER_00: So it is a stepwise progression, and I strongly encourage you to go back and watch those other ones so you have that foundation because I'm gonna refer to components in this lecture.
• 3:23
  SPEAKER_00: Now, where did all this stuff start?
• 3:27
  SPEAKER_00: I mean, probably the most famous group is Cochrane.
• 3:30
  SPEAKER_00: This logo is representative of the Cochrane group, a Cochrane collaboration historically.
• 3:38
  SPEAKER_00: And if you look at it, their logo is a forest plot, because they were instrumental in taking these studies and combining them in a systematic, reproducible way that generated important data points and perspectives.
• 3:58
  SPEAKER_00: Okay.
• 3:59
  SPEAKER_00: So what is that?
• 4:00
  SPEAKER_00: Well, there's two C's for C for Cochrane, but also to represent sort of the global perspective and their influence, and there's working groups all around the world that contribute to this.
• 4:15
  SPEAKER_00: But early on, this represents one of the first early meta-analysis that were done.
• 4:25
  SPEAKER_00: And you can see that the force plot, and you see the lines there, uh, represent studies.
• 4:32
  SPEAKER_00: And the first one was a randomized controlled trial back in 1972 looking at the benefit of steroids use in preterm uh labor and the impact on the outcomes of that baby, right?
• 4:50
  SPEAKER_00: And so as you go down through those lines, you see the lines crossing over the line of no effect, the vertical line, and then you see this diamond at the bottom.
• 5:00
  SPEAKER_00: Okay.
• 5:01
  SPEAKER_00: And at that, that's your point estimate that we're going to talk about in the forest plot.
• 5:07
  SPEAKER_00: And in this, they this meta-analysis said, look, look, um, the data is telling us that, you know what, there's a there's a chance that steroid use can be very beneficial.
• 5:22
  SPEAKER_00: And that led to then obviously studies that proved that point.
• 5:28
  SPEAKER_00: And that led to changes in standards of practice that have saved who knows how many people.
• 5:35
  SPEAKER_00: Right.
• 5:36
  SPEAKER_00: And so that's the basis and the rationale for generating now the more modern looking, obviously, and what in a real research paper publication looks like.
• 5:47
  SPEAKER_00: This meta analysis, I'm sorry, this for this meta-analysis, this is a forest plot.
• 5:52
  SPEAKER_00: Okay.
• 5:53
  SPEAKER_00: Now, obviously, a meta-analysis typically was done after the systematic review was done.
• 5:59
  SPEAKER_00: So there's that search process, how to include the studies, inclusion, exclusion criteria.
• 6:04
  SPEAKER_00: And you can see in this example of this forest plot, there were five studies included.
• 6:10
  SPEAKER_00: Okay, and I'm gonna walk you through it in detail.
• 6:13
  SPEAKER_00: We're gonna point out some of the landmarks of the forest plot so that you understand what the things are in there that are saying.
• 6:21
  SPEAKER_00: And if you lack some of the understanding about the model and the method, well then you need to go back to the key steps lecture that I did.
• 6:31
  SPEAKER_00: Okay, but let's just stop for a minute.
• 6:34
  SPEAKER_00: Where in the world did it get forest plot?
• 6:39
  SPEAKER_00: Well, there's a rumor that if you turn that thing, uh, it looks like I guess trees in a forest.
• 6:49
  SPEAKER_00: Yeah.
• 6:50
  SPEAKER_00: Anyway, uh that's sort of how it got its name from the rumors that I have understood.
• 6:57
  SPEAKER_00: Um, but let's get oriented to the individual components, okay?
• 7:02
  SPEAKER_00: So we zoom in on just the forest plot.
• 7:05
  SPEAKER_00: There's all that data to the left that's really important, and I'll come back to in a minute.
• 7:10
  SPEAKER_00: But what it's telling you a lot of information.
• 7:14
  SPEAKER_00: So get getting oriented.
• 7:16
  SPEAKER_00: That arrow right there is pointing to odds ratio.
• 7:21
  SPEAKER_00: That is telling you the very first step, uh, again, in doing a meta-analysis, part of the data analysis in a systematic review meta-analysis process.
• 7:32
  SPEAKER_00: What is the effect measure?
• 7:34
  SPEAKER_00: Is it an odds ratio?
• 7:36
  SPEAKER_00: Is it relative risk?
• 7:37
  SPEAKER_00: Is it a risk difference?
• 7:38
  SPEAKER_00: Is it a mean difference?
• 7:40
  SPEAKER_00: What is it?
• 7:41
  SPEAKER_00: Well, they're telling you it's an odds ratio in this one.
• 7:44
  SPEAKER_00: And then that is influenced by the type of data.
• 7:48
  SPEAKER_00: So there must be a numerator and a denominator.
• 7:51
  SPEAKER_00: Okay.
• 7:53
  SPEAKER_00: So it usually represents dichotomous endpoints, right?
• 7:57
  SPEAKER_00: So nominal data, yes or no.
• 8:00
  SPEAKER_00: And again, if you don't understand how to interpret the type of data, I have a whole lecture on how do you know something's nominal, ordinal, or continuous data if you need that help.
• 8:12
  SPEAKER_00: Then you see the model and the methods that were used by the investigators.
• 8:19
  SPEAKER_00: They're telling you.
• 8:20
  SPEAKER_00: Now, some of you may have just blown past all that in the past, but those what those that's what that is.
• 8:26
  SPEAKER_00: MH MH is one of the methods that is used in the mathematical weighting of each of these studies.
• 8:38
  SPEAKER_00: And we'll see the weight percent is part of that overall result.
• 8:44
  SPEAKER_00: And they followed a random effects model.
• 8:48
  SPEAKER_00: That's what random stands for.
• 8:50
  SPEAKER_00: So there's fixed effect model or there's random effect model, depending on how you interpret or assume the effect to be.
• 9:00
  SPEAKER_00: And then, of course, the 95% confidence interval.
• 9:04
  SPEAKER_00: Okay.
• 9:05
  SPEAKER_00: At the bottom, which you always need to look at, because this can change a lot.
• 9:13
  SPEAKER_00: On the right side, this is a uh ischemic stroke study looking at endovascular therapy after TPA.
• 9:22
  SPEAKER_00: So you can see on the right side of the vertical line, it favors endovascular therapy.
• 9:28
  SPEAKER_00: And you see that almost all the data is falling on the side of favor, and the point estimate that we'll talk about here in a minute is also on that side.
• 9:37
  SPEAKER_00: But you've got to be oriented because they'll flip that.
• 9:43
  SPEAKER_00: So the right and the left side of the line may be favorable, unfavorable, depending on what it is.
• 9:49
  SPEAKER_00: So always, always look at that.
• 9:52
  SPEAKER_00: And that line, that vertical line in the middle is the line of what I call no effect.
• 9:59
  SPEAKER_00: Okay, it's really where there's no statistical significance.
• 10:03
  SPEAKER_00: It's the null.
• 10:05
  SPEAKER_00: So if if a line crosses over and through that, a confidence interval, then that study's overall effect was not statistically significant.
• 10:19
  SPEAKER_00: So you can just visually look at that line, and the lines of those confidence intervals, 95% confidence intervals, and you can see that with the exception of the first one, which if you zoom in really close, is technically not touching that line of no effect.
• 10:36
  SPEAKER_00: Every single one of those studies favors endovascular therapy.
• 10:43
  SPEAKER_00: And all of them had statistical analysis that were statistically significant results in their odds ratio.
• 10:53
  SPEAKER_00: So the odds ratio, remember a numerator denominator, which would imply that if the the numerator and the denominator were the same number, then that would be one.
• 11:03
  SPEAKER_00: That's a line of no effect.
• 11:06
  SPEAKER_00: So that line would represent one.
• 11:08
  SPEAKER_00: If it's a difference in measures, right, like mean difference, right, then that would be zero.
• 11:17
  SPEAKER_00: And they tell you down here at the bottom, they're telling you to reinforce that concept, it's one.
• 11:23
  SPEAKER_00: So they're right.
• 11:24
  SPEAKER_00: You should look to make sure they're right.
• 11:28
  SPEAKER_00: That's what it means.
• 11:30
  SPEAKER_00: Okay.
• 11:33
  SPEAKER_00: Now, let's look at the individual studies.
• 11:36
  SPEAKER_00: So you see those little squares, they're called blobs.
• 11:40
  SPEAKER_00: Okay, and they represent that individual study's point estimate.
• 11:48
  SPEAKER_00: Obviously, each study has a overall degree of influence on this, which is the average all of those things combined, all the data.
• 12:00
  SPEAKER_00: Okay, and the size or the that's the area of that blob or square is influenced by the study's weight as it influenced on this pooled estimate.
• 12:16
  SPEAKER_00: And they published the weight.
• 12:18
  SPEAKER_00: It's a percent.
• 12:20
  SPEAKER_00: And you should look at it, it should be consistent with what you visually see.
• 12:25
  SPEAKER_00: So if I just look at this diagram, I see that the where the arrow is pointing has the largest square.
• 12:33
  SPEAKER_00: My assumption then is, and the interpretation of this is that that study has a greater weight of influence on the pooled estimate.
• 12:45
  SPEAKER_00: And interestingly, if you look at where the pulled estimate is, it's lining up.
• 12:51
  SPEAKER_00: Okay.
• 12:54
  SPEAKER_00: What about those lines going around on each side?
• 12:56
  SPEAKER_00: Well, that's the 95% confidence interval.
• 13:00
  SPEAKER_00: And obviously, the wider that is, the wider the confidence interval.
• 13:05
  SPEAKER_00: But interestingly, what do we typically see with small weighted or small sample size studies?
• 13:14
  SPEAKER_00: Like this one.
• 13:15
  SPEAKER_00: We see wide confidence intervals, don't we?
• 13:18
  SPEAKER_00: So the more people we include in the study, the confidence interval begins to narrow.
• 13:24
  SPEAKER_00: And so that visual should line up with the reality of those individual studies' data.
• 13:33
  SPEAKER_00: And so when you now add that information to the left, it makes sense.
• 13:40
  SPEAKER_00: So in this, for example, Mr.
• 13:42
  SPEAKER_00: Clean trial is the largest trial.
• 13:46
  SPEAKER_00: Look at the number of patients, 267.
• 13:49
  SPEAKER_00: And it carries the largest percentage of weight towards the total, which is this point estimate down here.
• 13:59
  SPEAKER_00: Okay.
• 14:00
  SPEAKER_00: 34% of that point estimate is being weighted by Mr.
• 14:07
  SPEAKER_00: Clean.
• 14:08
  SPEAKER_00: So we know it's the biggest, not just visually by the blah, but we know it's visually the biggest because it tells us.
• 14:15
  SPEAKER_00: I mean, the numbers say that.
• 14:21
  SPEAKER_00: Okay, now where did the weight come from?
• 14:24
  SPEAKER_00: Again, that's in the lecture on the key steps, but that's the m-h.
• 14:31
  SPEAKER_00: That's the method of how the data was weighted in the calculation.
• 14:36
  SPEAKER_00: And then they did a random effects model.
• 14:43
  SPEAKER_00: Okay, so you can see the largest trial.
• 14:46
  SPEAKER_00: What about the smallest trial?
• 14:48
  SPEAKER_00: Well, it's smallest blob, and again, you can see the confidence interval is huge, but it's still statistically significant.
• 14:57
  SPEAKER_00: Why is it so small?
• 14:58
  SPEAKER_00: There were only 35 people in this group.
• 15:00
  SPEAKER_00: Well, because in this particular topic or question clinically, when this paper Extend 1A was being conducted, the other studies in this area or topic were published.
• 15:15
  SPEAKER_00: And it was clear that endovascular therapy was a benefit to patients, that we should no longer debate or discuss it.
• 15:26
  SPEAKER_00: It was unethical to continue any other studies to answer that question.
• 15:31
  SPEAKER_00: And so that study was stopped prematurely and thus is the smallest sample size.
• 15:36
  SPEAKER_00: It's not because the researchers of Extend 1A didn't know what they were doing, didn't want to finish the study or whatever.
• 15:43
  SPEAKER_00: So that small blob and the wide confidence interval that you see for the smallest trial is simply because it was stopped early.
• 15:52
  SPEAKER_00: I know that.
• 15:53
  SPEAKER_00: You just read the data, but you may not know that, you would know why it's so small from just reading a forest plot, but I know that.
• 16:02
  SPEAKER_00: So I'm adding context to it.
• 16:05
  SPEAKER_00: But the whole point of a meta-analysis, right, that data analysis assessment is to create the pooled estimate.
• 16:12
  SPEAKER_00: Sometimes people use it as the overall summary effect.
• 16:16
  SPEAKER_00: Okay.
• 16:18
  SPEAKER_00: And so that's the diamond at the bottom.
• 16:22
  SPEAKER_00: Okay, and where it falls, right, and its width is influenced by its the impact that it has, right?
• 16:32
  SPEAKER_00: The precision.
• 16:34
  SPEAKER_00: So when you look at the individual components of the diamond, right, of the pooled estimate, you have the point estimate, which is right in the middle.
• 16:46
  SPEAKER_00: Where is that point estimate landing in that y on that x-axis down at the bottom in numbers?
• 16:55
  SPEAKER_00: And then how wide is this diamond?
• 16:59
  SPEAKER_00: That's your confidence interval.
• 17:02
  SPEAKER_00: Okay.
• 17:03
  SPEAKER_00: So again, as you add more people, that diamond becomes tighter and more bulky.
• 17:11
  SPEAKER_00: So if you look at pulled estimate number one over here on the left, which is smaller, versus pulled result uh estimate on the right, which is very wide, what would you say?
• 17:26
  SPEAKER_00: Well, I could I could simply say that, well, pulled one results one have a narrow, more narrow confidence interval.
• 17:37
  SPEAKER_00: Um and versus pool, I'd rather have that one than the than pool result two.
• 17:44
  SPEAKER_00: And so you you start to see these patterns in the visuals that the amount of wealth of information that these diagrams and plots give us, they orient us to so many things that the researcher has done for you in the systematic review meta-analysis, but you should still be able to interpret and understand what those labels mean and if they were done right.
• 18:11
  SPEAKER_00: Now, if you look at the bottom, you see the other part of the step when you after you've calculated it, not this isn't done part of the a priori process, this is done at the end, right?
• 18:24
  SPEAKER_00: Well, you should you have combined those studies.
• 18:28
  SPEAKER_00: So these are tests of heterogeneity, or you get the tau squared, and in this case it's zero, you get the chi squared or the Cochrane's Q, which is 2.93, and you compare it to the degrees of freedom, which again just I'll dive into in a minute here, and you get your p-value.
• 18:49
  SPEAKER_00: And then the lastly is the i squared.
• 18:51
  SPEAKER_00: And again, I I walk through those statistics in my lecture on the key steps of doing a meta-anal systematic review meta-analysis.
• 19:04
  SPEAKER_00: Okay, but you're really the they're trying to tell you is this data heterogeneous or not?
• 19:12
  SPEAKER_00: Okay, so let's let's zoom in further.
• 19:16
  SPEAKER_00: Okay, so I zoomed in, put that data up here at the top for you so you can see it clear.
• 19:25
  SPEAKER_00: And again, what are we doing here?
• 19:29
  SPEAKER_00: What is the Cochrane Q or the Chi-squared uh test of heterogeneity doing?
• 19:35
  SPEAKER_00: Well, it's trying to determine if the tests, uh, if the study results differ by more than expected by chance alone.
• 19:46
  SPEAKER_00: And that df or degrees of freedom, that's the number equals the number of studies, which in our case was five.
• 19:53
  SPEAKER_00: Okay.
• 19:54
  SPEAKER_00: So just this case was five.
• 19:56
  SPEAKER_00: And so there you subtract one from that, and so your degree of freedom up here is four.
• 20:04
  SPEAKER_00: Well, why do you need that?
• 20:07
  SPEAKER_00: Well, you use that to compare with the um chi-square or Cochrane Q, and when that Cochrane Q is less than the DF, right, typically the heterogeneity is very low or zero.
• 20:30
  SPEAKER_00: Okay, and in that case, it doesn't matter whether you use a fixed effects model or random effects model, you should still get the same point estimate, and that's about sensitivity analysis, which I'm gonna talk about here in a minute.
• 20:43
  SPEAKER_00: Okay, because you're trying to discern is the is the result that I'm seeing or the conclusions I'm finding, is it holding up if I put it under pressure?
• 20:53
  SPEAKER_00: If I change the circumstances, does it still say the same story?
• 20:59
  SPEAKER_00: So that's where all this is coming in.
• 21:02
  SPEAKER_00: Understanding what does the what do the studies look like?
• 21:05
  SPEAKER_00: What are the effects from these studies saying?
• 21:08
  SPEAKER_00: Is there a difference amongst them?
• 21:11
  SPEAKER_00: Okay.
• 21:12
  SPEAKER_00: What about that I squared statistics?
• 21:15
  SPEAKER_00: Again, it tells you, it doesn't tell you why the studies differ, it just says there is some difference.
• 21:20
  SPEAKER_00: Well, in this case, there's not, right?
• 21:24
  SPEAKER_00: And we saw that.
• 21:24
  SPEAKER_00: I mean, when you look at the force plot I showed you, all those studies showed favorable benefit, and they were all statistically significant in that particular result.
• 21:38
  SPEAKER_00: Why might a high percentage result?
• 21:41
  SPEAKER_00: Well, that can be from um different doses that were used, different populations that are not represented of the same.
• 21:50
  SPEAKER_00: There's maybe some bias, okay, which should have been assessed.
• 21:55
  SPEAKER_00: There's a bias in the outcome, their outcome definitions are different.
• 22:00
  SPEAKER_00: Right?
• 22:01
  SPEAKER_00: And so the higher that number gets, the greater the degree of heterogeneity.
• 22:08
  SPEAKER_00: And you know, the Cochran Handbook has a percent breakdown for these things.
• 22:13
  SPEAKER_00: They're just guides to try to give us something to wrap our minds around.
• 22:19
  SPEAKER_00: Is there a lot of this?
• 22:20
  SPEAKER_00: Is there a little bit of this?
• 22:21
  SPEAKER_00: Is this considerable enough to be concerned or not?
• 22:25
  SPEAKER_00: So they're not meant to be absolutes or black and white cutoffs.
• 22:30
  SPEAKER_00: It's meant to help us to understand this as we move into some of this sensitivity analysis.
• 22:37
  SPEAKER_00: So let's pull this together.
• 22:39
  SPEAKER_00: Okay.
• 22:41
  SPEAKER_00: Does endovascular thrombectomy add to standards of care?
• 22:46
  SPEAKER_00: What is a standard of care?
• 22:48
  SPEAKER_00: Well, you give the patient that qualifies for acute ischemic stroke TPA.
• 22:54
  SPEAKER_00: If we do that and we go on and give them and go do endovascular therapy, is there a benefit?
• 23:02
  SPEAKER_00: Okay, well, this is the first score up here is the modified ranking score.
• 23:09
  SPEAKER_00: This is a score of functionality.
• 23:11
  SPEAKER_00: And we see again, favors endovascular therapy.
• 23:14
  SPEAKER_00: We see that point estimate on the side of favor, and it doesn't cross the line of no effect or the null.
• 23:23
  SPEAKER_00: We know that's true.
• 23:24
  SPEAKER_00: We can just look at the odds ratio.
• 23:26
  SPEAKER_00: The 95% confidence interval does not go through and include 1.0.
• 23:34
  SPEAKER_00: So it's statistically significant.
• 23:37
  SPEAKER_00: Now remember, statistical significance and clinical relevance are two different things.
• 23:43
  SPEAKER_00: But look what happens when you look at 90-day mortality.
• 23:47
  SPEAKER_00: First of all, get oriented all over again.
• 23:51
  SPEAKER_00: Different outcome, different type of data.
• 23:55
  SPEAKER_00: Remember, up here we're doing odds ratios.
• 23:58
  SPEAKER_00: Down here, they're doing risk ratios.
• 24:01
  SPEAKER_00: So remember what the end point that you're talking about, you have to get the data to be the same.
• 24:08
  SPEAKER_00: Next thing that you have to do in getting oriented here is important is they put favors into vascular therapy on the other side, which is frustrating because that creates confusion.
• 24:19
  SPEAKER_00: I don't like that.
• 24:20
  SPEAKER_00: I think it's poor representation.
• 24:23
  SPEAKER_00: Because now I make an assumption that you're staying over on the right versus the left.
• 24:29
  SPEAKER_00: But you can see that now this pool decimant is not only wider, but is also crossing the line of no effect or non-significance or the null.
• 24:43
  SPEAKER_00: Because it includes one, it's a risk ratio.
• 24:45
  SPEAKER_00: So the numerator and denominator are the same.
• 24:48
  SPEAKER_00: Guess what?
• 24:49
  SPEAKER_00: Now the question is why?
• 24:53
  SPEAKER_00: Why do I have a benefit in the modified Rankin score, but I don't have it in the mortality?
• 24:58
  SPEAKER_00: Well, it depends on how the study was designed and powered, isn't it?
• 25:02
  SPEAKER_00: But you can see that for all the studies, looking at mortality, with the exception of escape, is the only one that had a statistically significant finding.
• 25:13
  SPEAKER_00: The rest are all crossing the line of no significance or the null.
• 25:19
  SPEAKER_00: So what is this sensitivity analysis?
• 25:23
  SPEAKER_00: Again, you want to see if the results are holding up under, I don't know, pressure.
• 25:29
  SPEAKER_00: If you change the variables around a little bit, what conclusion do you get?
• 25:33
  SPEAKER_00: Does it change something?
• 25:36
  SPEAKER_00: Right?
• 25:36
  SPEAKER_00: If it survives and shows the same pattern, then there's some degree of strength in the truth of that.
• 25:46
  SPEAKER_00: Okay, so that's basically rerunning the analysis under different reasonable decisions to see if those conclusions still remain the same.
• 25:58
  SPEAKER_00: What about running a different model?
• 26:01
  SPEAKER_00: Going from random effects model to the fixed effects model, and vice versa.
• 26:07
  SPEAKER_00: Does it what happens to the results?
• 26:09
  SPEAKER_00: Well, in this case, remember our tau squared was zero.
• 26:14
  SPEAKER_00: So in our case, really it isn't gonna change it very much.
• 26:19
  SPEAKER_00: So really, when you don't have heterogeneous data, especially at the endpoint you're talking about, it's not really gonna change that.
• 26:27
  SPEAKER_00: But it's still good to ask the question.
• 26:29
  SPEAKER_00: It's still still good to think about it.
• 26:32
  SPEAKER_00: What are some other things?
• 26:34
  SPEAKER_00: Well, you can re remove high-risk bias trials.
• 26:38
  SPEAKER_00: That's what you were supposed to do before you even got to the analysis.
• 26:43
  unknown: Okay.
• 26:45
  SPEAKER_00: Remove the largest trial, the one that's creating the greatest weight.
• 26:49
  SPEAKER_00: Switch from odds ratio to relative risk.
• 26:53
  SPEAKER_00: Okay.
• 26:54
  SPEAKER_00: All of these things can be done to look for and see if the conclusion sustain itself.
• 27:03
  SPEAKER_00: Okay.
• 27:05
  SPEAKER_00: The other thing that you want to look for is is a publication bias.
• 27:10
  SPEAKER_00: And you plot each study is against its precision.
• 27:12
  SPEAKER_00: And if you see asymmetry, it may suggest small study effects or publication bias being present.
• 27:21
  SPEAKER_00: Now you typically need more studies than 10, typically, uh, to do that.
• 27:29
  SPEAKER_00: Okay.
• 27:31
  SPEAKER_00: Our stroke example only at five trials, so it's not meaningful.
• 27:38
  SPEAKER_00: They're all showing positive.
• 27:42
  SPEAKER_00: Are there any out there that were negative?
• 27:45
  SPEAKER_00: Maybe they didn't make it.
• 27:47
  SPEAKER_00: Okay.
• 27:48
  SPEAKER_00: So the core concepts are to start with the um header, right?
• 27:54
  SPEAKER_00: What is the effect measure?
• 27:56
  SPEAKER_00: What was the method and the model that were used?
• 27:59
  SPEAKER_00: Look at the direction of your labels at the bottom.
• 28:02
  SPEAKER_00: Find the line of no effect.
• 28:04
  SPEAKER_00: Is it zero, one?
• 28:05
  SPEAKER_00: Is it line up with the effect measure?
• 28:08
  SPEAKER_00: Is it consistent?
• 28:10
  SPEAKER_00: Look at those individual studies.
• 28:12
  SPEAKER_00: Look at their blob, look at their confidence interval, where are they sitting?
• 28:16
  SPEAKER_00: Get the visual, make an interpretation about the result without looking at the point estimate.
• 28:23
  SPEAKER_00: Make your own assessment about what do you think the heterogeneity is?
• 28:28
  SPEAKER_00: So when you look at those results like Cochrane's Q, the I squared or tau squared, does it make sense?
• 28:38
  SPEAKER_00: Okay.
• 28:40
  SPEAKER_00: Was this sensitivity analysis done?
• 28:43
  SPEAKER_00: And if so, what did it show?
• 28:44
  SPEAKER_00: Did it hold up?
• 28:47
  SPEAKER_00: Those are things that are good for a critical appraisal.
• 28:51
  SPEAKER_00: Now there are tools that help us to do that.
• 28:55
  SPEAKER_00: And I do a lecture that covers not only how should a systematic review meta-analysis have been published, but also how do you appraise it.
• 29:07
  SPEAKER_00: Okay.
• 29:08
  SPEAKER_00: So I hope this episode in particular was helpful in pulling some things together, especially those of you that have already done the other two.
• 29:16
  SPEAKER_00: And understanding forest plots, how to interpret them, what do some of the things mean?
• 29:22
  SPEAKER_00: Understanding strength, consistency, clinical relevance of the evidence that are being made behind those conclusions.
• 29:30
  SPEAKER_00: Because as clinicians, we're expected to apply the best available evidence to patient care.
• 29:35
  SPEAKER_00: And learning how to quickly interpret a forest plot is an essential skill that helps you to decide whether meta-analysis should influence your clinical decisions or not.
• 29:46
  SPEAKER_00: So thank you for joining us on This Is Why.
• 29:49
  SPEAKER_00: This episode again is a part of our systematic review meta-analysis playlist, where we're building on the skills needed to critically appraise the highest levels of medical evidence.
• 30:00
  SPEAKER_00: And you can find a complete playlist at YouTube, also on our podcast channels, and over at thisiswhy.health.
• 30:06
  SPEAKER_00: I'm Dr.
• 30:07
  SPEAKER_00: Bustaye, and thanks again for joining me on this episode.
• 30:10
  SPEAKER_00: We'll see you next time.