Latest / Cardio Pod: Heart of the Matter / Contemporary Challenges in HF Clinical Trial Design

Transcript

Contemporary Challenges in HF Clinical Trial Design · Cardio Pod: Heart of the Matter

• 0:00
  Welcome everyone.
• 0:06
  I'm Chris O'Connor.
• 0:07
  I'm the principal investigator of the Heart Failure Collaboratory and we're here with
• 0:12
  Heart of the Matter podcast and I'm very excited to be with my colleagues today to talk about
• 0:18
  something really important.
• 0:19
  A meeting we recently had about statistical considerations in clinical trials.
• 0:24
  Now you may think, wow, that's going to be a dense, boring topic, but each and every
• 0:29
  one of you wake up every morning and use predictive analytics to get your day started because
• 0:34
  you look at the weather and you look at the probability of rain and you're integrating
• 0:39
  a lot of information into how you may be moving around, transporting yourself, using a lot
• 0:45
  of analytics.
• 0:46
  We're going to talk about how we look at clinical research, clinical trials, and cardiology.
• 0:52
  And before we get started, I'm going to ask my colleagues to introduce themselves and
• 0:56
  tell a little bit about their background.
• 0:58
  We'll start with Dr. Mona Fuzan.
• 1:00
  Hi everyone.
• 1:01
  I'm Mona Fuzan.
• 1:02
  I'm a pharmacist by training and associate professor of medicine at Duke University.
• 1:07
  We know that you've served as a drummer in a band, so you don't have to introduce that,
• 1:11
  but let's have Dr. Janet Wittes.
• 1:13
  Hi, I'm Janet Wittes.
• 1:15
  I'm the statistician.
• 1:17
  What's new about me right now is that I've worked for 50 years in large organizations,
• 1:23
  at small organizations, and now I'm an organization of one and I'm trying to decide how life
• 1:28
  goes on after one sort of retires.
• 1:31
  Janet, you never retire because I've seen you recently at a cardiorenal advisory panel
• 1:36
  meeting and 50 years, you must have started a statistician at age five.
• 1:43
  No, two.
• 1:44
  Two, okay.
• 1:45
  Eric Leifer.
• 1:46
  Hi, I'm Eric Leifer.
• 1:47
  I'm a statistician in the National Heart, Lung, and Blood Institute where I've been
• 1:52
  for about 22 years and I see a wide variety of trials.
• 1:56
  I worked with Dr. O'Connor on heart failure trials early in my career and kept it going.
• 2:01
  It's been a great experience.
• 2:02
  I guess for something unique, I'm actually still playing ice hockey, but now the people
• 2:07
  seem to be about half my age.
• 2:08
  Wow, so you understand risk and you understand hazard, so this is terrific conversation.
• 2:16
  We talk about your survival curve on the ice.
• 2:19
  Bill Abraham?
• 2:20
  Yeah, Bill Abraham.
• 2:21
  I'm a heart failure cardiologist at The Ohio State University and a clinical trialist.
• 2:26
  Today's topic is very important to me and I think to my patients.
• 2:31
  And today I'm going to go trick or treating with the kids as the wise old Al from the
• 2:37
  old Tootsie Pop commercial, if you remember that.
• 2:40
  Mitch?
• 2:41
  I'm Mitch Sotka.
• 2:42
  I'm a heart failure and transplant cardiologist at NOVA Health System in Falls Church, Virginia.
• 2:46
  And I will be Batman, so will my son tonight, for trick or treating.
• 2:51
  Oh, my Lord.
• 2:52
  This is amazing.
• 2:53
  Mitch, let's start with you.
• 2:55
  We dedicated a large part of one of our think tanks to talking about hazard ratios and hazard
• 3:01
  ratio drift.
• 3:02
  For the audience, what is a hazard ratio and why are we so interested in hazard ratios
• 3:09
  and components of hazard ratios?
• 3:11
  Fascinating.
• 3:12
  So hazard ratios are mainly in use and I'll defer to my statistical colleagues to truly
• 3:16
  answer this, but are mainly in use because of their statistical properties.
• 3:20
  They allow us to do lots of things in terms of the analysis and in terms of how we can
• 3:25
  treat them.
• 3:26
  They remain to have some statistical presence, but I think people find them useful because
• 3:31
  they get a sense that it talks about overall relative sorts of risks to people when they
• 3:36
  get treated with some sort of therapeutic and how they're going to respond to that.
• 3:40
  It lets people take that information home.
• 3:42
  I think that when we start thinking about what they mean and need to communicate that
• 3:48
  information to patients, it gets a little more difficult.
• 3:51
  And so I'm not sure that a patient understands when I say that there is a hazard ratio of
• 3:55
  0.9 for this treatment in a patient with heart failure that they know what that means.
• 4:00
  That wait, I'm 90% less likely to get a benefit or I'm 10% likely to do well over time or
• 4:08
  how much time?
• 4:10
  What does that mean if I take that for four years?
• 4:12
  What is the cost benefit?
• 4:15
  Am I 90% better?
• 4:17
  I think when you start delving into the details of how we communicate these patients, no one
• 4:21
  really knows what it means.
• 4:22
  And I think when you start talking about absolute benefits to patients over time or absolute
• 4:27
  benefits over a specified period of time, like five or 10 years, that becomes then much
• 4:32
  more likely to be something that patients can understand.
• 4:36
  To take it just a little bit further, when you start looking at the difference between
• 4:40
  relative benefits and absolute benefits, the absolute benefits are always much smaller.
• 4:45
  So industry sponsors tend to like relative benefits because it makes their therapeutic
• 4:49
  look better because you can say you have a relative risk reduction of 20, sometimes 30
• 4:54
  or even 40 or 50% for an outcome that you care about, such as dying or being hospitalized.
• 4:59
  But when you start looking at an absolute benefit, you start seeing things like a half
• 5:03
  a percent difference or a 1% difference over a year or two years in terms of an event,
• 5:08
  meaning you start seeing, well, one out of 100 patients maybe will live a bit longer
• 5:13
  with this, or if you use what's called the restricted mean survival time, which is another
• 5:18
  way of analyzing trial data that can tell you, you know, for a patient who has treated
• 5:22
  with this medication for five years, a patient treated with this is likely to have eight
• 5:27
  months longer life than a patient not.
• 5:30
  And I think that is something that patients can better understand, but is not typically
• 5:33
  used because it shows a quote unquote smaller otherwise treatment effect than others are
• 5:38
  used to.
• 5:39
  Dr. Justin Marchegiani We gotta unwrap a lot of that.
• 5:41
  There's a lot in there, Janet.
• 5:43
  Let's go back to I gotta dummy this down for myself.
• 5:46
  Hazard ratio is the probability of an event with a treatment divided by the control and
• 5:53
  integrates time.
• 5:54
  But explain to me and the difference between that and odds ratio and risk ratio, and then
• 5:59
  we'll get to we'll go around the table, but talk about time.
• 6:02
  Dr. Janet Levy All right, I think it's reasonable, Mitch,
• 6:06
  that patients don't understand what a hazard ratio is.
• 6:10
  Because what it's trying to do is integrate over time, the relative risk of an event.
• 6:17
  And that's really hard concept.
• 6:19
  And actually, what you're trying to do is describe what happens to people over time,
• 6:26
  one treatment compared to another, and summarize all of that complicated information into one
• 6:31
  number.
• 6:33
  And so it's really hard to know what that one number needs.
• 6:38
  I think what Mitch is describing in terms in terms of restricted means of vital time
• 6:43
  things that people can relate to it much more interpretable.
• 6:47
  My own feeling is the best way to look at the data is to show survival curves and hard
• 6:54
  failure.
• 6:55
  That's what we're interested in.
• 6:56
  What happens over time, and to show the survival curve the time to event in one treatment versus
• 7:03
  another and describe it.
• 7:05
  What happens at one year?
• 7:06
  What happens at two years?
• 7:07
  What happens at three years?
• 7:08
  And not try to summarize it in terms of hazard ratio.
• 7:12
  I agree that hazard ratio has nice statistical properties, but that's not sufficient for
• 7:20
  patients understanding it, for physicians understanding it, or I dare say even a statistician's
• 7:26
  understanding.
• 7:27
  That is a really puts the crux at this question, Bill.
• 7:31
  And you have a lot of patients.
• 7:33
  You've been in a lot of clinical trials.
• 7:36
  Tell us how you communicate the advantage of one therapy over another.
• 7:40
  Yeah, I think this is really the key, right?
• 7:43
  Clinicians want to understand this.
• 7:45
  And of course, we want to be able to communicate it as clearly as possible to our patients.
• 7:49
  So I think it's important to look at the totality of the data, certainly a primary endpoint
• 7:54
  with a time to event analysis and looking at a hazard ratio provide some information.
• 8:00
  But there are always other elements of clinical trials that tell us about how one treatment
• 8:06
  fares against another, for example, looking at quality of life scores, exercise ability,
• 8:13
  other patient centered or patient reported outcomes are very important as well.
• 8:18
  So I think you have to have a fairly holistic discussion with your patients.
• 8:21
  Now, I do try to put this in the context when I think about hazard ratios, what that means
• 8:27
  over some average period of follow up, what's the likelihood that you'll do better with
• 8:33
  this new therapy.
• 8:35
  In that regard, I usually do cite relative risks, just to give them some impression of
• 8:41
  what they might expect.
• 8:43
  It's not very precise.
• 8:45
  I think the limitations of that approach have already been explained by others, but at least
• 8:49
  gives patients some feeling for whether or not they're more or less likely to be improved
• 8:55
  and in some relative sense by how much.
• 8:58
  But I think that discussion in isolation is limited and talking about how much better
• 9:03
  they might feel on average, how much better they may be able to exercise, but they're
• 9:09
  going to be able to climb that flight of stairs that they can't climb today.
• 9:13
  For example, are they going to be able to perform more of their usual activities of
• 9:18
  daily living?
• 9:19
  And again, that only comes, I think, from a look at the totality of the data.
• 9:23
  No single number here tells the whole story.
• 9:26
  I think what you just said there, no single number tells the whole story.
• 9:30
  I like what you and Jana said is let's try to get the totality of all the information
• 9:36
  in there.
• 9:37
  And Eric, you've looked at a lot of data and we come in and we're proud.
• 9:41
  We tell you that we've got a therapy that reduces hard clinical outcomes, bad outcomes
• 9:47
  by 25%.
• 9:49
  We're proud of that.
• 9:50
  Mitch calculates the restrictive mean survival time and it turns out to be 60 days and we
• 9:56
  go, oh my gosh, this big expensive therapy and we only get 60 days of additional survival.
• 10:02
  How does that work out?
• 10:05
  How can you have such a big effect size by hazard ratio, but then such limited survival
• 10:10
  time?
• 10:11
  Well, that's an excellent question.
• 10:12
  And as Mitch said, what it really comes down to is looking at the relative reduction in
• 10:18
  the event rate versus an absolute reduction in the event rate.
• 10:21
  So if you start off with, for instance, something that's very low, say mortality, at that rate
• 10:27
  in absolute terms is very low.
• 10:29
  Even if you only have a 50% reduction, it's still going to be in absolute terms a small
• 10:35
  relative reduction.
• 10:36
  Nevertheless, I think the hazard ratio does give information about which treatment really
• 10:42
  does seem to be better.
• 10:43
  And remember, it is an average.
• 10:45
  So there's 60 days at your site.
• 10:47
  I mean, certainly there'll be certain patients who would benefit much more than 60 days.
• 10:51
  And so sometimes we just really need to drill into getting into certain subgroups of patients
• 10:56
  to get a better idea about the absolute reductions of bad event rates.
• 11:01
  So I think that having a single number can be useful, but it needs to be supplemented
• 11:06
  by talking about absolute event rates, looking at maybe one year out or two years out and
• 11:12
  looking at differences there.
• 11:14
  And even if the differences aren't that impressive, at least it gives physicians and patients
• 11:19
  some information as to what might be a better path to go on, at least at the start of their
• 11:23
  treatment and then going forward.
• 11:25
  Terrific.
• 11:26
  And so Dr. Fusata, if you had a drug A improve survival by 50% versus drug B, but the mean
• 11:35
  restricted survival time was 30 days and this drug costs $100,000 a year, do you think society
• 11:42
  should go forth with it?
• 11:44
  I'm an advocate for hope for patients and continued development in various therapeutic
• 11:51
  areas.
• 11:52
  So we've been a small advantage for me is important, but along the lines of what Bill
• 11:56
  was saying, I think as a pharmacist, one of the things that we often discuss with patients
• 12:02
  includes this kind of trade-off concept.
• 12:06
  So we have to be aware of how much benefit you may gain and what risks you might take
• 12:12
  to gain that benefit.
• 12:14
  And so I think Bill said that looking at that whole picture is really important, much more
• 12:20
  so than just the benefit alone.
• 12:23
  And again, the cost is something that isn't even in the equation when we talk about the
• 12:28
  risk and benefit ratios in numbers.
• 12:31
  So that's a very important point as well as do I really need to take another medicine,
• 12:36
  which can throw off patients entirely.
• 12:38
  So there are a lot of other issues outside of these statistical concepts that we have
• 12:44
  to consider.
• 12:45
  But I do think even small benefits, it's not been that popular.
• 12:49
  Some of the newer FDA approvals recently, but I think it's important for patients to
• 12:54
  have an option.
• 12:56
  And like Eric said, for some patients it works and for some patients it may not.
• 13:00
  But having that hope, especially in areas where there are very few options, is really
• 13:05
  important.
• 13:06
  Maybe what we're saying is that it would also help to know who gets the home run from benefit
• 13:12
  versus who gets the single.
• 13:13
  And when we look at this mean number, we need to delve in more.
• 13:18
  And maybe that's the way we can look at the totality of the information.
• 13:22
  One thing that happens in long trials of treatment A versus treatment B is those effect sizes
• 13:30
  begin to narrow, Janet.
• 13:33
  And we've talked about this concept called hazard ratio drift.
• 13:37
  What does that mean?
• 13:38
  Is that this number that we calculate blows in the wind and tumbles around and gets smaller
• 13:44
  because the trial is longer?
• 13:46
  It can get smaller or it can get larger.
• 13:49
  If you have a study that's a one year study and you calculate the hazard ratio, you'll
• 13:55
  get a number.
• 13:57
  If you have the same two treatments, treatment and control, and you look at it for two years
• 14:04
  and calculate the hazard ratio, you may get a different number.
• 14:08
  And if you look at the same for three years, you may still get a different number.
• 14:13
  And that's because the hazard ratio drifts.
• 14:17
  It changes over time.
• 14:19
  Why does it change over time?
• 14:21
  In some cases, maybe this doesn't happen in heart failure, but in lipid lowering trials,
• 14:26
  the farther apart the curves get, the better the hazard ratio is.
• 14:31
  And so the drift is in the direction of better and better, stronger and stronger evidence
• 14:37
  for benefit.
• 14:38
  In studies in which people drop off their therapy or the therapy stops working so well
• 14:44
  over time, then the curves come together in time, the hazard ratio gets lower and this
• 14:51
  drift in the other direction.
• 14:53
  And that's why I think we're all saying that one number doesn't tell the story.
• 14:59
  You have to look at the patterns over time and interpret them the best you can.
• 15:05
  I do have to say something about what Chris said.
• 15:08
  Obviously, what one wants to know is which patients benefit and which patients don't.
• 15:14
  I think that's a very, that's very hard to lean from clinical trials.
• 15:19
  We wish we could.
• 15:20
  It's always a mixture of all these things of who benefits and doesn't.
• 15:24
  It's never one or zero, it seems like.
• 15:27
  Mitch and Bill, I'm going to ask you as clinical trialists, how in the world do you know if
• 15:33
  your drift is going to going out like this, as Janet says, or coming in like this?
• 15:39
  You have to have some forecasting vision of what's going on with your therapy, right,
• 15:47
  Mitch?
• 15:48
  I think that this comes to the crucial aspect of trial design and you need to know in some
• 15:54
  level what forces are pushing your hazard ratios in different directions, what forces
• 15:59
  are pushing your treatment effects and what you're likely to see in different directions.
• 16:03
  And when we talk about differences in hazard ratios over time or we talk about this kind
• 16:08
  of change in the hazard ratio or the difference in the treatment effect that you see over
• 16:12
  time, one of my favorites is this issue of depletion of susceptibles.
• 16:16
  And to dumb it down or to say it more simply, this is that a patient who are sicker, more
• 16:22
  likely to have an event of interest, meaning like more likely to die or more likely to
• 16:26
  be hospitalized or more likely to meet some other endpoint, are the ones who are going
• 16:31
  to have it probably earlier in the study and also the ones who are more likely to then
• 16:37
  fall out of a time to first event analysis, such that as a trial is going on, those patients
• 16:44
  who are going to have an event tend to have it early in the control group if a intervention
• 16:50
  is not effective and tend to have it later in the treatment group if the intervention
• 16:54
  is effective.
• 16:55
  In a setting where an intervention is effective, those patients in the placebo group tend to
• 16:59
  have it earlier and then only the healthier patients tend to remain.
• 17:02
  As the trial goes on, your apparent hazard ratio, your apparent relative benefit of a
• 17:08
  therapeutic, it will shrink in every trial for which there is effect.
• 17:13
  Now that is one force that is always pushing down the hazard ratio.
• 17:17
  Janet mentioned one that is pushing up a hazard ratio.
• 17:20
  So if you have a treatment that becomes more effective the longer you take it, then the
• 17:24
  hazard ratio is going to move in opposite directions.
• 17:26
  So when you're planning your trial in terms of how long it should be, in terms of how
• 17:31
  sick the patients are that you're going to roll, this needs to go into that relatively
• 17:36
  complicated calculus of how to structure the trial, knowing that for an effective therapeutic,
• 17:41
  the hazard ratio will typically get smaller over time.
• 17:44
  Those are important concepts.
• 17:45
  But what we often get faced with in clinical trials is do we enroll more patients to get
• 17:52
  a lot more patients in and therefore the follow-up time might be shorter because the event rate
• 17:58
  is occurring or do because of the number of things that can happen, do we hold the sample
• 18:04
  size, the recruitment size and we try to extend the follow-up to get the number of events?
• 18:11
  That could be hazardous, right?
• 18:13
  It could be.
• 18:14
  And there's a lot that goes into the equation and that's why when one reads the literature,
• 18:18
  looks at these studies, it's important to look at all of the design aspects, right?
• 18:22
  Because sometimes we purposefully enrich an enrolled population for events.
• 18:28
  We use things such as a prior history of heart failure hospitalization or an elevated BMP
• 18:33
  or NT per BMP level to enrich a group for events so that we manage the sample size to
• 18:39
  be smaller and manage the follow-up period perhaps to be a bit shorter.
• 18:43
  When you enroll patients into clinical trials, then by virtue of sample size calculation
• 18:49
  and event accrual, you end up with larger sample sizes and more follow-up.
• 18:53
  So it's debatable which approach is better or more ideal.
• 18:57
  A lot of it depends on what you expect to show, how sick your patients are.
• 19:01
  Are you studying mainly class three and four patients versus class one and class two patients
• 19:07
  and so on and so forth.
• 19:08
  So I don't think I have a right answer here except to say that all of these considerations
• 19:13
  go into the design and then are subsequently important to interpreting the results, including
• 19:20
  this issue of hazard ratio.
• 19:22
  And Eric, you get the last word.
• 19:23
  We talked about the Stitch trial when we were in the think tank and obviously if we had
• 19:30
  kept the follow-up relatively short, just even two years, we would have missed the signal
• 19:35
  in Stitch.
• 19:36
  So as Janet was saying, there's an example where follow-up and getting diversion of those
• 19:43
  hazard ratios and the survival curves to get less drift collapse, so to speak, was a real
• 19:49
  advantage to the interpretation of that trial.
• 19:52
  So give us your final thoughts on hazard ratio drift and how to account for it.
• 19:58
  Well, Janet made a really excellent point.
• 20:00
  I think that what your hazard ratio is going to be is going to depend on whether most of
• 20:05
  your patients are followed for a short period of time, a medium period of time, and a longer
• 20:09
  period of time.
• 20:11
  I think hazard ratio drifts, I mean the longer we follow patients, I think we will see the
• 20:16
  hazard ratio vary over time and I think we just need to take that into account.
• 20:21
  It's funny, there's actually statistical software which will try to estimate the event rates
• 20:26
  over blocks of time and yet one of my mentors at the National Institutes of Health just
• 20:30
  said if you're going to do a sample size calculation, just think about what the average event rate
• 20:35
  is going to be during the follow-up.
• 20:37
  So if you think of it, if you take a step back and think about what the broader picture
• 20:41
  looks like, that might be a safer way to protect yourself against hazard ratio drift because
• 20:46
  you're going to say to yourself, okay, I can't really fully understand how things might,
• 20:52
  might expand or decrease over time as the trial goes along, but if I just think in average
• 20:57
  and act conservatively and when I design my trials, that might be a way for me to, for
• 21:03
  us to detect superior treatments and take, and have a little bit of robustness depending
• 21:08
  on how the drift might turn out which we can never foresee in advance.
• 21:11
  Well thank you Eric for bringing clarity.
• 21:13
  Thank you Janet and the whole team for bringing clarity to a topic that we continue to struggle
• 21:19
  with I think in our community, but it really emphasizes why statistics is so important
• 21:25
  and statisticians to be working with clinicians as we think about the goal of improving the
• 21:30
  ecosystem of therapeutic development.
• 21:32
  So fascinating discussion.
• 21:33
  I think if we had a longer follow-up time to this discussion, we'd probably even have
• 21:38
  a deeper discussion and we would probably drift this way and drift that way.
• 21:42
  But for now, the Heart of the Matter podcast will end today and we'll continue this conversation
• 21:48
  in the future.
• 21:49
  Good luck.