Latest / Cardio Pod: Heart of the Matter / The Role of AI in the Future of Clinical Trials and Therapeutic Development

Transcript

The Role of AI in the Future of Clinical Trials and Therapeutic Development · Cardio Pod: Heart of the Matter

• 0:00
  Good morning, good afternoon, good evening.
• 0:09
  I'm Chris O'Connor from the Heart Failure Laboratory and we're here with the Heart of
• 0:14
  the Matter podcast to discuss our latest think tank meeting in which we took a deep dive
• 0:20
  into the role of artificial intelligence in augmenting and enhancing clinical trial conduct.
• 0:28
  And I'm here with our distinguished panel.
• 0:30
  I'll let them introduce themselves.
• 0:32
  We'll begin with Dr. Martin Cowley.
• 0:34
  Hi, Chris.
• 0:35
  I'm Martin Cowley.
• 0:36
  I'm a cardiologist and Senior Vice President AstraZeneca in charge of all the late stage
• 0:41
  trials and cardiovascular.
• 0:43
  Great pleasure to be here.
• 0:45
  Thank you, Martin.
• 0:46
  Dr. Joanne Lindenfeldt.
• 0:47
  Hi, everyone.
• 0:48
  I'm Joanne Lindenfeldt, a cardiologist and heart failure physician at Vanderbilt University.
• 0:54
  Dr. Scott Solomon.
• 0:56
  Hey, Chris and everyone.
• 0:58
  I'm Scott Solomon.
• 0:59
  I'm a cardiologist and clinical trialist at Harvard Medical School in Brigham and Women's
• 1:03
  Hospital.
• 1:04
  I guess we're now called Mass General Brigham.
• 1:06
  Okay.
• 1:07
  We've got the merger.
• 1:09
  The Dr. Jonathan Cunningham.
• 1:12
  Hi, everyone.
• 1:14
  John Cunningham also from Brigham and Women's Hospital, Harvard Medical School.
• 1:17
  I'm a heart failure cardiologist.
• 1:19
  Dr. Mona Fuzon.
• 1:21
  Hi, everyone.
• 1:22
  I'm Mona Fuzon.
• 1:23
  I'm a co-chair of the Heart Failure Collaboratory and a heart failure investigator, a pharmacist
• 1:29
  by training, and my appointment is at Duke University.
• 1:33
  And Dr. Mitch Satka.
• 1:34
  Hi, I'm Mitch Satka.
• 1:36
  I'm section chief for heart failure and transplant at Innova Heart and Vascular, and then the
• 1:43
  digital health working group chair for the Heart Failure Collaboratory.
• 1:46
  Terrific.
• 1:47
  Well, let's dive right into this.
• 1:49
  The discussion was fascinating during our think tank, and it really spanned the whole
• 1:54
  broad spread of clinical trial conduct, patient identification to trial conduct to end points,
• 2:03
  and then to the dissemination of information.
• 2:07
  And I'll start with you, Mitch, talk about patient identification.
• 2:13
  You've been involved in some of the projects using artificial intelligence, which frankly
• 2:19
  is machine learning, to help with patient identification in a broad health system using
• 2:25
  the EHR.
• 2:26
  Could you describe how you think AI might help patient identification in clinical trial
• 2:32
  conduct?
• 2:33
  Absolutely.
• 2:34
  So, we know that a large burden of clinical trial enrollment comes in the form of patient
• 2:42
  screening, which has been traditionally performed by manual review of patient records, often
• 2:49
  those coming through a clinic or being seen in an outpatient or inpatient setting, and
• 2:56
  then performed going through the sometimes very lengthy list of inclusion and exclusion
• 3:01
  criteria that clinical trials have constructed to enroll the correct patients for their study
• 3:07
  design.
• 3:08
  And one of the ways that machine learning or algorithmic input into the electronic health
• 3:15
  record can assist with this process is by really identifying patients who meet the majority
• 3:21
  of these inclusion and exclusion criteria prior to manual review, thereby reducing the
• 3:27
  administrative burden for clinical trial enrollment.
• 3:31
  And perhaps really adding to patients who would not have been otherwise recognized because
• 3:37
  they are not being seen in a heart failure clinic or in a cardiology clinic or out in
• 3:42
  the world in the auspices of the electronic health record, but can be identified and then
• 3:48
  brought in.
• 3:49
  And so, there is both this opportunity to find patients who meet the criteria to be
• 3:54
  included more easily, but also to find patients who will not be otherwise found using the
• 3:59
  electronic health record and this kind of algorithmic identification of patients.
• 4:05
  That would be such an advance if we can improve this sort of identification of one patient
• 4:11
  per site per month that we're left in dealing with now.
• 4:15
  Martin, give us your broad thinking on as someone who's looking over a number of clinical
• 4:22
  trials from a sponsor standpoint, how you think AI might improve the ecosystem here?
• 4:29
  Well, I think there's enormous hope that it will do because it can take out a lot of the
• 4:34
  very slow tedious work and do it super fast.
• 4:37
  It never, you know, and it doesn't need a coffee break.
• 4:39
  It doesn't need a weekend off.
• 4:40
  It just keeps doing it, which is a huge advantage.
• 4:43
  So I think from site selection to patient identification and you know, things like protein
• 4:48
  urine, lots of cardiologists, I'm afraid, don't actually deal with urine.
• 4:53
  How can you predict from other markers in the medical record that the patient might
• 4:57
  have protein urine, therefore should be considered for screening at least, for example, and then
• 5:01
  all the way through to endpoint adjudication, which I know Scott has an interest in also
• 5:08
  where we've used our huge data sets to show that it's actually very good and easy to train
• 5:13
  AI to identify cardiovascular mortality from our data sets.
• 5:17
  And we can now adjudicate events within three minutes of finding them rather than three
• 5:21
  months from the conventional endpoint process.
• 5:24
  A little bit more tricky for heart failure, hospitalizations, as we all know, but that
• 5:30
  we'll learn as well.
• 5:32
  And I think also final point is that let's make sure that we understand that AI not necessarily
• 5:37
  just on its own, that it can be a degree of human supervision and AI may do your first
• 5:43
  best stab at it and then humans can add to that.
• 5:46
  So it's human plus artificial intelligence needs a faster, more efficient process.
• 5:50
  So we're very enthusiastic about it.
• 5:52
  Hopefully not just hype, but actually now translating into meaningful difference.
• 5:56
  Well, Martin, that's a really good point that there's a lot of fear out there that AI is
• 6:01
  going to replace everything and there won't be a need for humans.
• 6:05
  And your point that this is an enhancement, that they're working together, humans and
• 6:11
  AI machine learning, I think is a really important concept.
• 6:15
  Well, one of the really strong presentations at the think tank was around the endpoints.
• 6:23
  And if we quote Norman Stockbridge correctly, he said this may be one of the top priorities
• 6:29
  to tackle first would be the work that John and Scott have done in this space.
• 6:36
  Scott, why did you take on the endpoint question of using machine learning and AI?
• 6:44
  What did you find, Scott and John?
• 6:47
  I think the short answer is because we were working too many weekends.
• 6:54
  Though Martin doesn't think we do this on weekends.
• 6:57
  No, I mean, I think that first of all, as you said, Chris, the use of AI in medicine
• 7:04
  in general has exploded obviously in the last several years.
• 7:08
  And we're just starting to think about how we can use them to enhance our clinical trials.
• 7:13
  And I'm very excited about this as we try to make our trials more efficient.
• 7:19
  We've been adjudicating endpoints, as you know, for many, many years in clinical trials.
• 7:24
  I think you and I started on the Valiant trial together about 25 years ago.
• 7:31
  And we realized that it is enormously time consuming.
• 7:35
  It also can be even potentially inaccurate in some cases.
• 7:42
  We've spent a lot of time in the laboratory talking about endpoint definitions and the
• 7:48
  process of adjudicating endpoints.
• 7:51
  But we realized that if we can do this faster, we can do this more efficiently, if we can
• 7:56
  do it with more reproducibility, we can actually make a big difference.
• 8:02
  And so John had been very interested in this topic and was working to utilize natural language
• 8:11
  processing to essentially determine endpoints for heart failure in an EMR system.
• 8:21
  And the main reason for doing that, we don't do that as much in the EMR in individual healthcare
• 8:27
  systems as we do in trials.
• 8:30
  In trials, we spend a tremendous amount of time and effort adjudicating.
• 8:35
  In these large data sets, we mostly use ICD codes and things like that, and we realize
• 8:41
  there's a lot of information that's available in the medical records, even in the things
• 8:47
  that we type or dictate that can help determine if a patient has a particular disease.
• 8:53
  In this case, it's heart failure.
• 8:55
  So John started there and then moved on to clinical trials.
• 8:59
  And I'm going to turn to John and let him actually tell us a little bit about what we
• 9:05
  did.
• 9:06
  Thanks, Scott.
• 9:07
  So as Scott mentioned, we started off looking in our own electronic health record.
• 9:12
  I labeled a bunch of cases myself and we taught the computer to learn whether it was one that
• 9:18
  I was going to call a heart failure or not based on the 2017 SCTI criteria, the standard
• 9:25
  clinical trial criteria for heart failure hospitalizations.
• 9:29
  And then we took that model out of the box into the invested clinical trial and showed
• 9:33
  that across 157 sites in the US and Canada, the same model also worked well for picking
• 9:41
  out which hospitalizations would be called heart failure by a clinical events committee
• 9:47
  in the trial.
• 9:49
  And I think the excitement here is that we all know that high quality endpoint data is
• 9:55
  important for getting the right answer in any cardiovascular outcome analysis.
• 10:01
  And if we can improve the quality of data in our large scale data sets and improve the
• 10:06
  efficiency of getting high quality data in our adjudicated clinical trials, that's going
• 10:13
  to allow us to get better information faster for our patients.
• 10:18
  That's terrific and we know that these CECs are very cumbersome, very expensive.
• 10:25
  So you're tackling a big part of the core of what many of us are interested in.
• 10:30
  As I wear my health system hat, I also see an amazing opportunity here, Scott and John,
• 10:37
  I want you to think innovative of your ideas that as you know, penalties for readmission
• 10:43
  heart failure, readmissions, the accurate diagnosis of those readmissions is very complex
• 10:49
  and not well done.
• 10:50
  We've done adjudication of these readmissions that have gone towards the penalty fees and
• 10:55
  found that there's a lot of errors there and misclassifications.
• 11:01
  But then there's also the whole process of the morbidity and mortality committees that
• 11:05
  go on at every hospital.
• 11:07
  And in my view, it's asymmetric and there are biases in there and what cases are brought
• 11:14
  forth and how they're adjudicated.
• 11:17
  So having these standardized processes that may reduce bias could be very valuable to
• 11:24
  the health systems also.
• 11:26
  Well, Joanne, go ahead.
• 11:28
  I was going to say, I'm not very good at actually assigning ICD codes to my patients when I
• 11:36
  see them in clinic.
• 11:37
  I know that I'm just doing whatever I can as quickly as I can to get through to the
• 11:42
  next patient.
• 11:44
  This is a way potentially that healthcare systems could use what we write in our notes
• 11:49
  to do that kind of assignment.
• 11:53
  And I'm sure some of them are doing this already.
• 11:55
  And I'll just start with Chris.
• 11:57
  We got 60% in our analysis.
• 11:59
  So of the hospitalizations that have a primary code for heart failure, which would be the
• 12:06
  ones that the penalties would apply to, only 60% of those for us would be SCTI criteria,
• 12:15
  heart failure hospitalization.
• 12:16
  So I think there's a lot of cleaning to be done.
• 12:20
  Very nice.
• 12:21
  Great point.
• 12:22
  Joanne, you've served on a lot of data safety monitoring boards and what Martin said was
• 12:28
  fascinating and that is they can now get the desks identified, classified within minutes
• 12:35
  as opposed to this long lag time of monitoring the charts and getting them to the DSMC.
• 12:42
  How do you see AI helping trial conduct from a data safety monitoring committee?
• 12:49
  Well I think as you've said, we'll get the data much earlier and we'll be able to make
• 12:53
  decisions.
• 12:54
  You know, very often endpoints are outstanding.
• 12:55
  The adjudication has not been done when you meet.
• 12:57
  I think more than that, I think everything that's been said is exciting.
• 13:01
  I was excited by this paper and doing adjudicating the endpoints for several reasons.
• 13:05
  We all know on those committees that we keep a log.
• 13:08
  Okay, how are we going to decide this when we do it again?
• 13:10
  How are we going to decide this when we do it again?
• 13:13
  And it'll be much, much better to have consistency and after the first few trials do it this
• 13:18
  way then we'll learn if that's exactly the right way or not.
• 13:20
  So that will be enormously helpful.
• 13:23
  I mean I see as John said applying this back to our patients and saying, okay that last
• 13:28
  admission wasn't really our fair admission, you guys goofed.
• 13:32
  Think about it this way.
• 13:33
  But more than that there's just when we think about screening for patients there's a huge
• 13:38
  amount more that could go into this.
• 13:40
  So the whole thing is somebody has to screen, somebody has to look at value, get the criteria,
• 13:44
  can automatically correct for weight if you decide to do that.
• 13:47
  It can notify you who the primary physician is.
• 13:50
  Potentially if you check on yes screened for the study you could send a note to the primary
• 13:54
  physician say are you on board with having your patient be this study with a one paragraph
• 13:58
  or two sentence synopsis of the study and would you be agreeable to having your patient
• 14:02
  be called and do all of those things that take several phone calls and even to the point
• 14:07
  of saying your doctor has agreed to let us send you this through your My Health at Vanderbilt
• 14:11
  or whatever.
• 14:12
  You know this research study here's something about it click on this if you'd like more
• 14:15
  information we'd like to speak to you about the study.
• 14:18
  All of those things could be done in a very rapid period of time and that could really
• 14:23
  be exciting.
• 14:24
  I mean we could see how this could save weeks and months of time.
• 14:27
  So very exciting.
• 14:30
  Not to mention it could search the chart right when you say why aren't they on guideline
• 14:35
  directed medical therapy.
• 14:37
  It could search the chart for why aren't they on spironal lactone.
• 14:40
  There's probably reasons but sometimes they're buried way back in there and it takes the
• 14:43
  coordinators hours to find those things.
• 14:46
  But there's often reasons sometimes there's not but there often are and then you can get
• 14:49
  that and document it from the source code.
• 14:52
  So that will even go through the care everywhere the remote things that we get that just are
• 14:57
  enormously time consuming.
• 14:59
  So that's really exciting.
• 15:03
  The same technology that is used for the end point adjudication could be used at the front
• 15:09
  end for inclusion criteria could be used for adverse event reporting.
• 15:17
  It's really all different sides of the coin and could make every part of the process more
• 15:23
  efficient.
• 15:24
  And do you see a point John and Scott where you could say your patient has a 93% chance
• 15:30
  of not tolerating Entresto?
• 15:32
  No seriously.
• 15:35
  So we don't think you have to go there necessarily.
• 15:38
  Yeah you just need a label data set of people who had the same input, who had an outcome,
• 15:45
  who either did or did not tolerate it.
• 15:49
  Well and even to the point where I think this would help primary caregivers and general
• 15:53
  cardiologists who don't do it every single day to say all right you can start spiral
• 15:58
  lactone but you need to check potassium in four days or you need to check potassium in
• 16:02
  two weeks or just give you those from the criteria we see this is what you need to do
• 16:07
  and that will be enormously helpful to people like that.
• 16:10
  That's terrific.
• 16:11
  You're exactly right.
• 16:12
  I mean I think that's kind of the 2.0 of AI in our medicine practice right is guiding
• 16:19
  but-
• 16:20
  Yeah you have to be slightly careful because you know the concept of a digital twin where
• 16:25
  you can do trials in silico so let's just say we simulate humans and we do a trial in
• 16:30
  10 seconds and say well actually this drug would have this effect so let's not bother.
• 16:35
  I think that looks great on paper and on PowerPoint but actually there's nothing like empiric
• 16:39
  evidence to show that you don't actually understand this as much as you thought and that the patterns
• 16:45
  may be spurious.
• 16:46
  So I think and maybe as a trialist I would argue this that actually doing a trial is
• 16:51
  an important part of this process but AI can speed up that and get rid of a lot of the
• 16:57
  slow clunky processes and within my company for example we hope to submit instead of an
• 17:02
  average of 20 weeks after the database is locked down to three weeks ultimately so that
• 17:09
  gets science into medicine so much more quickly and actually the data is being used almost
• 17:14
  immediately to make a difference so for me I think that's more realistic and I feel it's
• 17:19
  a bit sci-fi all of this let's just simulate humans and see how they get on.
• 17:24
  Yeah I totally-
• 17:25
  But maybe I'm a bit on the end of the spectrum.
• 17:27
  I completely agree with that I don't think we want to skip the trials but I do think
• 17:31
  as we gain actual trial data with real people with the studies that some of these things
• 17:35
  will be predictable you know you're not likely to tolerate this and that will be I think
• 17:40
  valuable for people who do it less for my experience is that for instance general internist
• 17:47
  to do heart failure if you start spironolactone and you get really hyperkalemic one time that
• 17:51
  people tend to not do it again so but there are no there's just very rough guidelines
• 17:56
  in the guidelines for how to do it and what to expect and I think through these trials
• 17:59
  we're going to be able to get much more precise data for practitioners to be able to do those
• 18:04
  things and do them safely.
• 18:05
  Yeah experienced clinicians pattern recognition isn't it and AI is all about pattern recognition
• 18:11
  so can you make a new medical student actually behave like a 20-year experienced cardiologist
• 18:17
  probably with an AI tool maybe?
• 18:21
  I think Martin and Joanne make such important points that you know first of all we have
• 18:28
  to separate the hype of AI from the reality and the fact is it's just going to get better
• 18:36
  over the next several years but we have to be very cognizant of what it can do at the
• 18:42
  moment what we trust it with you know we've all played around with chat GPT we know that
• 18:49
  it hallucinates on the other hand it's gotten better in the last year I try to stump it
• 18:56
  all the time I edit a textbook of cardiology and we are incorporating AI chat GPT like
• 19:06
  interface to be able to query the textbook and get information from the book that hopefully
• 19:14
  is usable by clinicians so there's that aspect within the clinical trials realm I mean I'm
• 19:22
  really glad to hear a drug developer saying that they're not going to abandon clinical
• 19:27
  trials and just do things in silico because I you know we've all heard companies saying
• 19:33
  they can do this and we have to be very clear and I'm sure Mitch with your FDA hat on you
• 19:40
  feel this way that's very unlikely it's probably reasonable at the very early stages when you're
• 19:45
  trying to pick molecules but we don't have enough information to understand for the computers
• 19:51
  to actually get the biology right and pick out all the things that happen to people when
• 19:57
  you give them drugs and I just want to follow on what you just said Scott when I tried to
• 20:02
  write my chapter for your book using chat GPT it hallucinated all of the references
• 20:08
  that I wanted to use not one of them was a real reference and so just to be clear that
• 20:14
  really is at the nascent stage of using this technology particularly the productive aspect
• 20:20
  of this technology when it's being asked to produce something right it sounds very realistic
• 20:26
  but it typically that is where the technology is not really up to par and but I think the
• 20:32
  other way to think about it is we've been using predictive models in a lot of ways we've
• 20:36
  there I think Dr. Connors created a number of predictive models in the history of his
• 20:43
  manuscript writing and they are used to assign likelihoods of events happening or of things
• 20:51
  happening I think that those sort of predictive models we are starting to really incorporate
• 20:55
  more and more into clinical practice and as more data is collected as we've been discussed
• 20:59
  those predictive models can be more elaborate and perhaps more accurate in helping clinicians
• 21:06
  to make decisions but really again like Martin was saying not replacing the clinician in
• 21:12
  the activity but really just assisting them to make the right call at the right time.
• 21:19
  And if I could jump in you know one of the discussions that we had was around you know
• 21:25
  the role of the editors as a regulatory body because so much implementation is dependent
• 21:31
  on what's in the literature and how to you know ensure that the literature is valid you
• 21:39
  know right now they're at a stage of you know declaring if you use chat GPT but I don't
• 21:45
  think that's nearly enough and it's very dependent on the authors honesty somebody who's not
• 21:51
  willing to be honest I think could very easily be putting garbage into the literature and
• 21:57
  I think that's one of the downsides I don't know how an editor having been on that side
• 22:03
  of publishing would ever know how to distinguish a real paper from a false one.
• 22:14
  Well Mona thanks for jumping in on the editorial side the other aspect of this very cumbersome
• 22:21
  process of getting information out is peer review and could we imagine a world where
• 22:30
  peer review was doing what John showed us and Scott and that is right now we have we
• 22:37
  don't have enough adequately trained peer reviewers and we're burning the ones we have
• 22:42
  out but could we say okay every paper now being reviewed could be reviewed by the AI
• 22:50
  reviewer and one human reviewer in a hybrid models.
• 22:56
  What do you think?
• 23:00
  Well I don't know how to take that I mean I'm not sure we're quite there yet but to
• 23:08
  bring up this idea of a hybrid model and that's something that John mentioned the other day
• 23:14
  in his discussion in that you know you rather than just say is going to be all AI it's more
• 23:22
  like what I would call driver assist I have one of these cars that you can press a button
• 23:27
  and it'll pretty much steer for you but you've got to keep your hands on the wheel and you've
• 23:32
  got to be ready to take over and that's the kind of thing that we're thinking about in
• 23:39
  adjudication processes where you might use AI to help decide for example the cases that
• 23:47
  are definitely heart failure hospitalization for example or definitely not a heart failure
• 23:54
  hospitalization but the ones that for which the AI can't make that determination so easily
• 24:00
  there in the intermediate zone you would have human beings involved but it could still dramatically
• 24:09
  improve the work.
• 24:13
  One other question Chris if I may and Martin you may have some thoughts about this about
• 24:19
  the use potentially of AI in monitoring of clinical trials that's a place where you guys
• 24:25
  are spending so much money and I imagine this is a way that you could potentially identify
• 24:35
  fraud for example or identify certainly sloppy investigation I can think of a lot of ways
• 24:42
  in which AI might be helpful in that respect.
• 24:46
  I agree Scott and I think you know we're trying to do proportionate monitoring because you
• 24:50
  can't be a hundred percent intense on everything and I think AI will help us without looking
• 24:54
  for patterns of bad behavior and poor quality etc and picking that up earlier rather than
• 25:01
  later but also going back to your point about AI being you know let's say very likely or
• 25:06
  very unlikely to be something that's fine you can trust it in the middle that's where
• 25:10
  humans need to add their value to it the FDA seems very open to that and our discussions
• 25:15
  with them have been very positive very supportive very willing to explore it and say let's go
• 25:22
  on this journey together see where it takes us.
• 25:25
  I think other regulators are way behind that and still very fixed on humans being the ultimate
• 25:32
  judge of everything and that's the gold standard but I think all of us have been on endpoints
• 25:37
  committees and we know sometimes how much discussion there is about individual cases
• 25:42
  and I'm not even sure there's a ground truth for some cases it's very difficult in multimorbid
• 25:47
  elderly patients to know exactly what is going on so it is a balance of probabilities but
• 25:53
  that kind of mature nuanced attitude I think is quite difficult to communicate with regulators
• 25:58
  who like certainty so it's an interesting time but I think there's lots of people on
• 26:02
  this journey not just the trialists and clinicians.
• 26:05
  Yeah Chris one thing that I'd like to be disabused of this concern about AI that sometimes I
• 26:12
  don't hear is that as AI takes over the simple things we all do those things now you know
• 26:16
  we adjust the potassium we look at the but we do all those things so if we get a program
• 26:20
  that doesn't quite right humans feel comfortable saying wait a minute this doesn't seem quite
• 26:24
  right to me that is all of us start to have the simple things taken over and we never
• 26:28
  do them how do we become if you don't know how to do all the little parts of it how do
• 26:34
  you become the final adjudicator of what you get how do you do the driving when you don't
• 26:39
  understand you haven't been doing any of the turns or you know putting on your turn signalers
• 26:44
  I mean that just is the one thing I don't see a solution for I mean when you stop having
• 26:50
  all that baseline experiences have made you able to say I don't think this probably I'm
• 26:53
  going to step away from this program I'm not sure we'll be able to do it once there aren't
• 26:57
  a group of people that have the experience to say this isn't right.
• 27:02
  Well we've had that experience and that's called a plane crash we know that the pilots
• 27:07
  who haven't had the training on the manual adjustments of the instrument board they're
• 27:13
  relying on the completely on the electronics have stuttered or failed in their ability
• 27:18
  to compensate so you bring up a really good point I was going to go around the table and
• 27:22
  just in our conclusion and I'll count that as your concern but give us your top concern
• 27:32
  about AI in medicine and Joanne you so nicely articulated one but I'll go around the table
• 27:40
  here Mona.
• 27:41
  Misinformation one word misinformation.
• 27:46
  Okay one word Mitch.
• 27:50
  Well I'm going to follow in Mona's footsteps because I do think and going with Dr. Kalit
• 27:55
  that misinformation is one of the greatest challenges to the data that we create and
• 28:02
  utilizing it appropriately and knowing what is real and what is fake for not only the
• 28:07
  clinical trials but all the population.
• 28:10
  John I think portability I think we always have to worry that we trained in one area
• 28:16
  and then we're evaluating in a different time in a different place and that you know if
• 28:21
  it was a human doing the review then they would say wow this these documents really
• 28:26
  look different or something's wrong but the other computer just sort of flows forward
• 28:30
  and gives an answer anyway so we want to keep checking that keep our eyes on the data and
• 28:37
  keep making sure that you know the computer saying the right thing.
• 28:42
  Martin I think let's just make sure we're not just copying human biases you know you
• 28:50
  can use AI to train an interview panel to come up with exactly the same biases as the
• 28:54
  humans very predictably that's not what it's about it's supposed to be adding value not
• 28:59
  just simulating it so yeah let's just think about the use cases and what we're trying
• 29:04
  to achieve and as I said earlier AI plus human intelligence is what we're looking for.
• 29:10
  I love it I love it Scott and I worry about overhype and I think we have to figure out
• 29:18
  what AI can do for us at the moment we have to recognize that it's going to change dramatically
• 29:25
  very fast very different from drugs which take maybe 10 years to really change but the
• 29:33
  AI will change in months and we're going to update the algorithms very quickly so we have
• 29:40
  to be cognizant of that and adapt how we work based on how the AI changes.
• 29:48
  Ultimately I think it will be a good thing if we keep our hands on the wheel.
• 29:54
  Really well said and I agree with Martin that the biases could actually enhance disparities
• 30:02
  and care not reduce them so that's really my fear that the data sets that we have may
• 30:08
  not be robust enough to help us reduce disparities so a terrific conversation today gosh you're
• 30:15
  so shy and had no energy at you know four o'clock in the afternoon but wow AI is going
• 30:24
  to be with us it's going to be with us in medicine it's going to be with us in clinical
• 30:27
  trial development and I really appreciate your comments today and we're going to have
• 30:32
  more of this I guarantee you that Heart Failure Collaboratory will be taking this on strongly
• 30:36
  so thank you for being here with Heart of the Matter podcast have a great day.