Latest / Health Ethics Podcast / Artificial Intelligence (AI) in Clinical Decision Making: A Surgeon's perspective

Transcript

Artificial Intelligence (AI) in Clinical Decision Making: A Surgeon's perspective · Health Ethics Podcast

0:10
Welcome to the COVID Ethics Series podcast, a conversation with leading experts about bioethical
0:15
issues that have been exacerbated due to the COVID nineteen pandemic. I'm Brian Pilkington,
0:21
and I teach bioethics at Seatonhole University and at the Hackensack Marine School of
0:25
Medicine. I moderate the COVID Ethics Series in cooperation with the IHS Library and
0:30
IHS Student Life. This podcast is a spinoff collaboration with the IHS Library where
0:37
we'll have the opportunity to go deep with the experts from our series. Thanks
0:41
for joining our conversation. Hello everyone, and welcome to the COVID Ethics Series
0:58
podcast. I am Brian Pilkington, and as you all know, I teach
1:02
ethics at the Hackensack Meritian School of Medicine and also at Seaton Hall University and
1:06
the College of Nursing in the School of Health and Medical Sciences. It is
1:10
my great pleasure to be with friend and colleague, doctor Charles Binkley as we
1:15
discuss a host of ethical concerns sort of coming out of COVID with an eye
1:21
towards the future and considerations of artificial intelligence in medicine. Will also jump in
1:26
a bit to the recent work that he published in the New England Journal of
1:30
Medicine with a few colleagues, from the Eyeball test to the algorithm quality of
1:36
life, disability status and clinical decision making in surgery. Doctor Binkley, thank
1:42
you so much for joining me. Doctor Pilkington, it is an honor to
1:45
be here. Thank you for the invitation to join you. You are so
1:48
kind and I know so many of our audience will already know you, especially
1:52
our medical students, but for those who do not, Doctor Charles Binkley is
1:57
Director of Bioethics in the Central Region of Hackensack Meridian Health. He's Associate Professor
2:02
of Surgery in the Hackensack Meridian School of Medicine, trained as a liver in
2:07
pancreas surgeon, and he comes to this area from the markal A Center for
2:14
Bioethics, where he still holds a fellowship in bioethics, and he's doing a
2:17
ton of interesting work in artificial intelligence, big bioethics questions there and in medicine.
2:25
So again, Charles, great to have you here. Let's start with
2:30
sort of any big questions associated with it, but sort of rooted in the
2:35
paper, in the work you're doing, sort of thinking about artificial intelligence.
2:39
So in particular, what motivated you either to jump into the work on artificial
2:44
intelligence or in particular the artificial the work on artificial intelligence as it applies to
2:49
surgery and disability. Well, thank you, Brian, it's a really great
2:53
question. So I practiced Tipatobillery Deliver and pancreas surgery in San Francisco for about
3:00
fifteen years, and you know, I always you know, patients would come
3:06
to see me. Surgery is really the keystone to long term survival if you
3:12
have pancreated cancer, and patients would come to see me, and I would
3:15
spend a lot of time thinking about their candidacy for surgery, can I benefit
3:20
them, and particularly trying to weigh out in my mind and deciding whether to
3:24
offer surgery to an individual patient or not. You know, what is the
3:29
absolute risk, what is the absolute benefit, and what's the proportion of the
3:31
two, what's the balance? And you know, is this an acceptable or
3:36
even a reasonable plan? And so I always struggled, particularly with the patients
3:42
that I would say, you know, you're not a good candidate for surgery,
3:46
you know, how do I know that? And in some instances it's
3:50
very clear. For instance, if patients have a spread of their pain created
3:54
cancer, if they have a metastatic disease, we typically say that there's the
4:00
benefit is not there for surgery. Other things may extend survival, but we
4:03
don't really offer them surgery to resect to take out the primary cancer. And
4:09
so as I saw these patients, I would really struggle with the saying no,
4:14
that the telling people that they weren't candidates for surgery, because you know,
4:17
I just it was never clear to me that I had a full story
4:23
or that I that I knew positively surely that I would not benefit them in
4:28
some way. And so I started to think about how we could make better
4:31
surgical decisions. How could we predict who would and wouldn't benefit from surgery,
4:36
How could we do a better job at stratifying risk and also stratifying benefit.
4:42
And I was thinking about this and the context of surgical decision making, But
4:45
then I thought about all the other areas of clinical uncertainty that we have.
4:48
And typically when we have clinical uncertainty in many areas, I think about the
4:53
ICU. If we don't know which whether a patient's going to get better or
4:57
not, will do sort of a time trial, will treat them for you
5:00
know, X number of days, and then we'll get a sense of you
5:03
know, are they getting better, are they getting worse? What's the illness
5:05
trajectory? And I thought about, you know, could we potentially take some
5:10
of the guestwork out, could we predict better who's gonna benefit for surgery,
5:15
but also who would benefit from other interventions that we have to offer. Then
5:17
that was really where my interest in artificial intelligence came from. Was, you
5:21
know, this true sort of clinical conundrum of you know, how do we
5:27
know? And then how how can we know more? How can we know
5:30
better? We spend a lot of time doing retrospective reviews, We've spent a
5:34
lot of time trying to amassive data and use predictors and clinical calculators. But
5:44
the power that AI gives to really make these these predictions so much better than
5:50
we've been able to do before and potentially more unbiased. Uh. And I
5:57
think that's a that's a really important piece in terms of surgical selection because I
6:00
think we all, I think surgeons are potentially very biased and some of their
6:05
decisions that they make, and so that's really where the interest came from.
6:11
Can we can we do better at making predictions and can we take some of
6:15
our own biases out of the predictions that we make. Charles super helpful and
6:19
framing the broad question and your interest in AI, let's sort of focus it
6:27
a bit more with all of that in mind, sort of take us through,
6:32
just at a very high level, the basic argument in this New England
6:36
Journal of Medicine piece And for those we'll post links to it for the podcast,
6:41
But for those listening, this is from eyeball tests to the algorithm,
6:45
quality of life, disability status and clinical decision making in surgery. So Charles,
6:51
folks are going to read this. It's a great piece. It's also
6:54
as many New England Journal Medicine pieces, it's concise and to the point.
6:57
So I strongly recommend it to everyone. We don't need all the details,
7:00
but sort of what's the basic move in taking your interest in improving decision making
7:06
especially around surgical candidacy questions and disability? So why disability status? So I
7:15
wrote the paper with a couple of other researchers. Andy Schumann is another bioethicist
7:21
and a cancer surgeon at the University of Michigan. Andy and I have known
7:26
each other since we were both residents at the University of Michigan and Joel Reynolds.
7:30
Joel Reynolds is a professor of a philosophy at Georgetown University and is specifically
7:36
a disability ethicism. So I was teaching a healthcare ethics class at the market
7:41
List Center at Santa Claire University and wanted to do a segment or a class
7:46
on disability ethics. And I was inviting and we're all remote. This was
7:49
during COVID, and I invited Joel Reynolds, Professor Reynolds from Georgetown to come
7:56
to the class. I was inviting guests each week to comment and let this
8:00
interact with with really experts in the fielding. So Joel came and then he
8:03
and I started talking afterwards about gosh, you know, how are physicians biased
8:09
by disability status? Either an assumption about the quality of life of somebody and
8:16
saying, you know, somebody's quality of life is so poor is not justify
8:20
offering something or to say as a result of whatever I'm going to do,
8:24
whatever procedure I'm going to perform, your quality of life will be so poor
8:28
as to make this prohibitive is to justify saying no. And so as he
8:33
and I talked about that, we included Andy Schumann, the head and net
8:39
cancer surgeon at the University of Michigan the conversation, and you know, he
8:41
thinks very much about this in terms of offering extensive cancer resections to patients,
8:46
really is having the conversation with them, because then in many cases the reception
8:50
that he can offer patients may actually wind up making them blind, causing them
8:56
to be deaf, they're not able to speak as a result of some of
9:01
the receptions or even swallow. And so so you know, it was sort
9:05
of this trifecta of the three of us. You know, somebody who who
9:09
really is a surgeon deals with disability addresses disability, one who is a philosopher
9:15
who thinks about disability, and then me trying to think a little bit more
9:18
about how we can more precisely predict outcomes and deep bias them. And so
9:24
that was really how the paper was born. The overarching the overarching idea is
9:31
that you know, can we can we deep bias around disability and around quality
9:37
of life assumptions? That are associated with disability. As I described, either
9:41
somebody with a disability or quality of life is so poor as to as to
9:46
is to say, you know, the surgery or whatever the medical intervention is,
9:48
isn't going to be valuable to you, or is a result of the
9:52
surgery or the intervention your quality of life will be so poor as to not
9:58
make this valuable to you. That's Charles's super helpful, and also I think
10:03
nicely brings in again we have a diverse audience that listens to the podcast,
10:07
but lots of folks are either in a bioethical hass interest in bioethics, or
10:11
sort of engage in ethical reflection and other aspects of their lives. And so
10:15
the point about values is really interesting. And I'm going to quote a bit
10:18
from the paper and just sort of ask you to sort of flesh this out
10:22
a bit so you all note it is also and is just picking up the
10:26
theme you just raised. It is also imperative that algorithms not be biased and
10:31
I'm quoting here by assumptions about what a patient may or may not believe to
10:35
be an acceptable quality of life. Right, So it sounds like there may
10:41
be a challenge to surgeons or physicians making these kinds of decisions in general and
10:48
not taking into account particular values, sorry, values that particular patients have as
10:54
opposed to values that might be assumed by a practitioner that lots of folks have.
11:00
And you guys have a really helpful Yeah, you all have a really
11:03
helpful example about folks who are cited in vision loss. And so how assumptions
11:11
could there could be different assumptions based on what folks think relative to being cited
11:20
or not cited. So could you sort of help us think through quality of
11:24
life a little bit more relative to values that particular patients might have. Well,
11:28
absolutely, so we know that that you know, we as human beings
11:33
are fairly adaptable and resilient. And one of the things that winds up happening
11:39
that we have good empirical data about this is that if somebody has an event
11:46
that leaves them differently able, that removes some level of function, they may
11:50
have an advance of the event thought gosh, this is going to be so
11:52
horrible, My quality of life is going to be so poor. But then
11:56
after the event they we as humans adapt andre and we look back and say,
12:01
gosh, this isn't all that bad, you know anyhow I think about
12:05
you know, as a surgeon, there would be certain things like, you
12:07
know, if we had to talk to somebody about an amputation there, you
12:11
know, everyone would most people would say, oh gosh, this is this
12:13
is going to be really awful. It's going to be very hard for me.
12:18
I don't want to be a burden to my family. But then once
12:20
people get through and they learn to live after an amputation, they learned to
12:26
adapt to the prosthesis, they learned to sort of make it part of their
12:30
lives. They function completely normally, just as they would did prior to the
12:35
event. And again, we're to resilient, adaptive people, and so making
12:41
sure that it's the lived experience of the disability and not the way, not
12:46
the perception that the medical establishment or or the decision makers within healthcare, not
12:54
not our perceptions, but really the lived experience of the person with the discipl
13:00
so that we don't bring our own biases. You know, are we asking
13:03
ourselves, you know, what I want to live with an amputation, or
13:07
are we helping a patient adjust to you know, the potential need for an
13:13
amputation or their life after an amputation So what we wanted to be really careful
13:18
with here is that is that, you know, we not sort of make
13:22
decisions for people based on how we may perceive things or how we think most
13:28
people perceive things. And that's really where the AI piece comes in. If
13:33
we're able to, you know, accurately predict an outcome, or more accurately
13:37
predict an outcome, the patient can put their values on that outcome. And
13:43
if we're able to, you know, thinking about surgery, if we're able
13:46
to more accurately predict the risk, a patient can determine what's the right balance
13:52
between risk and benefit. How much risk is right for me in order to
13:56
achieve this particular benefit that I value. The benefit may be function, it
14:01
may be survival, it could be any number of things. But it's really
14:07
important for the patient to be able to determine the benefit, the value of
14:11
the benefit, and the what is for them the right balance between risk and
14:18
benefit really helpful. And I mean raises a host that this isn't a philosophy
14:22
podcast, but raises a host of interesting epistemological questions. We'll bracket those and
14:28
you and I can find about that another time. But yeah, I mean,
14:31
this is really interesting and I think really important work. So I want
14:37
to shift our focus a little bit and then we'll jump back in, but
14:41
point our gaze towards members of our audience who are in the health professions and
14:48
maybe a little bit more specifically to those training to be physicians and then hoping
14:52
in the future to train to be surgeons. And so I want to think
14:56
a bit about the surgeon's role in this with the a I assist and another
15:03
sort of helpful line from one that I focused on and reading from the paper
15:07
for those who have the paper with them as you listen to the podcast,
15:09
this is in the section improving on the eyeball test. And so you guys,
15:13
you all qualify the claims you say. We are not advocating that surgical
15:22
intervention should be offered indiscriminately. Rather, we believe that a patient's candidacy for
15:28
a proposed treatment should be based on an objective assessment of the likely outcome of
15:31
the treatment and the value that the patient rather than the physician, right,
15:35
just what you were saying, rather than the physician places on that outcome,
15:39
rather than on a flawed intuitive assessment. So this all sounds right, and
15:46
I think it's both good and important work. One question that arises in light
15:54
of the relationship between practitioners and patients, Between surgeons and patients, AI for
16:00
predicting outcome sounds super helpful. Is it fine grained? And I guess I'm
16:06
asking you to go beyond the article too, because I know you're doing a
16:07
lot of work in this area. Is it possible to have AI assists fine
16:11
grained enough to help determine candidacy relative to outcomes but also relative to the surgeons'
16:19
abilities and skill sets. Is that a fair and well posed question. If
16:25
not, I can, well, let me see if this sort of answers
16:30
that. So we can use We can use artificial intelligence both as a predictor
16:36
of patient outcomes. We can sort of train the system to make predictions based
16:41
on historic data. It's important to that historic data be unbiased, and so
16:45
we have to kind of analyze the data and scrub it to some extent.
16:48
But then is there also a role and I think this is what you're asking,
16:52
is there also a role potentially for using artificial intelligence to assess a surgeon's
16:59
skill So we we oftentimes will think of surgical complications as being largely patient determined,
17:07
so patients have complications because they we talked about you know, a very
17:12
common UH term that uses poor protoplasm. Basically, you know, this patient
17:19
had some underlying illnesses. They they were at high risk for a complication because
17:26
of you know, some factors that are completely patient centric. But what if
17:32
we were able to actually particularly surgeons using using the robot to operate performing robotic
17:41
surgery. What if the robot were to be able to analyze movements and determined
17:45
that some movements have a higher risk of complications than others. And so surgeons
17:51
who who have who displayed these sorts of movements, these sorts of kinetics.
17:56
It could be you know, time, it could be eye, hand,
18:00
to be any number of things. Again, one of the difficulties with AI
18:03
is that you really don't know what it's looking at. But what if what
18:07
if the AI system could analyze surgeon surgical kinetics and basically predict the likelihood of
18:14
a complication based on a combination of the patient factors, but also the surgeon's
18:19
contribution to patient's complication. Uh that you know, they're there number of these
18:25
systems are being engineered right now that are meant to be training systems and using
18:32
both AI and a virtual reality uh, in order to train surgeons to help
18:37
improve both for resident training and also to improve surgeon skills. But what if
18:44
what if a byproduct of this is also some sort of assessment and so there's
18:48
going to be an intrinsic assessment of the of the surgeon's skill and then perhaps
18:53
some some sort of module, some sort of training in order to remediate uh,
19:00
some technical aspects of the certain skill. Uh. And then what if
19:03
that what if that's not successful? What if we sort of set a bar
19:08
that you have to be able to perform higher than some level in order to
19:14
actually do this, because below this level, your your statistical risk of a
19:18
complication is unacceptably high. Uh. And again I'm sort of thinking about the
19:26
application of these training systems as an assessment system and also as a potential opportunity
19:33
to improve surgical quality. Excellent. Yes, yeah, you you picked up
19:41
on on where I was going and the training the training module framework. You
19:48
know, it's somebody teaches in a medical school. I think, uh,
19:52
sounds exciting and interesting and a possible I mean, it's a little bit of
19:56
fun here, but also a super useful tool. But it also raises potential
20:03
concerns and thinking about how some of the assessment data and processes might work in
20:08
the future if well, and I can imagine surgeons who are excellent and wonderful
20:17
and have very few complications and do very good work, but by and large,
20:22
maybe it takes them a little bit longer to perform particular kinds of surgeries
20:26
in light of specific patient features than others. And I guess, I wonder
20:33
this is guessing, right. I don't think you have a crystal ball,
20:34
but I could see assessment coming down in a way as employed by certain administrative
20:41
apparatuses which mean well, but sort of potentially hampering the surgeon's ability to just
20:48
do the surgery. Now we have time concerns because Charles is faster than bra
20:53
like, so how do we think about all of that data and could it
20:59
be used in a way sort of an incursion on the professionalism of the surgeon?
21:07
Is that possible? Am I? Is this? Like? Yeah?
21:08
Yeah, yeah, I think surgical autonomy, I think physician autonomy, clinician
21:14
autonomy. There's several potential fronts where artificial intelligence might affect impact the autonomy of
21:23
the clinician. You know it depends on how you want to frame this particular
21:29
instance of assessing surgical skills. I mean, some would see it as overall
21:33
a huge improvement in healthcare because again, you're gonna you're gonna take the best
21:38
people. Others would see it as not. I mean, right now,
21:41
you can basically be a surgeon if you if you want to be a surgeon
21:45
and you go through so there's no there's no exam, for instance, to
21:48
assess your technical skills before you enter into surgery residency, and you know,
21:55
should there be, can you teach it? I mean, these are all
21:56
really good questions, and I'm not sure we've ever really addressed them yet,
22:00
but I do think it's very interesting to think about a world in which there
22:03
may be some predictors of your success as a surgeon, your success in terms
22:11
of you know, patient outcomes, but even thinking more broadly about AI,
22:15
you know, there are at least three ways in which I think clinician autonomy
22:22
may well be rethought in the setting of of a I want is you know,
22:26
using AI for clinical decision making, there's going to be some level of
22:30
automation bias. So we know that just from the application of AI and other
22:34
areas when when individuals interact with AI systems for decision making, there's oftentimes this
22:45
sense of even though my decision may have been different, I will accept the
22:48
decision of the system because it's so much more powerful, it's so much more
22:52
well trained, it's you know, it's it's clear, it's tested. So
22:57
I think there that that automation bias may well be an issue. I think
23:04
another way in which autonomy might be impacted is payers may well not pay for
23:14
things that the prediction of the AI system if the AI system doesn't predict that
23:19
they're going to be beneficial to the patient. So, for instance, right
23:23
now, you know, many of us have healthcare plans that we have to
23:27
get pre approvable before we go to do something some procedure, and largely that's
23:33
that's approval about the appropriateness of the procedure. What if what if the payers
23:40
say, we're not going to pay for these things unless the system predicts that
23:42
they're going to be beneficial to you in some way. Now, there's a
23:48
lot of talk about potentially how much of our healthcare dollars in the United States
23:56
are spent on things that aren't valuable, and a push toward value based care.
24:04
But what if the system you only got reimbursed as a physician, as
24:11
a hospital system, you only got reimbursed if the system predicted that this was
24:15
going to be beneficial to the patient, And then how do you match that
24:18
benefit to patient values? Which is what we've talked about previously. So that's
24:22
a second area in which I think autonomy of the individual might be affected.
24:26
And the third is what happens when these systems become the standard of care,
24:30
the medical legal stated of care. So right now, if an individual you're
24:37
taking care of a patient, the patient suffers some sort of adverse outcome or
24:41
some sort of harm, and there is a claimmate, a legal claim made,
24:48
then the standard of care becomes the way in which an individual is assessed
24:52
for having either performed according to the standard of care or not. And right
24:59
now, or the way that you sort of a substinative care is that you
25:02
get experts in the field to say, this is what the standard of care
25:04
is, this is what the individual, how the individual performed, and this
25:08
was either a substandard of care or it meant the standard of care. What
25:11
if the standard of care becomes automated, What if the standardive care becomes the
25:15
AI system, and the system determines what the standard of care is. And
25:19
so if the system makes a prediction and an individual disagrees with the prediction,
25:26
then essentially you have to follow the prediction if it's the standard of care.
25:32
And that becomes important from a medical legal perspective, and it could get very
25:36
tricky from a moral perspective, because what if the AI system makes a prediction
25:42
and the individual clinician thinks that that is the wrong prediction and thinks that the
25:48
patient will be harmed if you follow if they follow that prediction, but they're
25:55
sort of they don't have any other choice because it's the standard of care.
25:57
And if you don't, if you're if you're wrong and the patient is harmed
26:03
and you acted contrary to the system, then you would be almost certainly be
26:07
liable if the system is the standard of care. Whereas if the system is
26:11
the standard of care, even if the patient is harmed, you're typically you
26:18
typically have have some basis for disputing that. You have some basis for saying
26:23
anyhow you're you're you're determined to be less liable for the harm if you follow
26:29
the standard of care, So you can see how that could pose a conundrum
26:33
for individuals if there is disagreement between the surgeon and the AI, right,
26:41
and if it becomes standard of care. I mean, what's interesting. I'm
26:45
not a clinician, a philosophy thinks about these things. What's interesting from the
26:48
outside is it's not as though there's it sounds like there's a rubric, right,
26:56
excuse my simplicity, Charles cut here, he ought to cut there,
27:00
or something like that, But it's you're actually appealing to another practical reasoner.
27:06
So it's the AI system making these kinds of determinations as to what standard of
27:11
care would be. I mean, suppose this happens, right, Suppose we're
27:14
there and that's what's going on. Is the AI system the surgeon or is
27:18
the flesh and blood human being the surg I mean, how do we think
27:22
about who's sort of running the show? Well, well, since you,
27:30
since you sort of were your surgeon hat for a second, I think my
27:33
philosopher hat just for a second, and then I'll take it off very promptly,
27:37
because it's a it's a it's like the crown. It's a little heavy
27:40
for my head. But so what so I think about you know, who
27:45
is the physician as an agent? What is the agency or the physician?
27:48
The agency the physician is really to make decisions, right the physicians clinicians,
27:53
we make decisions. We we even in offering something to a patient, even
28:00
in going through the range of options with the patient, even if we even
28:03
if we engage with a patient and shared decision making to make a final decision,
28:08
we make decisions in advance of that and acceding to that, in which
28:12
we decide what's the appropriate range of things to offer or what's the appropriate range
28:18
of things with which to engage in share decision making with the patient? And
28:22
so you know, I see the agency of the clinician as a decision maker.
28:30
But what if that decision making somehow is shared with a system. What
28:36
does that mean both for the system and also for the clindition. I think
28:40
when it comes to the assignment of liability or responsibility more generally, I don't
28:48
think it's going to be sustainable to simply have the credition always be responsible for
28:55
the decision that a system makes, either for executing it or for being responsible
29:00
for the decision itself, particularly if things go awry. I think there's probably
29:06
going to be something like some sort of corporate responsibility or shared responsibility. But
29:11
that's one of the things that that many people are trying to hash out now,
29:15
is you know, how does that responsibility book when a system is really
29:21
driving the decisions the clinicians are making. I'm noting all the questions I'm going
29:29
to ask you when I con you into coming back on the show. Thank
29:33
you so much for being here. I'll let you get out just pointing us
29:36
to any new projects, any particular work that our audience after they read this
29:41
can check out, or sort of what's next for you. Well, I
29:45
am in the process of writing a book. The University of California Press has
29:52
contacted me and asked me to write a book about some of the ethical issues
29:56
when AI is used for clinical decision make and so I am in the process
30:02
of writing that along with a co author, Tyler Loftus, who's a surgeon
30:06
at the University of Florida, And we're really looking at this from the perspective
30:10
of different stakeholders, including the patient, most especially the patient. So hopefully
30:14
that will be out before too long, And if your listeners have any interests,
30:18
they can take a look at the book excellent. Charles, thank you
30:22
so much for joining me. I really appreciate it. When the book comes
30:26
out, we'll have to get you back on the podcast because I'm sure folks
30:30
will have picked it up by then. So again, thanks very much for
30:33
the time, and we'll talk again soon. Great. Thank you, Brian.
30:36
All Right, have a good one. Thank you for listening to this
30:41
episode of the covid Ethics series podcast. Be sure to check out video versions
30:45
of our episodes online at library Shoe dot edu slash covid ethics, follow the
30:52
show on Twitter at Shoe Bioethics, and our moderator me at bcp ethics.
30:59
And if you're listening to this show on iTunes, Spotify, or wherever you
31:02
get your podcasts, please subscribe and leave a review. If there's a topic
31:07
you'd like to see discussed in the future, let us know