Latest / Code & Cure / #15 - When Algorithms Know Your End-Of-Life Wishes Better Than Loved Ones

Transcript

#15 - When Algorithms Know Your End-Of-Life Wishes Better Than Loved Ones · Code & Cure

• 0:01
  SPEAKER_01: Family members often make critical medical decisions in a crisis.
• 0:05
  SPEAKER_01: But what if their instincts are wrong?
• 0:08
  SPEAKER_01: New research shows that machine learning might be able to predict your preferences for CPR better than your partner could.
• 0:16
  SPEAKER_01: Is that a future we're ready for?
• 0:27
  SPEAKER_00: Hello and welcome to Code and Cure.
• 0:29
  SPEAKER_00: My name is Vasant Sarati, and I'm an AI researcher, and I'm with Laura Hagopian.
• 0:34
  SPEAKER_00: I'm an emergency medicine physician.
• 0:36
  SPEAKER_00: Yeah, and one of our top our topics today is I'm very excited about because it's to me a very sort of different way of thinking about AI.
• 0:43
  SPEAKER_00: We've been sort of talking about AI as a tool to help with automating tasks and those kinds of things.
• 0:50
  SPEAKER_00: But playing a very human-like role is what this topic is going to be about today.
• 0:56
  SPEAKER_00: And I honestly, when I read this paper, I was kind of like, whoa, that's a very creative, but potentially, you know, fraught uh use case for AI.
• 1:06
  SPEAKER_00: And so I think we let's let's just dive in, right?
• 1:08
  SPEAKER_00: Let's just dive into it.
• 1:09
  SPEAKER_00: Yeah, let's just do it.
• 1:10
  SPEAKER_00: Yeah.
• 1:10
  SPEAKER_00: So the paper is um about uh using a machine learning system to help with uh patient preferences for their end-of-life care.
• 1:21
  SPEAKER_00: Um and Laura, jump in if I'm using the wrong terminology.
• 1:24
  SPEAKER_01: No, no, you're using the right terminology.
• 1:26
  SPEAKER_01: It's just like uh a little uh technical mumbo jumbo there.
• 1:30
  SPEAKER_01: But it's basically like can a machine learning algorithm predict whether someone would want CPR or not?
• 1:38
  SPEAKER_01: And can it do a better job than someone's own partner?
• 1:40
  SPEAKER_01: Because oftentimes it's someone's partner who ends up having to make that decision in the moment if someone's incapacitated.
• 1:46
  SPEAKER_00: Well, that's the key, right?
• 1:47
  SPEAKER_00: They're incapacitated.
• 1:48
  SPEAKER_00: In the normal course, you would just ask the person, but for whatever reason, if they're not able to answer, then you're gonna have the proxy or someone else make the decision for them.
• 1:56
  SPEAKER_00: Which is like very tough in the moment if you're not sure.
• 2:00
  SPEAKER_00: Yeah.
• 2:00
  SPEAKER_00: And honestly, you would imagine that the human that you are have entrusted with that capability is going to be is going to stay true to your wishes and do the right thing in that, in that in that position.
• 2:11
  SPEAKER_01: And um what's I mean, that's interesting, right?
• 2:15
  SPEAKER_01: Because it's like if you've had that conversation, maybe they know what your wish actually is.
• 2:20
  SPEAKER_01: But what if you haven't had that conversation?
• 2:22
  SPEAKER_01: And even in this paper, I think they they actually went to people's partners and they said, Hey, do you think this person would want CPR?
• 2:30
  SPEAKER_01: Yeah.
• 2:31
  SPEAKER_01: And there were a decent number of people who said, I don't know.
• 2:36
  SPEAKER_01: And there were a decent number of people who were wrong.
• 2:40
  SPEAKER_00: Yeah.
• 2:40
  SPEAKER_00: So let's get into that.
• 2:41
  SPEAKER_00: Um, so what they did was they took a bunch of data.
• 2:44
  SPEAKER_00: I think they had about 1800 uh participants, and through a combination of survey questions and interviews, they were able to get a sense for uh various things.
• 2:54
  SPEAKER_00: So they had they got demographic information, the person's sex, age, you know, whether they have children, marital status, and that sort of thing.
• 3:00
  SPEAKER_00: Um, they got a lot of health-related information about their medical conditions, about uh their physiological, you know, things like um, do they smoke often and what's their body mass index and those sorts of things.
• 3:14
  SPEAKER_00: Um, I think they also have information about their clinical history, about their visits and um who they have as as their um who they got the vaccinations and that sort of thing.
• 3:24
  SPEAKER_01: Yeah, that they have their colon cancer screening, all of that.
• 3:26
  SPEAKER_00: So there's like the clin the demographics, yeah, the clinical, and then a third layer.
• 3:30
  SPEAKER_00: The third was the values.
• 3:31
  SPEAKER_00: They asked, they asked people a number of questions about the importance of having uh importance of sort of questions, things like importance of being a burden, or uh importance of not dying alone, or importance of having physical contact.
• 3:44
  SPEAKER_00: These are sort of gets to the very core values of who those people are.
• 3:48
  SPEAKER_00: Um, and so they asked those kinds of questions.
• 3:51
  SPEAKER_00: Um and beyond that, I think they asked other questions to explore other mental state uh considerations.
• 3:57
  SPEAKER_00: How often do they pray?
• 3:59
  SPEAKER_00: How extra how extroverted are they, how lonely are they, how open are they, and those sorts of things.
• 4:04
  SPEAKER_00: So they got all this data, and of course, uh they got data about whether or not they would choose certain end-of-life uh um conditions, right?
• 4:13
  SPEAKER_00: And um, this is the data they used to train machine learning models uh on.
• 4:19
  SPEAKER_00: And so there's 1800 participants, and actually what's interesting is of the 1800 participants, um about 800 of them were also uh there with uh partners.
• 4:30
  SPEAKER_00: So they were able to ask the partners about the potential end-of-life decisions as well.
• 4:34
  SPEAKER_00: So they got both the patient's answer and the partner's answer.
• 4:38
  SPEAKER_01: And the you know, AI's answer.
• 4:40
  SPEAKER_00: Well, the AI's answer not yet, right?
• 4:41
  SPEAKER_00: This is just the data collection piece, right?
• 4:43
  SPEAKER_00: So the AI, what they did with the AI was they had sort of three models of AI.
• 4:48
  SPEAKER_00: One that they trained basically on demographic information uh and where they live and that sort of thing.
• 4:54
  SPEAKER_00: And um, one that they trained on that demographic information plus all the clinical factors that we talked about before.
• 5:00
  SPEAKER_00: And then a third model that had the basic demographic information, the clinical information, and all the personalized things about their values and their personal um sort of dispositions.
• 5:13
  SPEAKER_00: Um and so those are the models they trained, and they tested it against, you know, they used standard machine learning techniques to train these models, and they tested them against both the patient's own answers and uh also compared them with their patient's partner's answers.
• 5:31
  SPEAKER_00: And what was striking was all three models were better than the on average than uh in accuracy than the patient's uh human partner.
• 5:41
  SPEAKER_00: Um, I think they were reaching the human partner had an accuracy of around 58%, which seems uh to me at least uh awfully low.
• 5:50
  SPEAKER_00: And the basic model that used just demographic information was above 60%, and the model that used all of the personalized information, I believe, was above 70%.
• 6:00
  SPEAKER_00: And um, which, you know, is is uh it blew my mind away that we would get this level of a difference.
• 6:07
  SPEAKER_00: Uh you know, we're used to seeing situations where the human is always right and the AI systems are getting better and better and getting closer to the human.
• 6:14
  SPEAKER_00: And here we have the flipped problem, which is the AI seems to be way better uh at predicting this.
• 6:20
  SPEAKER_00: And in fact, they did a uh study, uh, they did a sort of explainability type study to see um what features or what aspects of the patient are most influencing their the the accuracy of the results.
• 6:35
  SPEAKER_00: So what what were they?
• 6:36
  SPEAKER_00: What were the factors?
• 6:37
  SPEAKER_00: Demographics was the key thing, which is kind of like age, age and things you can't change, right?
• 6:43
  SPEAKER_00: How many kids someone has, yeah.
• 6:45
  SPEAKER_00: Maybe some of those things you can change, but age and sex you can't really.
• 6:48
  SPEAKER_00: And and so they're that that's the kind of information that was the most predictive.
• 6:52
  SPEAKER_01: Not the clinical history, not the all the not the sort of values, personalization, yeah, the demographics in and of themselves.
• 7:00
  SPEAKER_00: Uh was the was the strongest of all of these pieces, which again is another incredible thing, which you know gets to questions of free will and all of that stuff.
• 7:09
  SPEAKER_01: Because I know this is sort of like mind-blowing.
• 7:11
  SPEAKER_01: It feels like we're in this like a murky ethical territory, but I'm also sitting here like, oh my gosh, people's partners don't know what they want.
• 7:19
  SPEAKER_01: Right.
• 7:20
  SPEAKER_01: And I suppose what you want can change moment to moment with a new diagnosis or things like that.
• 7:26
  SPEAKER_01: But the rate that partners knew or didn't know whether or not someone wanted CPR is just strikingly low.
• 7:34
  SPEAKER_01: Yeah.
• 7:35
  SPEAKER_00: And it's not clear to me to what degree.
• 7:37
  SPEAKER_00: I don't know if they measured this.
• 7:38
  SPEAKER_00: This might be a question for further inspection, but I don't know if they looked at whether the partners and the degree to which the partners had prior conversations with the patient uh about these issues, right?
• 7:52
  SPEAKER_00: Or was it just like presented as here's my partner, they're part of the study, what would they do?
• 7:56
  SPEAKER_00: Right.
• 7:56
  SPEAKER_00: Versus them having a prior conversation about some of these pieces, which I would assume would up their accuracy, right?
• 8:02
  SPEAKER_00: Because they would get it right if they knew this is what they wanted, right?
• 8:04
  SPEAKER_00: This is what patients want.
• 8:07
  SPEAKER_00: So I find these aspects to be very interesting.
• 8:10
  SPEAKER_00: And again, that's one of the reasons why I wanted to talk about this paper.
• 8:13
  SPEAKER_00: Uh, because it also hits upon many very core human ethical concerns that one has with any AI technology.
• 8:20
  SPEAKER_00: And here's one where we're talking about respecting personal autonomy.
• 8:25
  SPEAKER_00: This these people are, you know, there's a preference that they have, that's their autonomy.
• 8:30
  SPEAKER_00: Being able to respect that and get that correctly is a huge piece.
• 8:34
  SPEAKER_00: But it's not just about the patient.
• 8:36
  SPEAKER_00: There's a stakeholder here who, you know, is the patient's partner who is going to have to deal with the after effects of their choices, right?
• 8:44
  SPEAKER_00: And and they're in in play as well.
• 8:46
  SPEAKER_00: So it's not just about a machine often making the more correct choice.
• 8:50
  SPEAKER_00: The more correct choice might be more challenging than that, right?
• 8:53
  SPEAKER_00: It needs to also do other things.
• 8:55
  SPEAKER_00: It can't just be correct, it has to also be respectful of the stakeholders involved.
• 8:59
  SPEAKER_01: This is an interesting one because like we know what the patient wanted in this situation.
• 9:06
  SPEAKER_01: We know what they wanted in this scenario and in this paper, but like in the real world, you might never find out what they actually had wanted.
• 9:13
  SPEAKER_01: No, right?
• 9:14
  SPEAKER_01: But you'll never know the answer.
• 9:15
  SPEAKER_01: You never know the answer.
• 9:16
  SPEAKER_01: Yeah.
• 9:16
  SPEAKER_01: Um, but like Well, maybe you do after if they survive.
• 9:20
  SPEAKER_01: If they suppose that's true.
• 9:23
  SPEAKER_00: Let's hope we know the answer.
• 9:24
  SPEAKER_01: But there are a lot of situations where you may not know the answer, um, which is like difficult for a partner to deal with if they're in that moment and have to make that call.
• 9:36
  SPEAKER_01: But at the same time, you're like, well, what if the machine learning algorithm like was a false negative?
• 9:44
  SPEAKER_01: Or what if it was a false positive?
• 9:45
  SPEAKER_01: Like, what if someone didn't get CPR and they would have wanted it?
• 9:48
  SPEAKER_01: Or what if someone would not have wanted CPR and we did it to them anyways?
• 9:53
  SPEAKER_01: Like that's, you know, right, not good.
• 9:57
  SPEAKER_00: Right, right.
• 9:58
  SPEAKER_00: And it's also, you know, there's also other uh the the the survey takes into account a lot of other factors about values and such, but um, there's also one thing, which is that these values change over time, right?
• 10:12
  SPEAKER_00: And so it's not static.
• 10:13
  SPEAKER_01: But it's interesting because a lot of the demographic stuff that we were talking about that feeds into this is static, right?
• 10:21
  SPEAKER_01: Like um your sex, your age, uh your education level, your language.
• 10:26
  SPEAKER_01: Um, you know, some things can can be adjusted, like your partnership status, um, having children, et cetera.
• 10:33
  SPEAKER_01: But a lot of those things are static.
• 10:35
  SPEAKER_01: So then it's like, well, if your preferences can change, but the main predictors in here, those don't change that much.
• 10:43
  SPEAKER_00: Yeah.
• 10:44
  SPEAKER_00: I mean, I I don't know if age was more than uh others, but age I could imagine is you could have a certain viewpoint when you're young and different viewpoint when you're older.
• 10:52
  SPEAKER_00: Yeah, absolutely.
• 10:52
  SPEAKER_00: Uh, but but you're right about the others, and they don't change.
• 10:54
  SPEAKER_00: So that's interesting um as well.
• 10:57
  SPEAKER_00: Um, but also there's also other cultural pieces, right?
• 11:00
  SPEAKER_00: To the what degree are they a certain religious background that requires them to have a certain viewpoint about these end-of-life decisions.
• 11:07
  SPEAKER_00: And so those, I don't know that the survey took into account that.
• 11:10
  SPEAKER_00: And I also also honestly, I don't know to what degree the the the participants were culturally varied in that sense as well.
• 11:17
  SPEAKER_01: Um, this was dead in one location in Sweden, right?
• 11:20
  SPEAKER_01: Switzerland, yeah.
• 11:21
  SPEAKER_01: That's all right.
• 11:23
  SPEAKER_00: Um, but but yeah, so I think there's some obviously some other factors, which which is all to say that the human partner is still important.
• 11:30
  SPEAKER_00: And you know, we don't want a situation where, I mean, well, I I don't think at least that we want a situation where the human partner is just replaced.
• 11:38
  SPEAKER_00: Now, one of the arguments made in the paper, and the paper acknowledges all of this, by the way, but one of the arguments made in this paper is that the human partner might experience a quite a bit of trauma from having to make that decision in the first place.
• 11:51
  SPEAKER_00: Absolutely.
• 11:51
  SPEAKER_00: And so and afterwards, right?
• 11:53
  SPEAKER_00: Right.
• 11:53
  SPEAKER_00: And so having them lean on an AI system or a machine learning system might be a good thing, right?
• 11:58
  SPEAKER_00: Um, that's sort of one way to look at it.
• 12:01
  SPEAKER_00: To have them sort of offload some of that trauma to another system that potentially takes into account things that they hadn't thought about in that moment.
• 12:08
  SPEAKER_00: Um, that if they had given thought, maybe they would agree with the machine, right?
• 12:12
  SPEAKER_00: And so it's not clear to me that the that, you know, having a machine isn't is necessarily bad.
• 12:18
  SPEAKER_00: It could actually be a helpful thing for the human.
• 12:20
  SPEAKER_01: Right.
• 12:21
  SPEAKER_01: And we talk about human in the loop all the time.
• 12:23
  SPEAKER_01: This is like a different context of that, right?
• 12:26
  SPEAKER_01: It's the partner in the loop in this situation where you know you don't want you don't want the AI making the decision alone.
• 12:36
  SPEAKER_01: That feels sort of like icky in a way, right?
• 12:40
  SPEAKER_01: And if you have both, like if you have both the machine learning algorithm and the partner, like do you choose between the two?
• 12:48
  SPEAKER_01: Do you let the two work together?
• 12:50
  SPEAKER_01: Uh it's sort of sort of an odd thing to think about.
• 12:54
  SPEAKER_00: Yeah.
• 12:54
  SPEAKER_00: And how do you make that available?
• 12:55
  SPEAKER_00: Presumably, the final decision maker is not going to be the machine, is going to be the human partner, at least in our system, right?
• 13:01
  SPEAKER_00: As as things stand.
• 13:02
  SPEAKER_00: So maybe it's a tool for the human partner to choose to use or not use as it as they are guided in their in their final decision, right?
• 13:10
  SPEAKER_00: Um, or is it uh are we suggesting that we take that partner out of the equation completely and have the AI system be substituted for the the patient, sort of a patient surrogate in a sense, right?
• 13:21
  SPEAKER_00: Which is the thing.
• 13:21
  SPEAKER_00: Yeah, although then it's weird.
• 13:23
  SPEAKER_01: It's it's weird because you're like, oh, where there's like an ick component to that of we don't have, do we not have free will?
• 13:29
  SPEAKER_01: Like uh, you know, what about wouldn't our proxy maybe like take into account more of the context or maybe have more knowledge of us?
• 13:39
  SPEAKER_01: I mean, they definitely showed in this article that like partners don't know.
• 13:43
  SPEAKER_01: But my big takeaway from that was like, why don't partners know?
• 13:48
  SPEAKER_01: So I read this and I was like, oh, this is a very interesting article that they can predict this, but like, why the heck don't partners know what patients want?
• 13:57
  SPEAKER_00: Great point.
• 13:58
  SPEAKER_00: And you know, it's to me, that sounds what you're saying sounds like is as much of as much as we're talking about how AI is very good at this, uh, we're also effectively saying that humans are really bad at this, actually.
• 14:08
  SPEAKER_00: Yeah, that's exactly a conclusion.
• 14:10
  SPEAKER_01: People don't want to talk about end-of-life stuff.
• 14:13
  SPEAKER_01: And so I was reading this and I was like, geez, okay, well, everyone should have a healthcare proxy, whether that's their partner or not.
• 14:20
  SPEAKER_01: They should have someone who can make decisions when they're incapacitated, right?
• 14:27
  SPEAKER_01: When they're not able to do so.
• 14:30
  SPEAKER_01: And also they should talk to their healthcare proxy, they should talk to their partner, they should talk to their provider about what they would want in a situation if they were incapacitated.
• 14:42
  SPEAKER_01: This is especially true for older individuals, but like, I don't know what what happens if you're young and in a car crash.
• 14:48
  SPEAKER_01: It's like, what would you want ahead of time, specifying that and letting them know?
• 14:53
  SPEAKER_01: That's like a hard conversation to have, though.
• 14:55
  SPEAKER_01: People don't want to have that conversation.
• 14:58
  SPEAKER_01: Um, and providers will well, primary care providers will will do this.
• 15:03
  SPEAKER_01: Um, there's like portable medical orders where you can specify, like, hey, if I were in a situation that I were incapacitated, I would or wouldn't want CPR.
• 15:14
  SPEAKER_01: I would or wouldn't want, you know, a breathing tube.
• 15:17
  SPEAKER_01: I would or wouldn't want other things like fluids or antibiotics, or, you know, I would or wouldn't want to focus on comfort and allow a natural death and get oxygen and suction, but allow more of a natural death.
• 15:31
  SPEAKER_01: Or I would or wouldn't want a feeding tube.
• 15:33
  SPEAKER_01: Those are all things that it's like, well, that that's that's like potentially an awkward conversation to have, a difficult conversation to have, but also a really important conversation to have at the same time.
• 15:46
  SPEAKER_00: Well, okay, so that's an interesting point because one reaction I have to that is, well, you could just ask the patient those questions and write down the answers.
• 15:56
  SPEAKER_00: Well, that's exactly what you're supposed to do.
• 15:58
  SPEAKER_00: Right.
• 15:58
  SPEAKER_00: But then what is the purpose of having the partner there?
• 16:01
  SPEAKER_00: The partner is there to sort of fill, to me at least, the part it seems like the human partner is there to fill in the gaps.
• 16:06
  SPEAKER_00: So you're in a current situation in which one of those existing patient answers don't fit nicely.
• 16:12
  SPEAKER_01: Oh, well, so first of all, people may not have that form filled out.
• 16:15
  SPEAKER_01: It may not be on file, right?
• 16:17
  SPEAKER_01: Like that's something you fill out with your provider and you both sign it and you put it in the record.
• 16:21
  SPEAKER_01: But like, what if what if it's not filled out?
• 16:23
  SPEAKER_01: Yeah.
• 16:24
  SPEAKER_01: Uh, what if your preferences might change over time?
• 16:27
  SPEAKER_01: Like maybe you wouldn't have wanted CPR for your uh if you had something related to your cancer, but if you were in a car accident, you would be okay with um with that or something something like that where the situation or the circumstance has changed.
• 16:42
  SPEAKER_01: And so it's like, you know, you kind of want a multi-pronged approach here.
• 16:46
  SPEAKER_01: And you want to make sure that both your provider and your proxy are sort of in the loop in terms of what you might want done.
• 16:54
  SPEAKER_01: These forms also don't cover all the things, right?
• 16:57
  SPEAKER_01: They don't cover like in X, Y, Z situation, this is what I would want done.
• 17:02
  SPEAKER_01: And that's why a proxy is like very important to have.
• 17:05
  SPEAKER_01: Yeah.
• 17:05
  SPEAKER_01: Because they're helping make those day-to-day decisions, and not all of those can be specified on this form.
• 17:10
  SPEAKER_01: It's like a limited number.
• 17:12
  SPEAKER_00: Yeah.
• 17:12
  SPEAKER_00: Yeah.
• 17:12
  SPEAKER_00: So I, you know, it seems to me that, yeah, right.
• 17:15
  SPEAKER_00: So the form is is sort of the first line of attack for uh somebody, you know, to to for a provider to decide whether or not a patient has wants this or not.
• 17:24
  SPEAKER_01: Oh, the provider's not deciding.
• 17:25
  SPEAKER_01: The patient's deciding.
• 17:26
  SPEAKER_01: Right.
• 17:27
  SPEAKER_01: The patient's deciding and the provider.
• 17:35
  SPEAKER_00: No, no, no, no.
• 17:38
  SPEAKER_00: Well, that's exactly why you have a a proxy.
• 17:41
  SPEAKER_00: Yes.
• 17:42
  SPEAKER_00: So the proxy comes in when there's nuance to the situation.
• 17:44
  SPEAKER_01: Well, they're yeah, they're gonna come in no matter what in these situations, because there's always going to be some sort of nuance to it.
• 17:52
  SPEAKER_01: Yes.
• 17:53
  SPEAKER_01: But I think one of the important things to note is that like your preferences can change over time, and a lot of people haven't filled out these forms.
• 18:01
  SPEAKER_01: So it's like, sure, a form could exist.
• 18:03
  SPEAKER_01: Like, is it on someone's record?
• 18:05
  SPEAKER_01: Who knows?
• 18:06
  SPEAKER_00: Yeah.
• 18:07
  SPEAKER_01: Which is why, you know, when I read this paper, I was like, ooh, people, people have not talked to their partners or their proxies about what they want.
• 18:16
  SPEAKER_00: That's what it is.
• 18:17
  SPEAKER_00: Right.
• 18:18
  SPEAKER_00: That's right.
• 18:18
  SPEAKER_00: That's right.
• 18:19
  SPEAKER_00: In fact, that's exactly right.
• 18:20
  SPEAKER_00: Because in a given, you're right, in a given situation, there is nuance always.
• 18:24
  SPEAKER_00: So you have to rely on that proxy to actually reason and think about that situation against what they believe the patient to want.
• 18:35
  SPEAKER_00: And unless the patient and the proxy have had those conversations, they would not be able to reason that at the time when they need to reason with that.
• 18:42
  SPEAKER_01: Right.
• 18:42
  SPEAKER_01: I mean, I'm gonna use you as an example.
• 18:44
  SPEAKER_00: Oh no.
• 18:45
  SPEAKER_01: Like do you have one of these forms on file with your doctor that says what you would want in a in this type of situation?
• 18:53
  SPEAKER_00: Maybe not.
• 18:54
  SPEAKER_01: I don't think so.
• 18:55
  SPEAKER_01: And then, like, have you talked to your partner or your healthcare proxy about what you would want if you were incapacitated?
• 19:03
  SPEAKER_00: No.
• 19:03
  SPEAKER_01: Well, that's the thing.
• 19:04
  SPEAKER_01: Like, that's you're like everybody else, though.
• 19:07
  SPEAKER_01: Right.
• 19:07
  SPEAKER_01: Yeah.
• 19:08
  SPEAKER_01: And so, you know, what would your partner do in that kind of situation?
• 19:13
  SPEAKER_01: Would they say, I'm not sure?
• 19:15
  SPEAKER_01: So let's pull the plug, or I'm not sure, let's just do CPR.
• 19:20
  SPEAKER_01: Uh, it feels like maybe we should do something, so let's just do it.
• 19:23
  SPEAKER_01: So it's it's really difficult for the partner in the moment to make that choice on behalf of you when they don't know what your preferences are.
• 19:32
  SPEAKER_01: And I think that's the point is we need to talk to our proxies about what our preferences are.
• 19:38
  SPEAKER_01: Yeah.
• 19:39
  SPEAKER_01: So that that was my main takeaway from this article is we're not doing a good enough job about discussing end-of-life care with our proxies.
• 19:46
  SPEAKER_00: Yes.
• 19:46
  SPEAKER_00: I I think I agree, I completely agree with that.
• 19:49
  SPEAKER_00: And it's less about the degree to which AI should be used, should not be used, and all of that is a separate, almost a separate question.
• 19:55
  SPEAKER_00: Because what I found interesting was from the study was also that a lot of the answers were an I don't know answer.
• 20:00
  SPEAKER_00: Now, what percentage of those I don't know answers could be turned into a yes or no answer if the proxies actually had conversations beforehand?
• 20:07
  SPEAKER_00: And then whatever is left over as truly I don't know, maybe those are the ones that we rely on a machine learning tool or something else to help us give us more information to give us to say, okay, this person also said this and this and this, or they would have said this and this.
• 20:21
  SPEAKER_01: Um Yeah, it's interesting that they allowed I don't know answers because when you're in a situation, there's no I don't know.
• 20:28
  SPEAKER_01: There's no I don't know.
• 20:29
  SPEAKER_01: There's either you do CPR or you don't do CPR.
• 20:33
  SPEAKER_01: And so you don't get to say, I don't know.
• 20:37
  SPEAKER_01: It's like a one or a zero, it's a yes or a no.
• 20:40
  SPEAKER_01: And so like the model wasn't allowed to say I don't know, but the human partners were allowed to say I don't know.
• 20:45
  SPEAKER_01: So I think that was an interesting piece of this study because in the real world you you're stuck deciding, even if you don't know.
• 20:54
  SPEAKER_01: Right.
• 20:55
  SPEAKER_01: Yeah.
• 20:55
  SPEAKER_00: Right.
• 20:55
  SPEAKER_00: More like the model in that sense.
• 20:57
  SPEAKER_00: Yeah, yeah.
• 20:59
  SPEAKER_00: No, yeah, just so many, so many um interesting questions here and does require a fair degree of self-reflection, at least for me, coming out of this, that maybe I need to have these conversations more and and get them get them clear.
• 21:12
  SPEAKER_00: Now, is there a way to do that?
• 21:14
  SPEAKER_00: I mean, is there forms?
• 21:15
  SPEAKER_00: Are there uh things that people is there a framework with which to have this conversation with the R with the proxy, or is that something we should look into and and and and figure out as a as a as a as a group?
• 21:27
  SPEAKER_01: Yeah, I mean, there are these forms that you fill out with your provider called pulse forms.
• 21:32
  SPEAKER_01: Um, they're like portable medical orders that you can discuss all the things that I was listing, like CPR, um breathing equipment, uh, feeding tubes, um treatments like IV fluids, that kind of stuff.
• 21:46
  SPEAKER_01: But you could also just sit down and have a conversation with your proxy so that you can let them know what you would want if you weren't able to make a decision in that moment.
• 21:59
  SPEAKER_00: Right, right, right, right.
• 22:02
  SPEAKER_00: Yeah, no, this is great.
• 22:04
  SPEAKER_00: Um I've just blown your mind.
• 22:07
  SPEAKER_00: You have.
• 22:08
  SPEAKER_00: I I've left you speechless.
• 22:10
  SPEAKER_00: Well, I again, like I said, I'm coming from a place where so far we've been talking about AI in this very almost mechanical way.
• 22:19
  SPEAKER_00: That is, there is a there's a human thing, humans get really annoyed doing this thing, and AI can come and do this thing for you.
• 22:25
  SPEAKER_00: Are they as good as humans?
• 22:26
  SPEAKER_00: No, but we're getting there.
• 22:28
  SPEAKER_00: And but we have to make sure that they are trustworthy, blah, blah, blah.
• 22:31
  SPEAKER_00: Right?
• 22:31
  SPEAKER_00: That's the the the the general narrative of health and AI.
• 22:35
  SPEAKER_00: Now AI, in my view, and and it's this is because thanks to you, um, I was able to see this, which is that now AI is saying, hey, look, here are all the gaps in human thinking, uh actually, and maybe not human thinking, but human behavior, and AI is just exposing them.
• 22:52
  SPEAKER_00: Here is one example where the proxies are are potentially not having conversations, which is why you have this poor performing proxy, right?
• 23:01
  SPEAKER_00: Or uh that means that they don't know their own uh their partners well enough, right?
• 23:05
  SPEAKER_00: To to actually answer this question.
• 23:07
  SPEAKER_00: Um, but the AI is the one that's doing the exposing.
• 23:10
  SPEAKER_00: So it's less about AI as a tool, uh AI being used to replace the human.
• 23:13
  SPEAKER_00: It's much more that it's informing us that the humans need to do better, which I think is is wonderful in a sense.
• 23:20
  SPEAKER_01: Yeah, I think it's definitely a different angle on this one.
• 23:24
  SPEAKER_00: Yeah.
• 23:25
  SPEAKER_01: So I think we can we can end here, and I hope that everyone listening has a healthcare proxy or gets one, and everyone has a discussion about end of life care decisions with them and their provider.
• 23:37
  SPEAKER_00: Agreed.
• 23:37
  SPEAKER_01: Thanks for joining us.
• 23:38
  SPEAKER_01: We'll see you next time on Code and Cure.