Latest / Code & Cure / #48 - Good Medicine Starts With Saying I Don’t Know

Transcript

#48 - Good Medicine Starts With Saying I Don’t Know · Code & Cure

0:00
SPEAKER_01: Hey Vasanth, what's the diameter of the moon?
0:03
SPEAKER_01: What?
0:04
SPEAKER_01: Yeah, just just give me like uh like an estimate where you're you know like 90% certain it's in that range.
0:11
SPEAKER_01: Don't don't do like a zero to a million or something, but just like you know.
0:16
SPEAKER_00: I don't know, somewhere between five and fifteen thousand miles.
0:22
SPEAKER_00: I don't know.
0:23
SPEAKER_00: I I want you to be like ninety percent confident though.
0:26
SPEAKER_00: Oh.
0:26
SPEAKER_00: Um yeah, I think it's between five and fifteen thousand miles.
0:30
SPEAKER_01: Interesting.
0:31
SPEAKER_01: You were uh overconfidently wrong there, or maybe you were underconfidently wrong.
0:35
SPEAKER_01: I don't know.
0:36
SPEAKER_01: You didn't sound very confident in your estimate.
0:38
SPEAKER_00: Oh boy.
0:39
SPEAKER_01: Um, but but the moon's diameter, uh the mean diameter is around 2,159 miles.
0:46
SPEAKER_01: Did you feel certain or uncertain there?
0:48
SPEAKER_00: Oh boy.
0:50
SPEAKER_00: I was definitely uncertain, and I had no idea what I was what I was doing.
0:53
SPEAKER_01: Did I trick you into like trying to trying to be certain?
0:59
SPEAKER_01: Can but the AI sounds certain when it does this kind of stuff.
1:02
SPEAKER_01: Yeah, I mean I I had to be certain, right?
1:03
SPEAKER_01: I had to say something.
1:04
SPEAKER_01: I forced you to.
1:05
SPEAKER_00: You did Hello and welcome back to Code and Cure, uh, the podcast where we discuss decoding health in the age of AI.
1:22
SPEAKER_00: My name is Vasant Sarathi.
1:23
SPEAKER_00: I'm an AI researcher and cognitive scientist, and I'm here with Laura Hagopian.
1:28
SPEAKER_00: I'm an emergency medicine physician.
1:29
SPEAKER_00: I have to say that moon example is interesting, um, you know, and it's it's kind of the the space of questions that we just don't have a starting point for, right?
1:39
SPEAKER_00: There are some questions that we're just uncertain about.
1:42
SPEAKER_00: We just don't know where to begin, but maybe that's okay.
1:46
SPEAKER_01: Well, I was gonna say, I actually think that's a good thing.
1:49
SPEAKER_01: Like uncertainty isn't a bad thing.
1:52
SPEAKER_01: In fact, certainty when you're wrong is way worse.
1:56
SPEAKER_01: That's true.
1:57
SPEAKER_01: And in medicine, it's not like you come in with wrist pain and I say, well, you have a scaphoid fracture by just, you know, with my x-ray vision.
2:06
SPEAKER_01: It's like, oh, well, what could this be?
2:08
SPEAKER_01: What's the differential diagnosis?
2:10
SPEAKER_01: What's the list of things it could be?
2:12
SPEAKER_01: And then I move things up and down the list based on, you know, what is the x-ray show or what what hurts when I touch it, or do I see a deformity or whatever it is?
2:25
SPEAKER_01: Um I'm, you know, moving things up and down that list.
2:29
SPEAKER_01: And I also kind of prioritize things on the list that are maybe more emergent, right?
2:33
SPEAKER_01: Like if you have an open wrist fracture or the bone's sticking out, that's that's something that needs to go to the operating.
2:41
SPEAKER_01: Obviously, yeah.
2:42
SPEAKER_01: And so I think uncertainty is a good thing.
2:45
SPEAKER_01: It forces us to try to narrow down, figure out what the next steps would be from a diagnostic perspective and also from a management perspective.
2:54
SPEAKER_01: And especially when you're a generalist like me, you can't know everything about everything, right?
2:59
SPEAKER_01: Right.
2:59
SPEAKER_01: So you have to realize, oh, hey, I'm uncertain.
3:03
SPEAKER_01: And maybe this other doctor who specializes in ex orthopedics or OBGYN or whatever knows more about this than me.
3:11
SPEAKER_01: Let me call them and see what they think about this.
3:14
SPEAKER_01: Let me have a consultation.
3:16
SPEAKER_01: And so I think uncertainty is a good thing.
3:18
SPEAKER_01: It's like you don't want to be sure about something that you're unsure about.
3:24
SPEAKER_00: Yeah.
3:25
SPEAKER_00: Yeah.
3:25
SPEAKER_00: Clearly.
3:26
SPEAKER_00: I mean, and this is this is the tension, right?
3:28
SPEAKER_00: We build these AI models and they are always giving us an answer and sometimes overconfident.
3:34
SPEAKER_00: And we've talked about that in the past too.
3:37
SPEAKER_00: And they don't have a sense of the degree of uncertainty that they should be worried about and maybe answer the question with a consult or or ask for a consult, um, or even not answer the question.
3:51
SPEAKER_01: Oh, yeah.
3:52
SPEAKER_01: I think that's a key thing.
3:53
SPEAKER_01: It's like always it's trying to please you, right?
3:55
SPEAKER_01: So then it answers your question.
3:56
SPEAKER_01: Yeah.
3:57
SPEAKER_01: And it could be answering it wrong.
3:58
SPEAKER_01: Or maybe there's like four different answers, but because, you know, as humans, we want this like binary yes or no, or like this is the diagnosis, that's often what it's giving.
4:09
SPEAKER_01: Yes.
4:10
SPEAKER_01: Because it feels good to be on the receiving end of that.
4:13
SPEAKER_01: Yes.
4:13
SPEAKER_01: Even though that may not be right.
4:15
SPEAKER_00: Yeah.
4:15
SPEAKER_00: And those are when we talk about trying to please you and things like that, that we're referring to the the LLM world, the the open AI chat, chat GPT type of world, right?
4:24
SPEAKER_00: Of AI systems that are training to speak to you.
4:29
SPEAKER_00: But there's also the world of machine learning systems that are not chatbots, and they are just meant to, they're just trained for a specific task of identifying something, identifying, you know, what what is what is an object in an image?
4:44
SPEAKER_00: Like is it a cat, is it a dog, and so on.
4:46
SPEAKER_00: So they have a list of things it could be, and they have to pick one.
4:49
SPEAKER_00: So it's not like they can just abstain, right?
4:53
SPEAKER_00: They are act they're they're it's not even they're thinking about it.
4:55
SPEAKER_00: It's not that they're not trying to please you, they're just doing the thing that they're trained to do, which is predict one of those things.
5:00
SPEAKER_00: And how do we care characterize uncertainty in those cases, right?
5:04
SPEAKER_00: Also.
5:05
SPEAKER_00: Um, and I think in this paper it really comes up in the context of um medical classification tasks.
5:11
SPEAKER_00: So like looking at doctors' notes and figuring out ICD 10 codes, for example, um, which are important, right?
5:17
SPEAKER_01: Yeah, I mean, we use them for billing purposes.
5:20
SPEAKER_01: Um, and so it's like, well, you know, if someone just going with the wrist pain example, I guess I was using earlier, it's like, hey, what are we billing for?
5:28
SPEAKER_01: Are we billing for just like a sort of general wrist pain?
5:31
SPEAKER_01: Are we billing for a scaphoid fracture?
5:33
SPEAKER_01: Are we billing for a distal radius fracture?
5:35
SPEAKER_01: Are we, you know, so the diagnosis is important to get.
5:39
SPEAKER_01: And can we scan the chart and get that information?
5:42
SPEAKER_00: Right, right, exactly.
5:43
SPEAKER_00: And you can imagine a machine learning system to be trained on lots of data and lots of ICD 10 codes.
5:49
SPEAKER_00: I mean, for each code, maybe there's a bunch of data that they can be trained on.
5:53
SPEAKER_00: And you can imagine that it's set up to do this automatically and effectively, maybe even.
5:58
SPEAKER_00: And then you wonder, okay, it's telling me that this specific example is this specific code.
6:03
SPEAKER_00: How confident is it?
6:04
SPEAKER_00: Right?
6:05
SPEAKER_00: Because if it's not confident, I want to be able to jump in.
6:08
SPEAKER_00: I want to have a system that has maybe a human or somebody in the loop that can take it, escalate the issue, and identify and and figure out that this is an uncertain outcome.
6:19
SPEAKER_00: So I need to do something, dig deeper, and figure out the right answer.
6:22
SPEAKER_01: Or like, could the model even abstain from doing it?
6:25
SPEAKER_01: Right.
6:26
SPEAKER_01: Instead of being like, this is ICD 10 code I11, it's going to be like, oh yeah, I don't know what the ICD 10 code is.
6:32
SPEAKER_01: Someone needs to review this thing because my certainty level is like under 50% or something.
6:39
SPEAKER_01: I don't know.
6:39
SPEAKER_00: Yeah.
6:40
SPEAKER_00: So I I thought that this could be a great opportunity to dive deep into uncertainty because we talk about uncertainty, we talk about confidence, we talk about what, you know, but what does that really mean in these contexts?
6:52
SPEAKER_00: And it's and we know some degree of uncertainty and randomness when we play dice games, right?
6:58
SPEAKER_00: You roll, you roll some dice, and it could be one of six options.
7:00
SPEAKER_00: And if it's a fair die, then you know that it's going to be, you know, one-sixth uh chance of getting playing with unfair die.
7:08
SPEAKER_01: Right.
7:09
SPEAKER_01: I mean, to the extent that it's what is happening here.
7:11
SPEAKER_01: Do you have trick dice that you play with?
7:13
SPEAKER_01: I'm just saying if there are trick dice, then it changes everything.
7:15
SPEAKER_01: But my point is like we're just doing a regular die with one, two, three, four, five, and six.
7:20
SPEAKER_01: Okay.
7:21
SPEAKER_01: Yeah.
7:21
SPEAKER_00: So there is there is randomness in that, right?
7:23
SPEAKER_00: And so you roll the dice and there's some randomness in that.
7:25
SPEAKER_00: And your uncertainty is already kind of set in stone because it's it's the same thing with the coin, right?
7:29
SPEAKER_01: There's like a one in six chance that you're gonna roll a five.
7:32
SPEAKER_00: Right, right, right, right.
7:33
SPEAKER_00: So that's that's one type of uncertainty.
7:34
SPEAKER_00: But there's the other type of uncertainty, which is we just talked about with the moon, right?
7:38
SPEAKER_00: The moon diameter.
7:40
SPEAKER_00: You can see that these are different.
7:41
SPEAKER_00: It's not like the moon diameter could be one of eight options, right?
7:45
SPEAKER_00: The the moon diameter is a thing.
7:47
SPEAKER_00: It it is it is a number and it's in a certain range and it exists.
7:51
SPEAKER_00: It's just that you don't know about it, right?
7:54
SPEAKER_00: It's an unknown piece of knowledge for you.
7:57
SPEAKER_01: Well, it's yeah, it's hard for me to even conceptualize.
7:59
SPEAKER_01: Like with a die, I know it's gonna roll one of those six numbers, but with the moon, especially when I was asking you, I was like, you can't do between zero and a million, because like I would have done that.
8:08
SPEAKER_01: Yeah, then I would have been really certain.
8:10
SPEAKER_01: But if I have to narrow it down, I'm like, I actually had to look it up for this for this episode.
8:15
SPEAKER_01: I didn't know it off the top of my head.
8:16
SPEAKER_00: Yeah, yeah, yeah, yeah.
8:18
SPEAKER_00: And it's really hard to do that.
8:19
SPEAKER_00: And so that's it, it's a different flavor of uncertainty.
8:22
SPEAKER_00: And I kind of wanted to talk about that.
8:24
SPEAKER_00: And and there's some fancy words that are going to be thrown here.
8:26
SPEAKER_00: One of which I love fancy words.
8:29
SPEAKER_00: One of which is alleatoric uncertainty.
8:31
SPEAKER_01: Alleatoric.
8:32
SPEAKER_00: And the other is epistemic uncertainty.
8:34
SPEAKER_00: Epistemic.
8:35
SPEAKER_00: Okay.
8:35
SPEAKER_00: The I think the easiest way um to think about this is the following, which is imagine looking at a blurry photo of a die.
8:44
SPEAKER_00: A die has been rolled, and somebody took a photo of it, and it's blurry, somewhat blurry, and some number came up, and your job is to predict what number came up.
8:54
SPEAKER_00: Um, and if the die was blurry, covered, tilted, or you know, genuinely hard to read, like it was really, if you're looking at it and you're like, I can't tell what this is, that is aleatoric uncertainty.
9:04
SPEAKER_00: The case that means that the uh information, the data itself is ambiguous.
9:10
SPEAKER_00: Okay.
9:11
SPEAKER_00: Um, but if they let's imagine that an AI system was used for this, something like this.
9:15
SPEAKER_00: And if an AI was trained on a let's imagine that the AI was trained to look at all photos of normal six-sided dies, right?
9:21
SPEAKER_00: But now in the blurry photo that you gave it, um, it is a strange-looking, uh, transparent die that has some gold dots on it or something, something weird, right?
9:31
SPEAKER_00: With some solid some other symbols in it.
9:33
SPEAKER_01: That's not even in its schema because it's how it been trained on that.
9:35
SPEAKER_00: That is epistemic uncertainty.
9:37
SPEAKER_00: So that is outside of the model's knowledge zone.
9:41
SPEAKER_00: So do you see the difference?
9:43
SPEAKER_00: So, like one is within the model's knowledge zone.
9:45
SPEAKER_01: But the data isn't there.
9:47
SPEAKER_01: Yeah.
9:47
SPEAKER_01: And the other is like, hey, this is outside of what it even trained on.
9:51
SPEAKER_00: Well, it's not even the data isn't there, it's just that there's so many so much noise and ambiguity and all that other stuff, right?
9:55
SPEAKER_00: Yeah.
9:56
SPEAKER_00: So, you know, for instance, in the ICD coding setting, uh, you could imagine that a clinical note is ambiguous or incomplete.
10:02
SPEAKER_00: And other times I I never do that.
10:04
SPEAKER_00: I don't know what you're talking about.
10:05
SPEAKER_00: But but the other times the other times the note is clear to a human expert, but the model has not seen enough examples of the disease.
10:12
SPEAKER_00: And so one, the second example is much more um about the model's knowledge.
10:17
SPEAKER_00: It just doesn't know.
10:18
SPEAKER_00: Yeah, okay.
10:19
SPEAKER_00: And the first is it does, it could know, but it just hasn't, you know, there's too many other factors in the data itself to make it hard to uh quantify.
10:26
SPEAKER_00: And these are two different kinds of uncertainties, and we need to be able to catch them both.
10:30
SPEAKER_00: Yeah, that makes sense.
10:31
SPEAKER_00: Yeah.
10:31
SPEAKER_00: So um, you know, and the solutions would be different for each of them, right?
10:35
SPEAKER_00: The solutions would be different for each of them.
10:36
SPEAKER_00: And I think it's worthwhile also like even taking this die example further to think even deeper, which is what is that, right?
10:44
SPEAKER_00: That that first blurry, noisy type of thing that we just talked about.
10:47
SPEAKER_00: Well, one thing is that photo is off a die roll.
10:51
SPEAKER_00: So inherently, the die roll has some uncertainty.
10:54
SPEAKER_00: We just talked about that, right?
10:55
SPEAKER_00: You roll there, it's a one-sixth or whatever, right?
10:57
SPEAKER_00: There's a genuine chance of the die rolling in any of those six numbers before it landed.
11:01
SPEAKER_00: So there's that.
11:02
SPEAKER_00: Um, that's the inherent uh variability of the dice itself.
11:06
SPEAKER_00: Um, and and this you can think of in the ICD setting as being something like disease trajectories or um the underlying clinical outcome itself could be variable, right?
11:15
SPEAKER_00: There's something beyond the notes that exists.
11:17
SPEAKER_00: That's the same.
11:18
SPEAKER_01: Yeah, for example, if you have a clavicle fracture, most of them can be managed with a sling, but depending on the location of it, some of them might need to be operated on.
11:30
SPEAKER_01: And potentially if you have some that are really close to the sternum, like they may need a further workup because that that could indicate the trauma was very severe.
11:39
SPEAKER_01: So I'm going into unnecessary details, but like there's different like not all clavicle fractures are managed the same way.
11:46
SPEAKER_00: Yeah, no, exactly.
11:47
SPEAKER_00: And so that's so then then there's the distinct ambigu ambiguous observation, right?
11:51
SPEAKER_00: In the dye uh photo example, the photo was blurry, right?
11:55
SPEAKER_00: So that's an example.
11:56
SPEAKER_00: Uh in the ICD 10 um coding example, maybe the notes are vague, right?
12:01
SPEAKER_00: Um, something was like, you know, the example, you know, you might have better examples, but um it doesn't specify enough details for you to make a judgment, right?
12:11
SPEAKER_00: In some level.
12:12
SPEAKER_01: Um yeah, and that's probably not uncommon in some of these notes.
12:16
SPEAKER_01: I guess maybe AI scribes could help improve that, but it's like, you know, when you're dealing with a patient, the most important thing in front of you is the patient and the management of the patient.
12:25
SPEAKER_01: And maybe not all the detail goes into the note.
12:28
SPEAKER_00: Yeah, yeah, yeah.
12:29
SPEAKER_00: Then then there's missing evidence.
12:30
SPEAKER_00: That's another type of allegatory uncertainty.
12:32
SPEAKER_00: Where in the dice example, maybe in the photo, your finger covers part of the dice.
12:37
SPEAKER_00: So you can see it's occlusion, it's occluded somehow, and you can't see it.
12:40
SPEAKER_00: And in the ICD10 example, it could just be that maybe some values were uh lab value, lab results were omitted, or something was left out, um, or or there's some details that was like, you know, timing information was left out or something, right?
12:53
SPEAKER_00: Okay.
12:53
SPEAKER_00: Um that's like an example of missing evidence that's also eleatoric.
12:57
SPEAKER_00: Um, you have noisy measurements.
12:59
SPEAKER_00: Um, in the photo, you know, in the photo, it's like bad lighting.
13:03
SPEAKER_00: Um, can distort how those little pips look on the dice.
13:06
SPEAKER_00: Um so that's an example.
13:07
SPEAKER_01: Oh, pips, that's a good word.
13:08
SPEAKER_00: It is a good word.
13:09
SPEAKER_00: I I learned that when I was um preparing for this podcast.
13:13
SPEAKER_00: You just like dropped it in there.
13:14
SPEAKER_01: You're like, I sound so smart.
13:16
SPEAKER_01: I know the word pips.
13:17
SPEAKER_00: Yeah, yeah, yeah.
13:18
SPEAKER_00: Um, but your clinical text could contain typos, abbreviations, um, maybe even contradictory statements that you were not aware of, right?
13:26
SPEAKER_00: And that's an example of a noisy, noisy measurement at some level.
13:31
SPEAKER_00: Um and if a die was tilted in the photo, you might two people might disagree on that.
13:38
SPEAKER_00: That's a ambiguity of the label itself, right?
13:41
SPEAKER_00: Of what the die says.
13:43
SPEAKER_00: Um, in your ICD 10, two quoters might disagree on what that is.
13:47
SPEAKER_00: Um, you have maybe uh like fuzzy boundaries between classes.
13:53
SPEAKER_00: This relates to kind of the ambiguity of what it looks like.
13:55
SPEAKER_00: Like a five kind of looks like a six in a dice.
13:57
SPEAKER_00: Uh, and you know, you couldn't miss a square, maybe not in a dice, but like the numbers.
14:00
SPEAKER_01: I was gonna say, I don't know.
14:01
SPEAKER_01: Are you are you talking about yourself?
14:02
SPEAKER_01: Because I don't think I'd mistake a five and a six on a die.
14:05
SPEAKER_00: Yeah, but you could imagine mistaking um, you know, there are things that you could leave out, like there could be parts that are kind of again, this goes back to the point of blurry, I suppose, but there is inherent ambiguity there too.
14:18
SPEAKER_00: Um and you could have potentially other types of noise that I'm not captured here, but you know, there might be in an ICD 10 case, there might be one thing that relates to multiple ICD 10 codes.
14:31
SPEAKER_01: Oh, sure.
14:32
SPEAKER_00: Right?
14:32
SPEAKER_01: So then that's yeah, and in the ICD 10 coding system, there's like often a parent code, and then there's like subclassifications.
14:38
SPEAKER_01: Yes, like uh like a tree almost, right?
14:41
SPEAKER_01: So it's like here is the main diagnosis code, you have a clavicle fracture, and then underneath it, it's like, oh, the fracture is in the proximal, middle, or distal clavicle.
14:51
SPEAKER_01: Yeah.
14:52
SPEAKER_01: So you get more specific and you can bill for it more appropriately if you've if you like hone in on the most specific code available.
15:00
SPEAKER_00: Got it.
15:00
SPEAKER_00: And and the way to think about this aleatoric uncertainty is that the medical note itself does not cleanly determine the code.
15:08
SPEAKER_01: Yeah, I mean, the code is for billing purposes.
15:10
SPEAKER_01: The medical note is to communicate something in the electronic health record.
15:14
SPEAKER_01: And sometimes those things overlap, but not always, right?
15:17
SPEAKER_00: Yes, yes, yes, yes.
15:19
SPEAKER_00: So that is the space of aleatoric uncertainty.
15:22
SPEAKER_00: You notice that like it's it's the problem with the with the with the inherent data item itself that you're looking at.
15:28
SPEAKER_00: And it's really hard to, you know, that there's that that's one type of uncertainty.
15:32
SPEAKER_00: And we'll get to exactly how we measure these in a second.
15:34
SPEAKER_00: But um, then there's the whole family of uncertainty measures that's called epistemic, which is just a philosophically fancy word for knowledge, belief, the idea that you there are things you don't know about.
15:45
SPEAKER_00: Um, and that also has a bunch of different ways that it can sort of surface.
15:50
SPEAKER_01: Hold on.
15:50
SPEAKER_01: Do LLMs actually like have things that they think they don't know about, or do they think they know about everything?
15:54
SPEAKER_00: Or do you have to like build it in?
15:57
SPEAKER_00: Um, that's a good question.
15:58
SPEAKER_00: I don't know.
15:59
SPEAKER_00: I don't I don't know.
16:00
SPEAKER_00: Oh, are you uncertain?
16:02
SPEAKER_01: Did I just like plant that in there unplanned?
16:05
SPEAKER_00: That was good.
16:06
SPEAKER_00: That was really good.
16:07
SPEAKER_00: Um, you know, saying I don't know is a good thing, right?
16:10
SPEAKER_00: We've talked about this before.
16:11
SPEAKER_00: Um, it and and saying that you do know about certain things more accurately than others is also a really good thing.
16:17
SPEAKER_00: And if you're correct.
16:18
SPEAKER_00: If you're correct, right.
16:20
SPEAKER_00: Um, but moving on to sort of epistemic uncertainty, and this is interesting because here you have the type of thing that we talked about with the weird 20-sided die, right?
16:29
SPEAKER_00: Your model has uh sees a weird 20-sided die and it's only been trained on six-sided dies, right?
16:35
SPEAKER_00: And that's kind of okay, I it doesn't know what to do with it.
16:39
SPEAKER_00: It's outside the realm.
16:40
SPEAKER_00: Outside the realm.
16:41
SPEAKER_00: And in the case of ICD 10, maybe that manifests itself as the model, it's a rare disease or an unusual procedure or novel clinical phrase phrasing that it's not been trained on.
16:50
SPEAKER_00: Yeah, okay.
16:51
SPEAKER_01: So there's just not been enough data that's been put into the system for this specific thing.
16:56
SPEAKER_00: Right.
16:57
SPEAKER_00: Um, or there's some, you know, it it could be some weird kind of unseen style.
17:01
SPEAKER_00: Maybe there's a different style of writing in one hospital versus another.
17:05
SPEAKER_00: Um, in the dice example, it could just be a different transparent dice, right?
17:09
SPEAKER_00: Or something weird looking like that.
17:11
SPEAKER_00: Um, the model might have seen very few examples.
17:13
SPEAKER_00: That's also another thing.
17:14
SPEAKER_00: Um, so maybe it's seen very few examples of the die phase four, right?
17:21
SPEAKER_00: Um, and so it doesn't just doesn't have enough knowledge about what fours look like.
17:24
SPEAKER_00: And a similar thing could happen in in ICD 10 codes too.
17:27
SPEAKER_00: Like maybe it's there's some codes that it just hasn't seen enough examples of.
17:30
SPEAKER_00: Um, and again, a problem that can be solved with more data.
17:34
SPEAKER_00: Um and notice that, right?
17:36
SPEAKER_00: You can solve these problems with more data, whereas the electoric one, it's like you can't solve the problems with more data because the data is the problem.
17:44
SPEAKER_01: Yeah, okay, that makes sense.
17:45
SPEAKER_00: Um, and you know, but the the the business of like you're you're the other thing that could happen is you might have um lighting that's off in in the dice photo example.
17:55
SPEAKER_00: Uh maybe in the ICD 10 example, you might have um training data that came from some old EHR, right?
18:01
SPEAKER_01: Or old system, and uh, you know, so they've got to be a little bit more than a lot of ICD9 codes instead of ICD 10.
18:06
SPEAKER_00: Right, right, right, exactly.
18:07
SPEAKER_00: Um, and there's other things like the model itself might have some limitations on the types of things it can learn um in in the AI case.
18:15
SPEAKER_00: And so that's you know, I think there is a challenge there.
18:17
SPEAKER_00: Maybe it has it can't deal with in a dice example, it can't deal with certain types of 3D rotations of how the dice looks.
18:23
SPEAKER_00: It just doesn't know what that means.
18:26
SPEAKER_00: Um because to it it doesn't even maybe look like a dye, you know, or a roll of a dice, right?
18:30
SPEAKER_00: And so um and so you can imagine this this whole world of unknown things that can often be solved by finding the right kind of data, but again, you have to know that it has low epistemic uncertainty on something or high epistemic uncertainty on something.
18:46
SPEAKER_01: Right.
18:46
SPEAKER_01: I mean, I think the underlying thing here is like when these models are s give you an answer and they sound confident, then how would you even know that they're uncertain?
18:55
SPEAKER_01: Yes.
18:56
SPEAKER_01: How like are is there any safeguard in place to get it to say, like, oh, I don't know, or I'm not going to answer that question because I don't have enough information?
19:04
SPEAKER_01: Because my experience using LLMs in general, and even even like the medical specific LLMs, like open evidence, is they like give you a very clear and discrete answer every time.
19:18
SPEAKER_01: Yeah.
19:18
SPEAKER_01: That they sound confident in.
19:20
SPEAKER_00: Yeah.
19:21
SPEAKER_00: Yeah.
19:21
SPEAKER_00: And it's it's even more challenging even with non-LLM systems, where if it's an ICD 10 predictor, you probably don't need an LLM, right?
19:28
SPEAKER_00: Because all it needs to output is one of the is is is a set of codes that it's found for a piece of information.
19:34
SPEAKER_00: So it's not like answering you by saying, Oh, that's a great question.
19:36
SPEAKER_00: Let me think about the codes and let me answer them.
19:39
SPEAKER_00: It's just giving you the answer because it's trained to look at the patterns of the data and tell you which code it matches, right?
19:45
SPEAKER_00: And even in that example, it's hard because uh if you think about it this way, there's uh six, seven choices, right?
19:53
SPEAKER_00: In a six choices in a depth in a in a roll of a dice.
19:56
SPEAKER_00: There's a lot more IZ ten codes than that.
19:58
SPEAKER_00: Sure, sure.
19:59
SPEAKER_00: And you know, and and and what these sometimes these models give you is what's called a probability distribution over those choices.
20:06
SPEAKER_00: So you provide some input, you provide the photo of the dice, right?
20:10
SPEAKER_00: And output says it's you know, 40% chance that this is a 50 five, uh, 10% chance that this could be a six, uh, two percent, and then that those numbers, it has something for all of the six choices.
20:23
SPEAKER_00: It might be zero, very close to zero.
20:25
SPEAKER_00: Zero percent chance it's a one, yeah.
20:27
SPEAKER_00: But but but those things add up to 100%.
20:29
SPEAKER_00: So it's telling you that here are my here is my that's called the probability distribution, and it tells you that this is my my current thought about that picture, that it's most likely a five or a six, uh, five, because I have that's the highest number, right?
20:45
SPEAKER_00: So there are these family of methods for uncertainty quantification that looks at it and says, um, how strong is my top guess?
20:51
SPEAKER_00: Yeah.
20:52
SPEAKER_00: Maybe it's not 40% isn't enough, right?
20:54
SPEAKER_00: Maybe you need maybe 80%.
20:55
SPEAKER_01: I mean, I don't I don't think 40% is enough when you're talking about something clinical like diagnosis or whatever.
21:00
SPEAKER_01: Yeah.
21:01
SPEAKER_00: Yes.
21:01
SPEAKER_00: And and so um so if this if this uh model's favorite answer is weak in that in that sense, it should be less trusted, right?
21:10
SPEAKER_00: In some sense.
21:12
SPEAKER_00: Um, the other way to think about this, so that was like how strong is my best answer?
21:15
SPEAKER_00: The other way to think about it is was there almost a tie between multiple ones?
21:18
SPEAKER_01: Maybe there was like 40% versus 39%.
21:22
SPEAKER_01: Nine percent.
21:23
SPEAKER_01: Yeah.
21:23
SPEAKER_00: And then all of a sudden, now it's different again because now the model has barely chosen one answer over another.
21:28
SPEAKER_00: But because it's given you one answer, you have no idea how close it was.
21:32
SPEAKER_00: Yeah.
21:33
SPEAKER_00: You just decided that the best one is the winner.
21:35
SPEAKER_00: Um, and similarly, the model might not have one or two clear winners, it might just be spread across everything, or multiple ones.
21:42
SPEAKER_00: And um and is if the if it's if that's the case, then kind of all bets are off, right?
21:48
SPEAKER_00: It's more of an I don't know in that case.
21:50
SPEAKER_00: Yeah.
21:51
SPEAKER_00: Um, and so you can think about uh these questions that I'm asking, which is is it almost a tie?
21:56
SPEAKER_00: Did it almost win?
21:57
SPEAKER_00: Um, when I sound confident, am I using Usually correct.
22:01
SPEAKER_00: Do slightly different versions of the model agree with me?
22:05
SPEAKER_00: After repeated noisy passes, how strong is my top answer?
22:10
SPEAKER_00: How much do the probabilities fluctuate across different times I run this thing?
22:14
SPEAKER_00: You know, how much is how similar is this example to what I've seen before?
22:18
SPEAKER_00: These are all questions that are answered by specific mathematical techniques that this paper talks about that are all targeted at different ways of looking at the same uncertainty problem.
22:29
SPEAKER_00: And they're all sort of either aleatoric or epistemic.
22:32
SPEAKER_00: Okay.
22:33
SPEAKER_00: Okay.
22:34
SPEAKER_00: So there's a lot here.
22:35
SPEAKER_00: And I know that this is technically heavy, but I want people to kind of think about uncertainty broadly.
22:42
SPEAKER_00: Um sort of to recap, you have aleatory and epistemic.
22:45
SPEAKER_00: Alleatoric is sort of inherent in the data itself.
22:48
SPEAKER_00: And epistemic is more about what you don't know.
22:51
SPEAKER_00: And uh the questions you ask to judge your choices, to decide something, will tell you how you are quantifying your uncertainty.
23:01
SPEAKER_00: And maybe you need all of them.
23:02
SPEAKER_00: Maybe you need to think through all of these different choices.
23:05
SPEAKER_01: And that is something that can be mathematically modeled.
23:07
SPEAKER_00: Yes.
23:08
SPEAKER_01: And then I guess my question becomes like, is that something that can be integrated into these systems where we ask, maybe for ICD Tenkodes, but what if you want help with a diagnosis or something?
23:20
SPEAKER_01: You it becomes more and more important to understand, okay, I think this is a hundred percent chance that this is a scaphoid fracture versus like, hey, 60% chance scaphoid fracture, 40% chance distal radius fracture, 20% chance it's both of those things together.
23:39
SPEAKER_01: Like that to me is like some something that I'd be clinically thinking about.
23:46
SPEAKER_00: Yeah, right.
23:47
SPEAKER_00: Exactly.
23:47
SPEAKER_00: You I think you were already thinking about both aleutoric and epistemic when a human thinks, a human expert thinks about this, right?
23:54
SPEAKER_01: Right.
23:54
SPEAKER_01: And so then it's like, well, how can we get the systems that are currently in place to do that?
23:59
SPEAKER_01: How can we build that into systems that we might be using for clinical purposes?
24:04
SPEAKER_00: Yeah, yeah.
24:05
SPEAKER_00: And and I want to I want to stress again that this is hard enough even on systems that are not LLMs, that are not just giving you diagnoses.
24:12
SPEAKER_00: This is just all it's doing is giving a piece of text, like a note, telling you what the ICD 10 codes are that are being acting, you know, that are relevant in that in that in that note.
24:22
SPEAKER_01: This is how like what was the example?
24:25
SPEAKER_01: They showed like COVID lungs and then a cat, and then the cat was a COVID lung.
24:29
SPEAKER_00: Yeah, that was a great example.
24:30
SPEAKER_00: I don't know if that's a great example of uncertainty, but that was a funny funny.
24:33
SPEAKER_00: It should have been uncertain.
24:35
SPEAKER_00: Oh, actually, this is a great example of uncertainty.
24:37
SPEAKER_01: Look, I did it.
24:38
SPEAKER_00: You're right, you're right, you're right.
24:39
SPEAKER_00: So that was an example where they had trained the model on, and I want to ask you a quiz question after this.
24:43
SPEAKER_01: Oh no, he's gonna ask me if it's aleatoric or epistemic.
24:46
SPEAKER_00: Uh-huh.
24:47
SPEAKER_00: Exactly.
24:47
SPEAKER_00: Okay, hold on, let me think while you while you're talking about.
24:49
SPEAKER_00: Let me let me tell the example uh again, which is that there was a model that was trained on on lungs, on COVID versus non-COVID lungs, and it was shown, you know, and and you could tell, given an X-ray, you could tell which of those two it was.
25:02
SPEAKER_00: And um, it was shown a picture of a cat, and it was 100% confident that it was a COVID lung.
25:09
SPEAKER_00: And so here are the two choices in this.
25:11
SPEAKER_00: It's not a six-sided diaret, so it's only two choices.
25:13
SPEAKER_01: So there's a 50% chance I'm gonna get it right.
25:15
SPEAKER_01: No, no, no.
25:16
SPEAKER_00: Well, it's a 50% chance that it's a COVID lung.
25:18
SPEAKER_00: And and so the question is, is it aleatoric or is it um epistemic?
25:26
SPEAKER_01: Okay, um, for$700, I am going, I'm gonna say what is epistemic uncertainty because it was outside of the system that it was trained on, right?
25:37
SPEAKER_01: Like it was not trained on cats.
25:38
SPEAKER_01: Yes.
25:39
SPEAKER_01: But then it then, like, you want the system to be like, hey, wasn't trained on cats.
25:44
SPEAKER_01: Yes.
25:45
SPEAKER_01: I can't answer this.
25:46
SPEAKER_01: The answer is I don't know.
25:48
SPEAKER_01: And it didn't do that.
25:49
SPEAKER_00: Exactly.
25:50
SPEAKER_00: And there are technical methods, and this paper talks about them.
25:53
SPEAKER_00: Um, they're called density-based methods that are exactly for this, where it should come back and say, This is looks like nothing like my training data.
26:02
SPEAKER_00: It this looks the the appearance and the shape and whatever else that's in here, it looks totally different.
26:07
SPEAKER_00: I don't know what it is.
26:08
SPEAKER_00: Of course, it doesn't know that it's a cat because it's only trained on COVID and non-COVID lungs.
26:13
SPEAKER_00: But it should be able to say, This is not something I've seen before.
26:17
SPEAKER_00: I'm going to abstain.
26:18
SPEAKER_00: I'm not going to answer it.
26:19
SPEAKER_01: And that is like, I think, really important.
26:22
SPEAKER_01: Where if a model doesn't know the answer, yeah, just like every human doesn't know the answer, to be able to say, hey, I don't know.
26:31
SPEAKER_01: I need more information, or I need to talk to a specialist, or I want to get an x-ray, or let's do some lab work, or it could be one of these six things, and five of them are all treated with a volar splint.
26:45
SPEAKER_01: So we're gonna do that.
26:46
SPEAKER_01: And for the six, we're ruling it out with an x-ray.
26:48
SPEAKER_01: So that's where it's like you're trying to make a decision about what to do next, and you want the information that goes into that decision to be appropriate.
26:58
SPEAKER_01: And if you are uncertain, that needs to be front and center so you can make the most appropriate decision.
27:06
SPEAKER_01: Exactly.
27:07
SPEAKER_01: Exactly.
27:08
SPEAKER_00: Uncertainty is a good thing.
27:10
SPEAKER_00: It is a good thing.
27:11
SPEAKER_00: It is a good thing to be uncertain.
27:12
SPEAKER_00: It's a good thing to be, it just means that there are questions left unanswered.
27:15
SPEAKER_00: It means that you have to raise new questions and explore more and figure out what it is.
27:20
SPEAKER_01: Yeah.
27:20
SPEAKER_01: And in medicine, sometimes something's straightforward, right?
27:23
SPEAKER_01: Yeah.
27:24
SPEAKER_01: You fell, you broke your arm, but a lot of times things are not so straightforward.
27:29
SPEAKER_01: And being uncertain about what's going on and what to do next can actually be helpful so that you expand what your tests are and then find a path forward and narrow in over time.
27:44
SPEAKER_01: Exactly.
27:45
SPEAKER_01: It's a good thing.
27:45
SPEAKER_01: Yes.
27:47
SPEAKER_01: All right.
27:48
SPEAKER_01: Well, uh, here's to uncertainty.
27:50
SPEAKER_01: And um, maybe, maybe, just maybe, hopefully, between a zero and a hundred percent chance, we will see you next time on Code and Cure.
27:59
SPEAKER_01: Thank you.