Latest / Code & Cure / #45 - How Machine Learning Improves Stroke Prediction With AFib

Transcript

#45 - How Machine Learning Improves Stroke Prediction With AFib · Code & Cure

0:00
SPEAKER_00: For decades, we've predicted stroke risk using static scoring systems.
0:05
SPEAKER_00: Machine learning may finally change that.
0:17
SPEAKER_01: Hello and welcome back to Code and Cure, where we discuss decoding health in the age of AI.
0:22
SPEAKER_01: My name is Vasant Sarathi.
0:24
SPEAKER_01: I'm an AI researcher and cognitive scientist.
0:27
SPEAKER_01: And I'm here with Laura Hagopian.
0:29
SPEAKER_00: I'm an emergency medicine physician.
0:31
SPEAKER_00: We saw a lot of um we're talking about atrial fibrillation today and stroke risk.
0:36
SPEAKER_00: That's probably like a lot of words that I can like, what are you talking about?
0:40
SPEAKER_01: I can't wait for you to unpack this because as we were preparing for this, those were words that I sort of kind of knew what they meant, maybe, but I'm really excited to learn about that.
0:50
SPEAKER_00: Yeah.
0:50
SPEAKER_00: And I mean, atrial fibrillation is very common.
0:53
SPEAKER_00: I saw a lot of it in the emergency room.
0:56
SPEAKER_00: Um, and there's a lot that goes along with it.
1:00
SPEAKER_00: But I think it probably makes sense to back up and and talk about sort of the clinical side of this before we dive in to the AI side of this and to the machine learning side of this.
1:09
SPEAKER_00: Um atrial fibrillation, uh, back to the beginning, what is it?
1:14
SPEAKER_00: It's an arrhythmia.
1:15
SPEAKER_00: It's the most common arrhythmia.
1:17
SPEAKER_00: It affects over 58 million people uh across the world.
1:21
SPEAKER_00: And it it's basically a form of an irregular heartbeat.
1:24
SPEAKER_00: And so what happens is that the upper chambers of the heart, the atria, they quiver or fibrillate.
1:31
SPEAKER_00: Um, and this often often happens quickly, it happens irregularly, and so your heartbeat is kind of uneven.
1:38
SPEAKER_00: It's probably the best way to describe it.
1:40
SPEAKER_00: It's uh it's more common as people get older, but other things can trigger it as well.
1:46
SPEAKER_00: Um, and and one of the main things we worry about with atrial fibrillation, it can cause a lot of a lot of issues, right?
1:55
SPEAKER_00: It can, especially if your heart is beating too fast and all these other things.
1:58
SPEAKER_00: But one of the main things we worry about is is a stroke risk.
2:03
SPEAKER_00: Um, because what's happening is when those upper chambers, those atria fibrillate rather than contracting in a coordinated fashion, then the blood kind of pools.
2:15
SPEAKER_00: Oh, it kind of sits in that one spot in the atria, right?
2:19
SPEAKER_00: Right.
2:19
SPEAKER_00: And it can then clot, right?
2:21
SPEAKER_00: It's because it's not moving like you want it to.
2:26
SPEAKER_00: Well, actually, the the clot doesn't do that.
2:29
SPEAKER_00: The clot can then travel.
2:31
SPEAKER_00: Oh, I see.
2:31
SPEAKER_00: Okay.
2:32
SPEAKER_00: So when your heart beats, that clot can travel anywhere, can travel anywhere in the rest of your body through your arteries.
2:39
SPEAKER_00: Your arteries lead to your legs, your arms, your gut.
2:43
SPEAKER_00: So you can get clots that go there.
2:45
SPEAKER_00: But one of the more common places, unfortunately, that that they can go with atrial fibrillation is the clots can go into your brain, causing a stroke.
2:53
SPEAKER_01: Got it.
2:54
SPEAKER_01: Okay.
2:55
SPEAKER_01: So just to take a step back though, a clot in this particular instance is just bad because it's a clot and it's going to go block something that shouldn't be blocked.
3:02
SPEAKER_00: Yeah, exactly.
3:03
SPEAKER_00: So when the blood pools in the atria it forms that clot, and that clot, because it's coming out of the heart with the rest of your blood, can travel into the blood vessels in the brain and cause a stroke.
3:14
SPEAKER_00: And so when we treat atrial fibrillation, we often do, you know, do we try to control the heart rate?
3:20
SPEAKER_00: We see if we can get them back to a normal rhythm.
3:23
SPEAKER_00: Those are all components of it.
3:24
SPEAKER_00: But the thing that I want to focus on today, based on the discussion we're having and the paper we reviewed, is hey, how do we figure out who's at high enough risk for strokes that we want to give them blood thinners?
3:36
SPEAKER_00: Because if you give them blood thinners, then they're much less likely to have a clot happen.
3:41
SPEAKER_00: Right.
3:41
SPEAKER_00: Much less likely to have that then go up into the brain and cause a stroke.
3:45
SPEAKER_01: So atrial fibrillation alone isn't enough to give people blood thinners.
3:51
SPEAKER_00: It's an interesting question.
3:52
SPEAKER_00: It's like, well, what is someone's risk, right?
3:55
SPEAKER_00: Because any treatment that you give someone, there's like pros and cons to it, right?
4:00
SPEAKER_00: Like uh something that thins your blood.
4:03
SPEAKER_00: What do you think might be a con of that?
4:05
SPEAKER_00: Oh my god, side effects.
4:07
SPEAKER_01: If you bleed anywhere, it's just not gonna clot at all.
4:09
SPEAKER_01: Right, exactly.
4:10
SPEAKER_00: So you could you could have like something sort of minor where you just get a big bruise where someone else might get some a small bruise, right?
4:16
SPEAKER_00: But you could have a bleed, I don't know, in your GI tract from an ulcer or something like that.
4:20
SPEAKER_00: You could have you you fall down and hit your head, and it's not a minor thing anymore.
4:24
SPEAKER_00: You could have bleeding inside the brain too.
4:26
SPEAKER_00: So there these things are, you know, there's there's like reasons to do it, and then there's reasons not to do it.
4:32
SPEAKER_00: And you want to make sure that the reasons to do it sort of outweigh the reasons not to do it because there's always risks associated with medications or interventions, you know.
4:43
SPEAKER_00: If you were to if you were to have a surgery, for example, you sign this big long consent form.
4:48
SPEAKER_00: You're like, okay.
4:49
SPEAKER_00: Long consent.
4:51
SPEAKER_00: Yeah.
4:51
SPEAKER_00: Well, it's like, here's why we're doing this, here's why it's important, here's why it's necessary, here are the bad things that could happen.
4:56
SPEAKER_00: They're rare, but they could happen, right?
4:58
SPEAKER_00: And so here it's the same thing.
5:00
SPEAKER_00: It's like, well, we want to understand who would benefit from blood thinners the most because we know that they can have some adverse effects too.
5:12
SPEAKER_00: And so there actually are prediction tools for this already and that are used in in in medicine.
5:19
SPEAKER_00: The the most common one that we use right now is called the Chad's to VASC score.
5:25
SPEAKER_00: And what that does is it calculates the stroke risk for patients with atrial fibrillation and it estimates that risk.
5:32
SPEAKER_00: And so very low scores may not need blood thinners.
5:36
SPEAKER_00: Um high scores should have blood thinners, but of course it's a discussion you end up having with your provider because there are things that you may need to consider uh about each person's individual situation.
5:50
SPEAKER_00: Sure.
5:50
SPEAKER_00: That the score may not pick up, right?
5:52
SPEAKER_00: Do they have frequent falls or do they have a history of multiple GI bleeds in the past or whatever it is?
5:58
SPEAKER_00: Um and so what the Chad's too vast score does, it's like simple addition.
6:04
SPEAKER_00: Um, and things score you different points, like zero, one point, or two points based on your age, right?
6:10
SPEAKER_00: So older, we know, is more like more at risk for stroke based on your sex, based on any uh history of certain medical conditions like congestive heart failure, hypertension, stroke, uh prior vascular disease, like having a heart attack in the past, or history of diabetes.
6:28
SPEAKER_00: And so what you do is you like answer yes or no to all these questions, and you get, you know, it adds it up for you, and then you just get this simple sort of score.
6:37
SPEAKER_00: And from that score, you and your provider can have a discussion around hey, is it do you recommend?
6:44
SPEAKER_00: Do we think this person should have blood thinners?
6:46
SPEAKER_00: Do we think this patient should get blood thinners to try to prevent them from having a stroke?
6:51
SPEAKER_01: Got it, got it.
6:52
SPEAKER_01: That's super helpful.
6:53
SPEAKER_01: Thank you.
6:53
SPEAKER_01: Yeah, that's that's great.
6:55
SPEAKER_00: But it is somewhat limited, right?
6:57
SPEAKER_00: It's not gonna necessarily capture like a complex pattern.
7:02
SPEAKER_01: Yeah, yeah.
7:02
SPEAKER_01: And we'll talk about that when we talk about the technical side of it.
7:05
SPEAKER_00: And it's the same for everyone, right?
7:07
SPEAKER_00: Everyone has that same score that's being calculated.
7:10
SPEAKER_00: That's right.
7:11
SPEAKER_00: And so the question they were asking in this paper was like, hey, can we make a machine learning model to predict stroke risk in patients with new onset atrial fibrillation?
7:22
SPEAKER_00: I think that's a very interesting question.
7:24
SPEAKER_01: Yeah, it is a very interesting question, and it's a really nice paper for people who are.
7:29
SPEAKER_01: I would highly recommend it for people who are learning machine doing machine learning or understanding how these models are applied and the challenges.
7:35
SPEAKER_01: This is a very nicely written paper.
7:37
SPEAKER_01: Um, and one of the big things you're tackling is um external validation and making sure that the data is good and all of that stuff, right?
7:44
SPEAKER_01: That we talked about before in the podcast.
7:46
SPEAKER_00: Yeah.
7:46
SPEAKER_00: So they train like they trained on one set of data, but then they tested on a whole nother set of data that was actually distinct, different.
7:53
SPEAKER_01: Yeah.
7:53
SPEAKER_00: Uh the patients were different, their demographics were different, and it still worked.
7:58
SPEAKER_01: Yeah.
7:58
SPEAKER_01: Yeah.
7:58
SPEAKER_00: You know, their machine learning model still worked.
8:01
SPEAKER_01: Yeah.
8:01
SPEAKER_01: And and and what's what's interesting also is the underlying machine learning techniques themselves are not super revolutionary.
8:07
SPEAKER_01: They're not necessarily new, but it's like I said, it's a great example of a machine learning algorithm and system that is well designed for the task at hand.
8:18
unknown: Yeah.
8:18
SPEAKER_00: So I'd love to have you dive into that a little bit more because what they did was they said, okay, well, you know what?
8:24
SPEAKER_00: We may not have what let's let's do a real-world scenario.
8:27
SPEAKER_00: You may not have um lab values.
8:30
SPEAKER_00: You may not have like a ton of information about these patients if they're being newly diagnosed, right?
8:35
SPEAKER_00: So let's figure out what would be sort of generally available, right?
8:40
SPEAKER_00: Like their age, like their medication, medication list and whatever, you know, past medical history they might have, whether that's like hypertension or diabetes or something else.
8:50
SPEAKER_00: Let's just take that information and see if we can use that to predict what someone's stroke risk is.
8:59
SPEAKER_01: Yeah, yeah, yeah, yeah.
9:00
SPEAKER_01: So let's, yeah.
9:01
SPEAKER_01: So let's talk about this.
9:02
SPEAKER_01: So they they are predicting the one-year stroke risk after an atrial fibrillation diagnosis.
9:08
SPEAKER_00: And you can call it AFib for sure.
9:10
SPEAKER_00: That sounded like so awkward when you were saying that.
9:12
SPEAKER_00: It was really funny.
9:13
SPEAKER_00: In my head, I was thinking the tricks.
9:17
SPEAKER_01: In my head, I was like, maybe I should just say AF to be all cool, but that would be.
9:20
SPEAKER_01: AFib.
9:21
SPEAKER_00: AFib.
9:21
SPEAKER_01: You can be cool if you say AFib.
9:23
SPEAKER_01: All right, I'll say AFib.
9:24
SPEAKER_01: So predict one year stroke risk after AFib diagnosis.
9:27
SPEAKER_01: Um, and the we talked about this, but the hand score, the Chad Tuvasque score is problematic.
9:33
SPEAKER_01: Well, not problematic necessarily, but it's it's good.
9:35
SPEAKER_01: It's still very good.
9:36
SPEAKER_00: Could it be better?
9:37
SPEAKER_01: But could it the question is could it be better?
9:38
SPEAKER_01: It's very rule-based, simplistic, and it has a set of fixed weights that people apply.
9:42
SPEAKER_01: And it's kind of just like going through the math and doing that.
9:44
SPEAKER_00: And it's not personalized, right?
9:46
SPEAKER_01: Yeah, but the nice thing is it makes sense.
9:48
SPEAKER_01: People understand it, it's interpretable.
9:50
SPEAKER_01: Um, and so there's some benefits to that.
9:51
SPEAKER_01: So that the question is, can we make it better?
9:54
SPEAKER_01: And this paper addresses that.
9:55
SPEAKER_01: So one of the things they do first is um figure out how we can take what should be the input to our model, right?
10:03
SPEAKER_01: We have all of these different pieces of information coming in, raw hospital data, uh, people's demographic information, all of this stuff.
10:11
SPEAKER_01: How do we take all of that into account?
10:13
SPEAKER_01: Let's say we had a data set that had all for every single patient, had a bunch of those things, right?
10:18
SPEAKER_01: And some of those things that you mentioned too, some other measures, right?
10:21
SPEAKER_01: They have all of that.
10:22
SPEAKER_01: And let's say you know you have a data set that tells you whether or not that patient patient had a stroke a year later or not, right?
10:28
SPEAKER_01: Let's say you had that.
10:30
SPEAKER_01: A lot of those measures are potentially not relevant to the uh an analysis.
10:35
SPEAKER_01: So one of the first steps in the machine learning process is something called feature selection, which is a method to basically figure out which of these variables that are put in, the inputs to the system, are actually useful for the calculation.
10:50
SPEAKER_01: Um because if you just include everything in, what can happen is that the results might seem really good, but then you're what and what ends up happening is you might be overfitting, which is uh it might work for this patient population, but it won't work for something else.
11:04
SPEAKER_01: Because you've given it too many details and it's made the patient very specific, and it might work for that very specific uh situation and causes noise when you go to a different setting.
11:13
SPEAKER_00: I could also say from like a clinical standpoint, if you have too much data that's going in, you you don't actually you may not have that data.
11:23
SPEAKER_00: You may not have that data on the next patient.
11:24
SPEAKER_00: Oh, that's true, yes.
11:25
SPEAKER_00: Right?
11:26
SPEAKER_00: Like you may have XYZ lab on this patient and JKL lab on the other patient, right?
11:31
SPEAKER_00: And so a lot of times, if you're taking clinical samples and you're looking at the population level, there's gonna be big gaps in the data.
11:38
SPEAKER_01: No, exactly.
11:38
SPEAKER_01: And so they use a technique called Bayesian information criterion or BIC.
11:43
SPEAKER_01: Um, and that is a mathematical trick or technique to figure out which of the ones, which of the variables actually matter.
11:52
SPEAKER_01: And it actually what it does is it it it it rewards cases where there's like you know, uh support with respect to the the variables correlation with respect to the output, and it it it penalizes uh too many variables or you know, uh uh it so so it accounts for this exact thing we're talking about.
12:09
SPEAKER_01: And so they use that technique, and what they did was they identified between nine and eleven core variables um that that they considered are important for this.
12:19
SPEAKER_01: And so that first step of feature selection, by the way, is a really hard problem in and of itself.
12:23
SPEAKER_00: There's they just like braced past that.
12:25
SPEAKER_00: They were like, by the way, we did this first.
12:27
SPEAKER_01: Yes.
12:28
SPEAKER_01: So they were, but but you know, it's also tied closely with the model you're choosing.
12:32
SPEAKER_01: So they chose two models they looked at.
12:34
SPEAKER_01: One is what's called a logistic regression model, and the other is called an XG boost model.
12:38
SPEAKER_01: And we'll talk about each of those models separately.
12:41
SPEAKER_01: But uh, for the logistic regression model, they picked nine features, like things like age and uh and some other, you know, prior case of prior stroke and chronic lung disease and some other things.
12:50
SPEAKER_00: Yeah, and medications were on this list too.
12:53
SPEAKER_00: Like, oh, are they on an antiplatelet agent?
12:55
SPEAKER_00: Are they on that's what APA stands for?
12:57
SPEAKER_00: Yeah, there you go.
12:58
SPEAKER_01: Yeah.
12:58
SPEAKER_01: So yeah, so they have all of these things, but they only figured out these nine features actually are are best uh predictive of um the data that they had.
13:07
SPEAKER_01: Um and this allows the logistic regression model to be more compact, right?
13:11
SPEAKER_01: Because your input's only nine things.
13:13
SPEAKER_01: Um, but logistic regression models are actually really good.
13:17
SPEAKER_01: And in fact, the one that they had had a very high performance, high performance.
13:20
SPEAKER_01: It did really well on this task.
13:22
SPEAKER_00: Better than the Chad Tuvask score.
13:24
SPEAKER_01: Yeah.
13:24
SPEAKER_01: Yeah.
13:25
SPEAKER_01: And it's this is really important because it's a very simple model and it does really well.
13:30
SPEAKER_01: And some might say it's interpretable because it kind of is.
13:33
SPEAKER_01: Because it's like your Chad Tuvask score, it also kind of sums up various of these nine variables and assigns sort of weights to them, which is more important, which is less important.
13:43
SPEAKER_00: Like, so if, for example, having a history of hypertension and diabetes, you know, you may find that those people are at higher risks for having a stroke with nuance at the end.
13:54
SPEAKER_00: And so you can go in and look at that.
13:55
SPEAKER_00: Like, here are the variables that are associated with it, here are the ones that are not.
13:58
SPEAKER_01: Yeah, yes, and it's a linear model, which means what they're trying to do is fit the data to a line, not literally a 2D line, but some higher dimensional line.
14:07
SPEAKER_01: And that's that's the core idea behind a linear regression model.
14:10
SPEAKER_01: Uh, but the problem is um some of the um behaviors and and dependencies between the different variables can be nonlinear.
14:20
SPEAKER_01: And so an XG boost model is a more is a newer technique uh that actually is very interesting.
14:26
SPEAKER_01: So the XG Boost is used heavily in cases where you have tabular data and you have you know kind of mixed data like that.
14:34
SPEAKER_01: And what it's based on something entirely different.
14:37
SPEAKER_01: So logistic regression was based on fitting things to a line.
14:41
SPEAKER_01: Like if you had a line, and if you know the mathematical equation for the line, and getting a little mathy here, but uh a line equation is y equals mx plus b, right?
14:51
SPEAKER_01: People might know that, people might not know that, but that's a like I remember that.
14:54
SPEAKER_01: Yeah, where x is the input and y is the output, and m and b are just the things that you can, the knobs that you can turn.
15:00
SPEAKER_01: And a linear model learns the m's and b's for the data you give it.
15:04
SPEAKER_01: And so then it finds the line and then it's able to predict for other x's what the y could be.
15:08
SPEAKER_01: Um that's broadly what it is, but uh uh XG boost model works completely differently.
15:14
SPEAKER_01: It starts from what's called a decision tree.
15:16
SPEAKER_01: And a decision tree is very straightforward.
15:18
SPEAKER_01: So, like if you think about it, if you just picked one of the variables that we talked about, maybe pick age, right?
15:24
SPEAKER_01: Maybe you have a rule that says if the age is greater than 75, yes to stroke.
15:29
unknown: Uh-huh.
15:30
SPEAKER_01: If it's less than 75, then check the HDN.
15:33
SPEAKER_01: And if that is high, um, then uh medium risk.
15:37
SPEAKER_01: And if that is low, then low risk.
15:39
SPEAKER_01: So you can imagine uh a decision logic that follows that keeps branching and creates a little tree.
15:46
SPEAKER_01: Now, if you have lots of variables, you have to branch quite a bit across and you'll get quite a quite quite a no, but it the the advantage of the the tree format is that it naturally captures all the thresholds you care about.
15:57
SPEAKER_01: If the the the data is kind of weird and non-linear, if there's like sudden changes because uh age really does matter, let's for let's say for instance, then you're able to capture those nonlinearities really well.
16:09
SPEAKER_01: You can capture interactions, like this and this needs to be greater than a certain value before you conclude something.
16:14
SPEAKER_01: So, like you can do things like that in uh a tree structure.
16:18
SPEAKER_01: Uh and trees and decision trees have been around for a really long time, but it's really hard in some ways to train them.
16:23
SPEAKER_01: They're very interpretable because at the end of it, you have a logic for how you arrived at the final.
16:27
SPEAKER_00: Yeah, which is always what you want to see with something like this before you make a clinical decision.
16:31
SPEAKER_01: Absolutely.
16:32
SPEAKER_01: Absolutely.
16:32
SPEAKER_01: And and XG Boost is a technique in which you learn the best tree by picking really crappy trees to start with, and then you improve on the error.
16:44
SPEAKER_01: You look at the error that was produced and you learn to change the tree to get better, lower and lower error.
16:50
SPEAKER_01: And it's just a technique that people use called gradient boosting.
16:53
SPEAKER_01: XG Boost stands for extreme gradient boosting.
16:56
SPEAKER_01: And it's essentially a way to get at the better, best tree through this incrementally picking the worst model and kind of working your way back upwards.
17:06
SPEAKER_01: Uh, so it can learn all the thresholds, it can learn all the feature interactions, it can learn uh conditional effects.
17:12
SPEAKER_01: For example, diabetes is only dangerous when the age is greater than 70 or something, which you'll notice a rule like that is really hard to express in logistic regression.
17:21
SPEAKER_01: Or that's not the chat.
17:24
SPEAKER_00: And they gave some examples in this paper.
17:27
SPEAKER_00: I thought this was very interesting, where it's like, oh, in this this patient, patient A, like being on a specific type of anti-arrhythmic drug increased their risk of stroke, but being on that same drug decreased the risk of stroke in a different patient who was a different age and had different health history.
17:44
SPEAKER_00: I think that's so interesting because it means like it's like truly personalized.
17:47
SPEAKER_01: That's right.
17:48
SPEAKER_00: And it's not just do you have X or Y, but how do X, Y, and Z interact to create a overall picture of your risk, which is like clinically what you want to see, but you're but that's not what you get out of uh like a more simple scoring model.
18:02
SPEAKER_01: Exactly.
18:03
SPEAKER_01: And it can handle heterogeneous tabular data really well because you might have mixed stuff, you might have noisy stuff, some might be like decimal numbers and some might be other things.
18:11
SPEAKER_01: It can trees are fantastic at handling that kind of thing.
18:15
SPEAKER_01: Uh, like we just talked about, we can get um you know, we can discover interactions between them.
18:20
SPEAKER_01: Um, and it there are techniques in XG Boost that prevent overfitting that we talked about before.
18:26
SPEAKER_01: They only used 11 of the features for the XG Boost.
18:28
SPEAKER_01: So they did the same feature selection thing we talked about at the beginning to arrive at um uh 11 things.
18:34
SPEAKER_01: XG Boost has been around for a little bit, and it's really a technique that's become very optimized.
18:38
SPEAKER_01: Um, things have been parallelized, it's very efficient, it's very fast, and it generally beats out a lot of the other models in most in many of these um cases where you have data and you have like this kind of data and you have this kind of task.
18:52
SPEAKER_00: And so instead of like going, so right now, for example, I go into a physician calculator MD Calc, and I'll just be like, okay, this is the age, this is the this, this is the that.
19:02
SPEAKER_00: Instead, I'd need to like use basically an application to do it because it's so complex.
19:07
SPEAKER_00: Yeah.
19:07
SPEAKER_00: Or I'd have to integrate it into the medical record somehow where it would pull all that data in.
19:12
SPEAKER_01: Yes, exactly.
19:13
SPEAKER_01: And again, I want to stress this point, which is that it adds in a linear model, you're always multiply because let's say you have a linear model that you've trained and you'll learn and it's really high performing, but it's figured out that any diabetes, sorry, any age factor is um multiplied by 0.5, right?
19:32
SPEAKER_01: Just making up a number, 0.5.
19:34
SPEAKER_01: Now, that weight that you add, the 0.5 that you add, the multiplier that you add, is a knob that was learned over the course of its training.
19:41
SPEAKER_01: That's fixed across all patients, right?
19:45
SPEAKER_01: It's the same amount of weight or risk you apply for a particular uh variable across all patients.
19:52
SPEAKER_00: And that's actually true in the Chaz 2 VASC score.
19:54
SPEAKER_00: The two in there gives like two points based on age greater than or equal to 75, whereas 65 to 74 is only one point.
20:01
SPEAKER_00: So it's like a real world thing that happens all the time.
20:04
SPEAKER_00: Yeah.
20:05
SPEAKER_01: Yeah.
20:06
SPEAKER_01: Um now on the flip side, XGBoost is wonderful and it does a really good job.
20:12
SPEAKER_00: And um both linear regression and XG Boost outperformed the calculator, the Chad's tubeass calculator.
20:20
SPEAKER_01: Yeah, yeah, yeah, yeah.
20:20
SPEAKER_01: So they also did some additional stuff.
20:22
SPEAKER_01: So there's additional um techniques that can take XG Boost's output and do more.
20:27
SPEAKER_01: So if you ask the question of why did this particular patient get a 39% risk score, why did that 39% risk specifically come from?
20:36
SPEAKER_01: Um, there is uh another technique called uh SHAP values or Shaply additive explanations.
20:43
SPEAKER_01: Um and it's a technique that derives from cooperative game theory, but it basically is able to figure out how much each feature contributed to this prediction and kind of walk down that tree for you a little bit.
20:54
SPEAKER_01: Um and we don't have to go into details here about the SHAP itself, but it allows you to get these kind of local explanations.
21:00
SPEAKER_01: Um, and it's useful when you want to targeted look at a specific result for a specific patient and look backwards.
21:07
SPEAKER_00: And I always want to see something that's explainable.
21:09
SPEAKER_00: It's like if you have a black box that's giving you an answer, it's like, well, how much do I trust it?
21:14
SPEAKER_00: But if you can look under the hood and be like, hey, it's your history of hypertension and diabetes along with your prior history of a stroke that is triggering such a high risk for you that's making me recommend the anti-coagulation, the blood thinners, then it it just feels better.
21:31
SPEAKER_00: It feels like, okay, I know why we're doing this thing that we're doing.
21:34
SPEAKER_01: Yeah.
21:34
SPEAKER_01: And and we're able to say things like, you know, younger age can reduce the risk and older age can increase the risk, but effect depends on patient profile, right?
21:41
SPEAKER_01: And so that's going to be separate for different patients.
21:43
SPEAKER_01: So I I'm really excited about this paper because it, like I said, it's obviously the performance is great, right?
21:48
SPEAKER_01: You have to do that.
21:48
SPEAKER_01: The performance is great.
21:49
SPEAKER_00: Yeah.
21:50
SPEAKER_00: It's truly letting you, especially with the XG boost model, personalize this stroke risk because it's not just about X, Y, Z, but About how X, Y, and Z interact.
22:02
SPEAKER_01: Yeah, exactly.
22:03
SPEAKER_00: Right.
22:03
SPEAKER_00: And it's the things that I would want to see clinically, which is like, hey, we don't need a ton of input features.
22:10
SPEAKER_00: We don't need everyone's lab values.
22:12
SPEAKER_00: We don't need 50 different things.
22:13
SPEAKER_00: We need 11.
22:15
SPEAKER_00: Um, and it's explainable.
22:17
SPEAKER_00: So we can go in and understand, hey, here are the things that are triggering this, and here's why.
22:23
SPEAKER_00: Here's why we would or would not recommend um anti-coagulation at this time.
22:28
SPEAKER_01: Yeah.
22:28
SPEAKER_01: And you know, the natural question that people ask always is like, why can't we use modern AI like ChatGPT or LLMs or something else like that?
22:34
SPEAKER_01: And we have to remember that those systems are neural networks and neural networks are black boxes.
22:39
SPEAKER_01: So we lose all of these.
22:41
SPEAKER_01: Um we can send in all the features and train it, and it probably will do fine performance-wise.
22:47
SPEAKER_01: But um, but do we need such big machinery for doing something that is so targeted?
22:51
SPEAKER_01: And when we have a great model like this already.
22:53
SPEAKER_00: And would it be explained it's not explainable then, right?
22:55
SPEAKER_00: At all.
22:55
SPEAKER_00: It's not explainable.
22:56
SPEAKER_00: How do you even know if it's like part of my problem is how do you even know if it's good?
23:01
SPEAKER_00: If you're like, well, it looks looks good, and then you like send out a bunch of people either on or off blood thinners, and they have something bad happen.
23:09
SPEAKER_00: Who do you go back to and say, well, hey, this didn't this didn't work out how we wanted it to?
23:14
SPEAKER_00: This person had a brain bleed and that person had a stroke.
23:16
SPEAKER_00: I mean, nothing in medicine is 100%, right?
23:19
SPEAKER_00: You're you're talking about stroke risk and whether or not you want to put them on blood thinners.
23:23
SPEAKER_00: Uh it's possible that someone you recommend no blood thinners to could still have a stroke, and someone you do recommend it to could have a bad effect from it.
23:31
SPEAKER_00: Yeah.
23:31
SPEAKER_00: But you want to understand like what's the underlying reasoning for doing this rather than just doing it because AI said so.
23:38
SPEAKER_01: Yeah, no, absolutely.
23:40
SPEAKER_01: And that's it.
23:40
SPEAKER_01: I think that's the key here, and that's the key contribution in this paper.
23:44
SPEAKER_00: I do think in general, when you think about risk scores for all sorts of things, this does open the door, though, to have uh just more, more personalization and more interaction between the different factors that are being considered, right?
24:00
SPEAKER_00: Yeah.
24:00
SPEAKER_00: Simple addition is nice and easy.
24:02
SPEAKER_00: It's great.
24:02
SPEAKER_00: You get there's tons of calculators online for it.
24:05
SPEAKER_00: But when you look at how well these models performed, both, you know, initially and with the external validation study, it was like, oh wow, like we can we can really hone in on this better and understand who are the patients that are the highest risk.
24:20
SPEAKER_00: How do the different uh pieces of their health history interact with each other in order to understand?
24:27
SPEAKER_00: Okay, this person does belong on anticoagulation, this person does not.
24:32
SPEAKER_00: And there, here's someone who's in between that we maybe need to have a discussion and figure out together what the best path forward is.
24:38
SPEAKER_00: Yep, exactly.
24:40
SPEAKER_00: So exciting article.
24:43
SPEAKER_00: Um, can you say, can you say apib now?
24:45
SPEAKER_01: I'll I'll try again.
24:46
SPEAKER_01: AFib.
24:47
SPEAKER_00: There we go.
24:50
SPEAKER_00: It's perfect.
24:50
SPEAKER_00: It's a perfect ending to this podcast.
24:52
SPEAKER_00: We will see you next time on Coding Cure.
24:54
SPEAKER_01: Thank you for joining us.