Latest / Code & Cure / #30 - From Reddit To Rescue: Real-Time Signals Of The Opioid Crisis

Transcript

#30 - From Reddit To Rescue: Real-Time Signals Of The Opioid Crisis · Code & Cure

• 0:01
  SPEAKER_00: What if the earliest warning signs of the opioid epidemic aren't in hospitals or public health reports, but buried in Reddit posts at 2 a.m.
• 0:19
  SPEAKER_01: Hello and welcome to Code and Cure, the podcast where we discuss decoding health and the age of AI.
• 0:25
  SPEAKER_01: My name is Vasant Sarathi, and I'm an AI researcher and cognitive scientist.
• 0:30
  SPEAKER_01: And I'm here with Laura Hagopian.
• 0:32
  SPEAKER_00: I'm an emergency medicine physician and I work in digital health.
• 0:35
  SPEAKER_01: Today we're going to talk about uh the opioid epidemic and specifically different ways of potentially sort of monitoring it.
• 0:44
  SPEAKER_01: Right.
• 0:45
  SPEAKER_01: Yeah.
• 0:45
  SPEAKER_01: Yeah.
• 0:46
  SPEAKER_00: I mean, oftentimes this is done by the CDC.
• 0:51
  SPEAKER_00: And they um they have some limitations.
• 0:54
  SPEAKER_00: I think one of the biggest problems with these public health agencies is that they're not fast.
• 1:00
  SPEAKER_00: These reports don't happen in real time.
• 1:02
  SPEAKER_00: They can even take months to come out.
• 1:04
  SPEAKER_01: Yeah.
• 1:05
  SPEAKER_01: And you know, I think so.
• 1:06
  SPEAKER_01: This is based off of a paper, right?
• 1:08
  SPEAKER_01: So one of the interesting things that I found was, yeah, I mean, there the the techniques that people use now to monitor for this sort of thing are um, you know, don't leverage all the available data out there.
• 1:22
  SPEAKER_01: And that's kinda, I think that that's kind of the point of this podcast, too.
• 1:26
  SPEAKER_01: But I think that as I learned about that, I learned how little, in fact, they use uh the potential of available data.
• 1:34
  SPEAKER_01: So, for example, they do surveys with the community epidemiologists, or uh they rely on transcriptions or whatever of forensic lab reports.
• 1:45
  SPEAKER_00: Or they cover certain geographies, but not all of them, because that would be too much information, right?
• 1:50
  SPEAKER_01: Right, right, right.
• 1:51
  SPEAKER_01: And it's so it's it's sparse, it doesn't have full coverage, but it's also kind of there's a huge time lag, right?
• 1:58
  SPEAKER_00: And there's potential for bias to be introduced uh in like this this type of a survey format, too.
• 2:04
  SPEAKER_01: That's true.
• 2:05
  SPEAKER_01: Yeah.
• 2:05
  SPEAKER_01: There's yeah, that's right.
• 2:06
  SPEAKER_01: And so it's very uh to me at least, it f I find it very interesting that the data, I mean, this is supposedly feeding into a predictive model of some sort uh to monitor what's happening and potentially figure out, okay, when the next, you know, kind of the next surge of these cases is going to go up or whatever.
• 2:23
  SPEAKER_01: And it it a model is gonna be very lacking if that's all the data that they that it has going in.
• 2:29
  SPEAKER_00: I mean, it it's interesting because you think, okay, what do you learn in school?
• 2:34
  SPEAKER_00: What do you learn in epidemiology?
• 2:36
  SPEAKER_00: And um oh, I've never I don't I don't have a master's in public health.
• 2:42
  SPEAKER_00: However, yeah, however, there's like the traditional way of doing things, right?
• 2:47
  SPEAKER_00: And now that you have AI, you can kind of flip that script if you want.
• 2:54
  SPEAKER_00: Yeah.
• 2:54
  SPEAKER_00: Right.
• 2:55
  SPEAKER_00: And there is this concept that, like, hey, we have all this extra data out there in the world now.
• 3:02
  SPEAKER_00: Can we harness it?
• 3:03
  SPEAKER_00: Do we need to use the traditional methods, or are there other methods out there that can mine the data and give us that same information and maybe do it in a way that, well, at least in some ways, could be better, right?
• 3:16
  SPEAKER_00: In this situation, it's like, well, if we know that an opioid overdose is going to be happening, or more, you know, the opioid epidemic is getting worse, then we can do something about it.
• 3:28
  SPEAKER_00: Yeah.
• 3:28
  SPEAKER_00: And so I think that's that's kind of the key here is hey, can we leverage different types of information that's not traditional information and use that to make decisions, yeah, to make changes, to intervene.
• 3:43
  SPEAKER_01: Yeah, and we've talked about this sort of thing before in the podcast too, where we use data in different ways and unexpected ways and so on.
• 3:49
  SPEAKER_01: Um, and here it's not super unexpected necessarily, but what they're doing is utilizing social media data, right?
• 3:55
  SPEAKER_01: And that was interesting too.
• 3:56
  SPEAKER_01: There was a whole section on the different social media platforms and how they are the data can be different across the board.
• 4:02
  SPEAKER_01: Like for instance, um, Twitter is really great in providing uh information, geolocation information, telling you where people are, when they're saying things, but they are restricted in terms of how much um text one can type in that, how much they can express in that, right?
• 4:20
  SPEAKER_01: So there's that issue.
• 4:21
  SPEAKER_01: Um, and it's kind of open-ended.
• 4:23
  SPEAKER_01: Whereas there's other social media platforms like Reddit, um, which are you can you can write longer posts, uh, but also it's community driven.
• 4:33
  SPEAKER_01: So Reddit has, for those who don't know, Reddit has what they call subreddits, which is essentially little communities, little groups within Reddit uh for different topics, different groups, and so on.
• 4:44
  SPEAKER_01: There's one for politics, and there's one for uh pen and ink drawings, and there's one for you know various other things.
• 4:51
  SPEAKER_01: There's I think there's one even for opioid um uh, you know, uh help.
• 4:57
  SPEAKER_00: And so Yeah, there's one like that's you know, a subreddit for addiction is one of the ones they called out in the article.
• 5:02
  SPEAKER_01: Right, right, right.
• 5:03
  SPEAKER_01: And and each of these subreddits is essentially a little community that you can join once you're part of the Reddit uh user base, and there's a moderator, there are rules, uh things you can and cannot say in that community.
• 5:14
  SPEAKER_01: And people, you know, usually have you know, Reddit accounts are somewhat anonymous, like you you your Reddit, you know, handle is whatever it is you want it to be, and so it's not your name or anything.
• 5:25
  SPEAKER_01: And so you get to be a little bit anonymous, but at the same time, there's a community of people, and it's not just the open world, it's people who are willing to follow the rules of that community when conversing with each other.
• 5:36
  SPEAKER_00: And I think that sort of pseudo-anonymity is like really important in this situation because there's a lot of stigma associated with opioid use, and so having the ability to to anonymously discuss that with a with a community of people is part of the appeal.
• 5:54
  SPEAKER_01: Yeah.
• 5:55
  SPEAKER_00: Um and one of the other things that's interesting about Reddit and true on some other social media platforms as well, is that the historical activity is is still there.
• 6:07
  SPEAKER_00: So sometimes I'll look something up on the internet and uh subreddit from you know 2006 will pop up with an answer to that question.
• 6:15
  SPEAKER_00: So the historical information is there, yes.
• 6:18
  SPEAKER_00: Um, which means that they were able to like study what happened in the past over like 10 or more years, what sort of drug discussions were happening, and and they tried to use geolocation as as much as they were able to here as well, and then kind of predict, okay, can this help predict what might happen in the future?
• 6:40
  SPEAKER_00: So what's or in real time?
• 6:42
  SPEAKER_01: Yeah, no, that's right.
• 6:43
  SPEAKER_01: And what's interesting also is that with social media, you get first-person reports.
• 6:47
  SPEAKER_01: So, unlike things like a forensic lab report or a survey performed on epidemiologists, here you have first-person reports discussing that drug.
• 6:57
  SPEAKER_01: And that is potentially extremely valuable, right?
• 7:00
  SPEAKER_01: Um, and so I think that that's the other piece here that social media brings that other forms of data don't bring.
• 7:06
  SPEAKER_01: And in this study, you can do this in a very large scale.
• 7:10
  SPEAKER_01: And I think that in this study, they selected over one and a half million people who they followed over 10 years across different locations, um, as those people, you know, typed on Reddit or whatever, and tracked that kind of conversation and they tried to extract out uh mentions, uh, which is whenever they said something about a particular drug.
• 7:33
  SPEAKER_01: And I think there's a list of drugs that they were specifically following too, right?
• 7:36
  SPEAKER_00: Yeah, exactly.
• 7:36
  SPEAKER_00: And it's interesting because it's not that they had to say, Oh, I used a drug or someone overdosed on the drug.
• 7:43
  SPEAKER_00: It was literally that they just commented about the drug, it was just a comment, simply that, and that still was enough to predict, help predict opioid overdoses.
• 7:53
  SPEAKER_01: Yeah, and and the thing that we're trying to predict was the CDC's overdose death rates, right?
• 7:58
  SPEAKER_01: And those are typically predicted by the things that I mentioned earlier, uh, the surveys and such.
• 8:03
  SPEAKER_01: Um, but they, and you know, you use what's called a time forecasting model to do something like this, which is nothing really that all that fancy.
• 8:10
  SPEAKER_01: It's a crew, it's used in financial prediction, all kinds of other prediction modes, but it's like it just looks at the data and looks at windows of the data as it moves and tells and then sort of tracks how it changed uh over time.
• 8:23
  SPEAKER_01: And it's able to do that, and it's able to predict based on these changes what is the most likely kind of next change coming up and going forward.
• 8:30
  SPEAKER_01: And um the survey data was fine for a lot of that, but um it obviously lagged, right?
• 8:36
  SPEAKER_01: It didn't, it didn't, it wasn't like frequent enough.
• 8:38
  SPEAKER_01: And then when they were able to overlay the Reddit data on top, the number of mentions of disease and such, it improved the quality of predictions quite a bit, uh, which I thought was super interesting.
• 8:48
  SPEAKER_00: Yeah, so basically, like the comments about opioids increased or around the same times that the overdose rates, death rates increased.
• 8:58
  SPEAKER_00: Yeah.
• 8:59
  SPEAKER_00: Except now, because of Reddit, if you were gonna use this like in the future, right, you would have that information in real time rather than in a six-month lag that you might get from the CDC, for example.
• 9:10
  SPEAKER_01: Exactly.
• 9:11
  SPEAKER_01: That's the big value here, yes.
• 9:13
  SPEAKER_00: And when you have that information, this is where my mind goes, you can do something about it, right?
• 9:19
  SPEAKER_00: It's like if you know that, for example, like fentanyl is being mentioned more.
• 9:26
  SPEAKER_00: This is a synthetic opioid, this is one of the ones from the study that that did very well with Reddit.
• 9:33
  SPEAKER_00: And um, if you know that it's being mentioned and you know that's fast acting, you know it's very strong, people overdose on it more often, then that's like an early warning sign that you need to do something about it.
• 9:47
  SPEAKER_00: And in this case, it could be like, hey, let's distribute a lot of naloxone or narcan in the community.
• 9:53
  SPEAKER_00: Let's make sure everyone has some on hand so that if someone does overdose, we can try to reverse it, right?
• 9:59
  SPEAKER_00: Right.
• 10:00
  SPEAKER_00: Or maybe we could have like uh distribution of drug testing strips so we can understand if if something that we thought was heroin is actually fentanyl and is stronger than we think it would be.
• 10:12
  SPEAKER_00: So there's a lot of like harm reduction methods that are out there to actually intervene and try to reduce the opioid death rate if you have some sort of reason to think that it's going out.
• 10:23
  SPEAKER_00: But in this case you would, right?
• 10:25
  SPEAKER_00: Because Reddit users are saying a lot about it.
• 10:28
  SPEAKER_01: Right.
• 10:29
  SPEAKER_01: And I think that to me, there's a conceptually interesting piece too, which is we're not hearing about the specific usage or the behaviors of those people, right?
• 10:40
  SPEAKER_01: Who are using the drugs.
• 10:41
  SPEAKER_01: We're just you observing linguistic mentions of those drugs in the in the Reddit post.
• 10:47
  SPEAKER_00: And that's all it took.
• 10:48
  SPEAKER_01: Well, that's all it took to improve the quality of the prediction, but could we do even better?
• 10:53
  SPEAKER_01: Is there something more being said that suggests, you know, that is more nuance, that potentially suggests more in the Reddit posts themselves?
• 11:00
  SPEAKER_00: Oh, this is a good question.
• 11:01
  SPEAKER_01: So, like if your intervention idea was a particular idea and it didn't, let's say it it didn't work, right?
• 11:08
  SPEAKER_01: Uh you did some intervention and nothing changed.
• 11:10
  SPEAKER_01: So, but but maybe the posts have actually information for why it didn't why nothing changed to give you the idea for the next intervention, as opposed to just randomly trying different things, right?
• 11:20
  SPEAKER_01: So I I I do wonder if there's more in there in those posts.
• 11:24
  SPEAKER_00: Um that you could zoom in on and say, oh, this core, like, can we sort of code the posts?
• 11:29
  SPEAKER_00: Like, oh, this one talked about an overdose, this one mentioned Narcan, this one XYZ, and like see which of those pieces correlates even more.
• 11:38
  SPEAKER_01: Yeah, because some posts might just be about it relaying somebody's experience with using it.
• 11:43
  SPEAKER_01: Some posts might be about um not relaying their personal experience using it, but maybe they have a question about it, right?
• 11:50
  SPEAKER_01: It could be anything.
• 11:51
  SPEAKER_01: And so I think even differentiating the type of content would be hugely valuable.
• 11:55
  SPEAKER_01: Maybe, I mean, I'm guessing it'll be valuable at least from an interventional perspective, if you're trying methods to reduce that, right?
• 12:01
  SPEAKER_00: Yeah, I mean, there's gotta be ways to like improve the model even more, essentially, is what you're saying, by taking a closer look.
• 12:09
  SPEAKER_00: And I'm I I think that makes a lot of sense.
• 12:12
  SPEAKER_00: In fact, when they were looking at this model, one of the things that affected it for the for the worse, I think, was COVID.
• 12:20
  SPEAKER_00: Which is interesting, right?
• 12:21
  SPEAKER_00: I mean, I guess it's not particularly surprising because when you have a monitoring system, you sort of expect it to keep plugging along at the status quo, right?
• 12:31
  SPEAKER_00: Yeah.
• 12:31
  SPEAKER_00: Um, and then when something major comes in and changes the world, or changes the world view, yeah, or changes people's behaviors, or is just kind of like a shock to the system, yeah, then it makes sense that the model's not going to perform as well because it's like operating in a different world than it was.
• 12:50
  SPEAKER_01: Yeah, it's a very sudden shock to the system.
• 12:52
  SPEAKER_01: And a sudden shock to the system, if you think about COVID itself, it caused people to stay at home more, it caused people to have issues, mental health issues, you know, it did a lot of other things that affected not just in this case, potentially not just the usage of the drugs, but maybe even how people talked about it.
• 13:10
  SPEAKER_00: Or the access to it, right?
• 13:12
  SPEAKER_00: There's so much going on there.
• 13:13
  SPEAKER_01: Yeah.
• 13:14
  SPEAKER_01: And that changed the underlying model completely.
• 13:16
  SPEAKER_01: And so it made this sort of slow prediction, if you think about the prediction as sort of this like time window that you're moving across the um uh over time to see, okay, what's my next likely number of mentions or my estimated, you know, CDC measures for overdose rate, death rates and such.
• 13:33
  SPEAKER_01: Uh maybe that just dramatically changes when you have a shock in the system, and now you can't rely on uh averaging it based on what you've seen before.
• 13:42
  SPEAKER_00: Yeah, I mean a model is only so good as the inputs into it, right?
• 13:48
  SPEAKER_00: And in this case, there's something that's affecting the inputs.
• 13:51
  SPEAKER_00: Yeah.
• 13:52
  SPEAKER_00: Right?
• 13:52
  SPEAKER_00: Because the whole world has changed.
• 13:54
  SPEAKER_01: Yeah, no, exactly.
• 13:55
  SPEAKER_01: Exactly.
• 13:56
  SPEAKER_01: So that's I think an interesting uh thing, and that's something maybe the content of the text might might might suggest, right?
• 14:01
  SPEAKER_01: That there is, in fact, a step change or a big change that's happened.
• 14:04
  SPEAKER_01: I mean, we know and an event happens, obviously, in the real world, but you know, that's something to keep in mind is you know how the distribution of the data changes itself.
• 14:13
  SPEAKER_01: And maybe the content of the text might give you more info there too.
• 14:16
  SPEAKER_00: Yeah.
• 14:17
  SPEAKER_00: I do think in general, it makes a lot of sense to mine whatever data is out there to get that public health information though, um, and to do it in a more real-time fashion.
• 14:31
  SPEAKER_01: Yeah.
• 14:32
  SPEAKER_00: And I think this is what it helped do.
• 14:33
  SPEAKER_00: It didn't, it worked really well for uh fentanyl, the synthetic opioid.
• 14:37
  SPEAKER_00: It didn't work quite as well for um heroin, interesting.
• 14:41
  SPEAKER_00: Natural and the combo of natural and semi-synthetic.
• 14:45
  SPEAKER_00: But I do think, like you said, maybe the model could be tweaked for certain things, like not just comments, but actual like specific types of mentions, like mention of overdose, mention of using Narcan, et cetera.
• 14:59
  SPEAKER_00: So I think there's adjustments that could be made.
• 15:02
  SPEAKER_00: But I think in general, this whole idea of hey, there's so much data out there, it's not data that we've necessarily thought of as public health data in the past.
• 15:11
  SPEAKER_00: But because we now have this larger compute power, we can aggregate all of it and it's publicly available.
• 15:18
  SPEAKER_00: Hey, we should be using this too and seeing if it can give us some of this information that can then lead us to intervene in a in a more time-sensitive fashion.
• 15:29
  SPEAKER_01: Yes.
• 15:30
  SPEAKER_01: That's huge.
• 15:30
  SPEAKER_01: I mean, this is the public health uh challenge, right?
• 15:33
  SPEAKER_01: And so we have we have the data now, so it's we should be using it, is what you're saying.
• 15:37
  SPEAKER_00: We should be using it and we should also be figuring out right how how can we use it best.
• 15:42
  SPEAKER_00: Yeah.
• 15:42
  SPEAKER_00: Right?
• 15:42
  SPEAKER_00: Because like you were saying, it's it may be the comments about opioids are one thing and it points you in the right direction.
• 15:48
  SPEAKER_00: But like maybe if we looked very specifically at types of comments, that would help even more.
• 15:56
  SPEAKER_00: Yeah.
• 15:56
  SPEAKER_00: And so it's like, well, how can we how can we harness that data to get the information faster, better, etc.?
• 16:05
  SPEAKER_00: And once we can create these predictions, what do we do about it?
• 16:09
  SPEAKER_00: And I think that what do we do about it question is really important.
• 16:13
  SPEAKER_00: Yeah, I agree.
• 16:14
  SPEAKER_00: Because predicting something, it means nothing if you can't do anything about it, right?
• 16:18
  SPEAKER_00: It's like, oh, well, that was interesting.
• 16:20
  SPEAKER_00: Next, but in this case, there are things.
• 16:22
  SPEAKER_00: And so you'd still have to have the public health infrastructure in place to actually like go get those Narcan kits, make sure that they're freely available, you know, distribute them at the public health department, um, you know, et cetera.
• 16:36
  SPEAKER_00: And so that piece is like the really important piece where the human needs to be there and and doing those interventions and making sure that that happens in real time too, because the knowledge is not enough.
• 16:48
  SPEAKER_01: But but also I think along the same lines, if you look back at the posts, you can you should be able to say, okay, why is it that there is this increase right now, right?
• 16:58
  SPEAKER_01: Why is that happening?
• 16:59
  SPEAKER_01: Is there more suggestion in the post itself, uh in those Reddit posts themselves, that tell you more about why something is happening the way it is, right?
• 17:07
  SPEAKER_01: So it's not Are you volunteering?
• 17:08
  SPEAKER_00: Are you volunteering to look through those 1.7 million users?
• 17:11
  SPEAKER_01: No, but I'm not you know, I think that that is a that is an excellent.
• 17:13
  SPEAKER_01: I mean, I think the paper themselves suggested using various AI and LLM-based approaches to mine the text for more meaning.
• 17:20
  SPEAKER_01: But my point to yours is is that it's not just the interventions, but you know, sort of what can you do with the predictions, but it is what why were the predictions the way they were to begin with?
• 17:30
  SPEAKER_01: Is there something systemically wrong in our system that would give us ideas for interventions based on why something was uh based on a pattern that we observe?
• 17:39
  SPEAKER_01: You know, and so that's kind of what I was getting at with this piece.
• 17:41
  SPEAKER_00: Yeah, no, that makes sense too.
• 17:43
  SPEAKER_00: It's like um kind of getting back to this reasoning piece of hey, how can we understand this better?
• 17:50
  SPEAKER_00: How can we understand the prediction that's happening better?
• 17:54
  SPEAKER_00: Yeah, the why behind it, and that can help us A, improve our model and B intervene at that level too, like in the before times.
• 18:03
  SPEAKER_01: Right, right, exactly.
• 18:04
  SPEAKER_01: I I'm excited by this research direction that uh you know the authors have been taking here, and I'm looking forward to seeing more uh, you know, what they do next with it, because there's a lot to be done here.
• 18:14
  SPEAKER_00: Yeah, and I think social media has so much information in it, and it can absolutely help monitor public health data so that we can intervene.
• 18:24
  SPEAKER_00: So we will leave you with that, and we will see you next time on Code and Cure.
• 18:28
  SPEAKER_00: Thank you for joining us.