Latest / Code & Cure / #22 - Hope, Help, and the Language We Choose

Transcript

#22 - Hope, Help, and the Language We Choose · Code & Cure

• 0:00
  SPEAKER_01: Could a large language model help rewrite the stigma around opioid use disorder?
• 0:14
  SPEAKER_00: Hello and welcome to Code and Cure, where we discuss decoding health in the age of AI.
• 0:19
  SPEAKER_00: My name is Vasant Sarathi.
• 0:21
  SPEAKER_00: I'm an AI researcher and cognitive scientist.
• 0:23
  SPEAKER_00: And I'm here with Laura Hagopian.
• 0:26
  SPEAKER_01: I'm an emergency medicine physician and I work in digital health.
• 0:30
  SPEAKER_01: And I'm very excited for today's topic because I saw a lot of opioid use disorder in the emergency department, both, you know, where I worked as an attending, but also where I trained.
• 0:42
  SPEAKER_01: We had a huge program.
• 0:42
  SPEAKER_01: We had a methadone clinic nearby.
• 0:45
  SPEAKER_01: We see we saw a lot of overdoses.
• 0:47
  SPEAKER_01: I mean, I had people who would show up to the emergency department and like sort of like knock on the door and dump their friends at the ambulance bay because they weren't breathing and you'd have to go out with the Narcan and uh, you know, a breathing apparatus and and essentially save their life.
• 1:03
  SPEAKER_01: So this was something that I saw so much of.
• 1:07
  SPEAKER_01: I saw it tear families apart.
• 1:08
  SPEAKER_01: I saw how hard it was on the patients, on the family members.
• 1:14
  SPEAKER_01: Um it's just, it's a very tough condition.
• 1:18
  SPEAKER_01: And and there is unfortunately a lot of stigma around it where people will point, they'll say, Oh, it's your fault, or you, you know, uh you did this to yourself, or um even the treatments, there was stigma around, right?
• 1:33
  SPEAKER_01: It was like, oh, if you're going on these meds, um it's it's almost as bad as as using drugs.
• 1:41
  SPEAKER_01: So I I think this is a this is a topic that's kind of close to me because I worked so hard on trying to intervene with um a lot of people who had opioid use disorder and trying to get them to a place where you know they were able to, for example, detox off the drugs or find medication assisted treatment.
• 2:06
  SPEAKER_01: Um but it it it's a tough one.
• 2:09
  SPEAKER_01: And I I think because there's one of the problems is that there is so much stigma around it.
• 2:15
  SPEAKER_00: Yeah, yeah.
• 2:15
  SPEAKER_00: And and in fact, that stigma is what pushes a lot of these people to go online where they can be anonymous and have built find their communities of people who share similar issues or are supportive of this, right?
• 2:31
  SPEAKER_00: I mean, you have all these TikTok TikTok communities, you have Reddit communities, and is that what you get online though?
• 2:35
  SPEAKER_01: Like, do you is it?
• 2:37
  SPEAKER_00: Well, that's I think that's what we're we're we're asking, right?
• 2:40
  SPEAKER_00: And so in this paper, which is incredible paper, is is all about whether or not at one level there's an AI piece too, which I'll get to in a second, but at one level, it's all about whether or not these communities, online communities, actually provide that kind of support.
• 2:53
  SPEAKER_00: And there's some suggestion that maybe not, because even in these online communities, uh, you know, there, and then people discuss things like um, you know, opioid use or alternative treatments or support or recovery, you know, support for recovery.
• 3:08
  SPEAKER_00: Um, there are still issues of unhelpful advice, uh, maybe anecdotal evidence that somebody's had some specific thing that may not may apply to them, but not someone else.
• 3:17
  SPEAKER_00: Um, there is a lot of incorrect and unverified information.
• 3:20
  SPEAKER_01: Yeah, and you can upvote things too, right?
• 3:23
  SPEAKER_01: Like that the top post may be the most interesting post, but is it correct, right?
• 3:29
  SPEAKER_01: Or um is it safe?
• 3:31
  SPEAKER_01: Is it is it something that a clinician might say when you're trying to help someone through opioid use disorder?
• 3:39
  SPEAKER_01: I not always, right?
• 3:40
  SPEAKER_01: It can it it could be that some of this text could actually marginalize people more, yeah, make people feel worse, you know?
• 3:48
  SPEAKER_01: And so this is tough because the the place where people may go to anonymously get information and ideas and support and support, I don't know.
• 4:00
  SPEAKER_01: Like, is it always supportive?
• 4:01
  SPEAKER_00: Well, this is also, you know, a reason why this paper was interesting to me was because uh they were interested in using the idea of LLMs and AI um as a sort of a moderator at a level, at some level.
• 4:13
  SPEAKER_00: And that's some work that I've done in the past where I've tried to build AI systems that are uh that provide improve the pro-sociality of a of a community like Reddit.
• 4:25
  SPEAKER_00: You see lots of you know um misinformation or toxic information out there, toxic language out there.
• 4:31
  SPEAKER_00: People use AI systems to reduce that, but maybe AI systems can also promote pro-social attitudes um in these communities, right?
• 4:38
  SPEAKER_00: And so that's some research work that I had done previously, and it was very for me, this paper was very interesting from that aspect as well, uh, personally.
• 4:46
  SPEAKER_00: Um, and I I think that the the study itself was was was well done and well focused, right?
• 4:51
  SPEAKER_01: Yeah, I think so.
• 4:52
  SPEAKER_01: I mean one of the things that's so interesting to me is how language can help define how we think about something.
• 5:01
  SPEAKER_01: Like, for example, I've been saying like someone who has opioid use disorder, that's very different than me saying addict, right?
• 5:11
  SPEAKER_01: Yeah, those are like they like in theory they're supposed to mean the same thing, but on Reddit you might see, you know, language that labels people as addicts.
• 5:20
  SPEAKER_01: Yeah.
• 5:21
  SPEAKER_01: It's like, do you say someone who has diabetes, or do you say that's a diabetic?
• 5:29
  SPEAKER_01: Do you know what I mean?
• 5:29
  SPEAKER_01: It's like there are certain things where they're health conditions, right?
• 5:34
  SPEAKER_01: They're not personal failings.
• 5:36
  SPEAKER_01: Um, and so if I said to you, oh, you know, I got a cold the other day, would you look at me and be like, well, that that was a that was a personal failure?
• 5:45
  SPEAKER_01: You you deserved that code and um uh cold, sorry.
• 5:50
  SPEAKER_01: And you know, you may never fully recover.
• 5:52
  SPEAKER_01: I would you say would you even would those things go through your head?
• 5:55
  SPEAKER_00: I mean that is stick that is the stigma piece there, right?
• 5:57
  SPEAKER_00: I mean uh the example that like cold, the example that comes to mind for me is always like food poisoning, right?
• 6:02
  SPEAKER_00: You go to your restaurant all the time that you go to, and then one time you go and get food poisoning, like no one is going to blame you for getting food poisoning, and no one is going to hold that against you.
• 6:11
  SPEAKER_00: No one is gonna define you by that, uh, or even decide that because you got food poisoning, you're this kind of individual.
• 6:17
  SPEAKER_01: Yeah, which is is not the same with opioid use disorder, right?
• 6:21
  SPEAKER_01: And we know that there are genetic components to it, for example, but that sort of gets ignored.
• 6:27
  SPEAKER_01: Yeah.
• 6:28
  SPEAKER_01: Especially when we label people with certain types of language, like the word addict versus, you know, actual person human who has opioid use disorder.
• 6:36
  SPEAKER_01: Yep, yep, yep.
• 6:38
  SPEAKER_01: So I think that that's where I saw a lot of opportunity here in this intervention is like, hey, we know, and we've talked about this on prior episodes, that LLMs really try to be helpful, right?
• 6:50
  SPEAKER_01: And they try to, you know, in a way, like make you like them.
• 6:56
  SPEAKER_00: Yeah, yeah, yeah, yeah.
• 6:57
  SPEAKER_00: So I mean, that's absolutely right.
• 6:58
  SPEAKER_00: And and that's where I think the the study started off with, right?
• 7:01
  SPEAKER_00: I was thinking that maybe there's a role here for for AI and LLM systems.
• 7:05
  SPEAKER_00: So that's, you know, what what did the study?
• 7:08
  SPEAKER_00: Can you summarize a little bit what the study talked about?
• 7:10
  SPEAKER_01: Well, so basically they looked at um attitudes to a few different kinds of stigma that are common with opioid use disorder.
• 7:19
  SPEAKER_01: One was around medication assisted, right?
• 7:21
  SPEAKER_01: So some people will go on uh a medication like Suboxone or methadone to um basically assist them in you know stop stopping the drugs that they were using.
• 7:33
  SPEAKER_01: So that's one.
• 7:34
  SPEAKER_00: The second And and sorry, the the stigma would there would be you're just replacing one drug for another.
• 7:39
  SPEAKER_00: And that's the stigmatizing um concept.
• 7:42
  SPEAKER_01: Yeah, I mean, there's lots of stigmatizing concepts that could go along with that, but that's probably the most common one is hey, you're like, are you replacing, you know, you're not you're still in you're still you still got opioid use disorder if you're still using an opioid, because that's what these are.
• 7:58
  SPEAKER_01: These are opioids, but they're used in a much more controlled fashion, right?
• 8:01
  SPEAKER_01: Um, you know, there are certain hallmarks of addiction where you might um, I don't know, like use all your rent money to get drugs.
• 8:12
  SPEAKER_01: And if you're on methadone and you're controlled, maybe you wouldn't do that.
• 8:16
  SPEAKER_01: And so this is like a harm reduction technique where you do something that um you're able to function function in daily life now because you're not spending your rent money and your food money on on opioids.
• 8:32
  SPEAKER_01: Right, exactly.
• 8:33
  SPEAKER_01: Um yeah.
• 8:35
  SPEAKER_00: And then then the second one they had there was And so sorry, one more thing.
• 8:38
  SPEAKER_01: Yeah There's there's lots of stigma that could be around this.
• 8:41
  SPEAKER_01: You named sort of the most common one, but like there are people who maybe think that these treatments aren't safe.
• 8:47
  SPEAKER_01: Um, or that they may not be the best way.
• 8:52
  SPEAKER_01: Um, you know, or that it's addictive too, right?
• 8:56
  SPEAKER_01: So there's different ways that you could have stigma towards medication-assisted treatment.
• 9:00
  SPEAKER_00: Got it, got it, got it, got so that was the first sort of dimension that they looked at.
• 9:04
  SPEAKER_00: And then another attitude they looked at was uh peep pee towards people, towards humans with the Leo disorder.
• 9:11
  SPEAKER_01: Yeah, it's like labeling, right?
• 9:12
  SPEAKER_01: Um oh, they're dangerous, oh, they're weak, whatever it is.
• 9:16
  SPEAKER_01: They don't, they this is like a personal failure, right?
• 9:18
  SPEAKER_01: Right.
• 9:19
  SPEAKER_01: And so um, you know, they're responsible for their own condition, that kind of stuff.
• 9:24
  SPEAKER_01: Um, none of which is true, right?
• 9:26
  SPEAKER_01: And then the third one was it was about attitudes toward the disorder itself, towards opioid use disorder, like, oh, um, you know, people will will never truly recover from it.
• 9:39
  SPEAKER_01: Um, it's it's not possible.
• 9:41
  SPEAKER_01: Uh, this is where kind of moral strength may the the concept may come in as well.
• 9:47
  SPEAKER_01: Or that it's like um treat treating people with opioid use disorder is futile.
• 9:52
  SPEAKER_01: Like those are the stigmatizing beliefs that are associated.
• 9:56
  SPEAKER_01: Um, and and so what they did in this paper was they said, okay, let's let's go on like Reddit, let's find some posts that are related to these things that we want to look at.
• 10:10
  SPEAKER_01: And often is a query of some sort, somebody's asking a question of some sort.
• 10:13
  SPEAKER_01: Yeah, exactly.
• 10:14
  SPEAKER_00: Yeah.
• 10:14
  SPEAKER_01: And we want to compare like an LLM generated response to a human response.
• 10:22
  SPEAKER_01: To the to the human response, like the most upvoted human response to like nothing, right?
• 10:28
  SPEAKER_01: To you know, you receive no new information.
• 10:31
  SPEAKER_01: And let's see what people's attitudes were before reading these and then after reading these.
• 10:36
  SPEAKER_00: Yeah.
• 10:36
  SPEAKER_00: And people start off with some baseline set of attitudes, right?
• 10:38
  SPEAKER_00: So they ask some questions beforehand to get a sense of the individual who is judging here.
• 10:43
  SPEAKER_01: Right, exactly.
• 10:44
  SPEAKER_01: And so you want to see if there's a change in that attitude.
• 10:46
  SPEAKER_01: And they did some other stuff here, like they said, oh, let's do it a single episode versus multiple episodes.
• 10:51
  SPEAKER_01: I don't think we need to get into all of that.
• 10:53
  SPEAKER_01: But at the end of the day, they found essentially that the LLM responses were way less stigmatizing.
• 11:02
  SPEAKER_00: At least for the first um sort of attitude of the um, you know, the m medic medications associated with OUD, right?
• 11:10
  SPEAKER_00: I mean, especially for that uh attitude, they had a significantly the LLMs were significantly better.
• 11:16
  SPEAKER_00: And so uh, yeah, I mean, that is that is promising, right?
• 11:19
  SPEAKER_01: And when when I I'm gonna read an example in a second, but they actually did like you know fancy linguistic evaluations of these, and they said, hey, these the ones generated by the LLMs, they were more optimistic.
• 11:32
  SPEAKER_01: They were more supportive, they gave people more of a sense of belonging.
• 11:36
  SPEAKER_01: Yeah.
• 11:37
  SPEAKER_01: And so it's a different experience to read something like that.
• 11:40
  SPEAKER_01: And I thought this was very interesting.
• 11:44
  SPEAKER_01: The um the distribution of pronouns was different too.
• 11:48
  SPEAKER_01: So the human written responses would say like I, I, I, I, I, I, this, I, me, uh, a lot of first person pronouns.
• 11:56
  SPEAKER_01: Whereas the LLM ones really didn't use that many first person pronouns.
• 12:01
  SPEAKER_00: I want to return to that one later.
• 12:02
  SPEAKER_00: I think that's an interesting one.
• 12:04
  SPEAKER_00: Oh, that was second person.
• 12:05
  SPEAKER_00: You know, no, but I want to return to that point later.
• 12:07
  SPEAKER_00: Uh, and I think because it's I want to sort of bookmark that point because it's such a good point.
• 12:12
  SPEAKER_00: But, you know, and and and I had more words here too.
• 12:14
  SPEAKER_00: They had, you know, they used the words like credible, informative, and resourceful as well for the LLM responses, which is super interesting from you know, um, just from the way how far AI has come.
• 12:25
  SPEAKER_01: But they get upvoted, that's another thing.
• 12:27
  SPEAKER_01: They didn't look at that.
• 12:28
  SPEAKER_01: I'm curious to see if they would get upvoted or not.
• 12:31
  SPEAKER_00: Yeah.
• 12:31
  SPEAKER_01: But I I'd love to give an example.
• 12:33
  SPEAKER_01: Yeah, please.
• 12:34
  SPEAKER_01: So here was here was a query.
• 12:36
  SPEAKER_01: The query, then this one is about the medication um assisted treatment.
• 12:40
  SPEAKER_01: So here's a query.
• 12:41
  SPEAKER_01: I have been dependent on Suboxone for three to four years.
• 12:44
  SPEAKER_01: I am worried that I am just trading one addiction for another.
• 12:48
  SPEAKER_01: I will end up addicted to both opioids and suboxone.
• 12:51
  SPEAKER_01: Do others feel the same?
• 12:55
  SPEAKER_01: And so here's a human response.
• 12:57
  SPEAKER_01: The human response that was upvoted the most is kind of paraphrased here, but yes, with Suboxone, I traded one addiction for another.
• 13:05
  SPEAKER_01: Two years later, I am employable, functional, and have money in the bank.
• 13:09
  SPEAKER_01: Yes, I'm addicted to Suboxone now, but this addiction is an angel compared to the devil that was opioid addiction.
• 13:15
  SPEAKER_00: That's pretty good.
• 13:16
  SPEAKER_00: That seems somewhat destigmatizing, right?
• 13:19
  SPEAKER_00: I mean, it's saying, yes, you know, while agreeing that it's, you know, while while agreeing with the wrong information that it's a substitute, at least it acknowledges the fact that there's a difference.
• 13:30
  SPEAKER_01: Yeah, it's interesting because from my clinician lens, I'm like, well, I I agree it sounds destigmatizing, but I I don't call Suboxone an addiction.
• 13:38
  SPEAKER_01: Right.
• 13:38
  SPEAKER_01: It's not, it's not the same.
• 13:39
  SPEAKER_00: It's the language, it's the labeling, right?
• 13:41
  SPEAKER_00: Right.
• 13:42
  SPEAKER_01: And it's not true.
• 13:43
  SPEAKER_01: Like you're using it as medication-assisted treatment.
• 13:46
  SPEAKER_01: That's a different, that's different entirely.
• 13:50
  SPEAKER_01: Um here's the LLM generated response.
• 13:53
  SPEAKER_01: And there's definitely less use of that first-person pronoun in here, so you can take notice while I read this.
• 14:01
  SPEAKER_01: It's not about treating one addiction for another, but rather about managing the disorder in a safer, controlled manner.
• 14:08
  SPEAKER_01: While it's true that physical dependence can occur with long-term use, this is not the same as addiction.
• 14:15
  SPEAKER_01: What do you think?
• 14:17
  SPEAKER_00: Yeah.
• 14:17
  SPEAKER_00: Uh you know, I think this is Yeah, this is very interesting for many reasons.
• 14:22
  SPEAKER_00: One, you know, I I I want to dive a little bit into the LLM itself because I'm curious.
• 14:26
  SPEAKER_00: I was curious about this, and I said, okay, what if you were designing an LLM or you were throwing in, you know, you're putting in this into Chat GPT, what would you say to Chat GPT to set it up?
• 14:36
  SPEAKER_00: And that's the prompt, right?
• 14:38
  SPEAKER_00: So it's worthwhile thinking about that for a second.
• 14:40
  SPEAKER_01: That's what they did too, right?
• 14:42
  SPEAKER_01: They went in and they did and they prompted.
• 14:43
  SPEAKER_01: Actually, when I looked at the prompt, I was like, this is way shorter than I expected.
• 14:47
  SPEAKER_01: I expected it to be this like big long thing where it said, Oh, be empathetic, be kind.
• 14:52
  SPEAKER_01: Right.
• 14:53
  SPEAKER_01: Make sure you're, you know, you're talking at like a sixth grade reading level.
• 14:56
  SPEAKER_01: It didn't do any, I'll let you read it.
• 14:58
  SPEAKER_01: It didn't do any of those things.
• 14:59
  SPEAKER_01: So I was I was shocked by how little information was in the prompt.
• 15:02
  SPEAKER_00: Me too.
• 15:03
  SPEAKER_00: Me too.
• 15:03
  SPEAKER_00: It's the prompt, which actually I think makes a study even stronger.
• 15:06
  SPEAKER_00: Uh, which uh the prompt basically said, uh, I'm gonna read it out.
• 15:10
  SPEAKER_00: You are a Reddit user, you actively browse through different subreddits, which are online communities in Reddit, to gather health-related information.
• 15:17
  SPEAKER_00: While browsing through these communities, you also frequently answer posts sharing opinions or information on opioid use disorder.
• 15:24
  SPEAKER_00: Write a comment, which could be posted on the subreddit, answering the following question taken from the same subreddit for the for context.
• 15:31
  SPEAKER_00: The and then they sort of you can sort of plug in the subreddit name.
• 15:35
  SPEAKER_00: Um blank subreddit is described as and then they provide the actual description of the subreddit.
• 15:41
  SPEAKER_00: And that's it.
• 15:42
  SPEAKER_00: And then they post the question.
• 15:43
  SPEAKER_00: And so to me, the prompt is very simple and very direct, and uh it doesn't assume anything, it doesn't provide any kind of um directions for pro-sociality, uh, it doesn't talk about stigma at all.
• 15:56
  SPEAKER_00: It's just in response to these comments.
• 15:59
  SPEAKER_00: Uh it is very fluent, which is wonderful.
• 16:02
  SPEAKER_00: Um, and and that's typically with these these LLMs, they speak, they can speak like in the same style as the community that so but so telling it that you're a Reddit user is really important because you then tell it how to speak like one would speak on Reddit, which is different from say, well, how one would speak on Facebook or how one would speak on Twitter or Right.
• 16:21
  SPEAKER_00: So the information's there.
• 16:22
  SPEAKER_00: Yeah.
• 16:23
  SPEAKER_00: So that's the there's that piece of information they provide to the internet.
• 16:25
  SPEAKER_01: And you know that you're gonna go to the subreddit on Sabox and or the subreddit on opiates or the subreddit on opiate recovery.
• 16:32
  SPEAKER_01: Like that's that that gives you like a lot of important context, right?
• 16:35
  SPEAKER_00: And the subreddit descriptions often have rules for people part of the subreddit.
• 16:42
  SPEAKER_00: So it'll include here are the things you want to talk about, and then say, here's the tone we want to maintain in this community.
• 16:47
  SPEAKER_00: And Reddit's very good about policing and about making sure that people in the community behave in the appropriate way of that community.
• 16:54
  SPEAKER_01: Do they use AI for that?
• 16:55
  SPEAKER_00: For what?
• 16:56
  SPEAKER_00: To to monitor.
• 16:58
  SPEAKER_00: Oh, well, that's the thing.
• 16:59
  SPEAKER_00: That's that was the research I did a few years ago on monitoring that.
• 17:02
  SPEAKER_00: And there are AI bots out there, but again, the sophistication isn't isn't there because the challenge is human language use is so complex and so nuanced and culturally nuanced that some of these things are not so straightforward.
• 17:15
  SPEAKER_00: So you could have a general rule by saying don't be rude, but what don't be rude actually means is a whole host of things.
• 17:22
  SPEAKER_00: You could say a completely normal, calm comment, but in the context of what was said before, could mean something absolutely rude.
• 17:30
  SPEAKER_00: And so that would be would necessarily need to be flagged as rude, right?
• 17:34
  SPEAKER_00: But my point is that these moderators set the rules up front and and then they kind of human moderated most of this, um, with the help of some AI bots as well to catch like toxic language and such.
• 17:46
  SPEAKER_00: But you still need some human moderators, and and they're not catching all the stigma stuff because the stigma things are very much very much nuanced.
• 17:52
  SPEAKER_00: Um but that also to your point about language choices, I think this, and then going back to the pronouns, right?
• 17:58
  SPEAKER_00: There's a distinction here, which is that um LLMs don't have OUD, they don't have opioid use disorder.
• 18:06
  unknown: Right.
• 18:06
  SPEAKER_00: They have never had that personal experience.
• 18:08
  SPEAKER_00: So the LLMs can and should never have to say, I, you know, had did this and this or I had this experience.
• 18:15
  SPEAKER_00: Uh if they did that, that would be misleading.
• 18:18
  SPEAKER_00: And yes, there might be LLMs that do that for the purposes of you know helping, being supportive, whatever.
• 18:23
  SPEAKER_00: And and you will see this sometimes, right?
• 18:24
  SPEAKER_00: I mean, there's these commercials, there used to be at least for Alexa and these other Amazon devices, where you would talk to it and it and you would say things like, Oh, how was your weekend?
• 18:33
  SPEAKER_00: And it would make up something, right?
• 18:34
  SPEAKER_00: It would say, Oh, I went hiking or whatever.
• 18:36
  SPEAKER_00: And of course it didn't actually go hiking.
• 18:38
  SPEAKER_01: Right.
• 18:38
  SPEAKER_01: And then you lose trust in it because you're like, that doesn't make any sense.
• 18:41
  SPEAKER_00: Yes, all you you lose trust or you just you you you place less importance into what it says.
• 18:45
  SPEAKER_00: You downvote it.
• 18:46
  SPEAKER_00: You doubt yeah, you downvote it either mentally or actually, right?
• 18:50
  SPEAKER_00: And and so there is that.
• 18:51
  SPEAKER_00: So it can't just come out and say, I did this.
• 18:53
  SPEAKER_00: That's that seems disingenuous.
• 18:55
  SPEAKER_00: Um, so it has to come off in a different way, in a different tone.
• 18:58
  SPEAKER_00: So I think you have another example.
• 19:00
  SPEAKER_01: Yeah, I do have another example.
• 19:01
  SPEAKER_01: So this one is from is the query for this one was fellow opioid addicts, where do you see yourself five years from now?
• 19:08
  SPEAKER_01: First of all, you can see already, even in the question stem, directly points at somebody directly points at somebody and labels them as an addict.
• 19:16
  SPEAKER_01: It's not like, hey, hey, um, other people who have opioid use disorder or whatever.
• 19:21
  SPEAKER_01: Like, I'm not doing a good job here, but you see what I'm saying.
• 19:23
  SPEAKER_01: It's like the labeling's even in the question stem.
• 19:26
  SPEAKER_01: So the human written response here was I hope to be clean from any medication-assisted treatment drugs.
• 19:32
  SPEAKER_01: Again, labeling.
• 19:33
  SPEAKER_01: What's this word clean?
• 19:34
  SPEAKER_01: Uh, you know, why are why are medication-assisted treatment drugs wrong or bad?
• 19:39
  SPEAKER_01: It's the last hurdle for me clean and sober with a better paying job and a home, one thing at a time.
• 19:44
  SPEAKER_01: There's so much labeling there, right?
• 19:46
  SPEAKER_00: Yeah.
• 19:47
  SPEAKER_01: And and that might be how that individual person feels.
• 19:49
  SPEAKER_01: But then you have to think about all the people who are reading this online and saying, oh, that's that's what I need to do too.
• 19:55
  SPEAKER_01: And they start labeling themselves, right?
• 19:58
  SPEAKER_01: So interestingly.
• 20:00
  SPEAKER_01: This is what the LLM generated response was.
• 20:02
  SPEAKER_01: And it does use first person pronouns, right?
• 20:04
  SPEAKER_01: Because the question was, where do you see yourself in five years?
• 20:07
  SPEAKER_01: And so the LLM says, I just wanted to say that it's great you're thinking about the future.
• 20:13
  SPEAKER_01: It's a crucial step towards recovery.
• 20:15
  SPEAKER_01: I hope that in five years you see yourself in a healthier, happier place, free from the chains of addiction.
• 20:21
  SPEAKER_00: Interesting.
• 20:23
  SPEAKER_01: Very interesting.
• 20:24
  SPEAKER_00: Doesn't answer the question, I guess.
• 20:25
  SPEAKER_00: You know.
• 20:26
  SPEAKER_01: No, it doesn't really answer the question.
• 20:28
  SPEAKER_00: It's just changing the temperature of the conversation.
• 20:31
  SPEAKER_01: Mm-hmm.
• 20:31
  SPEAKER_01: And it also, it's like, how does how like back to your point?
• 20:35
  SPEAKER_01: How does a bot say, where do you see yourself in five years?
• 20:38
  SPEAKER_01: It can't answer.
• 20:39
  SPEAKER_01: It like literally cannot answer the question.
• 20:41
  SPEAKER_01: And so it doesn't.
• 20:42
  SPEAKER_01: You would lose trust in it if you realized it was making something up.
• 20:46
  SPEAKER_01: Right.
• 20:46
  SPEAKER_01: Right.
• 20:46
  SPEAKER_01: It is making something up because that's what they do.
• 20:48
  SPEAKER_01: But like if if it said, Oh, I see myself in this in this place in five years, it does it's nonsensical.
• 20:55
  SPEAKER_01: Whereas at least this is sensible.
• 20:57
  SPEAKER_01: It's like congratulating them for thinking about the future, you know, um, as a a means of motivation and like wishing them luck.
• 21:06
  SPEAKER_01: So it's not, you know, it's not, you're right.
• 21:08
  SPEAKER_01: It's not answering the question, but I think it would actually be worse if it did.
• 21:12
  SPEAKER_00: Yeah, yeah.
• 21:12
  SPEAKER_00: And and the problem is, of course, that once you have a Reddit user that's an AI system like that, and it's answered one question, how is it going to handle follow-ups to that?
• 21:23
  SPEAKER_00: If people ask it further questions, how is it going to respond correctly?
• 21:26
  SPEAKER_00: Is it going to be consistent across its answers?
• 21:28
  SPEAKER_00: These are all difficult things.
• 21:30
  SPEAKER_00: And a lot of people in the AI research space design personas for these things, right?
• 21:34
  SPEAKER_00: It's not just enough like in the prompt to say that you're a Reddit user, but you have to set it up so that it has a proper character background so that it actually is able to answer follow-up questions consistently.
• 21:45
  SPEAKER_01: If it were to continue to engage.
• 21:47
  SPEAKER_01: Yes, yes, yeah.
• 21:48
  SPEAKER_01: And this was just like a you know, parts of this paper they did do some longitudinal stuff.
• 21:52
  SPEAKER_01: But in this example, it was just like a single, a single response, and then they would say, Hey, like, what do you what's your attitude after we knew what your attitude was before reading this, what's your attitude after?
• 22:02
  SPEAKER_01: Yeah.
• 22:02
  SPEAKER_01: And with the LLMs, the attitudes, uh, you know, this the stigma decreased.
• 22:07
  SPEAKER_01: But it that didn't always happen with the human ones, which is interesting.
• 22:13
  SPEAKER_01: We saw some of the human ones though, right?
• 22:15
  SPEAKER_01: Like there was labeling, um, you know, there was there was stigma in some of them.
• 22:21
  SPEAKER_01: And so in some of those instances, people, especially people who who didn't have much stigma to begin with, they might read the human written response and it it kind of backfired.
• 22:31
  SPEAKER_01: Like their stigma level went up.
• 22:32
  SPEAKER_01: Oh, interesting.
• 22:33
  SPEAKER_01: So it's like, well, maybe it would have been better if if we did nothing instead of that.
• 22:39
  SPEAKER_01: And especially for that subset of people who started off without a lot of stigma.
• 22:44
  SPEAKER_01: Like, why, why intervene on them, right?
• 22:47
  SPEAKER_01: Save your interventions for someone who has has a much higher level of stigma.
• 22:52
  SPEAKER_00: But of course, you don't know who's reading those Reddits.
• 22:54
  SPEAKER_00: I know.
• 22:54
  SPEAKER_00: That's the challenge, right?
• 22:55
  SPEAKER_00: You know what I'm saying?
• 22:56
  SPEAKER_01: Custom Taylor thing that you really it's it's open to anyone to read.
• 22:59
  SPEAKER_00: Yeah, I mean, maybe the goal of this would be to up the uh change the attitudes in the community in general.
• 23:04
  SPEAKER_00: And for those people whose stigma levels were low to begin with, at least there's now increased awareness.
• 23:10
  SPEAKER_00: And although it might have temporarily gone down, uh gone up, it's possible to bring it back down.
• 23:14
  SPEAKER_00: And so you might that might be the the the approach here.
• 23:17
  SPEAKER_00: But there is value in having these sorts of discussions of generally improving pro-sociality in the community, but also um more specifically, um, reducing stigma with respect to opioids here.
• 23:30
  SPEAKER_01: Yeah, I mean, I think it would help people with opioid use disorder so much if they weren't constantly battling with stigma on top of the condition that they have, right?
• 23:42
  SPEAKER_01: If they if there's less stigma, that means like it's easier to get help if you don't feel like you're being judged all the time, if you don't feel like you're being blamed, if you don't feel like you're being dismissed.
• 23:53
  SPEAKER_01: Um, stigma obviously affects like the quality of care someone gets too.
• 23:58
  SPEAKER_01: Like what, you know, the what the healthcare system or what the general public thinks about them.
• 24:04
  SPEAKER_01: It can um reducing stigma makes people less isolated.
• 24:07
  SPEAKER_01: Yeah.
• 24:08
  SPEAKER_01: I think that's why people turn to these communities, right?
• 24:11
  SPEAKER_01: Because they they feel so isolated.
• 24:13
  SPEAKER_01: Right.
• 24:13
  SPEAKER_01: Um, and there's a lot of social withdrawal there.
• 24:16
  SPEAKER_01: And of course, we talk about policy all the time, but when people see opioid use disorder as a health condition, that means that it has policies and resources allocated to it.
• 24:29
  SPEAKER_01: Right.
• 24:30
  SPEAKER_01: And so the whole point here is that if LLMs can help reduce stigma, they can help remove barriers, they can improve care, they can support recovery, and that can all save lives.
• 24:44
  SPEAKER_00: I think with that we can we can end here.
• 24:46
  SPEAKER_01: Yes, we will see you next time on Code and Cure.
• 24:48
  SPEAKER_01: Thank you for joining us.