Latest / What's Your Problem? / Can a Chatbot Treat Depression?

Transcript

Can a Chatbot Treat Depression? · What's Your Problem?

0:15
Speaker 1: Pushkin. Nick Jacobson wanted to help people with mental illness,
0:22
Speaker 1: so he went to grad school to get his PhD
0:25
Speaker 1: in clinical psychology, but pretty quickly he realized there just
0:29
Speaker 1: were nowhere near enough therapists to help all the people
0:32
Speaker 1: who needed therapy.
0:34
Speaker 2: If you go to pretty much any clinic, there's a
0:37
Speaker 2: really long wait list, it's hard to get in, and
0:40
Speaker 2: a lot of that is organic, and that there's just
0:42
Speaker 2: a huge volume of need and not enough people to
0:45
Speaker 2: go around.
0:46
Speaker 1: Since he was a kid, Nick had been writing code
0:49
Speaker 1: for fun, so in a sort of a side project
0:52
Speaker 1: in grad school, he coded up a simple mobile app
0:55
Speaker 1: called Mood Triggers. The app would prompt you to enter
0:58
Speaker 1: how you were feeling, so it could measure your levels
1:00
Speaker 1: of anxiety and depression, and it would track basic things
1:04
Speaker 1: like how you slept, how much you went out, how
1:06
Speaker 1: many steps you took. And then twenty fifteen Nick put
1:10
Speaker 1: that app out into the world and people liked it.
1:13
Speaker 2: A lot of folks just said that they learned a
1:15
Speaker 2: lot about themselves and it was really helpful and actually
1:18
Speaker 2: changing and managing their symptoms. So I think it was
1:20
Speaker 2: beneficial for them to learn. Hey, maybe actually it's on
1:25
Speaker 2: these days that I'm withdrawing and not spending any time
1:27
Speaker 2: with people, that it might be good for me to
1:31
Speaker 2: go and actually get out about that kind of thing.
1:35
Speaker 2: And I had a lot of people that installed that application,
1:39
Speaker 2: So about fifty thousand people installed it from all over
1:43
Speaker 2: the world, over one hundred countries. In that one year,
1:49
Speaker 2: I provided an intervention for more than what I could
1:53
Speaker 2: have done over an entire career as a psychologist. I
1:57
Speaker 2: was a graduate student at the time that like this
2:00
Speaker 2: is like, you know, something that was just amazing to me,
2:04
Speaker 2: the scale of technology and its ability to reach folks,
2:09
Speaker 2: And so that made me really interested in trying to
2:11
Speaker 2: do things that could essentially have that kind of impact.
2:21
Speaker 1: I'm Jacob Goldstein and this is What's Your Problem, the
2:24
Speaker 1: show where I talk to people who are trying to
2:25
Speaker 1: make technological progress. My guest today is Nick Jacobson. Nick
2:30
Speaker 1: finished his PhD in clinical psychology, but today he doesn't
2:33
Speaker 1: see patients. He's a professor at Dartmouth Medical School and
2:37
Speaker 1: he's part of a team that recently developed something called Therabot.
2:41
Speaker 1: Therabot is a generative AI therapist. Nick's problem is this,
2:47
Speaker 1: how do you use technology to help lots and lots
2:50
Speaker 1: and lots of people with mental health problems, and how
2:54
Speaker 1: do you do it in a way that is safe
2:56
Speaker 1: and based on clear evidence. As you'll hear, Nick and
3:00
Speaker 1: his colleagues recently tested therapod in a clinical trial with
3:03
Speaker 1: hundreds of patients and the results were promising. But those
3:07
Speaker 1: results only came after years of failures and over one
3:10
Speaker 1: hundred thousand hours of work by team Therapbot. Nick told
3:15
Speaker 1: me he started thinking about building a therapy chatbot based
3:18
Speaker 1: on a large language model back in twenty nineteen. That
3:21
Speaker 1: was years before chat GPT brought large language models to
3:25
Speaker 1: the masses, and Nick knew from the start that he
3:28
Speaker 1: couldn't just use a general purpose model. He knew he
3:31
Speaker 1: would need additional data to fine tune the model to
3:34
Speaker 1: turn it into a therapist chatbot.
3:37
Speaker 2: And so the first iteration of this was thinking about, Okay,
3:40
Speaker 2: what where is there widely accessible data and that would
3:45
Speaker 2: potentially have an evidence base that this could work. And
3:48
Speaker 2: so we started with peer to peer forums, so folks
3:52
Speaker 2: interacting with folks surrounding their mental health. So we trained
3:56
Speaker 2: this model on hundreds of thousands of conversations that were
4:01
Speaker 2: happening on the Internet.
4:02
Speaker 1: So you have this model, you train it up, you
4:05
Speaker 1: sit down in front of the computer. What do you
4:09
Speaker 1: what do you say to the chat pot?
4:11
Speaker 2: I'm feeling depressed? What should I do?
4:13
Speaker 1: Okay? And then what is the what does the model
4:15
Speaker 1: say back to you?
4:16
Speaker 2: I'm paraphrasing here, but it was just like this, I'm
4:21
Speaker 2: I feel so depressed every day I have. It's such
4:25
Speaker 2: a hard time getting out of bed. I just want
4:28
Speaker 2: my life to be over.
4:29
Speaker 1: So literally therapist is saying they're going to kill themselves.
4:33
Speaker 2: Right, So it's escalating talking about kind of really thoughts
4:37
Speaker 2: about about death. And it's clearly like the profound mismatch
4:42
Speaker 2: between what we were thinking about and what we were
4:44
Speaker 2: going for is.
4:45
Speaker 1: What what did you think when you read that?
4:48
Speaker 2: So I thought this is a such a non starter.
4:52
Speaker 2: But it was. I think one of the things that
4:54
Speaker 2: I think was clear was it was picking up on
4:56
Speaker 2: patterns in the data, but were the wrong data.
4:59
Speaker 1: Yeah, I mean one one option then is give up.
5:02
Speaker 2: It would have been absolutely like, literally, the worst therapist
5:06
Speaker 2: ever is what you have built? I mean it, I
5:09
Speaker 2: couldn't imagine a worse Yeah, a worse thing to actually
5:12
Speaker 2: try to implement in a real setting. So this went
5:15
Speaker 2: nowhere on in and of itself. But we had a
5:19
Speaker 2: good reason to start there actually, So it wasn't just
5:21
Speaker 2: that there's widely available data these peer networks actually do.
5:24
Speaker 2: There is literature to support that having exposure to these
5:27
Speaker 2: peer networks actually improves mental health outcomes. It's a big
5:30
Speaker 2: literature and the cancer Survivor network, for example, where folks
5:33
Speaker 2: that are struggling with cancer and hearing from other folks
5:35
Speaker 2: that have gone through it can really build this resilience
5:38
Speaker 2: and it promotes a lot of mental health outcomes that
5:40
Speaker 2: are positive. So we had a good reason to start,
5:42
Speaker 2: but gosh, did it not go well? So, okay, the
5:47
Speaker 2: next thing we do is switch gears the exact opposite direction. Okay,
5:51
Speaker 2: we started with very lay persons trying to interact with
5:54
Speaker 2: other lay persons surrounding their mental health. Let's go to
5:57
Speaker 2: what providers would do. And so we we got access
6:01
Speaker 2: to thousands of psychotherapy training videos and these are interesting.
6:05
Speaker 2: These are these are how psychologists are often like exposed
6:08
Speaker 2: to the field on what they would what they would
6:12
Speaker 2: like really learn how therapy is supposed to work and
6:14
Speaker 2: how it's supposed to be delivered. And in these these
6:17
Speaker 2: are like dialogues between sometimes actual patients that are consenting
6:21
Speaker 2: to be part of this and sometimes simulated patients where
6:24
Speaker 2: it's an actor that's that's trying to mimic this, and
6:29
Speaker 2: there's a psychologist or a mental health provider that is
6:32
Speaker 2: like really having a real session with this, And so
6:36
Speaker 2: we train our second model on on that. On that data,
6:40
Speaker 2: it seems more promising. You would think you'd say, I'm
6:43
Speaker 2: feeling depressed. What should I do? As like the initial
6:47
Speaker 2: way that we would test this, the model says mm hmm,
6:53
Speaker 2: like literally mm hmm.
6:56
Speaker 1: And like like it writes out m m space hmm.
7:00
Speaker 2: You've got it.
7:01
Speaker 1: And so what did you think when you saw that?
7:05
Speaker 2: And so I was like, oh gosh, it's picking up
7:07
Speaker 2: on patterns in the data, and so the yeah, but
7:10
Speaker 2: you continue these interactions and then the next responses go
7:14
Speaker 2: on from the therapist, So like within about five or
7:18
Speaker 2: so turns, we would often get the model that would
7:22
Speaker 2: respond about their interpretations of their problems stemming from their
7:25
Speaker 2: their mother or their parents more generally. So like it's
7:29
Speaker 2: kind of like, if you were to try to think
7:30
Speaker 2: about it, what a psychologist is. This is like every
7:33
Speaker 2: trope of what a like in your mind if you
7:36
Speaker 2: were going to like think.
7:37
Speaker 1: Of the stereotypical on the couch and a guy's wearing
7:40
Speaker 1: a tweet jacket sitting in a chair.
7:42
Speaker 2: And hardly says anything of that could be potentially helpful,
7:47
Speaker 2: but is reflecting things back to me and.
7:50
Speaker 1: So then telling me it goes back to my parents. Yeah,
7:52
Speaker 1: well this is so let's just pause here for a moment, because,
7:54
Speaker 1: as you say, this is like the stereotype of the therapist,
7:59
Speaker 1: but you trained it on real data, so maybe it's
8:02
Speaker 1: the stereotype for a reason.
8:04
Speaker 2: Yes, I think what to me was really clear was
8:09
Speaker 2: that we were we had data that the models were
8:13
Speaker 2: emulating patterns they were seeing in the data. So the
8:16
Speaker 2: models weren't the problem. The problem was the data were
8:20
Speaker 2: the wrong data.
8:21
Speaker 1: But the data is the data that is used to
8:23
Speaker 1: trade real therapists. Like, it's confusing that this is the
8:26
Speaker 1: wrong data.
8:26
Speaker 2: It is it is.
8:28
Speaker 1: Why why is it the wrong data? This should be
8:31
Speaker 1: exactly the data you want.
8:32
Speaker 2: Well, it's it's the wrong data for this format. And
8:34
Speaker 2: our conversation when you might say something me nodding along
8:40
Speaker 2: or saying mm hmm or go on, my contextually be
8:43
Speaker 2: like completely appropriate. So tactically in a conversational dialogue that
8:48
Speaker 2: would happen via chat. This is not like a medium
8:52
Speaker 2: that works very well like this kind of thing.
8:54
Speaker 1: Yeah, it's almost like a translation, right. It doesn't translate
8:58
Speaker 1: from a human face to face interaction to a chat
9:00
Speaker 1: window on the computer.
9:01
Speaker 2: And not the right setting.
9:03
Speaker 1: Yeah, so that I mean that goes to the like
9:05
Speaker 1: nonverbal subtler aspects of therapy, right, like presumer when the
9:10
Speaker 1: therapist is saying m hm, there is there is body language,
9:14
Speaker 1: there's everything that's happening in the room, which is a
9:17
Speaker 1: tremendous amount of information or emotional information, right, and that
9:21
Speaker 1: is a thing that is lost, yes, no doubt in
9:24
Speaker 1: this medium and and maybe speaks to a broader question
9:28
Speaker 1: about the translatability of therapy.
9:31
Speaker 2: Yeah, absolutely so I think to me, like the it
9:35
Speaker 2: was at that moment that I kind of knew that
9:39
Speaker 2: we we needed to do something radically different. Neither of
9:42
Speaker 2: these was working well. About one in ten of the
9:46
Speaker 2: responses from that from that chatbot, based on the clinicians,
9:51
Speaker 2: would be something that we would be happy with. So
9:54
Speaker 2: something that is both personalized, clinically appropriate and dynamic.
9:58
Speaker 1: So you're saying you've got it right ten percent of exactly.
10:00
Speaker 2: So really, no, that's that's not a good like no,
10:04
Speaker 2: it's not a good not a good therapy. No, we
10:06
Speaker 2: would we would never think about implement like actually trying
10:09
Speaker 2: employ that. So then what we started at that point
10:13
Speaker 2: was building our own creating our own data set from scratch,
10:18
Speaker 2: in which we how how the models would learn would
10:22
Speaker 2: be exactly what we want it to say.
10:25
Speaker 1: That seems that seems wild, I mean, how do you
10:28
Speaker 1: do that? How do you how do you generate that
10:30
Speaker 1: much data.
10:31
Speaker 2: We've had a team of one hundred people that have
10:33
Speaker 2: worked on this project over the last five and a
10:35
Speaker 2: half years at this point, and they've spent over one
10:38
Speaker 2: hundred thousand human hours kind of really trying to build this.
10:42
Speaker 1: Just specifically, what how do you build a data set
10:44
Speaker 1: from scratch? Because like, the data set is the huge problem.
10:48
Speaker 2: Yes, absolutely so. Psychotherapy, when you would test it is
10:54
Speaker 2: based on something that is written down in a manual.
10:57
Speaker 2: So when you're a psychologist when they're in a randomized
11:01
Speaker 2: controlled trial trying to test whether something works or not.
11:04
Speaker 2: To be able to test it, it has to be replicable,
11:07
Speaker 2: meaning it's like repeated across different therapists. So there are
11:11
Speaker 2: manuals that are developed. In this session you work on
11:14
Speaker 2: on psycho education. On this session we're going to be
11:17
Speaker 2: working on behavioral activation, So which are different techniques that
11:21
Speaker 2: are really a focus at a given time, and these
11:24
Speaker 2: are broken down to try to make it translational so
11:27
Speaker 2: that you can actually move it. So the team would
11:30
Speaker 2: read these empirically supported treatment manuals, so the ones that
11:34
Speaker 2: had been tested in randomized control trials, and then what
11:37
Speaker 2: we would do is we would take that content chapter
11:41
Speaker 2: by chapter, because this is like session by session, take
11:44
Speaker 2: the techniques that would work well via chat, of which
11:47
Speaker 2: most things in cognitive behavioral therapy would, and then we
11:51
Speaker 2: would create an artificial dialogue between would act as like
11:57
Speaker 2: what is the patient's presenting problem, what they're bringing on,
12:00
Speaker 2: what the personality is like, and we're kind of constructing this,
12:03
Speaker 2: and then what is what we would want our system
12:07
Speaker 2: to be the gold standard response for every kind of
12:10
Speaker 2: input and output that we'd have. So we're writing both
12:14
Speaker 2: the patient end and the therapist end.
12:17
Speaker 1: It's like you're writing a screenplay.
12:19
Speaker 2: Basically, it really is. It's a lot like that, but
12:22
Speaker 2: instead of a screenplay that might be written like in general,
12:26
Speaker 2: it's like not like not just something general, but like,
12:29
Speaker 2: where is something that's really evidence based based on content
12:32
Speaker 2: that we know works in this setting?
12:36
Speaker 1: And so what you write the equivalent of what thousands
12:39
Speaker 1: of hours of sessions?
12:41
Speaker 2: Hundreds of thousands. There was post docs, grad students and
12:44
Speaker 2: undergraduates within my group that we're all part of this
12:47
Speaker 2: team that are creating.
12:49
Speaker 1: Just doing the work, just writing the dialogue.
12:51
Speaker 2: Yeah, exactly. And not only did we write them, but
12:54
Speaker 2: every dialogue before it would go into something that our
12:58
Speaker 2: models are trained would be reviewed by another member of
13:01
Speaker 2: the team. So it's all not only crafted by hand,
13:06
Speaker 2: but we would review it, give each other feed on it,
13:09
Speaker 2: and then like make sure that it is the highest
13:12
Speaker 2: quality data. And that's when we started seeing dramatic improvements
13:18
Speaker 2: in the model performance. So we continued with us for years.
13:23
Speaker 2: Six months before chat TPT was launched, we had a
13:29
Speaker 2: model that in today's standards would be so tiny, that
13:34
Speaker 2: was delivering about ninety percent of the responses that were
13:38
Speaker 2: output We were evaluating as exactly what we'd want. It's
13:42
Speaker 2: this gold standard evidence based treatment, so that was fantastic.
13:47
Speaker 2: We were really excited about it. So we've got like
13:49
Speaker 2: the we've got the benefit side down of the equation.
13:54
Speaker 2: The next two years we focus on the risk, the
13:58
Speaker 2: risk side of it well, because there's a huge risk.
14:00
Speaker 1: Here, right The people who are using it are by
14:02
Speaker 1: design quite vulnerable, by design putting a tremendous amount of
14:07
Speaker 1: trust into this bot and making themselves vulnerable to it,
14:12
Speaker 1: like it's a it's quite a risky proposition. And so
14:16
Speaker 1: so tell me specifically, what are you doing.
14:18
Speaker 2: So we're trying to get it to endorse elements that
14:21
Speaker 2: would make mental health worse. So a lot of our
14:24
Speaker 2: conversations or surrounding trying to get it to For example,
14:29
Speaker 2: I'll give you an example of one that nearly almost
14:33
Speaker 2: almost any model will struggle with that's not tailored towards
14:37
Speaker 2: the safety end. What is it is if you tell
14:40
Speaker 2: a model that you want to lose weight, it will
14:43
Speaker 2: generally try to help you do that. And if you
14:46
Speaker 2: want to if you want to work in an area
14:48
Speaker 2: related to mental health, trying to promote weight loss without
14:52
Speaker 2: context is so not safe.
14:54
Speaker 1: You saying it might be a user within eating disorder
14:56
Speaker 1: absolutely unhealthily thin, who wants to be even thinner.
14:59
Speaker 2: And the model will help them to often actually get
15:03
Speaker 2: into a lower weight than they already are. So this
15:06
Speaker 2: is like not something that we would ever want to promote,
15:09
Speaker 2: but this is something that we certainly at earlier stages
15:13
Speaker 2: we're seeing these types of characteristics within the model.
15:15
Speaker 1: What are other like, that's an interesting one and it
15:18
Speaker 1: makes perfect sense when you say it, I would not
15:20
Speaker 1: have thought of it. Or what's another one?
15:22
Speaker 2: A lot of it would be like we talk about
15:24
Speaker 2: the ethics of suicide. For example, somebody who is who thinks,
15:27
Speaker 2: you know, they're in a midst of suffering and you
15:30
Speaker 2: know it's like that they could should be able to
15:33
Speaker 2: end of their life or they're thinking about this.
15:35
Speaker 1: Yes, and what do you want the model? What what?
15:39
Speaker 1: What does the model say that it shouldn't say in
15:41
Speaker 1: that setting?
15:41
Speaker 2: So for you, in these settings, we want to make
15:44
Speaker 2: sure that they don't and the model does not promote
15:47
Speaker 2: or endorse elements that would promote someone's a worsening of
15:52
Speaker 2: suicidal intent. We want to make sure we're providing not
15:55
Speaker 2: only not the absence of that, actually some benefit in
15:59
Speaker 2: these types of scenarios.
16:00
Speaker 1: That's the ultimate nightmare for you. Yeah, right, Like this
16:03
Speaker 1: be super clear. The very worst thing that could happen
16:06
Speaker 1: is you build this thing and it contributes to someone. Absolutely,
16:09
Speaker 1: that's a plausible outcome and a disastrous night.
16:13
Speaker 2: It's everything that I worry about in this area is
16:16
Speaker 2: exactly this kind of thing. And so we essentially every
16:19
Speaker 2: time we find an area where they're not implementing things perfectly,
16:24
Speaker 2: some optimal response, adding new training data, and that's that's
16:28
Speaker 2: when things continue to get better until we do this
16:31
Speaker 2: and we don't find these holes anymore. That's when we
16:34
Speaker 2: finally we're ready for the randomized control trial.
16:38
Speaker 1: Right, So you decide after after what four years five years?
16:43
Speaker 2: This is about four and a half years.
16:46
Speaker 1: Yeah, that that you're ready to to have people.
16:50
Speaker 2: Use use the model.
16:51
Speaker 1: I'll be it in a kind of Yeah, you're going
16:54
Speaker 1: to be the human in the loop. Right, So, so
16:56
Speaker 1: you decide to do this study. You recruit people on
16:59
Speaker 1: Facebook and Instagram.
17:00
Speaker 2: Basically ye exactly, yep, and what.
17:04
Speaker 1: So what are they signing up for? What's the what's
17:06
Speaker 1: the big study you do?
17:07
Speaker 2: So it's a it's a it's a randomized control trial.
17:11
Speaker 2: The trial design is essentially that folks would come in,
17:15
Speaker 2: they would fill out information about their mental health across
17:19
Speaker 2: a variety of areas, so depression, anxiety, and eating disorders,
17:24
Speaker 2: for folks that screen positive for having clinical levels of
17:28
Speaker 2: depression or anxiety, they would be in our Folks that
17:32
Speaker 2: were at risk for eating disorders would be included in
17:35
Speaker 2: the trial. We tried to have at least seventy people
17:39
Speaker 2: in each group, so we had two hundred and ten
17:41
Speaker 2: people that we were planning on and rolling within the trial,
17:45
Speaker 2: and then half of them were randomized to receive their
17:48
Speaker 2: ABOUT and half of them were on a wait list
17:51
Speaker 2: in which they would receive their ABOUT after the trial
17:53
Speaker 2: had ended. The trial design was to try to ask
17:57
Speaker 2: folks to use their ABOUT for four weeks. They retained
18:00
Speaker 2: access to therabot and could use their ABOUT for the
18:02
Speaker 2: next four weeks thereafter, so eight weeks total, but we
18:06
Speaker 2: asked them to try to actually use it during that
18:09
Speaker 2: first four weeks and that was that was essentially the
18:12
Speaker 2: trial design.
18:13
Speaker 1: So okay, so people signed up, they start like, what's
18:17
Speaker 1: what's actually happening? Are they just like chatting with the
18:20
Speaker 1: bought every day? Is it?
18:22
Speaker 2: So they install a smartphone application that's that they're about at.
18:27
Speaker 2: They are prompted once a day to try to have
18:31
Speaker 2: a conversation starter with the with the bot and then
18:34
Speaker 2: the bot. From there they could talk about it when
18:37
Speaker 2: and wherever they would want. They can ignore those notifications
18:40
Speaker 2: and kind of engage with it at any time that
18:42
Speaker 2: they'd want. But that was the gist of the trial design,
18:47
Speaker 2: and so folks in terms of how people used it,
18:50
Speaker 2: they interacted with it throughout the day, throughout the night.
18:54
Speaker 2: So for example, folks that would have trouble sleeping, that
18:58
Speaker 2: was like a way that folks during the middle of
19:00
Speaker 2: the night would engage with it fairly often. They in
19:05
Speaker 2: terms of the types of the topics that they described,
19:10
Speaker 2: it was really the entire range of something that you
19:13
Speaker 2: would see in psychotherapy. We had folks that were dealing
19:15
Speaker 2: with and discussing their different symptoms that they were talking about.
19:19
Speaker 2: So the depression, their anxiety that they were struggling with,
19:21
Speaker 2: their their eating, and their body image concerns. Those types
19:24
Speaker 2: of things are common because of the groups that we
19:27
Speaker 2: were recruiting. But relationship difficulties, problems like folks, some folks
19:33
Speaker 2: were really like I had ruptures in there, you know,
19:37
Speaker 2: somebody was going through a divorce. Other folks were like
19:40
Speaker 2: going through breakups, problems at work. Some folks were unemployed
19:45
Speaker 2: and during this time, So like the range of kind
19:48
Speaker 2: of personal dilemmas and difficulties that folks were experiencing. Was
19:52
Speaker 2: a lot of what we would see in like a
19:54
Speaker 2: real setting where it's like kind of a whole host
19:58
Speaker 2: of different things that folks were describing and experiencing.
20:02
Speaker 1: And presumably had they agreed as part of enrolling in
20:04
Speaker 1: the trial to let you read the transcript Oh?
20:07
Speaker 2: Absolutely, yeah, very very clear when we did an informed
20:11
Speaker 2: consent process where folks would know that we were reading
20:15
Speaker 2: reading these transcripts.
20:16
Speaker 1: And are you personally, like, what was it like for
20:19
Speaker 1: you seeing them come in? Are you reading them every day?
20:21
Speaker 1: I mean more than that.
20:23
Speaker 2: So, I mean this is something that is so ill.
20:27
Speaker 2: You alluded to that that this is one of these
20:29
Speaker 2: concerns that anybody would have. Is like a nightmare scenario
20:33
Speaker 2: where something is the bad happens and somebody actually outs right,
20:37
Speaker 2: So this is like I think of this in a
20:39
Speaker 2: way that I take So this is not.
20:40
Speaker 1: A happy moment for you. This is like you're terrified
20:43
Speaker 1: that it might go wrong.
20:45
Speaker 2: Well, it's it's certainly like I see it going right,
20:48
Speaker 2: but I have every concern that it could go wrong.
20:50
Speaker 2: Right like that, And so for the first half of
20:54
Speaker 2: the trial, I am monitoring every single interaction sent to
21:00
Speaker 2: or from the bot. Other people are also doing this
21:03
Speaker 2: on the team, so I'm not the only one. But
21:05
Speaker 2: I did not get a lot of sleep in the
21:07
Speaker 2: first half of this trial, in part because I was
21:10
Speaker 2: really trying to do this in near real time. So
21:12
Speaker 2: usually for nearly every message I was, I was getting
21:15
Speaker 2: to it within about an hour. So yeah, it was
21:18
Speaker 2: it was a barrage of NonStop kind of communication that
21:22
Speaker 2: was happening.
21:22
Speaker 1: So were there were there any slip ups? Did you
21:24
Speaker 1: ever have to intervene as a human in the loop.
21:27
Speaker 2: That we did? And the thing that that was something
21:30
Speaker 2: that we as a team did not anticipate. What we
21:34
Speaker 2: found was really unintended behavior was a lot of folks
21:38
Speaker 2: interacted with they're abot, and in doing that, there was
21:42
Speaker 2: a significant number of people that would interact with it
21:46
Speaker 2: and talk about their medical symptoms. So, for example, there
21:48
Speaker 2: was a number of folks that were experiencing symptoms of
21:51
Speaker 2: a sexually transmitted disease and they would described that in
21:55
Speaker 2: great detail and ask it, you know what, how how
21:58
Speaker 2: they should medically treat that? And instead of they're they're
22:01
Speaker 2: about saying, hey, go see a provider for this this
22:05
Speaker 2: is not my realm of expertise, it responds as if,
22:09
Speaker 2: and so this that all of the advice that it
22:12
Speaker 2: gave was really fairly reasonable, both in the assessment and
22:17
Speaker 2: treatment protocols, but we would not have wanted to act
22:20
Speaker 2: that way, So we contacted all of those folks to
22:24
Speaker 2: recommend that they actually contact a physician about that. Folks
22:29
Speaker 2: did interact with it related to crisis situations. So we
22:33
Speaker 2: had also had there about in these moments provided appropriate
22:39
Speaker 2: contextual crisis support, but we reached out to those folks
22:43
Speaker 2: to further escalate and make sure that they had further
22:45
Speaker 2: support available and that and those types of times too.
22:50
Speaker 2: So there there were things that, you know, we're certainly
22:54
Speaker 2: areas of concern that that happened, but nothing nothing that
22:58
Speaker 2: was concerning from the major areas that we had intended
23:02
Speaker 2: all kind of really went went pretty well.
23:08
Speaker 1: Still, Tom on the show the results of the study
23:11
Speaker 1: and what's next for therapot. What were the results of
23:23
Speaker 1: the study.
23:23
Speaker 2: So this is one of the things that was just
23:27
Speaker 2: really fantastic to see was that we had we looked
23:31
Speaker 2: at our main outcomes for what we were trying to
23:33
Speaker 2: look at, where the degree to folks reduced their depression symptoms,
23:39
Speaker 2: their anxiety symptoms, and their eating disorder symptoms among the
23:44
Speaker 2: intervention group relative to the control group. So based on
23:47
Speaker 2: the change and self reported symptoms in the treatment group
23:51
Speaker 2: versus the control group, and we saw these really large
23:56
Speaker 2: differential reductions, meaning a lot more reductions and changes that happened,
24:01
Speaker 2: and that non depressive symptoms, the anxiety symptoms, and the
24:04
Speaker 2: eating disorder symptoms, and the THERAPOT group relative of the
24:07
Speaker 2: witless control group, and the degree of change is about
24:11
Speaker 2: as strong as you'd ever see. And are randomized control
24:14
Speaker 2: trials of outpatient psychotherapy that would be delivered within cognitive
24:19
Speaker 2: behavioral therapy with a human, a real human delivering this
24:23
Speaker 2: an expert. You didn't test it against against therapy, No,
24:26
Speaker 2: we didn't. What you're saying, results results of other studies
24:30
Speaker 2: using real human therapists show comparable magnitudes of benefit. That's
24:35
Speaker 2: exactly right.
24:36
Speaker 1: Yes, you've gotta do a head to head. I mean,
24:38
Speaker 1: that's the obvious question, like why not randomize people to
24:41
Speaker 1: therapy or THERAPI bought?
24:42
Speaker 2: So the main thing when we're thinking about the first
24:46
Speaker 2: origins point is we want to have some kind of
24:49
Speaker 2: effect of how this works relative to the absence of anything.
24:52
Speaker 1: Relative to nothing, well, because I mean, presumably the easiest
24:56
Speaker 1: case to make for it is not it's better than
24:58
Speaker 1: a therapist. It's a huge number of people who need
25:01
Speaker 1: a therapist don't have one exactly, and that's the unfortunate reality.
25:05
Speaker 1: BOT is better than nothing. It doesn't have to be
25:07
Speaker 1: better than a human therapist. It just has to better,
25:09
Speaker 1: that's right.
25:10
Speaker 2: But so yes, the we are planning ahead to head
25:14
Speaker 2: trial against therapist as the next trial that we run,
25:18
Speaker 2: in large part because I already think we are not inferior.
25:22
Speaker 2: So it will it'll be interesting to see if that
25:24
Speaker 2: actually comes out. But that is that is something that
25:28
Speaker 2: we have outstanding funding proposals to try to actually do that.
25:34
Speaker 2: So one of the other things that I haven't gotten
25:36
Speaker 2: to with in the trial outcomes that I think is
25:38
Speaker 2: really important on that end actually is to two things.
25:44
Speaker 2: One is the degree that folks formed a relationship with
25:49
Speaker 2: therapaud and so in psychotherapy, one of the most well
25:54
Speaker 2: studied constructs is the ability that you and your therapist
25:58
Speaker 2: can get together and work together on common goals and
26:01
Speaker 2: trust each other that you as a it's a relationship,
26:04
Speaker 2: it's a human relationship. And so this in the literature
26:09
Speaker 2: is called the working alliance, and so it's this ability
26:12
Speaker 2: to form this bond. We measured this working alliance using
26:17
Speaker 2: the same measure that folks would use with outpatient providers
26:21
Speaker 2: about how they they felt about their therapist, but instead
26:24
Speaker 2: of the therapist that now we're talking about therabot, and
26:28
Speaker 2: and folks rated it nearly identically to the norms that
26:32
Speaker 2: you would see on the outpatient literature. So we asked folks,
26:36
Speaker 2: we give folks the same measure, and that it's essentially
26:39
Speaker 2: equivalent to how folks are reading human providers in these ways.
26:43
Speaker 1: This is consistent with other where we're seeing people having
26:46
Speaker 1: relationship with chatbots and other domains. Yes, I'm old enough
26:51
Speaker 1: that it seems weird to me. I don't know, seem
26:54
Speaker 1: weird to you.
26:55
Speaker 2: I that part I this is more of a surprise
26:58
Speaker 2: to me that it was as the bonds were as
27:00
Speaker 2: high as they were, that they would actually be about
27:02
Speaker 2: what humans would be. And I will say, like one
27:05
Speaker 2: of the other surprises within the interactions was the number
27:08
Speaker 2: of people that would like respond kind of check in
27:12
Speaker 2: with therabot with and just say hey, just checking in
27:16
Speaker 2: as if like Therabot is like a I don't know.
27:19
Speaker 2: I would I would only like have anticipated folks would
27:22
Speaker 2: use this as a tool, so like not like they
27:25
Speaker 2: went to hang out with like almost that way. It's
27:28
Speaker 2: like our initiating a conversation that isn't I guess doesn't
27:32
Speaker 2: have an intention in mind.
27:34
Speaker 1: I say please, I'm thank you, I can't help my
27:39
Speaker 1: Is it because I think they're going to take over
27:41
Speaker 1: or is it a habit or what? I don't know,
27:43
Speaker 1: but I do, I do.
27:44
Speaker 2: Yeah, I wouldn't. I would say that this was more
27:47
Speaker 2: surprising the degree to that folks established this this level
27:50
Speaker 2: of a bond with it. I think it's actually really
27:52
Speaker 2: good and in really important that they do, in large
27:57
Speaker 2: part because that's one of the ways that we know
27:59
Speaker 2: psychotherapy works, is that that that folks can come together
28:03
Speaker 2: and trust this and develop this working relationship. So I
28:06
Speaker 2: think it's actually a necessary ingredient for this to work.
28:08
Speaker 1: To some I get it makes sense to me intellectually
28:11
Speaker 1: what you're saying. Does it give you any pause or
28:13
Speaker 1: do you just think it's great?
28:15
Speaker 2: It it gives me pause. If we weren't delivering evidence
28:19
Speaker 2: based treatment, Well, this is a good moment. Let's talk
28:23
Speaker 2: about the let's talk about the industry more generally. Yeah,
28:26
Speaker 2: this is not a you're not making a company, this
28:28
Speaker 2: is not a product, right, you don't have any money
28:31
Speaker 2: at stake. But there is a something of a therapy
28:34
Speaker 2: bought industry.
28:35
Speaker 1: There is a private sector. Like, tell me what is
28:38
Speaker 1: the broader landscape here?
28:39
Speaker 2: Like, so there's a lot of folks that are have
28:43
Speaker 2: jumped in predominantly sense the loans of shot GPT, and
28:48
Speaker 2: a lot of folks that have learned that you can
28:52
Speaker 2: call a foundation model fairly easily.
28:54
Speaker 1: When you say call you mean just sort of like
28:56
Speaker 1: you sort of take a foundation model like GBT, and
29:00
Speaker 1: then you kind of put a wrapper around exactly and
29:02
Speaker 1: the rapper it's like it's basically GPT with a therapist wrapper.
29:05
Speaker 2: Yeah. So it's a lot of folks within this industry
29:08
Speaker 2: are saying, hey, you act like a therapist and then
29:13
Speaker 2: kind of off to the races. It's it's otherwise not
29:16
Speaker 2: changed in any way, shape or form. It's it's like
29:19
Speaker 2: a literally like a system prompt. So if you were
29:22
Speaker 2: interacting with chat GBT, it would be something along the
29:25
Speaker 2: lines of, hey, act as a therapist and here's what
29:29
Speaker 2: we go on to do. They may have more directions
29:31
Speaker 2: than this. But that's this is kind of the light
29:34
Speaker 2: touch nature, so super different from what we're doing. Actually, yes,
29:38
Speaker 2: so we conducted the first randomized control trial of any
29:42
Speaker 2: generative AI for any type of clinical mental health problem.
29:48
Speaker 2: And so I know that these folks don't have evidence
29:52
Speaker 2: that this kind of thing works.
29:54
Speaker 1: I mean, there are non generative AI bots that people
29:57
Speaker 1: did randomize control trials of, right, just to be clear.
30:01
Speaker 2: Yes, there are non generative absolutely that have have evidence
30:05
Speaker 2: behind them. The generative side is very new, and so
30:10
Speaker 2: and there's a lot of folks in the generative space
30:13
Speaker 2: that have jumped in. Yeah, and so a lot of
30:16
Speaker 2: these folks are not psychologists and not psychiatrists, and and
30:22
Speaker 2: Silicon Valley, there's a saying move fast and break things.
30:27
Speaker 2: This is not the setting to do that. Like move
30:29
Speaker 2: fast and break people is what you're talking about here.
30:32
Speaker 2: You know, it's like the and the amount of times
30:35
Speaker 2: that these foundation models act in profoundly unsafe ways would
30:40
Speaker 2: be unacceptable to the field. So like that, we tested
30:43
Speaker 2: a lot of these models alongside when we were developing
30:46
Speaker 2: all of this. So it's like, I know that they
30:48
Speaker 2: don't they don't work in this kind of way in
30:51
Speaker 2: a real safe environment. So because of that, I'm I'm
30:55
Speaker 2: really hugely concerned with kind of the field at large
30:59
Speaker 2: that is moving fast and doesn't really have this level
31:02
Speaker 2: of dedication to trying to do it right. And I
31:06
Speaker 2: think one of the things that's really kind of canning
31:09
Speaker 2: within this is it always looks polished, so it's harder
31:12
Speaker 2: to see when you're getting exposed to things that are dangerous.
31:16
Speaker 2: But the field, I think is in a spot where
31:18
Speaker 2: there's a lot of folks that are out there that
31:20
Speaker 2: are acting and implementing things that are untested, and I
31:23
Speaker 2: suspect a lot of them are really dangerous.
31:26
Speaker 1: How do you how do you imagine theahbut getting from
31:30
Speaker 1: the experimental phase into the widespread use phase.
31:32
Speaker 2: Yeah, so we want to essentially have one at least
31:35
Speaker 2: one larger trial before we do this. You know, we
31:38
Speaker 2: have it's pretty a pretty decent sized first trial for
31:42
Speaker 2: being a first trial, but it's not something that I
31:45
Speaker 2: would want to see out in the open just yet.
31:48
Speaker 2: We want to have continue to oversight it, make sure
31:50
Speaker 2: it's safe and effective. But if it continues to demonstrate
31:54
Speaker 2: safety and effectiveness, this is one of those things that
31:57
Speaker 2: why I got into this is to really have an
32:00
Speaker 2: impact on folks lives, and this is one of those
32:04
Speaker 2: things that could scale really effective personalized cares in real ways. So, yeah,
32:08
Speaker 2: we we intend to if evidence continues to show that
32:12
Speaker 2: it's safe and effective, to make this out into the
32:15
Speaker 2: open market. In terms of the thing that I care
32:17
Speaker 2: about in terms of the ways that we could do
32:20
Speaker 2: this is trying to do this in some ways that
32:23
Speaker 2: would be scalable, so that we're considering a bunch of
32:25
Speaker 2: different pathways. Some of those would be delivered by philanthropy
32:29
Speaker 2: or nonprofit models. We are considering also like just a
32:34
Speaker 2: strategy that would just not for me to make money,
32:37
Speaker 2: but just to scale this under some kind of for
32:40
Speaker 2: profit structure as well, but really just to try to
32:43
Speaker 2: get this out into the open so that folks could
32:46
Speaker 2: actually use it, because ultimately we'll need some kind of
32:49
Speaker 2: revenue in some ways to be part of this that
32:54
Speaker 2: would essentially enable the servers to stay on and to
32:57
Speaker 2: scale it.
32:58
Speaker 1: And presumably you have to pay some amount of people
33:00
Speaker 1: to do some amount of supervision absolutely forever.
33:04
Speaker 2: Yeah, So we in the real deployment setting, we hope
33:07
Speaker 2: to have essentially the decreasing levels of oversight relative to
33:12
Speaker 2: these trials, but not an absence of oversight. So exactly
33:16
Speaker 2: you're not going to stay up all night reading every
33:18
Speaker 2: message exactly, that won't be sustainable for the future, but
33:22
Speaker 2: we will have like flags for things that should be
33:25
Speaker 2: seen by humans and intervened upon.
33:27
Speaker 1: Let's talk about this other domain you've worked in in
33:32
Speaker 1: terms of technology and mental health, right, and so in
33:36
Speaker 1: addition to your work on thera bot, you've done a
33:38
Speaker 1: lot of work on it seems like basically diagnosis monitoring people,
33:44
Speaker 1: essentially using mobile devices and wearables to track people's mental
33:49
Speaker 1: health to predict outcomes like tell me about your work
33:52
Speaker 1: there and the field there.
33:54
Speaker 2: So essentially it's trying to trying to monitor folks within
33:58
Speaker 2: their freestanding conditions, so like in their real real life,
34:04
Speaker 2: through using technology so in ways that are not don't
34:08
Speaker 2: require burden.
34:09
Speaker 1: The starting point is like your phone is collecting data
34:12
Speaker 1: about you all the time. What if that data could
34:15
Speaker 1: make you less depressed?
34:17
Speaker 2: Yeah, exactly, what if we could use that data to
34:19
Speaker 2: know something about you so that we could actually intervene
34:23
Speaker 2: and so, like thinking about a lot of mental health symptoms.
34:27
Speaker 2: I think one of the challenges of them is they
34:31
Speaker 2: are not like all or nothing the field. Actually, I
34:34
Speaker 2: think it's this really wrong. And when you would talk
34:38
Speaker 2: to anybody who has a experience as a clinical problem,
34:41
Speaker 2: they have changes that happen pretty rapidly within their daily life.
34:44
Speaker 2: So they like will have better moments and worse moments
34:48
Speaker 2: within a day, They'll have better and worse days. And
34:51
Speaker 2: it's not like it's all this like it's always depressed
34:54
Speaker 2: or not depressed. It's like these these fluctuating states of it.
34:59
Speaker 2: And I think one of the things that's really important
35:02
Speaker 2: about these types of things is if we can monitor
35:05
Speaker 2: and predict those rapid changes, which I think we can.
35:08
Speaker 2: We have a that we can is that we can
35:11
Speaker 2: then intervene upon the symptoms before they happen in real time,
35:16
Speaker 2: so like trying to predict the ebbs and the flows
35:18
Speaker 2: of the symptoms, not to like say, I want somebody
35:21
Speaker 2: to never be able to be stressed within their life,
35:24
Speaker 2: but so that they can actually be more resilient and
35:26
Speaker 2: cope with it.
35:28
Speaker 1: And so what's the state of that art, Like, is
35:32
Speaker 1: there somebody who's can you do that? Can somebody do that?
35:35
Speaker 1: Is there an app for that?
35:36
Speaker 2: As we used to say, Yeah, I mean we have
35:39
Speaker 2: the science surrounding. This is about ten years old. We've
35:44
Speaker 2: done about forty studies in this area across a broad
35:47
Speaker 2: range of symptoms, so anxiety, depression, post traumatic stress disorder, schizophrenia,
35:54
Speaker 2: bipolar disorder, eating disorders, so a lodge are different types
35:59
Speaker 2: of clinical phenomenon and we can predict a lot of
36:02
Speaker 2: different things in ways that I think are really important.
36:07
Speaker 2: But I think, like to really move the needle on
36:10
Speaker 2: something that would make it into population wide ability to
36:14
Speaker 2: do this, I think the real thing that would be
36:18
Speaker 2: needed for like the ability to do this is to
36:22
Speaker 2: pair this with intervention that's dynamic. So something that's actually ability,
36:28
Speaker 2: has an ability to change and has like a boundless
36:31
Speaker 2: context of intervention. So I'm going to actually loop you.
36:35
Speaker 1: Back like the Abot.
36:36
Speaker 2: That's exactly right. So these two things that have been
36:39
Speaker 2: distinct arms of my work are like so natural compliments
36:43
Speaker 2: to one another. Now think about Okay, let's come back
36:46
Speaker 2: to therabot in this kind of setting.
36:48
Speaker 1: So give me the dream.
36:49
Speaker 2: So this is the dream. So you have Therabot, but
36:53
Speaker 2: instead of like a psychologist that's completely unaware of what happens,
36:57
Speaker 2: is reliant on the patient to tell them everything that's
37:00
Speaker 2: going on. In their life. Yeah, all of a sudden,
37:03
Speaker 2: there butt knows them knows hey, oh this they're not
37:06
Speaker 2: sleeping very well for the past couple days. They haven't
37:11
Speaker 2: left their home this week, and this is a big
37:14
Speaker 2: deviation from them and how they normally would live life
37:18
Speaker 2: Like this can be targets of intervention that don't wait
37:22
Speaker 2: for this to be some sustained pattern in their life
37:25
Speaker 2: that becomes entrenched and hard to change. Like, no, let's
37:29
Speaker 2: actually have that as part of the conversation, where we
37:31
Speaker 2: don't have to wait for someone to tell us that
37:34
Speaker 2: that they didn't get out of bed. We kind of
37:35
Speaker 2: know that they haven't left their house, and we can
37:39
Speaker 2: actually make that a content of the intervention. So that's like,
37:42
Speaker 2: I think these these ability to like intervene proactively in
37:46
Speaker 2: these risk moments and not wait for folks to come
37:49
Speaker 2: to us and tell us every aspect of their life
37:53
Speaker 2: that they may not know and so like because of this,
37:55
Speaker 2: it's that's that's where I think there's a really powerful
38:01
Speaker 2: pairing of these two.
38:02
Speaker 1: I can see why that combination would be incredibly powerful
38:05
Speaker 1: and helpful. Do you worry at all about having that
38:09
Speaker 1: much information and that much sort of personal information on
38:13
Speaker 1: so many dimensions about people who are by definition vulnerable.
38:16
Speaker 2: Yeah, I mean, in some ways, I think it's the
38:19
Speaker 2: real ways that folks are already collecting a lot of
38:22
Speaker 2: this type of data already on these same populations, and
38:26
Speaker 2: now that we could put it to good use. Do
38:28
Speaker 2: I worry about kind of yet falling into the wrong hands. Absolutely.
38:32
Speaker 2: I mean we have like really big tight data security
38:35
Speaker 2: kind of protocols surrounding all of this to try to
38:38
Speaker 2: make sure that only folks that are established members of
38:41
Speaker 2: the team have any access to this data. And so yeah,
38:44
Speaker 2: we are really concerned about it. But yeah, no, if
38:47
Speaker 2: there was a breach or something like that that could
38:49
Speaker 2: be hugely impactful, something that would be greatly worry.
38:56
Speaker 1: We'll be back in a minute with the lightning round. Hey,
39:09
Speaker 1: let's finish with the lightning round.
39:11
Speaker 2: Okay.
39:14
Speaker 1: On net, have smartphones made us happier or less happy?
39:19
Speaker 2: Less happy?
39:21
Speaker 1: You think that you think you could change that, You
39:23
Speaker 1: think you could make the net flip back the other way.
39:25
Speaker 2: I think that we need to meet people where they are,
39:28
Speaker 2: and and so this is we're not like trying to
39:32
Speaker 2: keep folks on their phones, right, like, we're trying to
39:34
Speaker 2: actually start with where they are and intervene there, but
39:38
Speaker 2: like push them to go and experience life in a
39:41
Speaker 2: lot of ways.
39:42
Speaker 1: Yeah, Freud overrated or underrated?
39:48
Speaker 2: Overrated?
39:49
Speaker 1: Still okay, who's the most underrated thinker in the history
39:55
Speaker 1: of psychology? Oh?
39:56
Speaker 2: My, I I mean to some degree, Skinner was like
40:06
Speaker 2: really operant conditioning is like at the heart of most
40:11
Speaker 2: clinical phenomenon that deal with emotions, and I think it's
40:15
Speaker 2: probably one of the most impactful. Like it's so simple
40:18
Speaker 2: in some ways that behavior is shaped by both positive
40:23
Speaker 2: essentially benefits and like drawbacks, so rewards and punishments and
40:29
Speaker 2: these these types of things are the simplicity of it
40:33
Speaker 2: is is so simple, but like the how meaningful it
40:36
Speaker 2: is and daily life is so profound, we.
40:39
Speaker 1: Still underrate it. I mean when I the little bit
40:41
Speaker 1: I know about Skinner, I think of the black box, right,
40:44
Speaker 1: the like, don't worry about what's going on in somebody's mind,
40:46
Speaker 1: just look at what's going on on the outdoit. Yeah.
40:48
Speaker 2: Yeah, And with.
40:49
Speaker 1: Behavior, I mean in a way it sort of maps
40:52
Speaker 1: to your wearable's mobile devices thing, right, like just look,
40:57
Speaker 1: if you don't go outside, you get sad, and so
40:59
Speaker 1: go outside.
41:00
Speaker 2: Sure exactly. I am a behaviorist at heart, So this
41:04
Speaker 2: is part of part of what however you way.
41:07
Speaker 1: I mean, I was actually think briefly before we talked
41:10
Speaker 1: that wasn't gonna bring it up, But since you brought
41:11
Speaker 1: it up, it's interesting to think. Like the famous thing
41:14
Speaker 1: people say about Skinner is like the mind is a
41:16
Speaker 1: black box, right, we don't know what's going on on
41:18
Speaker 1: the inside and don't worry about it.
41:19
Speaker 2: Yeah.
41:20
Speaker 1: It makes me think of the way large language models
41:23
Speaker 1: on black boxes, and even the people who build them
41:25
Speaker 1: don't understand how they work.
41:27
Speaker 2: Right. Yeah, absolutely, I think psychologists in some ways are
41:30
Speaker 2: best suited to understand the behavior of large language models,
41:34
Speaker 2: because it's actually the science of behavior absence the ability
41:38
Speaker 2: to like potentially understand what's going on inside, Like neuroscience
41:43
Speaker 2: is a natural compliment, but in some ways a different
41:46
Speaker 2: different lens in which you view the world. So like
41:48
Speaker 2: trying to develop a predictable system that is shaped. I
41:51
Speaker 2: actually think we're not so bad in terms of folks
41:54
Speaker 2: to be able to take this on.
41:59
Speaker 1: What's your go to karaoke song?
42:01
Speaker 2: Oh, don't stop believing. I'm a big karaoke person too.
42:04
Speaker 1: Somebody just sent me that just the vocal from stop believing.
42:09
Speaker 2: Ah, yeah, no, it's it's it's like a meme.
42:13
Speaker 1: It's amazing, it is.
42:15
Speaker 2: Uh.
42:17
Speaker 1: What's one thing you've learned about yourself from a wearable device?
42:21
Speaker 2: Mm hmm. One of the things that I would say,
42:24
Speaker 2: like my ability to understand recognize when I've actually had
42:29
Speaker 2: a poor night's sleep or a good night's sleep has
42:31
Speaker 2: gotten much better over time. Like I think, as humans
42:35
Speaker 2: were not very well calibrated to it. But as you
42:38
Speaker 2: actually start to wear them and get understand you can
42:43
Speaker 2: you are you become a better self reporter.
42:45
Speaker 1: Actually I sleep badly. I assume it's because I'm middle aged.
42:50
Speaker 1: I do most of the things you're supposed to do.
42:52
Speaker 1: But give me one tip for sleeping. Well, I get
42:54
Speaker 1: to sleep, but then I wake up in the middle
42:56
Speaker 1: of the night.
42:57
Speaker 2: Yeah. That. I think. One of the things that a
42:59
Speaker 2: lot of people will do is they'll worry, particularly in bed,
43:04
Speaker 2: or use this as a time for thinking, so a
43:07
Speaker 2: lot of a lot of the effective surrounding that, or
43:10
Speaker 2: to try to actually give yourself that same time that
43:14
Speaker 2: would be that unstructured time that you would be dedicated
43:17
Speaker 2: that you might experience in bed.
43:19
Speaker 1: You tell me I should worry it ten at night
43:22
Speaker 1: instead of three in the morning. If I worry, if
43:24
Speaker 1: I say it ten at night, okay, worry now, then
43:26
Speaker 1: I'll sleep through the night.
43:27
Speaker 2: There there's literally evidence surrounding scheduling your worries out and
43:31
Speaker 2: I love during the day and it does work. So yeah,
43:33
Speaker 2: that's okay. If it's got some.
43:35
Speaker 1: Worries, I'm gonna worry it ten tonight, I'll let you
43:38
Speaker 1: know tomorrow morning.
43:39
Speaker 2: If it were just don't do it in bed. Yeah, okay, okay.
43:45
Speaker 1: If you had to build a chatbot based on one
43:48
Speaker 1: of the following fictional therapists or psychiatrists, which fictional therapist
43:54
Speaker 1: or psychiatrist would it be? A Jennifer Milthy from The Sopranos,
44:00
Speaker 1: B Doctor Krokowski from The Magic Mountain, see Fraser from Fraser,
44:06
Speaker 1: or d Hannibal Lecter.
44:08
Speaker 2: Oh god, okay, I would probably go with Frasier, a
44:12
Speaker 2: very different style of therapy than but I think his
44:15
Speaker 2: demeanor is at least generally decent, So yeah, mostly appropriate
44:20
Speaker 2: with most of his clients from what I remember in
44:22
Speaker 2: the show.
44:22
Speaker 1: Okay, it's a very thoughtful response to an absurd question.
44:28
Speaker 1: Anything else we should talk about?
44:30
Speaker 2: You've asked wonderful questions one thing I will say, maybe
44:34
Speaker 2: for folks that might be listening, is a lot of
44:37
Speaker 2: folks are already using generator AI for their mental health treatment,
44:43
Speaker 2: and so I will I'll give a recommendation if folks
44:47
Speaker 2: are doing this already, that they just treat it with
44:51
Speaker 2: the same level of concern they would have the Internet.
44:54
Speaker 2: They there may be benefits they can get out of it. Awesome, great,
44:59
Speaker 2: but just don't work on changing something within your daily
45:02
Speaker 2: life surrounding particularly your behavior, based on what these models
45:07
Speaker 2: are doing, without some real thought on making sure that
45:10
Speaker 2: that is actually going to be a safe thing for
45:13
Speaker 2: you to do.
45:20
Speaker 1: Nick Jacobsen is an assistant professor at the Center for
45:23
Speaker 1: Technology and Behavioral Health at the Geissel School of Medicine
45:27
Speaker 1: at Dartmouth. Today's show was produced by Gabriel Hunter Chang.
45:31
Speaker 1: It was edited by Lydia Jean Kott and engineered by
45:34
Speaker 1: Sarah Brugier. You can email us at problem at Pushkin
45:38
Speaker 1: dot FM. I'm Jacob Boldstein, and we'll be back next
45:41
Speaker 1: week with another
45:42
Speaker 2: Episode of What's Your Problem.