Latest / Code & Cure / #26 - How Your Phone Keyboard Signals Your State Of Mind

Transcript

#26 - How Your Phone Keyboard Signals Your State Of Mind · Code & Cure

• 0:00
  SPEAKER_01: Mood leaves a digital footprint and it's hiding in your keystrokes.
• 0:05
  SPEAKER_01: The question is, what can we do with this insight?
• 0:18
  SPEAKER_00: Hello and welcome to Code and Cure, the podcast where we discuss decoding health in the age of AI.
• 0:25
  SPEAKER_00: My name is Vasant Sarathi, and I'm an AI researcher and a cognitive scientist, and I'm here with Laura Hagopian.
• 0:31
  SPEAKER_00: I'm an emergency medicine physician and I work in digital health.
• 0:34
  SPEAKER_00: Today's topic is very interesting because it's not, you know, our typical or our usual AI being used to solve some healthcare problem or to make something more efficient or anything like that.
• 0:47
  SPEAKER_00: It's not an LLM and it's not a diffusion model.
• 0:51
  SPEAKER_00: It's not machine learning.
• 0:53
  SPEAKER_00: Well, maybe there is a little bit of that, but for the most part, it's not those things.
• 0:56
  SPEAKER_00: And it's not, you know, some complex medical thing, you know, imaging technique or anything like that or any situation like that.
• 1:03
  SPEAKER_00: Instead, it's about behavioral health.
• 1:06
  SPEAKER_00: It's about mood.
• 1:07
  SPEAKER_00: And the techniques are so simple, but they're so interesting.
• 1:10
  SPEAKER_00: And so we found this paper that we thought we wanted to share with you all.
• 1:14
  SPEAKER_00: And um it's all about how typing and your keystrokes can actually be a potential digital biomarker for moods.
• 1:24
  SPEAKER_01: And it's not like exactly what you're typing, it's not the content of what you're typing, right?
• 1:29
  SPEAKER_01: It's not the message.
• 1:30
  SPEAKER_01: In this study, they tracked the metadata.
• 1:32
  SPEAKER_01: So it's like, oh, how fast was someone typing?
• 1:35
  SPEAKER_01: How many errors did they make?
• 1:37
  SPEAKER_01: How many, like, how long were their pauses?
• 1:39
  SPEAKER_01: How long was their typing session in total?
• 1:41
  SPEAKER_01: So it was all this information that had nothing to do with the message itself that basically can they found contained um a good amount of information about someone's mood.
• 1:51
  SPEAKER_00: That is, I mean, absolutely fascinating to me because I mean, first of all, you know, this is a device that's a modern device, right?
• 1:58
  SPEAKER_00: It's a phone, it's got specific specific affordances of a phone, and you know, you're typing on a phone and you have a keyboard on there.
• 2:05
  SPEAKER_00: And to me, it's fascinating that the way your fingers move to convey ideas and messages and so on is a little bit of a hint into what your brain and what your mind is actually doing right now.
• 2:16
  SPEAKER_01: Right?
• 2:17
  SPEAKER_01: It's it's like it's very cool.
• 2:19
  SPEAKER_01: And what they did was it it was a sample of people who happened to say, okay, like I'm gonna participate in this and I'm gonna download this keyboard that looks like your typical iOS or in some cases Android keyboard.
• 2:32
  SPEAKER_01: Yeah.
• 2:32
  SPEAKER_01: And but it allows tracking of the metadata.
• 2:35
  SPEAKER_01: So it's gonna look, you know, you could you can go and do it.
• 2:38
  SPEAKER_01: You can go onto buyfact.com and like say, I I want to download this keyboard so that they can do research on me too.
• 2:47
  SPEAKER_00: Oh, that's cool.
• 2:48
  SPEAKER_00: I'm yeah, I might do that.
• 2:49
  SPEAKER_00: That's a good idea.
• 2:50
  SPEAKER_00: I I think that it's you know, maybe it's worthwhile taking a step back for a second and seeing, okay, we're talking about mood.
• 2:57
  SPEAKER_00: What is what is mood and in in this in this context and how is it currently being measured?
• 3:05
  SPEAKER_01: Yeah, so I think we can we can talk mostly about sad, sad mood or depressed mood right now, uh, because that's what they were tracking in this study.
• 3:14
  SPEAKER_01: And from a clinical standpoint, a lot of times we screen for mood disorders and we track mood with a questionnaire called the PHQ.
• 3:22
  SPEAKER_01: And many people here have probably done the PHQ.
• 3:25
  SPEAKER_01: You might get it at your checkup.
• 3:27
  SPEAKER_01: Um, you know, if you have a mood disorder, you might be asked more frequently to do it, you know, every three months, every six months, etc.
• 3:35
  SPEAKER_01: Um, but it's intermittent, right?
• 3:38
  SPEAKER_01: And it requires the patient to report things.
• 3:40
  SPEAKER_01: And, you know, depending on how you feel that day or what time of day it is, you might have different responses.
• 3:46
  SPEAKER_01: But it asks like, hey, um, how often have you been bothered by these problems?
• 3:52
  SPEAKER_01: Feeling down or depressed, um, trouble falling or staying asleep, poor appetite or overeating, um, feeling bad about yourself.
• 4:01
  SPEAKER_01: There's a whole series of questions in this PHQ or patient health questionnaire that's looking at mood.
• 4:09
  SPEAKER_01: Yeah.
• 4:09
  SPEAKER_00: Yeah.
• 4:10
  SPEAKER_01: And so that's, you know, it's something that you could do electronically.
• 4:13
  SPEAKER_01: It's something you could do at your doctor's office.
• 4:16
  SPEAKER_01: But I think the key points are, you know, it takes time to do, right?
• 4:19
  SPEAKER_01: Yeah.
• 4:20
  SPEAKER_01: Uh, it's it's active, it's not passive, and it's pretty intermittent.
• 4:23
  SPEAKER_01: Like you're not gonna do the PHQ every day.
• 4:28
  SPEAKER_00: Yeah, and even even if you do it right before a doctor's visit, it's right before a doctor's visit.
• 4:34
  SPEAKER_00: There's all this context that you already have about it, and that day might be very specific in many ways, right?
• 4:39
  SPEAKER_00: You might it might not be representative of the rest of the time you're experiencing those feelings.
• 4:43
  SPEAKER_01: Right.
• 4:43
  SPEAKER_01: And it does say in the questionnaire, like over the last two weeks, how how often have you been bothered by these symptoms, right?
• 4:49
  SPEAKER_01: It does ask you to have that recall.
• 4:52
  SPEAKER_01: But of course, there's there's bias in terms of recall and in terms of what you've been feeling more recently, for example.
• 5:00
  SPEAKER_01: And so it it's a great tool.
• 5:02
  SPEAKER_01: It's been well studied, it's validated.
• 5:05
  SPEAKER_01: Um, but I think the question they were starting to ask in this study was a good one because um when you're using someone's keyboard strokes, it's passive, right?
• 5:17
  SPEAKER_00: They're not even aware that you're doing that, which is actually a good thing in this particular instance because you're it's better than consciously recalling something.
• 5:25
  SPEAKER_00: You're actually tracking, you're potentially tracking something directly.
• 5:28
  SPEAKER_01: I mean, it could be, right?
• 5:28
  SPEAKER_01: That's the that's kind of the question that we're we're asking the study, but it's not like obtrusive.
• 5:33
  SPEAKER_00: Yeah.
• 5:34
  SPEAKER_01: It's something that is just happening in the background and it can be happening all the time.
• 5:37
  SPEAKER_01: So you're not getting a result every three months or six months or year or whatever it is.
• 5:42
  SPEAKER_01: Like you're able to track the ups and downs over time if there is something going on there, and we're able to find that that type of correlation.
• 5:50
  SPEAKER_01: Um, the other thing is like, I don't know, this is not, it's not like a super long questionnaire.
• 5:56
  SPEAKER_01: It's eight or nine questions long, depending on the version that you give.
• 6:00
  SPEAKER_01: Um, but it takes time to do, it takes energy to do, right?
• 6:06
  SPEAKER_01: Uh, and so there is this component of like, wouldn't it be nice if our phones could like sense us instead of us having to tell this is how I feel.
• 6:18
  SPEAKER_00: Yeah, I agree.
• 6:20
  SPEAKER_01: I think there's there's definitely this component of it where it makes it easy.
• 6:23
  SPEAKER_01: Like our phones send so many things about us to begin with.
• 6:26
  SPEAKER_01: Like if you go to if you go over to someone's house, you start getting retargeted with ads for their toothpaste because you logged into their Wi-Fi, right?
• 6:33
  SPEAKER_01: Yeah.
• 6:33
  SPEAKER_01: Like there's so much sensing going on with phones.
• 6:36
  SPEAKER_01: Like, can we harness some of that?
• 6:39
  SPEAKER_00: I mean, we have heart rate monitors and other things on our phones that are sending information.
• 6:43
  SPEAKER_00: If you ask me, is my heart racing right now?
• 6:45
  SPEAKER_00: I might give you a weird answer, but my phone's just my my my my phone, my my watch is is is measuring all of that automatically.
• 6:53
  SPEAKER_00: It's kind of the same thing, but for moods, right?
• 6:55
  SPEAKER_00: I mean, that's it's That's the concept, right?
• 6:57
  SPEAKER_01: Yeah of course there could be errors with something like this too.
• 7:00
  SPEAKER_01: Sure.
• 7:00
  SPEAKER_01: Right.
• 7:01
  SPEAKER_01: Um you know, in the physician community, we we joke about Fitbit syndrome.
• 7:05
  SPEAKER_01: It's it's right after New Year's.
• 7:07
  SPEAKER_01: Lots of people get their new Fitbits over the holidays, and then they start coming in saying, oh, you know, I got I had this abnormality detected on my Fitbit.
• 7:18
  SPEAKER_01: I had this abnormal.
• 7:20
  SPEAKER_01: And sometimes they're real, and sometimes it's you know, yeah, it's meant to detect certain things, it's not so great at detecting other things.
• 7:27
  SPEAKER_01: So it's you know, we you want to make sure that if you're saying this works, that it truly does work, that it truly does correlate.
• 7:35
  SPEAKER_01: And that's what they were kind of exploring in this study.
• 7:39
  SPEAKER_01: So what they looked at was a few different variables.
• 7:45
  SPEAKER_01: Um, and they they took a sample of people and they said, Hey, we're gonna find out your PHQ depression mood scores.
• 7:54
  SPEAKER_00: This is like a few hundred people, right?
• 7:55
  SPEAKER_01: Yeah, a few couple hundred people.
• 7:57
  SPEAKER_01: And then we're gonna follow your typing.
• 7:59
  SPEAKER_01: And it was the metadata from the typing.
• 8:01
  SPEAKER_00: So again, not the actual content of what they were saying, but like the the timing of when a key was pressed and when another key was pressed and so on.
• 8:10
  SPEAKER_01: Yeah.
• 8:10
  SPEAKER_01: So like if you're spelling out a word, like word, how long does it take you to go from W to O to R to D?
• 8:17
  SPEAKER_01: Yes.
• 8:17
  SPEAKER_01: Right.
• 8:18
  SPEAKER_01: Um, and then if you're pausing, like to think or what whatever, how long is that pause?
• 8:25
  SPEAKER_01: And what's the sort of variability in there too?
• 8:28
  SPEAKER_01: Is it fast and then slow and then fast and then slow, or is it kind of more uniform?
• 8:33
  SPEAKER_01: And then of course, uh I use autocorrect all the time.
• 8:37
  SPEAKER_01: Like they checked the accuracy, how many autocorrect instances were needed or used.
• 8:42
  SPEAKER_00: Yeah.
• 8:42
  SPEAKER_00: And I think they they had an autocorrect ratio that was against all the number of characters you typed or something.
• 8:48
  SPEAKER_01: And they tried to correct for other things too, right?
• 8:50
  SPEAKER_01: Like they said, oh, uh, we notice the the time of day is important.
• 8:55
  SPEAKER_01: People are better at typing in the middle of the day.
• 8:58
  SPEAKER_01: Yes.
• 8:58
  SPEAKER_01: Uh, they get worse at typing at the end of the day.
• 9:01
  SPEAKER_01: And midnight to 2 a.m.
• 9:02
  SPEAKER_01: It's like pretty bad.
• 9:03
  SPEAKER_00: Don't send emails after midnight, right?
• 9:05
  SPEAKER_00: Same sort of thing, yeah.
• 9:06
  SPEAKER_01: Um, and there were variations with age too, right?
• 9:10
  SPEAKER_01: So, you know, older people were sometimes slower, maybe took longer pauses, et cetera.
• 9:15
  SPEAKER_01: So they they tried to correct for some of those things.
• 9:17
  SPEAKER_01: And some people type with one hand versus two hands.
• 9:19
  SPEAKER_00: I mean, there's that too, right?
• 9:21
  SPEAKER_01: And they did some modeling to try to correct for all of those things.
• 9:24
  SPEAKER_00: Yeah.
• 9:24
  SPEAKER_01: Um, and I'm sure that, you know, it's impossible to correct for all the metrics that are out there.
• 9:31
  SPEAKER_01: But what they were able to find um is that the people with worse mood, like higher PHQ scores, they compare those to the people with PHQ scores of zero, like very good moods.
• 9:44
  SPEAKER_00: Yes.
• 9:45
  SPEAKER_00: And it's worthwhile to note that this was this study is is a few years old now, and there's been follow-ons.
• 9:50
  SPEAKER_00: But this original, this was the original study where they had the uh it was more exploratory.
• 9:55
  SPEAKER_00: So they they they wanted to see, okay, not every single nuance of this of this correlation between um typing speed and mood, but more in terms of uh large-scale effects.
• 10:06
  SPEAKER_00: So what they did was they focused on PHQ scores of zero versus the worst, right?
• 10:12
  SPEAKER_00: So the the best mood versus the worst mood in a sense, and try to see Yeah, not the stuff in the middle.
• 10:17
  SPEAKER_00: Not the stuff in the middle, yeah, because that's all like follow-on nuance, but they wanted to see if there's an effect at this very gross level, right?
• 10:23
  SPEAKER_01: Yeah, and they found that people who were sad, who had a more depressed mood, they actually typed faster.
• 10:30
  SPEAKER_00: Interesting.
• 10:31
  SPEAKER_01: Which we'll get into that.
• 10:32
  SPEAKER_00: Interesting.
• 10:33
  SPEAKER_01: But at the same time, they had more pauses and they had more sort of variability in their typing, fast to slow to fast to slow.
• 10:42
  SPEAKER_01: Um and and their sessions were shorter.
• 10:47
  SPEAKER_01: So they typed for like a a shorter amount of time.
• 10:50
  SPEAKER_01: Interesting.
• 10:51
  SPEAKER_01: So they typed quickly and for a shorter amount of time.
• 10:53
  SPEAKER_01: With more variability, with more variability and more pauses, and their typing was less accurate.
• 10:59
  SPEAKER_00: Okay.
• 11:00
  SPEAKER_01: So they used, they ended up using autocorrect a lot more frequently than the people with um, you know, a a good mood.
• 11:08
  SPEAKER_00: Yeah, right.
• 11:09
  SPEAKER_00: People with zero PHQ score um uh nine scores.
• 11:12
  SPEAKER_00: Yeah.
• 11:13
  SPEAKER_01: They use the PHQ eight actually, because they they took away the suicidality question.
• 11:16
  SPEAKER_01: Oh, I see.
• 11:17
  SPEAKER_01: Okay.
• 11:17
  SPEAKER_00: Um that's fascinating to me.
• 11:23
  SPEAKER_00: And I want to dig a little deeper into this business of typing faster.
• 11:29
  SPEAKER_00: Because it seems like that's uh maybe counterintuitive in some level.
• 11:34
  SPEAKER_00: Um you know, it seems like mood slows people down, but maybe not.
• 11:40
  SPEAKER_00: Maybe there is a connection between depression and um these sorts of motor skill motor slash thinking skills.
• 11:50
  SPEAKER_01: Yeah, well, so I if you asked me to summarize, like, oh, what is this what does this mean?
• 11:56
  SPEAKER_01: Like the the longer pauses, the high variability, the fast typing speed, I'd say it means like it it's a little bit more erratic.
• 12:04
  SPEAKER_00: Yeah.
• 12:04
  SPEAKER_01: And that does not actually surprise me because depressive disorders have a bunch of like neurocognitive and psychomotor mechanisms associated with them, right?
• 12:16
  SPEAKER_01: So you could have motor slowing.
• 12:19
  SPEAKER_01: That's you know, that makes sense in your head, right?
• 12:22
  SPEAKER_01: You're gonna slow down a little bit.
• 12:23
  SPEAKER_01: You're when you're down and when your mood is down.
• 12:26
  SPEAKER_01: But there's also this like erratic motor control.
• 12:31
  SPEAKER_01: Um and that includes for things like typing.
• 12:34
  SPEAKER_01: So it's not it's not surprising to me that the typing speed is altered, but like it's it's the whole picture that makes it feel like, oh, this makes sense to me because there are longer pauses, there's higher variability, there's uh more errors, and the typing speed is is altered too.
• 12:56
  SPEAKER_00: Yeah, yeah.
• 12:58
  SPEAKER_01: And in addition to that, I'm not surprised that there are more errors, right?
• 13:04
  SPEAKER_01: Because it's erratic.
• 13:06
  SPEAKER_01: And you know when you have that faster typing, right?
• 13:10
  SPEAKER_01: Right.
• 13:10
  SPEAKER_01: And there have been studies that show that people with depression um may be even more sensitive to errors, and then so you get this like speed accuracy trade-off that essentially happens.
• 13:23
  SPEAKER_01: Um, and so it's not surprising that someone who's typing so much faster loses accuracy with it.
• 13:31
  SPEAKER_01: And so those things actually do make sense in my mind.
• 13:35
  SPEAKER_01: They're not slowing down, they're not checking for errors, right?
• 13:40
  SPEAKER_01: They're not correcting things, they're letting autocorrect come in and do it.
• 13:44
  SPEAKER_00: Yeah, and and the erratic piece also kind of fits in, maybe because there's a fine balance between in these mood situations where maybe that you have a lot of things to say versus you don't want to say anything.
• 13:55
  SPEAKER_00: And and having a lot of things to say forces you to sp to write faster, type faster, think faster, but also go make all those mistakes, right?
• 14:03
  SPEAKER_01: Um yeah, and the other thing that I'll mention is they did not look at anxiety, but there's a lot of times where people who have depressive mood features have overlapping anxiety.
• 14:16
  SPEAKER_01: They might have, you know, some agitation.
• 14:19
  SPEAKER_01: Yeah.
• 14:20
  SPEAKER_01: And so they may end up trying to because they're cognitively a little bit slower from their depressed mood, they might try to like overcome that by typing faster.
• 14:34
  SPEAKER_00: Oh, interesting.
• 14:36
  SPEAKER_01: And so, of course, in that situation with that cognitive slowing and the fast typing, again, not surprising that you're gonna see more errors here.
• 14:46
  SPEAKER_00: Yeah, and I think that would be interesting to see because what we're seeing so far in the study is correlations.
• 14:52
  SPEAKER_00: And the question is, why is this happening, right?
• 14:54
  SPEAKER_00: And we are speculating or thinking about potential connections, and I'm sure uh there is research that's starting to be done about that, about exploring and diving deeper into these connections.
• 15:04
  SPEAKER_01: Yeah.
• 15:05
  SPEAKER_01: I and I think at the end of the day, it's like it doesn't necessarily need to be a causal thing, right?
• 15:13
  SPEAKER_01: You're trying to, we're trying to figure out, hey, can we monitor someone's mood in real time by using this metadata passively?
• 15:21
  SPEAKER_00: Yeah.
• 15:22
  SPEAKER_01: And, you know, it it's not like a huge study here, but the beginning, there's a beginning of a yes answer.
• 15:29
  SPEAKER_01: And I think that is fascinating because you can start to build passively this like concept of a digital phenotype of someone, right?
• 15:38
  SPEAKER_01: Like you get to know who an individual is based on some things that you can passively sense.
• 15:46
  SPEAKER_01: In some ways, like putting on a wearable, you're like, okay, I know that I'm wearing it, right?
• 15:51
  SPEAKER_01: I know that I'm wearing uh, you know, a Fitbit or whatever it is.
• 15:56
  SPEAKER_01: But but your typing speed, you're not like thinking about that.
• 15:59
  SPEAKER_01: You're not, you know, you can sign up for this study for sure, but you're not you're not considering that all the time.
• 16:05
  SPEAKER_01: Right.
• 16:06
  SPEAKER_01: And I think the idea here is there's there's a rich amount of data, and we can harness that to learn more and more about individuals in a passive way and get that more continuous feed of data so we can make adjustments in real time.
• 16:24
  SPEAKER_00: Yeah.
• 16:24
  SPEAKER_00: That's I think that's a very exciting piece of it.
• 16:27
  SPEAKER_00: But I do want to caution our listeners because it's very easy to go from there's a correlation to a causation that is specifically here where you you you can see the connection between the sort of typing data and mood, and you're like, okay, you know, does that mean that I can now predict someone's mood based on observing their how erratic they get in typing or whatever?
• 16:49
  SPEAKER_00: And that is not what the paper is saying at all, right?
• 16:52
  SPEAKER_00: That's I think it's really important to recognize that difference because you know, we don't know which direction the causality works, we don't know which way or if there's any other factors involved, but we don't know how these two the typing and and the mood, we don't know how they um uh uh uh influence each other exactly.
• 17:10
  SPEAKER_00: We just know that there's a correlation between them.
• 17:13
  SPEAKER_01: Yeah, exactly.
• 17:15
  SPEAKER_01: Um but I do think it's promising.
• 17:19
  SPEAKER_00: Yes, yes.
• 17:19
  SPEAKER_01: And I think it's it's really interesting.
• 17:21
  SPEAKER_01: Like I never until we read this paper, I never really thought about, oh geez, I wonder if I use autocorrect.
• 17:28
  SPEAKER_01: What does that say about my mood?
• 17:30
  SPEAKER_00: Yeah, well, that that that's it though.
• 17:31
  SPEAKER_00: That's the question.
• 17:32
  SPEAKER_00: That's the causality question that I'm hoping that we will get to get more insight on in the future.
• 17:37
  SPEAKER_01: Yeah, exactly.
• 17:38
  SPEAKER_01: And I do think there's a component of even if you aren't able to establish causality, can this be used to track?
• 17:46
  SPEAKER_00: Yeah, yeah.
• 17:47
  SPEAKER_00: Well, that's fair.
• 17:48
  SPEAKER_00: That's true.
• 17:48
  SPEAKER_00: Can this be used to track?
• 17:49
  SPEAKER_00: Correct.
• 17:50
  SPEAKER_01: Because I think that was the goal here is like, can we understand how typing typing metadata and mood go along together with each other?
• 18:02
  SPEAKER_01: Like, can we predict from this?
• 18:04
  SPEAKER_01: Yeah, even if there's not causality, can we use this as a monitoring mechanism?
• 18:09
  SPEAKER_01: And yeah, the answer is maybe.
• 18:12
  SPEAKER_00: Yeah, and the and the whole thing also, I think, that I I got really excited about was the fact that I didn't think about that there could be a connection here, but that's that's because I'm not an expert in this specific area.
• 18:22
  SPEAKER_00: But um, what else can we be tracking in this way, right?
• 18:26
  SPEAKER_00: There's so the sort of the world is our oyster, right?
• 18:30
  SPEAKER_01: Right.
• 18:30
  SPEAKER_01: How else could we build out a digital phenotype on each of us as individuals?
• 18:35
  SPEAKER_01: And I think uh when we were before we had that started the podcast, we were actually talking about, hey, wouldn't it be interesting if you compared yourself at one moment to yourself at another moment?
• 18:47
  SPEAKER_01: Like, what's my baseline typing speed and accuracy?
• 18:50
  SPEAKER_01: Yeah.
• 18:51
  SPEAKER_01: And if I'm having like a bad day, what does it look like compared to that?
• 18:54
  SPEAKER_01: So I think there's a lot of opportunity here to really dig in, get more personalized, whether it's different phenotypic markers or it's within this typing metadata and understanding like what is an individual's baseline and how does that vary when they're stressed, when they're anxious, um, when they have Alzheimer's disease.
• 19:15
  SPEAKER_01: Right.
• 19:15
  SPEAKER_01: Like there's there's so much that we could gather from this.
• 19:18
  SPEAKER_00: Yeah.
• 19:18
  SPEAKER_00: Yeah.
• 19:19
  SPEAKER_01: All right.
• 19:20
  SPEAKER_01: And with that, uh, we will see you next time on Code and Cure.
• 19:24
  SPEAKER_01: Thank you for joining us.