Latest / The Road to Accountable AI / Munmun De Choudhury (Georgia Tech): Conversational AI and Mental Health

Transcript

Munmun De Choudhury (Georgia Tech): Conversational AI and Mental Health · The Road to Accountable AI

• 0:09
  Kevin: Hi, I'm Kevin Werbach, Professor of Legal Studies and
• 0:11
  Business Ethics at the Wharton School of the University of Pennsylvania.
• 0:15
  For decades, I studied emerging technologies, from
• 0:17
  broadband to blockchain.
• 0:20
  Today, AI is promising to transform our world, but AI needs accountability,
• 0:26
  mechanisms to ensure it's developed and deployed in responsible,
• 0:29
  safe, and trustworthy ways.
• 0:32
  On this podcast, I speak with the experts leading the charge for accountable AI.
• 0:41
  Many millions of people now turn to conversational AI for
• 0:44
  emotional support, even when tools weren't built for that purpose.
• 0:47
  Concerns about AI's mental health impacts, up to and including
• 0:50
  suicides, are increasingly significant on the public policy agenda.
• 0:55
  My guest is Munmun De Choudhury, professor at Georgia Tech, who's
• 0:59
  been studying the interaction of digital technology and mental health
• 1:02
  as long as anyone and is one of the best cited researchers in the field.
• 1:05
  We talk about what conversational AI is actually doing to users, how different
• 1:10
  stakeholders need to address the risks and opportunities of AI for mental
• 1:14
  health, what researchers do and don't know, and what responsible development
• 1:19
  of conversational AI for well-being looks like when technology and use
• 1:23
  cases are both changing so quickly.
• 1:27
  Munmun, thank you so much for being with me on the road to accountable AI.
• 1:32
  Munmun: My pleasure.
• 1:32
  Thank you for having me today.
• 1:34
  Kevin: Tell me first a little bit about how you as a computer
• 1:37
  scientist got into the, uh, issue of mental health, uh, in people's
• 1:43
  interaction with computer systems.
• 1:45
  Munmun: That's a great question, and I would say, uh, my story
• 1:49
  goes back to the times when I was still a, a PhD student, um- Mm-hmm
• 1:54
  at Arizona State, and, um, I was studying how social media emerged as
• 2:00
  a new platform for communication, and this is sort of the late two thousands.
• 2:05
  And, you know, Twitter was a big thing, uh, in that era.
• 2:10
  And I started to look at how sort of the digital traces that people
• 2:14
  leave behind on these platforms can tell so much about people.
• 2:18
  But a lot of the focus at that time was sort of understanding communities
• 2:22
  and populations, not as much about understanding the individual.
• 2:25
  So by the time I f- came to finish my PhD and I, uh, joined Microsoft
• 2:30
  Research for a, a brief while, I started to ask that question.
• 2:35
  Can we actually help individuals with their well-being, with how they are
• 2:40
  feeling, what they're experiencing in their everyday lives by looking at
• 2:44
  what they're doing on these platforms?
• 2:46
  Uh, and that brought me close to mental health because turns out that
• 2:50
  a lot of these platforms are used by p- people when they are feeling
• 2:55
  in a cert- a certain way, right?
• 2:56
  And that could be- What is happening in their life or
• 2:59
  what is happening around them.
• 3:02
  And that came, uh, to me as an opportunity to explore uh, in the mental health space.
• 3:07
  Mm-hmm ... but broadly as a spectrum to think about wellbeing, think about
• 3:12
  happiness, satisfaction, all the way to when people are having struggles.
• 3:17
  And then the last decade, I would say, um, was spent thinking about
• 3:21
  how that question could be answered as the platforms changed over time.
• 3:26
  But what became also apparent is that these platforms are
• 3:29
  not just monoliths, right?
• 3:31
  Mm. They influence, uh, our lives in positive and negative ways, and that
• 3:35
  led to the most recent iteration of my research, which is kind of around
• 3:40
  understanding what can we understand about our mental health by looking to
• 3:44
  these platforms and what influence and impact are these platforms having on
• 3:49
  our mental health, uh, whether they're helping our mental health- Mm-hmm
• 3:52
  whether they're worsening our mental health, and so on.
• 3:54
  So that's kind of the bigger story here.
• 3:57
  Kevin: Absolutely.
• 3:57
  So let's just start back with the original social media work.
• 4:00
  What were you able to find just looking at people's transcripts
• 4:04
  of their interactions with things like Twitter that was predictive
• 4:08
  of their mental health outcomes?
• 4:10
  Munmun: So obviously, there are reasons of mental health when
• 4:13
  people go to these platforms, right?
• 4:15
  Mm-hmm.
• 4:15
  When p- uh, when we have moments of vulnerability, when we, uh,
• 4:18
  feel- Mm ... uh, struggling in our lives, when we feel lonely.
• 4:22
  But what was apparent from, uh, the data that we were looking at is that sometimes
• 4:27
  there are these implicit cues in the way that we behave on these platforms,
• 4:32
  in the way we express ourselves on these platforms that can hold a lot of
• 4:37
  information about our mental health.
• 4:39
  So to give you an example a particular aspect of our language is, you know,
• 4:43
  there are these content words, right, which are like different topics.
• 4:47
  Mm ... but there are all these other words in the English language
• 4:50
  that are called non-content words, or these are kind of words like
• 4:54
  the pronouns and the articles- Mm
• 4:56
  and the conjunctions and so on that we use- Mm ... which we
• 4:59
  don't consciously manipulate.
• 5:01
  These are words that we need to construct our sentences or to express ourselves.
• 5:05
  Mm. And what we saw is that those non-content words, those
• 5:09
  language signals, held a lot of information about how we are feeling.
• 5:13
  So people with mental health struggles often would use a
• 5:16
  lot of first-person pronouns.
• 5:18
  Kevin: Mm.
• 5:18
  Munmun: Singular pronouns, so I's and me's and myself's.
• 5:22
  And psychologically speaking, that happens when a lot of our focus
• 5:26
  is- is inward, uh, directed, right?
• 5:29
  We are thinking about ourselves- Mm-hmm ... and we are talking about
• 5:31
  ourselves, and this was something that psychologists has ob- have observed
• 5:35
  to be the case in the speech of people with mental health struggles.
• 5:40
  So we, what we started to find is that those kinds of signals in the language
• 5:44
  could tell us a lot about how people are feeling, not just somebody explicitly
• 5:48
  saying that they're feeling depressed, which does- Mm ... happen, but also how
• 5:52
  they're saying that they are struggling in their life was the real signal there.
• 5:58
  Kevin: And, uh, broadly speaking, to connect it up with AI, uh, in
• 6:02
  social media, i- i- you're saying something publicly, whereas if
• 6:07
  you're interacting with a chatbot then the AI is your interlocutor.
• 6:10
  Uh, w- do you see similar kinds of patterns in people's
• 6:15
  communication in those two contexts?
• 6:18
  Munmun: People certainly feel more candid in their interactions when they know that
• 6:23
  this is a platform that is, uh, private.
• 6:26
  And there are certain kinds of social media that are like that as well.
• 6:29
  Mm-hmm.
• 6:30
  So we looked at uh, in the past, we have looked at public social media such as-
• 6:33
  Mm ... Twitter, but we have also looked at private conversations that happen on other
• 6:38
  types of social media, such as Instagram.
• 6:40
  Instagram, Facebook, and these platforms have the direct messaging
• 6:45
  mechanism where you can message another person or a group of people.
• 6:48
  Mm. Mm. And these are inherently very private conversations.
• 6:51
  So we did see differences even at the time across social media where
• 6:56
  people are more publicly facing and when people are feeling that
• 7:00
  they are in a more private setting.
• 7:02
  So obviously these are, these are information that people chose to share
• 7:05
  with us later on for research purposes.
• 7:08
  Mm-hmm.
• 7:08
  But that carries on to AI as well.
• 7:11
  When somebody is interacting with a bot or when somebody is, you know, searching
• 7:14
  on a search engine, that has a perception that it's a very private activity.
• 7:18
  Kevin: Mm-hmm.
• 7:19
  Munmun: We see people being more disinhibited, more, uh, le- less
• 7:22
  guarded, um, more- Yeah ... candid in those conversations- Mm
• 7:25
  and talk about, uh, maybe things that they may not be sharing in a public platform.
• 7:31
  But public social media kind of gives us other sets of insights
• 7:35
  which you don't see sort of in these private AI, uh, interaction contexts.
• 7:39
  Mm. For instance, how does a person engage with the larger community and
• 7:43
  s- how do they socialize with others?
• 7:45
  And those are a unique set of signals that you see in the public contras- context.
• 7:50
  So yeah, to your question, a lot of these behaviors do shift when we are
• 7:54
  talking about human-AI interaction.
• 7:57
  Kevin: And obviously the, the big challenge with conversational AI
• 8:00
  systems is that on the one hand they can be a huge boon for mental health,
• 8:05
  that it's people who don't have access to therapy can now talk to someone.
• 8:08
  B- but on the other side, we've seen, uh, all these examples that, seems to cause
• 8:13
  psychoses or worsening mental health.
• 8:14
  So ju- just broadly speaking, you know, how do you evaluate those
• 8:19
  two sides in your work, and how do you think about that challenge?
• 8:23
  Munmun: It is truly, uh, a double-edged sword.
• 8:26
  That's how I think about it.
• 8:27
  And none of these technologies are sort of all good or all bad- Mm-hmm ... whether
• 8:31
  it's social media or conversational AI.
• 8:34
  And what we need to do is to really identify where the
• 8:40
  points, where the benefits are.
• 8:42
  Can we quantify them?
• 8:43
  Could we understand them in a more systematic, principled way?
• 8:47
  And the same goes for the harms and the risks.
• 8:49
  Hmm.
• 8:50
  Where are these platforms uh, failing, uh, to you know, serve our mental health?
• 8:55
  And can we quantify, can we identify those situations?
• 8:58
  And then the question is this trade-off, right?
• 9:01
  And even for the same person, it's not a matter of AI works for
• 9:04
  some people and doesn't work for some people or harms some people.
• 9:09
  We know from social media that even for the same person, the same technology
• 9:14
  can sometimes be helpful and can sometimes- Mm-hmm ... be harmful.
• 9:19
  So the path forward is, I would say it's a, it's a multi-stakeholder
• 9:24
  responsibility base, where we are kind of thinking about how different stakeholders
• 9:29
  in this ecosystem counterbalance that trade-off of benefits- Hmm
• 9:32
  and harms.
• 9:34
  Kevin: Say a little bit more what you mean by the different stakeholders.
• 9:36
  Munmun: Yeah.
• 9:37
  So, you know, when we think about technologies like AI, we often
• 9:41
  think- Mm-hmm ... about the companies that build these technologies.
• 9:45
  Mm-hmm.
• 9:45
  We think about the people who are using them.
• 9:47
  But that ecosystem is very rich.
• 9:51
  It's not just those two parties that are part of the system, but we have
• 9:56
  the broader society and context.
• 9:58
  We have other people in the person's life who is interacting
• 10:01
  with the conversational AI.
• 10:03
  And we have, uh, governments and regulators and policymakers- ... as well.
• 10:09
  Uh, so what I meant to say is that, you know, this trade-off
• 10:11
  between these technologies being beneficial or harmful is something
• 10:16
  that is important to be investigated for each of those stakeholders.
• 10:21
  Mm-hmm.
• 10:21
  And I would say that each of those stakeholders have a unique role to play
• 10:25
  in how they counterbalance this trade-off.
• 10:28
  Kevin: Mm-hmm.
• 10:29
  How much do we understand today about, uh, addressing that, that challenge you
• 10:33
  talked about earlier, that there are certain patterns in these interactions
• 10:39
  a- a- and these chatbots that, you know, will lead to more healthy, uh,
• 10:43
  results versus the ones that will push people towards depression and self-harm?
• 10:49
  Munmun: The understanding is evolving.
• 10:51
  The good news is that it's evolving rapidly.
• 10:54
  The bad news is that it is evolving and- Mm-hmm ... we don't have a set
• 10:58
  understanding, okay, these are the benefits, this is where people are getting
• 11:03
  help, and these are the risks and harms where, you know, bad things are happening.
• 11:07
  That understanding is evolving because the technologies themselves
• 11:12
  are changing very rapidly.
• 11:14
  We have a new version of, uh, a chatbot from some- Mm-hmm ... uh, platform
• 11:19
  or the other pretty much every month.
• 11:21
  Um-
• 11:22
  Kevin: Mm-hmm
• 11:23
  ... Munmun: and the capabilities of these tools are also changing.
• 11:27
  But let's also not forget the way people use these platforms
• 11:31
  is also evolving, right?
• 11:33
  Kevin: Mm-hmm.
• 11:33
  Munmun: And the analogy I often use is a very old technology called
• 11:38
  platform called Google Flu Trends- Mm-hmm ... which Google released in the
• 11:42
  late 2000s, and then- Yep ... it was kind of taken off in the early, uh, 2010s.
• 11:48
  That lesson has a lot of meaning and value and relevance to this moment as well.
• 11:55
  Google Flu Trends, which at some point was the poster child of big data, and
• 11:59
  it was able to predict influenza cases before the CDC numbers came out, um,
• 12:06
  started first underestimating and then overly overestimating these trends.
• 12:10
  Kevin: Mm-hmm.
• 12:11
  Munmun: And one of the postmortems was that the way
• 12:14
  people were using Google changed.
• 12:16
  Yes.
• 12:16
  And this is a challenge for AI as well.
• 12:19
  The way people use AI is not fixed.
• 12:22
  Not only are the AI tools themselves changing, but the way and the reasons
• 12:27
  why people use them are also changing.
• 12:29
  So this complicates our understanding of where the benefits are and
• 12:33
  where where the harms are.
• 12:35
  Kevin: Mm-hmm.
• 12:36
  Munmun: But here is, I think, a positive thing.
• 12:39
  For something like social media, it took us a long time to acknowledge and
• 12:44
  recognize that there are real harms that could be happening, uh, to people.
• 12:48
  It took us almost a decade to be able to study those harms.
• 12:54
  Kevin: Mm-hmm.
• 12:54
  Munmun: I think for conversational AI and chatbots, this is already happening.
• 12:59
  A lot of it is happening in academia, uh, but there are also nonprofits
• 13:03
  who are invested in that for specific applications of these platforms.
• 13:08
  And I'm so, uh... I feel very good to see that that attention is being- Mm-hmm
• 13:14
  ... given to conversational AI and chatbots early on in, in the life cycle of use of
• 13:19
  these technologies, rather than waiting for significant harm to happen later.
• 13:25
  Kevin: No, absolutely.
• 13:25
  I agree with you completely, and I'm glad you mentioned Google Flu Trends.
• 13:28
  I've taught that for a number of times, and it's-- in some ways it's
• 13:30
  even more, uh, complicated than you describe because there was, reflexivity
• 13:34
  involved that people read media coverage about Google Flu Trends and went to
• 13:38
  Google and said, "Oh, let me try this."
• 13:40
  And then that threw off the, you know, the, the dataset as well.
• 13:43
  So, so no, it's a really great example.
• 13:45
  I guess the question is, given what you've described, that we're dealing
• 13:50
  with something where everything is, is moving and changing, a-and yet we have
• 13:54
  this big challenge how to mitigate the extent to which these chatbots are, you
• 13:59
  know, worsening mental health problems.
• 14:02
  You know, what are the, the research questions we need to answer
• 14:05
  or, or what are the things that stakeholders really need to focus
• 14:09
  on to be able to address that issue?
• 14:12
  Munmun: So I think we need some standards, right?
• 14:15
  Mm-hmm.
• 14:15
  And we see that in, in the non-tech sectors- ... widely speaking, right?
• 14:20
  We see that in pharma, we see that in, uh, medical devices- where there
• 14:26
  are industry standards, and any product that is-- or artifact that
• 14:31
  is developed- Mm ... needs to meet.
• 14:34
  It's sort of like a minimum set of criteria.
• 14:36
  The standards is like you can always do better- Mm ... than that, but
• 14:39
  this is what you sh- your product should be doing at the minimum.
• 14:43
  That is lacking in the, the AI space.
• 14:47
  Uh, so that is something that'll be needed, and that'll
• 14:49
  be particularly relevant from the tech company's perspective.
• 14:52
  Mm-hmm.
• 14:52
  Of course, this makes the assumption that the tech companies as a coalition
• 14:57
  would come forward and, and develop what those industry standards could look like.
• 15:02
  Again, it happened, uh, for pharma and other, other- Mm-hmm
• 15:05
  uh, industries.
• 15:06
  So that is something that I would say is focused more on the tech company side.
• 15:12
  The other thing I would say is that, you know, tech companies, it'll
• 15:14
  be useful for them to acknowledge and recognize that wellbeing is an
• 15:19
  important- ... uh, criteria, even from their bottom line perspective.
• 15:23
  Mm-hmm.
• 15:23
  Oftentimes that is not, uh, given as much importance compared to other facets
• 15:29
  of- Mm ... a product that might impact the bottom line or the next quarterly
• 15:32
  revenues, uh, or profits of a company.
• 15:35
  But you know, if, if the users are not feeling well, then people are gonna
• 15:40
  abandon the platform sooner or later.
• 15:41
  So I would- Mm ... love for the companies to consider wellbeing as an
• 15:45
  important aspect of how they operate.
• 15:48
  Speaking of, uh, stakeholders such as, you know, our society or the
• 15:53
  people who use these platforms, I think we need AI literacy.
• 15:58
  Mm. And that needs to start early on, I would argue in
• 16:02
  middle school or high school.
• 16:05
  And the example that I often cite is let's talk about programming, right?
• 16:11
  When a lot of, uh, the previous generation was growing up, we all said
• 16:15
  that, you know, programming is, is such an important skill set that we want
• 16:19
  kids to learn it from early on because, you know, it, it's not just about the
• 16:23
  actual activity of writing the code- Mm
• 16:25
  but it's also about helping develop analytical- Yeah ... thinking,
• 16:29
  logical ways to proceed and analyze situations and problems, and
• 16:33
  these are important life skills.
• 16:34
  I think we need something like that for AI as well.
• 16:37
  Again, AI literacy is not just about how do I use the AI-
• 16:41
  But how does use of the AI impact our other aspects of life, critical
• 16:45
  thinking how we manage our well-being, what AI can and cannot do for us.
• 16:50
  And all of these are important things that I think will be necessary
• 16:55
  for the next generation to learn because the world that they are gonna
• 16:58
  see is gonna look very different.
• 17:02
  One in which AI is deeply enmeshed in everything that- Mm-hmm ... they
• 17:06
  do or the way they lead their life.
• 17:09
  Mm. Um, and that means that, uh, you know, as caregivers, as parents-
• 17:14
  Mm-hmm ... as teachers you know, as, as college prof-professors, we all have
• 17:19
  something to do in that space and in how we help the next generation of people
• 17:28
  work with AI not abandon AI, not- Mm-hmm
• 17:33
  you know, totally give up or surrender to AI.
• 17:36
  But what are good ways to, to, to use AI, you know?
• 17:39
  And mental health is a great example of that.
• 17:41
  Like, when is AI and, uh, a conversational AI good for mental
• 17:45
  health help, and when is it that-
• 17:47
  ... People need to defer to other people for getting- Mm-hmm
• 17:50
  getting help.
• 17:51
  And then finally governments and policymakers.
• 17:54
  Mm-hmm.
• 17:54
  You know, we, we can turn to Europe, for instance, where we see that there
• 17:59
  is a lot of effort in a top-down fashion from-- coming from policymakers and
• 18:04
  in terms of holding the technologies accountable to the harms that
• 18:09
  they might be causing to people.
• 18:11
  And while that is not happening in, in the United States at the moment,
• 18:16
  but this is- Right ... something that will be needed, in my opinion-
• 18:19
  Mm-hmm ... going forward, along with those other approaches that I talked about.
• 18:24
  And again, any one of those is not gonna address the challenge that we are seeing
• 18:30
  or the moment that we are passing through.
• 18:32
  I think we need all of us to work together- Mm-hmm ... and
• 18:36
  adopt different strategies.
• 18:38
  Kevin: Related specifically to the next generation, um, y- many of these cases,
• 18:43
  at least the, you know, the suicide cases where people are suing AI companies
• 18:46
  involve teenagers or, or young people.
• 18:49
  You know, how different is the, the mental health context of adolescents?
• 18:54
  What are your thoughts on, uh, the notions that we should restrict
• 18:57
  access to some of these technologies a- at certain age levels?
• 19:01
  A- a- and more broadly, how do we confront that challenge?
• 19:04
  Munmun: Yeah.
• 19:05
  So I would mention that, uh, Australia is one of the countries- Mm-hmm ... who
• 19:09
  has adopted a social media ban, and, uh, many countries are hoping- Mm-hmm
• 19:14
  to follow s- suit.
• 19:15
  Uh, so I am actually on the advisory board of the Australian government's-
• 19:19
  Mm-hmm ... e-safety panel.
• 19:20
  And I would say that, you know, we are waiting to see how that pans
• 19:24
  out, and that'll be- Mm-hmm ... a very valuable lesson- Mm-hmm ... for
• 19:27
  other kinds of technologies on whether this kind of age-based
• 19:31
  restriction or other forms of ban-
• 19:33
  Can actually help.
• 19:34
  For instance, in the state of Georgia, a lot of the public school systems are,
• 19:38
  uh, implementing cell phone ban- bans.
• 19:41
  Mm-hmm.
• 19:41
  And this is happening in a number of other states- Mm-hmm ... in the US as well.
• 19:45
  So again any-- I feel any regulation or policy that is implemented like that,
• 19:50
  we need to pay very careful attention to whether implementation of that policy
• 19:55
  is actually having the intended effects.
• 19:57
  Um- Mm-hmm.
• 19:58
  And we n- I would say that we need to go at it, uh, with an open mind because
• 20:02
  at least the preliminary evidence shows that it is very complicated.
• 20:07
  I'll give you the example of the cell phone bans in middle schools.
• 20:10
  Mm-hmm.
• 20:10
  And some of the emerging work says that, whether the cell phone ban
• 20:15
  is good or bad, it really depends upon what is it that you care about.
• 20:18
  If you care about whether the kids are gonna be bullied less online,
• 20:23
  you know, cell phone bans are not gonna help because, yes, they don't
• 20:26
  get bullied during the school hours.
• 20:28
  They're gonna get bullied in the morning, later in the evening, and so on.
• 20:32
  So if that's the, the goal of the ban, then this is not a good policy.
• 20:36
  But if the goal is let's have the kids pay more attention to the teacher
• 20:41
  and socially interact with each other during school hours, then the
• 20:45
  cell phone ban makes a lot of sense.
• 20:48
  And it can have positive effects.
• 20:49
  So as we think about these types of policies of bans or age verification
• 20:56
  and so on, we need to pay very careful attention to what is it that we are
• 21:00
  hoping to achieve with this policy.
• 21:03
  Because this kind of a blanket policy will not help us solve all of the 23
• 21:08
  problems that we might be hoping to solve.
• 21:10
  Right.
• 21:10
  I wish that was the case.
• 21:12
  It's gonna help resolve some issues and not gonna help resolve some others, and
• 21:17
  we need to bring that approach where we lean on the evidence, we lean on what
• 21:22
  the impacts of the implementation of the policy are and then go from there.
• 21:27
  Um, so yeah.
• 21:28
  Mm-hmm.
• 21:28
  So there is no golden answer, I would say- Yeah ... to whether
• 21:30
  these policies are good or bad.
• 21:33
  Kevin: Yeah, and cellphone bans and social media bans are one thing that, as you
• 21:37
  say, are being explored, but now we've got chatbots and things like ChatGPT,
• 21:42
  which are not entertainment services.
• 21:44
  People are using them to do their schoolwork and to do information
• 21:48
  gathering, but also in some cases having these long conversations, uh,
• 21:52
  which may have a mental health effect.
• 21:54
  How does any of this relate to the generative AI situation, where now we've
• 21:58
  got these general purpose chatbots?
• 22:01
  Munmun: That really complicates it.
• 22:02
  Yeah.
• 22:02
  Because if you look at the history of AI, for the last 30 or 40 years, we r-
• 22:09
  as a, as a society, we really excelled at building AI for specific purposes and
• 22:15
  Kevin: needs.
• 22:16
  Munmun: Right?
• 22:17
  Or you, we, we want to we want to help humans evaluate- Mm. ... radiology
• 22:22
  reports- Mm-hmm ... for cancer patients.
• 22:24
  Let's build a custom AI- Yeah ... for that, that, that only can read
• 22:29
  radiology reports, and we found that that can be pretty good.
• 22:32
  I mean, there are studies that show that the efficacy of these AI models-
• 22:38
  ... That can detect, let's say, cancer cells from a radiology report can
• 22:43
  work at par with human radiologists.
• 22:46
  But all of this changed with the generative AI paradigm, where we
• 22:50
  are talking about these general purpose AI, which flips the whole
• 22:54
  pyramid- Mm-hmm ... that, you know, we are making these AI more and more
• 22:58
  niche and more and more specialized.
• 23:00
  Now we are saying that maybe it's an AI that is not gonna be amazing- Mm
• 23:04
  at everything, but it's gonna do reasonably well- ... in you
• 23:09
  know, a number of different tasks.
• 23:11
  The way this poses a problem is- Now these benefits and harms
• 23:16
  are very much task-dependent, or how- Mm-hmm ... somebody's using
• 23:20
  these platforms depend on that.
• 23:22
  And mental health is a great example, right?
• 23:24
  I mean, none of these platforms were actually designed-
• 23:27
  Kevin: Mm-hmm
• 23:27
  ... Munmun: to, help a person with mental health struggles.
• 23:30
  Although I would say that there are now companies who are trying to build
• 23:33
  therapy bots- Yeah ... and so on.
• 23:34
  So I'm not talking about those per se.
• 23:36
  I'm talking about the general purpose conversational AI tools, which can
• 23:40
  do a number of different things- Yeah
• 23:42
  from being your, being a, a kid's homework, homework buddy to, you
• 23:47
  know, being a, a soulmate, um- Mm-hmm ... to, uh, to, uh, being
• 23:51
  a counselor during mental health struggles and everything in between.
• 23:56
  And it poses new types of challenges because now we don't know how
• 24:00
  somebody is gonna use the platform.
• 24:02
  Mm-hmm.
• 24:03
  And when we don't know how they're gonna use the platform, we don't know
• 24:06
  where the harms and benefits may lie.
• 24:09
  So this is part of the larger issue that, you know, we, we are having sort
• 24:13
  of these general purpose monoliths, and we as users of these platforms have very
• 24:18
  little understanding of how it helps us.
• 24:23
  It, it caters to a variety of- Mm-hmm ... different tasks and needs
• 24:26
  and questions that we might have.
• 24:28
  And that lack of transparency and lack of understanding further complicates
• 24:32
  uh, how we navigate this issue.
• 24:36
  Kevin: Mm-hmm.
• 24:37
  What is the research agenda for scholars like you in this area to understand
• 24:43
  uh, those mental health issues around these general purpose chatbots?
• 24:47
  Munmun: Yeah.
• 24:47
  So I would say that there are three different directions.
• 24:50
  Uh, one where there, there is a lot of interest already, and I
• 24:53
  mentioned earlier that it's great that academics are working to- Mm-hmm
• 24:57
  uncover these harms and risks.
• 25:00
  Sure.
• 25:00
  And that needs to continue.
• 25:02
  As I said, these issues are evolving, these technologies are
• 25:06
  evolving, so we continue to invest.
• 25:08
  It's not gonna be like day after tomorrow we would have figured out
• 25:12
  all the possible harms of these tools, and now we can, close that research
• 25:16
  program, move on to the next one.
• 25:18
  I wish that was the case, but it is not gonna be the case.
• 25:22
  So we continue-- we need to continue to look at and probe that space, and there
• 25:26
  are a few different ways to go about it.
• 25:28
  There's theoretical work that is happening from researchers on proposing,
• 25:32
  let's say, a framework or a taxonomy of harms so that somebody, uh, coming
• 25:37
  into it can have a general sense- Mm-hmm
• 25:39
  of where these things land when somebody uses these tools.
• 25:44
  Um, there's also work on what is known as red teaming, and red teaming- Mm-hmm
• 25:48
  ... is the concept of adversarially using these tools in a way that would, you
• 25:54
  know, uncover where the pitfalls are.
• 25:57
  Uh, so both of those things need further investment going forward.
• 26:01
  Uh, the second direction I would say is- A deeper engagement with the people.
• 26:06
  As I said- Mm ... there are so many stakeholders in the space,
• 26:10
  and solutions to the harms are not gonna come by interacting or
• 26:14
  looking at any one stakeholder alone.
• 26:16
  We need to be able to understand where the challenges are, how these tools are
• 26:23
  being used, but also the users of these tools don't exist in a vacuum, right?
• 26:27
  Kevin: Mm.
• 26:28
  Munmun: We know that, sometimes interactions with AI can lead to
• 26:33
  displacement of actual human connections.
• 26:36
  Um, we hear that, we read it in the news that somebody started to spend so
• 26:41
  much time interacting with the chatbot that they abandoned their family.
• 26:45
  So there are real impacts in our everyday lives as well, and I think
• 26:48
  more research is needed to look at what we can call the spillover effect.
• 26:53
  Mm-hmm.
• 26:53
  So it's not just the harms that are happening in the interaction
• 26:56
  between the human and the AI, but what is happening beyond that in
• 27:01
  our society and our communities.
• 27:03
  And then the third, uh, research agenda I would say is about, okay, we, we uncover
• 27:09
  all these problems, but as researchers, we also like to provide s-solutions.
• 27:13
  Mm-hmm.
• 27:14
  And I think that is much needed here is okay, so let's say we do the red
• 27:19
  teaming, we identify where the harms are, so how do we move forward from there?
• 27:24
  What needs to change in the design of these- Mm-hmm ... platforms, in the
• 27:27
  way that we build these AI tools that can, amplify the benefits and- Mm-hmm
• 27:33
  uh, minimize, if not mitigate, the harms?
• 27:35
  So kind of leading that path forward- ... and defining what that path
• 27:39
  would look like would go a long way.
• 27:42
  And the hope is that enough research happening in this space would eventually
• 27:47
  make its way into the tech sector as well when- Mm-hmm ... when companies
• 27:52
  small and big see that, okay, there are paths forward that look pr-promising,
• 27:58
  uh, and hopefully that those would eventually get integrated into the
• 28:03
  next generation of AI that is built.
• 28:06
  Kevin: Mm-hmm.
• 28:06
  And what about, uh, tools that are built specifically to provide mental health
• 28:10
  support based on conversational AI?
• 28:13
  What, what have you found in, in looking, uh, at how people interact with those?
• 28:20
  Munmun: Most of the tools that I have seen in that space are really putting
• 28:24
  an added layer on top of a general-
• 28:26
  purpose AI bot.
• 28:27
  And while it is an easy and quick way to uh, build something and bring help
• 28:34
  to people- Mm ... what we have to recognize that even at fundamentally
• 28:38
  Those tools are still that general purpose AI bot, which, you know,
• 28:41
  was not designed for the purposes of providing mental health support.
• 28:46
  So we need a fundamental rethinking, a revamping of how we approach that.
• 28:52
  I don't think today's AI is ready to provide therapy, for
• 28:56
  instance- Mm-hmm ... uh, to people in lieu of actual human beings.
• 29:01
  But AI can be a way to provide or fill in support when human
• 29:08
  support may not be available.
• 29:09
  Mm. So I feel like it-- for those uh, actors who are building these
• 29:14
  tools, particularly for mental health support, do two things.
• 29:18
  One is fundamentally rethink where AI can actually be helpful, and I think
• 29:24
  that AI- Mm-hmm ... is best used in moments where other forms of human
• 29:28
  help are not available- Mm ... not as a replacement for human help.
• 29:32
  And then the second thing is how can these models be, these foundation models be
• 29:38
  built ground up with principles from the mental health space, principles of- Mm
• 29:43
  psychotherapy, principles of how humans support each other.
• 29:48
  I often say that, you know, um, these tools are so good
• 29:52
  at problem-solving, right?
• 29:53
  These AI conversational- Mm ... AI agents are good at problem-solving.
• 29:56
  But mental health often is not about problem-solving.
• 29:59
  Kevin: Yep.
• 29:59
  Munmun: Mental health help, if you talk to an expert like a counselor or- Mm
• 30:05
  therapist, uh, or peer supporter, they would say that my job is
• 30:09
  to give the person the tools so that they can navigate-
• 30:14
  Kevin: Mm
• 30:14
  ... Munmun: their life challenges better.
• 30:16
  Mm-hmm.
• 30:16
  So my question for those actors who are building mental health AI is
• 30:21
  that can AI take on that ro-role?
• 30:23
  Instead of being- Sure ... a problem solver, what would it mean for AI
• 30:28
  to provide the human being with the tools that they, that they need
• 30:32
  to empower themselves to better take care of their mental health?
• 30:38
  Kevin: Well, that's actually a great place to land on.
• 30:40
  Uh, that's the challenge going forward.
• 30:42
  Uh, Mona, thank you very much for being with me.
• 30:44
  Munmun: Thank you so much for having me.
• 30:47
  Kevin: This has been The Road to Accountable AI.
• 30:50
  If you like what you're hearing, please give us a good review and
• 30:53
  check out my Substack for more insights on AI accountability.
• 30:56
  Thank you for listening.
• 31:01
  If you wanna go deeper on AI governance, trust, and responsibility with me and
• 31:05
  other distinguished faculty of the world's top business school, sign up for the
• 31:09
  next cohort of Wharton's Strategies for Accountable AI online executive education
• 31:14
  program, featuring live interaction with faculty, expert interviews, and
• 31:18
  custom designed asynchronous content.
• 31:21
  Join fellow business leaders to learn valuable skills you can
• 31:24
  put to work in your organization.
• 31:26
  Visit execed.wharton.upenn.edu/acai for full details.
• 31:34
  I hope to see you
• 31:34
  there.