Latest / CareTalk: Healthcare. Unfiltered. / The Trust Problem With Healthcare AI

Transcript

The Trust Problem With Healthcare AI · CareTalk: Healthcare. Unfiltered.

• 0:07
  David: So what do you see in terms of people working alongside some technologies like AI and also robotics, uh, in care delivery?
• 0:15
  David: Is that something we should be enthusiastic about or is it, is it scary?
• 0:20
  Sarah Matt: For me, it's exciting, but at the same time I've been building my whole career, so I've been seeing this come to bear over several years.
• 0:28
  Sarah Matt: So.
• 0:30
  Sarah Matt: I don't think, for me it's scary.
• 0:32
  Sarah Matt: It's a calculated risk and ultimately, if you understand what we're.
• 0:37
  Sarah Matt: Taking data from what we're doing with it, how it's going to improve things.
• 0:41
  Sarah Matt: It has built trust with me in certain circumstances.
• 0:44
  Sarah Matt: What I'd say for patients and for providers for that matter, is that ultimately a lot of the AI is not what I'd call explainable.
• 0:52
  Sarah Matt: Um, there's a lot of black boxes and if you're a provider putting your license on the line to take care of patients, ultimately you're gonna wanna know where those tools are coming from, how they're being utilized, and.
• 1:04
  Sarah Matt: Where your liability starts and ends, and as a patient, you're gonna want information and we need to show patients that there's value in their data, but by giving us their data, they gain value from it.
• 1:16
  Sarah Matt: As an example, I give Google Maps my information all the time.
• 1:20
  Sarah Matt: I'm sure you do too.
• 1:21
  Sarah Matt: And I find value in giving them that data.
• 1:24
  Sarah Matt: On the healthcare side, I don't think we've been providing true value to patients by utilizing their data just yet.
• 1:30
  David: You know, you mentioned a, a black box, um, from some of the earlier times when AI is starting to be used, there's an awareness that some of the training data is coming from.
• 1:40
  David: You know, from bias data sets.
• 1:42
  David: And then, so on the one hand you may naively, naively go and say, Hey, we're gonna get rid of bias because, you know, the, the AI could be objective in a way that maybe humans can't.
• 1:51
  David: But they say, well, wait a minute, how did Theis learn?
• 1:54
  David: And you could actually potentially echo in the AI world, what's in the real world, or even entrench.
• 2:01
  David: Even further, where are we?
• 2:03
  David: How big of a problem is that, uh, algorithmic bias?
• 2:06
  David: And are we doing anything to correct it or make it worse
• 2:10
  Sarah Matt: right now?
• 2:11
  Sarah Matt: One of the things I think is interesting is that by creating these algorithms, we're finding our existing biases within the system.
• 2:20
  Sarah Matt: That data that's been collected has probably been collected in a fashion that maybe wasn't the best in the first place, but it was taken as you know.
• 2:28
  Sarah Matt: This is our, our central database.
• 2:31
  Sarah Matt: This is great.
• 2:31
  Sarah Matt: We've done this.
• 2:32
  Sarah Matt: Um, but people are biased.
• 2:34
  Sarah Matt: Systems are biased.
• 2:36
  Sarah Matt: Processes are biased.
• 2:38
  Sarah Matt: I think right now, because of the ability of these new technologies to use so much information and do it so quickly and change and evolve so quickly, folks are getting a little bit more, I'd say, anxious about the fact that.
• 2:50
  Sarah Matt: They can't put their finger on how it's done.
• 2:53
  Sarah Matt: They can't put their finger on all the training data.
• 2:56
  Sarah Matt: What I'd suggest is that most of the data we've probably collected over the course of, you know, we'll say US history in terms of healthcare, has probably been taken in fashions that weren't ideal.
• 3:08
  Sarah Matt: And so if we can identify why things might be swayed one way or the other.
• 3:12
  Sarah Matt: I think that's a great first start because it shows us which populations should be included now that we know that they aren't, or that perhaps scoring or other diagnostics are not working well for specific populations.
• 3:26
  Sarah Matt: So what are we using these for?
• 3:28
  Sarah Matt: Who are they working for best?
• 3:30
  Sarah Matt: What populations perhaps we need more data on, and let's work to fill that gap.