Latest / CareTalk: Healthcare. Unfiltered. / Is AI Helping or Hurting Healthcare Right Now?

Transcript

Is AI Helping or Hurting Healthcare Right Now? · CareTalk: Healthcare. Unfiltered.

• 0:00
  There's three things that I heard last week.
• 0:02
  I had a Nighthawk Radiology owner tell me that AI is actually gonna put him out of business, an insurance executive who told me that the richest hospitals are using AI to upcode in a very clever way, and the residents at Mass General Brigham, they don't even have primary care themselves.
• 0:18
  So what is AI doing in healthcare right now, and where is it leading?
• 0:32
  Welcome to Care Talk, America's home for incisive and insightful debate about healthcare business and policy.
• 0:37
  I'm David Williams, president of Health Business Group.
• 0:40
  And I'm John Driscoll, the chair of the UConn Health System.
• 0:43
  David, the, the bots have not replaced us yet.
• 0:47
  What does that mean?
• 0:48
  No.
• 0:48
  You know, there was an article back, I think in 2016, that was talking about how, you know, radiologists were gonna be gone and, you know, everything's gonna be automated.
• 0:55
  Geoffrey Hinton, the father of AI, famously said that within a few years, um, all, uh, radiologists, those guys who do the scans for the people in the hospitals using technology that generates images, images that can be analyzed by computers, would be out, would, would, would, uh, be without jobs within, I think, five years.
• 1:20
  And Geoffrey Hinton, I think he may have even won the Nobel Prize in technology.
• 1:25
  He's the father of something called, uh, convoluted neural networks, and convoluted neural networks and, which is the fundamental computer architecture that all of these AI super, uh, frontier models are currently running on.
• 1:42
  So the guy, the brain behind the brains turned out to be wrong.
• 1:47
  'Cause David, are there radiologists still being em- being employed?
• 1:51
  John, there are, and I think that, you know, whoever said that, uh, you know, 10 years later and the radiologists are still driving their gas cars to, to work.
• 1:58
  So you know, it's like when they talk about predicting the stock market, you know, they say like, "If you're gonna give a number, don't give a date." Better, David.
• 2:04
  We actually employ more radiologists today.
• 2:08
  We-- So with all of the technology, it's sort of the ul- the ultimate problem with healthcare, the better we get, the more we spend.
• 2:15
  Yeah.
• 2:15
  And in the case of radiology, the more we, the more people we employ.
• 2:19
  All right.
• 2:19
  So John, let's, let's go back to, you know, let's go back to what I heard though from this guy.
• 2:25
  So he's had a lot of time.
• 2:26
  He has a, a nighthawking business.
• 2:28
  So what that means is that when someone comes into the emergency room, typically there's a CT scan done, and someone needs to read it immediately, but there may not be anybody around at the emergency room.
• 2:39
  They may be busy, whatever m- the case may be.
• 2:41
  So there's a way, you know, you outsource it and you have someone else take a look at it, and they do a preliminary read, and then radiologists may come in and do the r- the, the full read the next morning.
• 2:50
  So what I'm hearing, this is from a guy who's made a ton of money doing this over the past decade or so, is that, you know what?
• 2:56
  Now the AI is actually good enough, and he thinks he's gonna be out of business, that the AI can do the preliminary reads as well as the, uh, the radiologist.
• 3:05
  So we actually do think it's happening, so I'm gonna put down and say the time, time is actually coming.
• 3:10
  Not a buyer.
• 3:11
  Not a buyer, Dave.
• 3:13
  I think that the reality is we aren't producing enough doctors.
• 3:17
  We have an aging population.
• 3:19
  We've got a number of physician and nursing and care deserts in America, increasingly in rural and urban areas.
• 3:27
  I think what AI is gonna allow us to do is get more out of the ra- many radiologists, the… And we will need more because the other interesting phenomenon is that more and more technologies, scanning, uh, PET scanning, CAT scanning, X-ray machines, all of the innovation around insights that come from scans are gonna create more and more of a need for a doctor to be in the middle because we are, we are far from w- from the day, and I don't think I'm ever gonna be comfortable with w- where a human is not a do- a, a physician, a clinical expert, a human isn't in the loop around decisions like do you have cancer?
• 4:10
  Do you have a, a brain aneurysm?
• 4:13
  I think you really always want that, that, that interpret- that m- that human doctor in the middle to interpret it, and I think that we're gonna need more of them.
• 4:21
  We're not gonna have enough of them, and technology's gonna be our friend.
• 4:24
  Well, w- relax, Dave.
• 4:27
  You should, you should, you should embrace some of this with some warmth.
• 4:31
  Well, John, the warmth is there.
• 4:33
  You know, I, uh, I told my wife once that she should relax, and you know what she said?
• 4:36
  She says, "I'm not the relaxing type." So, uh, maybe it rubbed off a little bit.
• 4:41
  I, I feel I am a relaxing type, John.
• 4:42
  I think you, I think you may need some relationship guidance.
• 4:44
  I, I think- Yeah telling your wife to relax is probably not a wise long-term- I only- … str- strategy for you at least.
• 4:49
  I only mentioned it once.
• 4:50
  Now, John, yeah, but I'm also feeling like there's a good opportunity, which is I wanna see if maybe you'll buy this radiology company, uh, since there seem to be different expectations.
• 4:58
  But let's talk about this- Not me, man.
• 5:00
  Let's talk about this business of having, you know, always having the need for physicians.
• 5:04
  Now, one, I can't mention who the radiologist was, but I can mention where I was at this conference and, uh, Zach Kohane from, uh, Boston Children's, who's, uh, a real leader in the whole AI world, FHIR, uh, et cetera, data.
• 5:21
  He mentioned that at Mass General Brigham, the residents themselves don't have primary care physicians, right?
• 5:28
  So we talked about this before when we talked about having, you know, the AI-based physicians do the triage for somebody that doesn't have a primary care physician.
• 5:35
  So the, the doctors themselves in training, they can't get primary care physicians, so it certainly puts them face to face right into this, uh, AI world.
• 5:44
  First of all, there's a lot to be worried about when we have models that are being adopted really fast and, you know, 60 to 70% of all doctors are using tools like OpenEvidence and Doximity and others to actually in the… Or, or even the frontier models like Open, Op- uh, uh, Chat or Anthropic's tools to answer questions and to, and to help them accelerate getting answers to what they're doing.
• 6:10
  So we're sort of in the middle of a, uh, a flood of AI tools before we've thought through the regulatory and oversight gates to make sure that they are providing the right advice at the right time.
• 6:25
  And so I think there's-- That to me raises some questions as to whether in this rush to scale that the AI companies are doing, whether it's worth us spending a bit more time thinking about how to do it right.
• 6:39
  Let's, so let's-- let me make that point.
• 6:42
  But part of the reason why people are accessing the tools so quickly, individually and from an enterprise level, 'cause hospitals are also buying a lot of AI stuff, ambient listening tools, revenue cycle software, um, some of which, you know, one of the companies I'm involved in, Waystar.
• 6:59
  Um, and it's because they are, uh, they are, they are struggling with their own abilities to serve the people they care for with the tools available and the talent available, and they're using technology to augment what they're doing.
• 7:14
  And so as long as it's augmentation as opposed to replacement, Dave, I'm kinda for it.
• 7:21
  Um, what's your alternative to a world where we don't have enough supply of clinicians and there's a, there's a, there's a growing need for clinical support?
• 7:32
  Yeah, I mean, it really is interesting to tease out the differences, say, among the different professions within healthcare and then over time.
• 7:40
  So let's look at it now.
• 7:41
  So we talked about radiologists, right?
• 7:43
  And there's a different view about whether there's still a shortage of radiologists.
• 7:46
  Is that 'cause people aren't going into it 'cause they think the end is near, or is it really upon us?
• 7:51
  Then in, we're talking about primary care, which we know is a big issue, and it's not really theoretical.
• 7:55
  Like if you, you can't get an appointment or you can't even get a doctor, never mind, you know, just the, the appointment.
• 8:01
  So you're gonna use AI, plus it may give you a clearer and faster, um, answer.
• 8:07
  But then you've got other areas like with nursing.
• 8:11
  So the nurses union has been behind some of these moves in California, for example, to regulate or reduce the use of AI, citing patient safety, but perhaps also concerned about their own role or their own job security.
• 8:23
  And then you've got in Boston, the nurses from Mass General Brigham, who are highly paid by any kind of objective standard, talking about going on an open-ended strike in a couple weeks.
• 8:34
  So I think the, the, it's washing across us in, you know, in different ways.
• 8:39
  I, I mean, I think that the nurses unions have a legitimate beef in that The clinician should have a, a say in how the tools are used to augment their work, and that you don't have some big brother, you know, one-eyed computer-driven babysitter kind of managing their work, and that they have a-- they're at the table to negotiate what an appropriate amount of support, again, human support is needed because I think humans need humans.
• 9:10
  And I, I, I, I kind of think the unions, regardless of how well they're paid, need to be at the table to, to negotiate the terms of how that augmentation happens, and I think docs should as well.
• 9:22
  The, uh, again, we're in the midst of a flood of AI use and AI tools, and I do worry.
• 9:30
  I mean, that's, that's one of the benefits of America, you know, open market and all that.
• 9:33
  But I think when you deal, when you're talking about clinical care and the potential elimination of clinical connection and human connection, I think we ought to be really careful.
• 9:43
  I, I'm, I'm sympathetic to the nurses.
• 9:45
  Uh, what, what, what do you-- You sort of are suggesting that you're, you're, you are less.
• 9:50
  And how do you, how do you, how do you think about that?
• 9:52
  What, what role, I guess, Dave, should the nurses play in helping figure out how to use this technology in a way that helps and grows the amount of support we provide for patients?
• 10:04
  I'm not super familiar with all the things going on in, in California, but I did see that Governor , I think, signed three AI-related bills that, that have to do with healthcare in the past week and, and vetoed a couple of others.
• 10:17
  And as I understand it, um, you know, some of this is to try to, uh, prevent AI from representing itself as a licensed clinician, which it's been doing.
• 10:28
  So that sounds, that sounds reasonable.
• 10:30
  What?
• 10:32
  You're not allowed to do that.
• 10:32
  You're not supposed to do that Dr. AI?
• 10:35
  Well, he didn't say that.
• 10:35
  He may just call it Dr., Dr. Al, uh, or something like that and, and not notice how you spelled it.
• 10:41
  But no, there's, um, you know, there's--'cause that started to, to happen in some places, and you know, when you're online, you deal… It happens all the time in customer service interactions.
• 10:49
  You think you're talking to a person, but it's really a bot.
• 10:52
  But it seemed as though there was an area for when you're talking about behavioral health, where there maybe it wasn't-- He vetoed a bill there which had sought to put in a similar restriction, and it wasn't as strict, and I think that has to do with perhaps just the, the huge shortage on the mental health side.
• 11:08
  So it's one thing to say, "Hey, don't step in here in front of a nurse we may have in front of us," but it's another one to say, "Huh, if we regulate this too tightly, maybe we're not gonna be able to actually, um, get care 'cause there's no people around." And then there's the question about bias, right?
• 11:24
  So that's a concern that nurses have, and I think anybody at the front line of care and, and also on the policy side is to say, you know, depending on how these systems are trained, it may exacerbate existing disparities.
• 11:35
  It may introduce, um, new ones.
• 11:38
  So these, these are the sort of things that at least in California are being debated and legislated, whereas I think in Washington it's basically, uh, hands off.
• 11:46
  Well, there's a, there's definitely a, a substantial amount of red and blue state interest at the state level in Putting in some sort of guidelines and regulatory, um, guardrails around the unlimited use of AI.
• 12:00
  I have not heard of anyone replacing a doctor with a bot, um, you know, as a licensed physician.
• 12:07
  I mean, those, those bots haven't gone through medical school.
• 12:10
  Um, there is, uh, in Utah, there was some regulatory flexibility around, um, reordering, um, uh, prescription drugs, yeah, th- through Doctronic, but I have not heard of the, you know, the, the, the, the Franken bot of a, of an AI doc.
• 12:31
  Um, but I, I think that the, the, that, that restricting that makes a great deal of sense.
• 12:37
  Again, I'm very focused on, like, let's make sure the humans are in the loop here while we're still trying to figure this stuff out.
• 12:45
  The frontier models, the technology of the frontier models is such, using convoluted neural networks or these, these, the architecture and very fast chips, um, are taking all of the knowledge they can s- relevant knowledge they can scrape from the internet to come up with an answer or response.
• 13:02
  And the mo- models and the technologies are ge- are moving very, very fast.
• 13:05
  As anyone who's following the news can tell you, we are the fastest growing technol- it's one of the fastest growing technologies the world has ever seen.
• 13:14
  Um, the challenge there is that it is hard to back propagate or understand many times exactly how the, the bot or the AI tool got to the conclusion it got to.
• 13:28
  And, and it, in healthcare, you know, the difference between being 95% right could be that you're dead.
• 13:34
  I mean, like the, the, the, the, the challenge is you really do wanna know how, what the rationale and sourcing and, uh, and you can… There are some tools that do that, but many do not.
• 13:45
  They just want to give you an answer as fast as the, the bot honestly wants to perform to your expectation, which is, "I ha- I have a question.
• 13:52
  I'm asking it right now.
• 13:54
  Please give it to me." Um, you know, that's a, that's a, that's a challenge.
• 13:59
  You al- and in healthcare, you almost have to go to the actual, get the bot to go back to the sources.
• 14:04
  So I'm actually in favor of some amount of eleme- of elementary guardrails in the clinical area.
• 14:11
  On the bias area, there's actually been nu- a nu- numerous examples, at least in the last 46 months, where if you ask a, a, a straightforward question, you do come up with some element of bias, and that would make sense given the fact that the bots are learning off of the internet, and the internet isn't, you know, uh, biased even if you, even, even co- uh, conservatively, uh, measured has been shown to pop up, uh, in a way that could distort answers for different populations or subpopulations.
• 14:42
  You know, the thing, Dave, is that in, in healthcare, um, it's not, uh, the, the, the, the, the simple things you worry about.
• 14:51
  It's all the edge cases where someone's got two comorbidities or they're… You've got a rare and orphan disease or you're showing up with a new symptom.
• 15:00
  That's the hard stuff that healthcare needs to solve every day when someone shows up in the emergency room and they have a pain and, and, and is it something that's an appendectomy or they, or they ate something wrong?
• 15:11
  And I think that's the kind of clinical intermediary where we don't wanna take the doctors out, and I think we do need some guardrails John, I'm just shuddering as I hear you talk about that, you know, like that the bot is basing its diagnosis on some crazy that you've read on, on, uh, on Reddit, you know, that's gonna be, that's gonna be out there.
• 15:28
  Let, let- Not me, man Let's, let's s- That was your subreddit.
• 15:31
  Yeah.
• 15:31
  Not mine.
• 15:31
  Let's, let's, uh, let's separate it a little bit into some gross categories.
• 15:35
  Let's say diagnosis, treatment, and then some of the administrative stuff, like on the, on the billing- Sure … and coding.
• 15:41
  A diagnosis is an area where you feel like, you know, there's this kinda diagno- diagnostic odyssey that people go through, and there's a lot of data out there, and I see that's kind of the clearest place where I don't necessarily want the bot just to like diagnose me and that's just, that's it, but I wa- actually want clinicians to rely on that.
• 15:56
  I wanna get all my data in.
• 15:57
  And that's where I think you see in some places like radiology, where the diagnosis is less about judgment and more about dealing with the digital data where you, where you see that first.
• 16:06
  The cl- the treatment side of things, and that's maybe, let's just say grossly where we're talking about the nurses.
• 16:11
  So that changes the role, but it also, in a way, makes the nurses more important relatively.
• 16:17
  The, the doctors ironically may, uh, be surplus before the nurses, and that's part, I think, why we see the nurses, uh, flexing some of their, uh, some of their power.
• 16:26
  And then on the administrative side, where we really are seeing things in a way that's hitting the, the bottom line and the top line, um, there was an article I think in The New York Times recently, the Blue Cross Blue Shield Association… I, like I say, I think, because I don't really read The Times anymore, but the, um, they, they said, the Blue Cross said it's about, you know, almost a billion dollars of added costs over the couple years with no change of care from essentially AI-based upcoding.
• 16:50
  And I was talking to another insurance executive who was mentioning that some of the, uh, the richest hospitals that are already well reimbursed are investing well, and he, they gave an example of in the emergency room, a patient that had a finding where they were, you know, some shortness of breath or whatever, and it was like an incidental finding of hypoxia, but it turns out that codes very well.
• 17:10
  And so you see these sort of upcoding that goes on because the, uh, 'cause the bot noticed it and pushed it.
• 17:15
  That's the area where I think you're seeing a lot of, of real action and impact on medical inflation now.
• 17:21
  And I think, in fact, Dr. Oz has said that AI is gonna start off with having an inflationary, uh, impact.
• 17:27
  Well, let's, let's break it into pieces.
• 17:29
  I think in diagnosis, again, I don't think we wanna just rely on the bot because doctors spend a lot of time making judgment calls about complex situation, and there is no simple situation if it's your family's health.
• 17:44
  Um, in terms of treatment, where I think the, the… And I think the bots can help provide information to doctors who are making that diagnosis.
• 17:53
  In terms of treatment options, I do think that's an area where since it takes over a decade for a new, um, idea, approach, technology, surgery, what have you, to actually become common practice after it has been accepted by as best practice in any medical field I think that's an area where we could use technology to accelerate doctors' and clinicians' learning.
• 18:18
  Uh, in terms of the administrative side, I'm somewhat sympathetic to the insurance companies saying, "Oh my gosh, you know, we're getting bigger bills for what look like the same patients." But let's be honest, the insurance companies have spent a lot of times deferring, denying, delaying those same bills since I think it was something like, you know, it, 90% of, uh, of prior authorizations end up being approved.
• 18:45
  So exactly what are we doing other than deferring stuff?
• 18:48
  So I think what's happening, I know hospitals are finding more Uh, uh, thing, more items that they can bill for when a f- a, a patient shows up and they're sick.
• 19:01
  But these are things that the insurance companies has said they are permitted to be billed for, otherwise they wouldn't be paying it.
• 19:08
  So I think that the, the, there is an arms race going on right now between the insurance company's AI and the hospital's AI, but I think that the, uh, you know, historically the, the hospitals have been on their back foot, and perhaps this is more of a, an evening of the score as it, as, as we get to a more reasonable middle than it is really the, the hospitals being the bad guys here.
• 19:32
  But I speak as the chairman of a hospital system, so perhaps I'm slightly biased.
• 19:37
  Perhaps, John, but not as biased as some of the, uh, the AI based on your, your Reddit posts, which I, which I go back to.
• 19:43
  I wanna take issue a little bit on the diagnosis side, and I am a proponent of AI's use in diagnosis because yeah, there's physicians that spend a lot of time in medical school learning things, but there's a lot of things that are, are rare or are nuanced where we're having a lot of data together is actually helpful.
• 20:02
  And I know, you know, anecdotally many patients who, you know, they find at the end of some number of years, aha, actually they're being treated for the wrong thing, or this is the key to what they were doing.
• 20:13
  And, and that's, those anecdotes are supported by the data as well, and common sense, which is that there's just so much information out there that's all hard to bring together.
• 20:21
  We've now got it digitized, and you wanna make sure that, you know, you're gonna run something through diagnosis.
• 20:26
  One of the issues, uh, early on with, uh, clinical decision… All right, John, go ahead.
• 20:31
  You just keep throwing all this stuff at me and thus not giving me a chance to respond.
• 20:36
  But remember the example in Bob Wacht-, Dr.
• 20:38
  Bob Wachter's book, um, on AI and healthcare, where he, he documented the fact that in Australia, you know, uh, your r- your buddies in radiology reach a certain level of competence without technology.
• 20:53
  When they added technology, they reached a higher level of, of competence, but when they took the technology away, that competence went below the original level of competence 'cause they'd lost the, the comfort and the skills that they had honed on their own.
• 21:10
  So really the question isn't either/or, it's both/and.
• 21:12
  I think you wanna train doctors to get good at diagnosis and then train them with technology to get better.
• 21:19
  But if you don't get that balance right, uh, we'll have dumb doctors and eventually bad models.
• 21:24
  I mean, I think that, that, that's, that's what that research would suggest.
• 21:28
  So I think that actually your desire to kind of pin me as the, uh, as the, as the, as the, as the v- as the AI Luddite is, is, is you, you missed on that one, that shot I was trying to pin you as the Reddit queen.
• 21:43
  John, that brings up so many interesting issues, um, you know, that go in, in, in different directions here.
• 21:48
  So one is related to the de-skilling, potential de-skilling of physicians.
• 21:52
  Which we gotta worry about.
• 21:53
  And this is also tr- yeah, which is also true when I was seeing this, um, point about AI more broadly.
• 21:59
  MIT did a study, and I think they coined the term cognitive surrender, which is when you hit a wall, when you just hit, like the first time you hit something hard, you're like, "Well, we'll just put it in and see what, you know, what Claude or ChatGPT has to say about it." I, I notice the same thing even from, you know, from driving and using the GPS.
• 22:15
  I think this is also well documented.
• 22:17
  Um, now, you know, you, once you use the GPS all the time, you forget how to drive if you, if you had to do that.
• 22:23
  On the other hand, you can go a lot more places and it's more efficient.
• 22:26
  So you want the physicians to use the technology well.
• 22:29
  You still want them to be there and in the loop, but it's not so obvious exactly which, you know, which technologies are gonna be the most useful and which direction we actually wanna go, so that it's less about maybe replacing the physician and saying, "Hey, I can do that better," but tools to actually augment what people can do.
• 22:48
  And I think that's actually kind of the decision in AI development.
• 22:51
  Well, I think the most important thing you've suggested, although it took you some time to get there, is that we need to train and learn and determine whether, whether what we're missing is something that really is lost or whether we wanna maintain it.
• 23:06
  Like I can still read a map even if you can't.
• 23:09
  Um, and I think that that's because I spend a lot more time perhaps in the outdoors where I, the GPS is a little tricky But in, when I'm driving in a car, actually I kind of am happy to defer it, it sounds like as you are, to the car.
• 23:23
  But when we get into healthcare, I do think we, as fast as this technology is moving, I think studying it so we understand those nuances and can make those choices is critical.
• 23:34
  You know, before we go f- too far down the, uh, the rabbit or Reddit hole here, the, if you just look even before AI, you hear an older generation of physicians who were trained before the ubiquity of CT scanners, ultrasound, and so on, and being able to order tests, was they could do more kind of hands-on and actually understand what's going on with a patient.
• 23:55
  And the older physicians will, will say,"Geez, some of these younger guys just know how to," or guys and gals- Order tests … just know how to order a test.
• 24:02
  So these are things to, to think about.
• 24:04
  And John, the good news is we have this show on every week, and I'll be, you know, in addition to reading all of your posts, I will be, uh, uh, happy to, uh, to interact with you again here face to face, if that in fact is your face, um, and discuss AI and where it, where it goes.
• 24:20
  So before we go any, any greater or worse, John, I'm gonna say that's it for another episode of Care Talk.
• 24:25
  We've been talking about the best and the worst of AI, what's happening, uh, in healthcare, and what we learned, uh, this week.
• 24:32
  I'm David Williams, President of Health Business Group.
• 24:34
  And I'm John Driscoll, the chairman of the UConn Health System.
• 24:37
  If you like what you heard or you didn't, we'd love you to subscribe on your favorite service.