Latest / Becker’s Healthcare Podcast / Beyond the Doomerism: What Safe Healthcare AI Looks Like in Practice

Transcript

Beyond the Doomerism: What Safe Healthcare AI Looks Like in Practice · Becker’s Healthcare Podcast

• 0:00
  - Hello, everyone. This is Molly Gamble with Becker's
• 0:01
  - Healthcare. Thank you for tuning in to the
• 0:03
  - Becker's Healthcare podcast.
• 0:05
  - Today, we're going to talk about beyond the
• 0:07
  - doomerism,
• 0:08
  - what safe health care AI looks like in
• 0:11
  - practice.
• 0:12
  - Joining me for today's conversation is Munjal Shah,
• 0:15
  - cofounder and CEO of Hippocratic AI. Munjal, thanks
• 0:19
  - for joining me today and being my participant,
• 0:22
  - my guest for this big topic.
• 0:24
  - Oh, yeah. I'm, really excited to be here.
• 0:26
  - Thanks for having me.
• 0:28
  - Likewise.
• 0:29
  - For our audience who hasn't caught up with
• 0:31
  - us before, Munjal, can you share a bit
• 0:33
  - more about yourself and your work in health
• 0:35
  - care just to get them up to speed?
• 0:38
  - Yeah. I'm a serial entrepreneur. This is actually
• 0:40
  - my fourth company. So,
• 0:42
  - I'm a glutton for startup punishment, but,
• 0:45
  - and
• 0:46
  - what we've built here at Hippocratic is really
• 0:48
  - we build,
• 0:50
  - agents that call patients on the phone and
• 0:52
  - conduct clinical patient facing activities,
• 0:55
  - largely within the scope of a registered nurse
• 0:57
  - and and below. So we don't diagnose and
• 1:00
  - we don't prescribe, but, but we do everything
• 1:02
  - else.
• 1:03
  - Okay.
• 1:04
  - And we are sitting down to discuss this
• 1:06
  - at a pretty interesting time. The conversation about
• 1:09
  - AI safety
• 1:10
  - right now is is so loud,
• 1:13
  - and a lot of this discussion and these
• 1:15
  - headlines
• 1:16
  - and consternation, it's happening in altitude
• 1:19
  - among Frontier Labs, policymakers
• 1:22
  - that are somewhat removed from the realities of
• 1:25
  - delivering care.
• 1:26
  - You've seen leaders from OpenAI and Anthropic specifically
• 1:30
  - call for shared regulation
• 1:31
  - to ensure greater safety.
• 1:34
  - You, on the other hand, have built Hippocratic
• 1:37
  - specifically around deploying AI safely in health care.
• 1:39
  - This has been part of your your mission,
• 1:42
  - your charter since day one. So maybe to
• 1:44
  - to start us off, how is the work
• 1:46
  - you and your partners are doing with clinical
• 1:48
  - AI agents
• 1:50
  - just fundamentally different from what we're hearing out
• 1:52
  - of the Frontier Labs lately?
• 1:55
  - Yeah. You know, we took a very different
• 1:56
  - approach.
• 1:57
  - I think when we started the company, you
• 1:59
  - know, we named it Hippocratic. We made the
• 2:00
  - tagline do no harm. And we said... I
• 2:03
  - actually said to the investors, I said, I'm
• 2:05
  - not going to ship until it's safer than
• 2:07
  - a human.
• 2:09
  - And, they're like, great. When when is that?
• 2:11
  - And I was like, I'm... I don't know.
• 2:13
  - I'm like, and if you're not comfortable with
• 2:15
  - that, don't give me your money.
• 2:17
  - And I think, you know, today,
• 2:21
  - and and if you look at the Frontier
• 2:22
  - Labs, you know, they're like, well, we need
• 2:23
  - to all slow down and make it safe.
• 2:25
  - I'm like, you coulda just made it safe
• 2:27
  - yourself.
• 2:28
  - Like, that's what we did. We just didn't
• 2:30
  - ship. It took us eighteen months to actually
• 2:32
  - cross the threshold of being safe enough to
• 2:34
  - even begin our first patient facing calls. And,
• 2:38
  - it's just a a personal responsibility.
• 2:41
  - Like, I don't I don't think you need
• 2:43
  - to have some... You know, it's like it's
• 2:45
  - like peer pressure in in middle school. Like,
• 2:47
  - you don't need to do the right thing
• 2:49
  - only because if the other guy does the
• 2:50
  - right thing, you just need to do the
• 2:52
  - right thing. And so we approach this problem
• 2:55
  - very differently from a place of personal responsibility
• 2:58
  - and commitment because... Look. I mean, we're all
• 3:01
  - patients. We all have family members who are
• 3:03
  - patients.
• 3:04
  - Like, you you just wanna build something that
• 3:07
  - truly helps people and doesn't hurt them, and
• 3:10
  - and the only way to ensure that is
• 3:12
  - is to take the care upfront.
• 3:15
  - Let's unpack that a little bit. When you
• 3:16
  - say
• 3:18
  - safe, let's make it more concrete. You mentioned
• 3:20
  - it took you eighteen months to to cross
• 3:22
  - a threshold into what you would deem as
• 3:24
  - a safe product or safe AI,
• 3:27
  - with the name Hippocratic and do no harm
• 3:29
  - in... Intact.
• 3:30
  - But when you say you're engineering for safety
• 3:32
  - back in that that time and still today,
• 3:35
  - what does that actually involve? Can you speak
• 3:37
  - to that in a bit more detail?
• 3:39
  - Yeah. You know, safety is a multilayer
• 3:41
  - thing. In fact, you know, Jensen said this
• 3:43
  - best when this whole debate was occurring, and
• 3:45
  - he said,
• 3:46
  - safety is about engineering.
• 3:48
  - If you want safety, you engineer safety. But
• 3:50
  - the way you do it is redundancy
• 3:53
  - in every way you can think of. So
• 3:55
  - we start first where every employee takes the
• 3:57
  - Hippocratic
• 3:58
  - Oath,
• 3:59
  - which is a version of the original one,
• 4:01
  - but one we made for for kind of
• 4:04
  - our company.
• 4:05
  - But it says, hey. Look. I'll I'll make
• 4:06
  - sure I put patients first. Why? Because you
• 4:08
  - want people in the right psychology.
• 4:11
  - Second, we decided that,
• 4:13
  - what we would do is limit our scope.
• 4:15
  - You know? People say they wanna be safe.
• 4:17
  - I'm like, look. The best measure of being
• 4:18
  - safe is
• 4:20
  - self editing. You know? And we said we're
• 4:22
  - not gonna diagnose. We're not gonna prescribe, but
• 4:24
  - we also went deeper. We said we're not
• 4:26
  - gonna do kids under two. We're not gonna
• 4:28
  - do high risk pregnancies. We're not gonna do
• 4:30
  - mental health. We're not gonna do hospice care.
• 4:33
  - Like, there's just areas that we knew
• 4:35
  - it just wasn't the right,
• 4:38
  - you know, area for AI to be used.
• 4:40
  - And then we said, alright. We're not gonna
• 4:43
  - build one LLM. We're gonna build 31 LLMs
• 4:45
  - and have 30 of the LMs double check
• 4:48
  - the main LLM all the time. In fact,
• 4:49
  - some of them double check another LM that
• 4:51
  - double checks the main LM.
• 4:54
  - And we said you need this redundancy of
• 4:56
  - systems. I mean, that's how you design an
• 4:57
  - airplane. You don't just have one way to
• 4:59
  - control the rudder. You know, you have fly
• 5:01
  - by wire. You also have a mechanical connection
• 5:03
  - in some planes. You have a wired connection.
• 5:05
  - Like, you just because if one goes down,
• 5:07
  - you wanna make sure the other two were
• 5:09
  - there. And so we took a similar approach.
• 5:11
  - Then we said, hey.
• 5:13
  - That's not enough. Let's make sure it'll transfer
• 5:15
  - the call to a human if needed if
• 5:17
  - you're having chest pains or shortness of breath.
• 5:19
  - And so we built an escalation engine that'll
• 5:22
  - actually transfer to humans. We've now done that
• 5:24
  - 38,000
• 5:26
  - times. So we've done, you know, 300,000,000
• 5:28
  - plus patient interactions,
• 5:31
  - but,
• 5:32
  - in those interactions, 31,000 times, we had to...
• 5:35
  - Or 38,000 times, we had to transfer it
• 5:37
  - to a human. And so,
• 5:39
  - and then we said, let's output test. Let's
• 5:41
  - hire 7,000 nurses to make,
• 5:44
  - test calls to the AI, and let's not
• 5:46
  - ship until,
• 5:48
  - and in those test calls, one out of
• 5:49
  - 50 times, they got a human on the
• 5:50
  - other line, and we said, alright. We're not
• 5:52
  - shipping till we're safe as the human.
• 5:55
  - And and then we monitor the first thousand
• 5:58
  - calls, every single call within an hour,
• 6:01
  - and call the patient back. If there's a
• 6:03
  - problem, we sample half a percent of the
• 6:05
  - calls after it's live. Like, we've just done
• 6:07
  - every last thing.
• 6:10
  - You know, this is belt, suspenders,
• 6:13
  - whatever comes after suspenders on your jacket. Like,
• 6:17
  - this is this is every last thing we
• 6:19
  - could think of to just ensure safety.
• 6:21
  - Yeah. It sounds like restraint and redundancy has
• 6:24
  - been
• 6:25
  - two really important things among others. But I
• 6:28
  - I guess just to turn and flip the
• 6:29
  - seats a little bit, Munjal, like, if you
• 6:31
  - were sitting in a CMIOs
• 6:33
  - or CIO seat, what are the questions you'd
• 6:36
  - want a vendor to answer before letting their
• 6:38
  - agents anywhere near patients?
• 6:41
  - Yeah. You know, I think that,
• 6:44
  - there's this trope out there that's constantly, like,
• 6:47
  - all the models are commodities.
• 6:49
  - I don't think so.
• 6:51
  - Like, there's a reason Harvey in legal is
• 6:53
  - doing really well
• 6:56
  - despite Claude legal launching because people want a
• 7:00
  - built for purpose system. Because when you get
• 7:02
  - a built for purpose system,
• 7:04
  - it closes the last two, 3%,
• 7:07
  - last five, 10%, which in safety is a
• 7:10
  - big deal.
• 7:11
  - Like, you can't tell your boss, hey. I
• 7:13
  - I bought this thing. It was half the
• 7:14
  - price, but I'm gonna kill 2% more people.
• 7:16
  - Like like, that's not okay.
• 7:18
  - It might be okay in, you know, taking
• 7:20
  - your order at Taco Bell with the voice
• 7:22
  - AI. Like, okay. 2% of the time, I
• 7:24
  - gave you three tacos instead of two. Okay.
• 7:27
  - Fine. Like, you know, it's not the end
• 7:28
  - of the world. But I think for us,
• 7:31
  - the first question they need to ask is,
• 7:33
  - you know, what's the architecture?
• 7:35
  - Is it a multimodel safety architecture?
• 7:38
  - How has it been tested? Did they do
• 7:40
  - output testing?
• 7:41
  - So some of the points I just made.
• 7:43
  - But one of the ones that I find
• 7:45
  - everybody is missing at the moment
• 7:48
  - is they've forgotten why this revolution happened.
• 7:51
  - It's called the large language model.
• 7:55
  - And yet, so many times today, people are
• 7:58
  - not asking how big are the models you're
• 8:00
  - using.
• 8:01
  - So,
• 8:02
  - you know, most people don't know this, but...
• 8:04
  - Or have forgotten, but we we measure models
• 8:07
  - by the number of parameters they have. The
• 8:09
  - number of parameters to use is quite literally
• 8:12
  - the number of connections between the neurons in
• 8:14
  - the neural network.
• 8:16
  - And just like the human brain, the more
• 8:18
  - neurons an animal has,
• 8:20
  - the more connections it has and the smarter
• 8:22
  - it is. Right? Meaning,
• 8:24
  - you know, your mouse brain is not as
• 8:26
  - smart as your dog brain, is not as
• 8:28
  - smart as your dolphin brain,
• 8:30
  - and obviously onto the chimpanzee and and humans.
• 8:33
  - And so
• 8:35
  - this is actually the breakthrough that happened where
• 8:37
  - we got the first models at 7,000,000,030
• 8:40
  - parameters that really created breakthroughs, and then CHAT
• 8:43
  - GPT came
• 8:44
  - and was a much higher number. But, you
• 8:46
  - know, today, the the fable you use is
• 8:49
  - 10... Is rumored to be. They haven't published
• 8:51
  - the numbers, which is why we lost track,
• 8:53
  - but it's rumored to be 10,000,000,000,000
• 8:55
  - parameters.
• 8:57
  - Now why do you care in health care
• 8:58
  - about the size of the brain that's being
• 9:00
  - used in the model? In fact, a lot
• 9:02
  - of companies are running around saying, we have
• 9:04
  - small language models. You know they're really better.
• 9:06
  - I'm like, no. They're not. The whole darn
• 9:07
  - breakthrough was large language models, guys.
• 9:10
  - And the reason is that we found the
• 9:12
  - large language models catch subtlety.
• 9:15
  - So if I ask you, you know, Molly,
• 9:17
  - are you short of breath? And you don't
• 9:19
  - say yes. You don't say no. You say,
• 9:21
  - feels like there's an elephant on my chest.
• 9:24
  - Our model better darn well say that's a
• 9:27
  - yes.
• 9:28
  - Shoot. You know? And,
• 9:30
  - what we found is the small models will
• 9:32
  - miss that. So there's a lot of subtly,
• 9:34
  - a lot of ambiguity,
• 9:36
  - that's critical in health care that you want.
• 9:38
  - And so you want this intelligence. In fact,
• 9:40
  - the new mythos,
• 9:42
  - one that's finding all those bugs, is rumored
• 9:44
  - to be 27,000,000,000,000
• 9:45
  - parameters.
• 9:46
  - And so, you know, at Hippocratic, we're about
• 9:49
  - now 5,100,000,000,000
• 9:51
  - parameters, which
• 9:53
  - I would use more except you have to
• 9:54
  - make it fast enough for voice, and so
• 9:56
  - that's why we use 30 in parallel. So,
• 9:59
  - is we we basically push the number of
• 10:01
  - parameters up by firing them in parallel because
• 10:04
  - the voice latency means you you can't take
• 10:06
  - too long. And Fable takes a long time.
• 10:08
  - Like, you put in something. The other day,
• 10:09
  - I put in something, and,
• 10:11
  - you know, I had enough time to go
• 10:12
  - get a cappuccino and come back before it
• 10:14
  - was done. Like, you can't use that in
• 10:16
  - voice, so you you have an inherent trade
• 10:18
  - off there. But the size of the model
• 10:21
  - matters because it's basically the amount of intelligence
• 10:23
  - you're bringing to bear.
• 10:25
  - Mhmm. You just touched on something, which is
• 10:27
  - that this work... This... The the work of
• 10:29
  - safety
• 10:30
  - comes intention to other goals to speed and
• 10:33
  - or... And urgency being one of them.
• 10:36
  - But Munjal, I mean, nearly every AI company
• 10:39
  - in health care right now, it... It's making
• 10:41
  - some version of of the safety claim. And
• 10:43
  - for a leader
• 10:45
  - trying to separate
• 10:46
  - real rigor, like you're discussing, from good marketing,
• 10:50
  - are there any sniff test or litmus test
• 10:53
  - that you would put forward as a way
• 10:54
  - of verifying what what you're... They're actually hearing
• 10:57
  - about safety?
• 11:00
  - The thing about safety is
• 11:03
  - it... It's only measurable backward looking at some
• 11:06
  - point. Right? So,
• 11:08
  - like, you are safe because you have been
• 11:10
  - safe.
• 11:11
  - And so,
• 11:13
  - you know, we've now done these 300,000,000 patient
• 11:15
  - interactions,
• 11:17
  - and we've not had one major safety incident.
• 11:20
  - So I'm like, you can bet on somebody
• 11:22
  - who maybe
• 11:23
  - hasn't proven that, but you don't. Like, why
• 11:26
  - take that chance if you don't have to?
• 11:31
  - Second,
• 11:32
  - I think that
• 11:33
  - there's another element,
• 11:36
  - and you can get into the architecture, and
• 11:37
  - you can get into the model size, and
• 11:39
  - you can get into some of those properties.
• 11:41
  - But
• 11:42
  - in the end, to me, there's only two
• 11:44
  - things that really will help you. One, did
• 11:46
  - we do output testing?
• 11:48
  - Did we test what's coming out of the
• 11:49
  - model, not just say it's safe because of
• 11:51
  - what went in?
• 11:52
  - You know, drugs are done this way. They're
• 11:54
  - not... The FDA is not like, oh, yeah.
• 11:55
  - You you built that drug. You're just following
• 11:57
  - the process, so, therefore, we approve it. Like,
• 11:59
  - they're like, no. No. No. We're gonna test
• 12:01
  - the output.
• 12:02
  - Mhmm. So look for the output testing. Look
• 12:04
  - for the history of it. But, you know,
• 12:07
  - this is a new field, so there's a
• 12:08
  - ton of history for anybody.
• 12:12
  - And then I think that, you know, the
• 12:14
  - other element,
• 12:16
  - if you're a leader just trying to parse
• 12:18
  - through it all,
• 12:21
  - is, you know, have your own teams
• 12:25
  - go at it clinically
• 12:27
  - and look for corner cases
• 12:30
  - that are there. I mean, I'll I'll tell
• 12:31
  - you an example. We had a patient calling
• 12:32
  - in
• 12:34
  - on,
• 12:35
  - he was, calling in a schedule with a
• 12:37
  - neurologist.
• 12:39
  - He said, oh, I need an appointment with
• 12:40
  - a neurologist, and and the AI is like,
• 12:42
  - hey. Why why do you need an appointment?
• 12:43
  - And he's like,
• 12:45
  - or or what? Why do you need an
• 12:46
  - appointment, and what type of appointment do you
• 12:47
  - need? And he's like, oh, well, I was
• 12:49
  - struck by lightning yesterday at work.
• 12:52
  - Like, so you would think scheduling doesn't need
• 12:54
  - a clinically safe thing, but, actually, every now
• 12:57
  - and then, it does.
• 12:58
  - And you don't... You know? And that that
• 13:00
  - one's not so easy because it's not like
• 13:01
  - the answer is he should've called 911.
• 13:04
  - It's the next day. Like, he's he's okay.
• 13:07
  - Mhmm.
• 13:08
  - But at the same time, you better not
• 13:09
  - book him an appointment with a neurologist six
• 13:11
  - weeks from now.
• 13:13
  - Like, somebody's gotta squeeze this guy in tomorrow.
• 13:15
  - Right? So,
• 13:18
  - I think that there's a lot of different
• 13:20
  - elements, but in the end,
• 13:22
  - you really wanna build a safer system. You
• 13:24
  - really need to realize almost every call in
• 13:26
  - health care
• 13:28
  - can... Has a risk of being a clinical
• 13:30
  - call. So even when you think
• 13:33
  - it's a nonclinical use case, you're you're not
• 13:36
  - always right, and so you you want this
• 13:38
  - capability to be there at any time to
• 13:40
  - jump in.
• 13:41
  - Mhmm. Mhmm.
• 13:42
  - The stakes are just so much... They're already
• 13:45
  - higher in this... In in health care. And
• 13:47
  - then even there, there can still be, like,
• 13:50
  - being struck by lightning. My goodness. Like, what
• 13:52
  - a
• 13:53
  - cascading... The stakes are just even higher than
• 13:56
  - you could even imagine.
• 13:57
  - And and this is where you need the
• 13:59
  - judgment. Right? Because Right. You you think struck
• 14:02
  - by lightning means scheduled this way is anywhere
• 14:05
  - in the protocol?
• 14:06
  - Like like, there's no business rule that was,
• 14:08
  - if struck by lightning, then do this. Like,
• 14:11
  - you know, even the escalation protocols don't have
• 14:14
  - probably struck by lightning. So
• 14:16
  - the model just had to have judgment. And,
• 14:18
  - basically, the bigger the model, the better the
• 14:20
  - judgment.
• 14:22
  - Working off of of that story, I mean,
• 14:24
  - so much of the public,
• 14:27
  - narrative or commentary or attention
• 14:31
  - right now is really looking at advanced AI
• 14:33
  - as something to fear.
• 14:35
  - It sounds like from these stories and the
• 14:37
  - work that you've been putting forward at Hippocratic
• 14:40
  - AI, you and your team are just clearly
• 14:42
  - operating from a different premise altogether.
• 14:44
  - What would you say, Manjal, is the counter
• 14:46
  - case you'd make to that type of doomerism?
• 14:49
  - And maybe more to the point, what are
• 14:51
  - you seeing in live deployments that actually supports
• 14:54
  - optimism?
• 14:56
  - Yeah.
• 14:57
  - This thing drives me nuts.
• 15:00
  - You know, the... For... I'm I'm gonna start
• 15:02
  - at the highest level.
• 15:04
  - Why are we listening to doomers? These people
• 15:06
  - are owing 5,000
• 15:07
  - in the history of the world. They've never
• 15:09
  - been right because we're all here.
• 15:12
  - I'm like, oh, but this time for sure,
• 15:15
  - like, you know, it's it's... There's must be
• 15:17
  - something inherent about human beings beings that we
• 15:20
  - want to believe the doomer, framework because literally
• 15:22
  - these people have always been wrong. And I
• 15:25
  - I I thought about it deeply. I was
• 15:27
  - like, why are they always wrong? I mean,
• 15:30
  - as it's on the surface, I could see
• 15:32
  - why some of these arguments, you know, might
• 15:34
  - play out. It's... I actually think it's because
• 15:36
  - you can never predict the secondary and tertiary
• 15:39
  - implications,
• 15:41
  - only the first. So you're like like, let's
• 15:43
  - say you and I had owned farms on...
• 15:46
  - When the tractor came out in the industrial
• 15:48
  - revolution. People might have said, oh my god.
• 15:50
  - The farmer's gonna do so amazing.
• 15:53
  - You guys own all the good fertile land.
• 15:55
  - Like, you're gonna make a fortune now. You're
• 15:57
  - gonna make food for so much less, and
• 15:59
  - you're gonna make so much more of it.
• 16:01
  - Did the farmers end up super rich in
• 16:02
  - society from the tractor? No. Because they all
• 16:05
  - made a ton more food, and the cost
• 16:08
  - of food dropped.
• 16:09
  - And, actually, they ended up some of the
• 16:10
  - worst off. And so... But people could anticipate
• 16:14
  - kind of the first phase of it, but
• 16:15
  - not the second implication and the third implication.
• 16:18
  - And I think we keep
• 16:20
  - missing that. The second thing that I've seen
• 16:23
  - is we're wrong about the job loss thesis.
• 16:27
  - In fact, I'll I'll tell you one of
• 16:28
  - the key learnings that we've seen with Hippocratic
• 16:30
  - is we have this framework called Copilot, Autopilot,
• 16:33
  - Infinipilot.
• 16:34
  - And Copilot is things you do with the
• 16:36
  - human in the loop. Autopilot is things you
• 16:38
  - do today that we can now do cheaper,
• 16:40
  - can staff where you couldn't staff. And infinite
• 16:42
  - pilots, things you never thought to do till
• 16:44
  - you had an infinite supply of clinicians, you
• 16:46
  - know,
• 16:47
  - at at a very low cost. And
• 16:50
  - almost all of our... We knew Copilot was
• 16:52
  - gonna be hard because the human in the
• 16:54
  - loop, you only get 10% efficiency gain, and
• 16:56
  - it kind of evaporates on you. So we
• 16:59
  - focused on autopilot. Actually, every one of our
• 17:01
  - autopilot use cases has been a bit of
• 17:03
  - a struggle.
• 17:04
  - Like, we've gotten people there, but it's kinda
• 17:07
  - ground warfare. Like, we have had to fight
• 17:09
  - and fight to make the use cases work.
• 17:12
  - And every one of the infinite pilot, the
• 17:14
  - things you never thought to do that are
• 17:16
  - emergent have worked brilliantly.
• 17:19
  - Mhmm.
• 17:20
  - And
• 17:21
  - that's where I realized, okay, the job loss
• 17:23
  - thesis is false. So I'll give you one
• 17:24
  - example. You know, we recently did a loss
• 17:27
  - to follow-up,
• 17:29
  - for a large health system where we called
• 17:31
  - 1,700 patients that had not gone in for
• 17:34
  - their low dose CT scans for lung cancer.
• 17:37
  - And we didn't get all of them, but
• 17:38
  - they were gonna be at zero because they
• 17:40
  - gave up. That's what loss to follow-up means.
• 17:42
  - And,
• 17:43
  - they had tried to text. They had tried
• 17:44
  - an email. They had tried to letter. And
• 17:46
  - we got 02:50 to schedule. We got a
• 17:48
  - 180 to show up. We found nine significant
• 17:51
  - findings, three cancers, like,
• 17:54
  - you know, like... And one of them was
• 17:56
  - caught so early. It only needed a re
• 17:58
  - resection. It did not actually need a,
• 18:02
  - chemo or radiation. I mean,
• 18:05
  - you know, what a great
• 18:07
  - outcome
• 18:08
  - that we only could've done with the abundance
• 18:11
  - of of AI. And so
• 18:13
  - I think that the company should make great
• 18:15
  - products and not put them out till they're
• 18:16
  - safe.
• 18:17
  - They should also, by the way, take liability,
• 18:20
  - which we do at Hippocratic.
• 18:22
  - We're saying, look. We'll stand behind our products.
• 18:24
  - And I think any vendor that doesn't
• 18:27
  - is is a vendor one shouldn't use because
• 18:30
  - taking liability means I'll make my product better
• 18:33
  - because
• 18:34
  - I'm forced to financially as well as morally.
• 18:38
  - So,
• 18:39
  - anyways, those are those are some of the
• 18:40
  - things I think that, you know, we really
• 18:42
  - should look at the balance of payments because
• 18:44
  - we do that with every other technology in
• 18:46
  - health care.
• 18:47
  - Mhmm. I think it's a really good point
• 18:49
  - of our... Just human psychology and and biases
• 18:52
  - we might have to our, like, not
• 18:54
  - not anticipating those secondary and tertiary effects like
• 18:57
  - you had said in your opening.
• 18:59
  - The part of that response, Minjal, it's very
• 19:01
  - important to to consider. I love the farmer's
• 19:03
  - analogy.
• 19:05
  - As we head into 2027,
• 19:07
  - maybe we can wind down here and leave
• 19:09
  - our listeners with one thing that you'd want
• 19:11
  - the health care c suite to understand about
• 19:13
  - adopting AI safely.
• 19:15
  - And then the second part of that... My
• 19:17
  - question is what you think will separate leaders
• 19:19
  - who get this right from the ones who
• 19:21
  - who don't.
• 19:23
  - Yeah. So I think one part is,
• 19:26
  - don't
• 19:27
  - think there's such a thing as clinical and
• 19:29
  - nonclinical.
• 19:31
  - We call this omnidopic.
• 19:34
  - It says, hey. When a patient calls in
• 19:35
  - to schedule, if you have them on the
• 19:37
  - phone,
• 19:39
  - you better darn well
• 19:41
  - talk about
• 19:42
  - the discharge follow-up that they needed to do
• 19:45
  - today,
• 19:46
  - even if that's not why they called in.
• 19:48
  - And you can only do that if you
• 19:50
  - have a clinical and nonclinical
• 19:52
  - single platform, single framework. You should look for
• 19:55
  - that in the market and try to build
• 19:57
  - that because it'll take advantage of kind of
• 19:59
  - the opportunity,
• 20:00
  - much better. I think that the second element
• 20:02
  - is,
• 20:04
  - you know,
• 20:05
  - architect...
• 20:06
  - Look for things that are architected for safety
• 20:08
  - from day one.
• 20:11
  - Everybody thinks, oh my god. These frontier models
• 20:13
  - are amazing, and they do everything. I think
• 20:16
  - they've forgotten one very important insight we've gathered,
• 20:19
  - which is...
• 20:20
  - I call this the jagged intelligence insight.
• 20:23
  - These models can solve a 150 year old
• 20:25
  - math proofs.
• 20:26
  - But the other day, by the way, one
• 20:28
  - of these frontier models we were using,
• 20:31
  - we were testing out actually was, like, it
• 20:33
  - offered a woman a pap smear over telemed.
• 20:36
  - Said, ma'am, for your pap smear, would you
• 20:37
  - like telemed or in person?
• 20:39
  - And I'm like, pretty sure you can't do
• 20:41
  - a pap smear over telemed.
• 20:43
  - But,
• 20:45
  - these models are really smart and really dumb
• 20:48
  - at the same time. It's just we don't
• 20:49
  - realize the dumb things because we don't have
• 20:51
  - evals that tell us those those troughs.
• 20:54
  - And so you really wanna look for things
• 20:56
  - that,
• 20:57
  - that go... But I think you will see
• 20:58
  - the specialist companies in general.
• 21:02
  - Really, the question you're gonna wanna ask them
• 21:04
  - is what are all the troughs they fixed,
• 21:06
  - versus just, you know, what's your absolute score
• 21:09
  - on how you can solve, you know, the
• 21:10
  - next math proof. And so
• 21:12
  - I think that that's a... That's an element.
• 21:14
  - And then the last part is
• 21:17
  - patients are getting used to talking AI.
• 21:19
  - Mhmm. This question that you used to ask
• 21:21
  - about, oh my god. Are they really gonna
• 21:23
  - talk to it?
• 21:24
  - They're doing it all the time now. They're
• 21:26
  - talking to MetaMuse.
• 21:28
  - I mean, the other day, I I ordered
• 21:29
  - flowers for a family member on one of
• 21:31
  - the on on MetaMuse,
• 21:33
  - and,
• 21:35
  - they're talking to it in in Taco Bell
• 21:37
  - drive thrus. They're talking to it on their
• 21:39
  - bank. Like, they are very much getting used
• 21:42
  - to it. And so
• 21:44
  - you're soon gonna see in the next, I
• 21:46
  - bet, in '27,
• 21:48
  - every single consumer you'll start seeing will know
• 21:51
  - the difference, say, hey. This is not like
• 21:53
  - talking to IVR. It's way smarter. It really
• 21:55
  - answers my question.
• 21:57
  - It really completes my action,
• 22:00
  - and they'll just be comfortable. In fact, even
• 22:03
  - prefer it over many other forms of interaction,
• 22:05
  - and I think that folks need to get
• 22:07
  - ahead of that,
• 22:08
  - because the patients are doing it all day
• 22:10
  - long. I mean, what, 25%
• 22:13
  - of all searches on ChattGPT now are health
• 22:15
  - care? So people are like, are they gonna
• 22:17
  - really use AI in health care? I'm like,
• 22:19
  - they're already using it.
• 22:22
  - Right. And, by the way, there's almost a
• 22:24
  - billion weekly act of uniques. That means 250,000,000
• 22:28
  - people are doing health care searches
• 22:31
  - today. Like, what do you mean? Like, it's
• 22:34
  - already happening. And so I think they need
• 22:36
  - to get ready for this new world,
• 22:39
  - and think about what sort of... How does
• 22:41
  - that change access? How does that change follow-up?
• 22:44
  - And then the next phase is now already
• 22:46
  - coming, which is they're not even gonna call
• 22:48
  - you. Their agent, their MetaMuse is gonna call
• 22:51
  - you
• 22:52
  - and book the appointment and
• 22:54
  - follow-up on things.
• 22:57
  - And are you ready to have an interface
• 22:59
  - where you can talk to their agent,
• 23:02
  - and their agent will talk to the patient?
• 23:05
  - I love this. So for our listeners, it's
• 23:07
  - just twenty twenty seven omnitopics, so not distinguishing
• 23:10
  - or not separating the clinical versus nonclinical.
• 23:13
  - I love that challenge for those interactions. If
• 23:15
  - you have a patient on the line,
• 23:17
  - everything in... Is somewhat... Is is in clinical
• 23:20
  - in nature.
• 23:21
  - And then look for things that are built
• 23:23
  - with safety
• 23:25
  - day one. Beware of that jagged intelligence.
• 23:28
  - AI can be so brilliant and so
• 23:30
  - so dumb. Like you remind us with that
• 23:32
  - PAP smear anecdote, Mujal. And then patients getting
• 23:35
  - used to talking to AI. Sometimes it feels
• 23:37
  - like there's a lag to how we're still
• 23:39
  - seeing AI and moving into this industry. But
• 23:41
  - to your point, consumers,
• 23:43
  - patients, they're not only using it and talking
• 23:45
  - to it, but sometimes at length,
• 23:48
  - much much longer and with more detail than
• 23:51
  - you might assume.
• 23:52
  - Munjal, I just wanna thank you for your
• 23:54
  - time and your insights today. Anything you wanted
• 23:56
  - to add in closing here?
• 23:59
  - No. I, you know, I I think that...
• 24:02
  - Look. These things are really great helps in
• 24:04
  - health care. You know, I had a family
• 24:06
  - member diagnosed with stage four cancer actually this
• 24:09
  - week, and it was awful.
• 24:12
  - But I will tell you the large language
• 24:13
  - models are... Have real... I mean, in one...
• 24:17
  - In literally the last seventy two hours, I
• 24:19
  - was able to
• 24:21
  - look at every clinical trial, look at, you
• 24:23
  - know, every treatment plan, understand what it means,
• 24:26
  - and prepare for my meeting with the oncologist,
• 24:30
  - later on today, actually,
• 24:33
  - in a way that I could have never.
• 24:35
  - Like, you know, I could have never... It
• 24:36
  - would have taken me a month of research
• 24:39
  - to get what I got.
• 24:41
  - And,
• 24:42
  - I truly... And even... It prepared me a
• 24:45
  - 71
• 24:46
  - bulleted checklist
• 24:48
  - to to,
• 24:49
  - go through and make sure... And even took
• 24:51
  - the 10 that I gotta make sure I
• 24:53
  - ask the oncologist today.
• 24:55
  - Like, it's really helped me and my family
• 24:58
  - zero in on what's needed. And,
• 25:02
  - this is,
• 25:03
  - you know, this is a gift we never
• 25:05
  - had before, and I I think the whole
• 25:07
  - world already kind of is starting to get
• 25:09
  - that if they've been through a health crisis
• 25:11
  - since these LMS came out.
• 25:13
  - Mhmm.
• 25:16
  - In response to some of the doomerism is,
• 25:18
  - unfortunately, when health care hits close to home
• 25:20
  - like it has for you and your family,
• 25:22
  - which I'm sorry to hear,
• 25:23
  - Munjal, about your family member, and I I
• 25:25
  - wish them and you the best.
• 25:28
  - But nothing can quite counterdumerism,
• 25:29
  - like, being reminded of what's capable now, with
• 25:32
  - the technology like this at the fingertips of
• 25:35
  - of consumerisms
• 25:35
  - and patients and families and organizations.
• 25:38
  - So thanks for sharing that.
• 25:40
  - Listeners, you can tune into more podcasts from
• 25:43
  - Becker's Healthcare by visiting our podcast page at
• 25:45
  - beckershospitalreview.com.
• 25:48
  - Munjal, thank you again. Always a pleasure hearing
• 25:51
  - from you and learning more about the work
• 25:52
  - you and Hippocratic AI are building and doing
• 25:54
  - with organizations.
• 25:56
  - Thank you for having me, Molly.