Latest / Becker’s Healthcare Podcast / Agentic orchestration: A new approach to enterprise AI

Transcript

Agentic orchestration: A new approach to enterprise AI · Becker’s Healthcare Podcast

• 0:00
  - Everyone. This is Molly Gammel, Becker's Healthcare. Thank
• 0:02
  - you so much for tuning in to the
• 0:03
  - Becker's Healthcare podcast series. Today, we're going to
• 0:06
  - talk about agentic orchestration,
• 0:08
  - a new approach to enterprise AI.
• 0:10
  - Joining me for today's discussion is Munjal Shah,
• 0:12
  - cofounder and CEO of Hippocratic AI. Munjal, thanks
• 0:16
  - for joining me today.
• 0:17
  - Thanks for having me, Molly. So agentic orchestration,
• 0:20
  - Munjal, this is a a new term to
• 0:22
  - me, but it's something you and the Hippocratic
• 0:24
  - AI team have a a lot of depth
• 0:26
  - in, and you're also
• 0:28
  - launching
• 0:29
  - or or agentic orchestrators or this orchestration layer
• 0:32
  - that moves from individual AI agents
• 0:34
  - who handle discrete tasks to coordinated teams of
• 0:38
  - agents managing more complex workflows.
• 0:41
  - What does that evolution look like in practice,
• 0:43
  - and and what challenges are you seeing in
• 0:45
  - health care that really require a shift like
• 0:47
  - this?
• 0:48
  - Yeah. You know, I think when we started
• 0:50
  - the company, we thought we were gonna build
• 0:52
  - an LLM.
• 0:53
  - And then we all realized that nobody wants
• 0:55
  - to take an LLM and and have to
• 0:57
  - turn it into something you use for your
• 1:00
  - health care,
• 1:01
  - tasks. And so we actually were one of
• 1:03
  - the first to say we're gonna build an
• 1:05
  - agent.
• 1:06
  - And to do that, we had to fine
• 1:07
  - tune an LOM to be able to provide
• 1:09
  - the underlying the underpinnings. We had to
• 1:12
  - we then announced a set of agentic skills.
• 1:15
  - We said, oh, you know, you need a
• 1:16
  - skill to be able to talk to patients
• 1:18
  - that are post stroke and and have slurred
• 1:20
  - speech, and you need a skill to be
• 1:21
  - able to deal with patients that have mild
• 1:24
  - cognitive impairment and need a lot of reminding.
• 1:26
  - And so we then announced that we were
• 1:28
  - building a bunch of skills, and we built
• 1:30
  - hundreds of them, frankly,
• 1:31
  - to deal with things we saw in real
• 1:33
  - life, you know, in the I think we
• 1:35
  - now have 250
• 1:37
  - some odd million patient interactions, and and we
• 1:40
  - just saw a lot of problems that had
• 1:41
  - to be solved.
• 1:42
  - But as we did that, we realized that
• 1:44
  - isn't quite
• 1:46
  - how health care like, it's not enough to
• 1:49
  - just give you an agent to do x
• 1:51
  - because then you gotta say, okay. Run this
• 1:53
  - agent now. Oh, wait. I want another agent.
• 1:55
  - So let's take something like readmission prevention.
• 1:58
  - You know, it isn't just
• 2:01
  - one agent that does a discharge call.
• 2:04
  - It might be an agent inpatient that is
• 2:07
  - educating you before you leave.
• 2:10
  - It might be one you know, we're actually
• 2:12
  - working on putting our our agents into a
• 2:14
  - robot that might bring meds to beds for
• 2:17
  - you
• 2:18
  - and actually deliver the medication to you, the
• 2:20
  - first dose or the second dose, which is
• 2:22
  - common. Right? Because that helps to ensure patients
• 2:25
  - will start taking their medication.
• 2:28
  - We then will build another agent that'll call
• 2:30
  - you, you know, at some regular interval
• 2:33
  - that your care plan says. And so it
• 2:35
  - says, you know, on day three, you're supposed
• 2:36
  - to change your band, and it's really important
• 2:38
  - that you're you know, so you don't get
• 2:39
  - infected.
• 2:40
  - Like, our AI will make sure we call.
• 2:42
  - Now what is all that? That is a
• 2:44
  - group of agents designed and focused on creating
• 2:47
  - an outcome.
• 2:49
  - And that is what we really call an
• 2:50
  - orchestrator now, which is that
• 2:53
  - we think that, you know, we're not here
• 2:55
  - just to deliver agents that do tasks that
• 2:58
  - may or may not deliver the outcome.
• 3:01
  - You know, health care wants outcomes. We want
• 3:03
  - to change patient lives,
• 3:05
  - and we want to improve KPIs on which
• 3:08
  - we measure how well we're delivering the quality
• 3:10
  - of care.
• 3:11
  - And it's time to align this AI world
• 3:14
  - with that.
• 3:15
  - So many health systems have already seen success
• 3:17
  - using AI for individual workflows, but orchestration,
• 3:24
  - patient journeys. If I'm in a CEO or
• 3:27
  - COO shoes and looking through their perspective,
• 3:30
  - Munjal, what are some new operational opportunities that
• 3:33
  - creates that weren't possible with stand alone agents?
• 3:36
  - You know, I think, you know, COO or
• 3:38
  - CEO,
• 3:39
  - you love to manage by objective. Right? Please
• 3:41
  - lower the readmission rate.
• 3:43
  - Please get my loss to follow-up
• 3:45
  - closed and and help us increase better care
• 3:47
  - for these patients.
• 3:49
  - Please avoid leakage.
• 3:50
  - And I think that for the first time,
• 3:52
  - orchestrators
• 3:53
  - offer a solution based approach to using AgenTic
• 3:56
  - AI.
• 3:57
  - You just indicate the objective you want and
• 4:00
  - the solution you're really after, the area of
• 4:02
  - focus, and and you do that. So we're
• 4:04
  - now providing orchestrators that help you improve all
• 4:07
  - of your onboarding, improve all of your leakage,
• 4:09
  - improve all of your loss to follow-up,
• 4:12
  - improve your pharmacy,
• 4:13
  - business, improve your inpatient experience, your ambulatory experience,
• 4:18
  - reduce your readmissions, and improve your heated scores
• 4:21
  - and your stars. And so, you know, if
• 4:23
  - you think about it, it allows you to
• 4:24
  - operate on an objective level rather than just
• 4:27
  - on a task level. And I think until
• 4:29
  - now, we really couldn't
• 4:31
  - really
• 4:32
  - ask our AI to achieve an objective and
• 4:34
  - know it would go and figure out all
• 4:36
  - the things needed to to do that. There's
• 4:38
  - actually three systems where they had us do
• 4:41
  - discharge calls and try to avoid, readmissions.
• 4:44
  - And we realized how many different agents we
• 4:47
  - had to build to be able to achieve
• 4:49
  - the goal.
• 4:50
  - And we were in in actually, two of
• 4:52
  - the systems, both of them, we ended up
• 4:54
  - lowering readmission rates by 4%. One was 18
• 4:57
  - to 14,
• 4:58
  - which if you know anything about readmission rate
• 5:00
  - reduction was so big a change, nobody believed
• 5:03
  - it.
• 5:04
  - Like, we didn't believe it. Our client health
• 5:06
  - system client didn't believe it. And in fact,
• 5:09
  - they made us rerun the number seven times
• 5:11
  - on our side.
• 5:13
  - We can we actually even we reran them
• 5:15
  - because we're like, this can't be true. This
• 5:16
  - has gotta be impossible.
• 5:18
  - And it turned out that I mean, after
• 5:20
  - seven times, we all realized, wow. It it
• 5:22
  - was really true. And then we did it
• 5:24
  - again in another health system and saw another
• 5:25
  - four point drop. And that's when we realized,
• 5:27
  - oh, wow. Like, this is actually working.
• 5:31
  - Absolutely. A a range of of settings and
• 5:33
  - needs that you just highlighted there. And I'm
• 5:35
  - I'm putting myself in the in the position
• 5:37
  - of our our audience who I think hearing
• 5:39
  - some of these things, Minjal, they might share
• 5:41
  - that early sense of of disbelief or just
• 5:43
  - like they're we need to reorient ourselves thinking
• 5:46
  - about this in terms of what orchestration can
• 5:48
  - do.
• 5:49
  - You've shared an example. You talked about some
• 5:51
  - of the discharge calls, but another type of
• 5:53
  - call I've heard about is this example of
• 5:55
  - an academic medical center where your agents contacted
• 5:58
  - more than 1,700
• 5:59
  - patients who had open CT scan orders in
• 6:03
  - three days, and the result was two hundred
• 6:05
  - and forty nine scheduled lung cancer screenings.
• 6:09
  - So that was that was one agent in
• 6:10
  - one workflow.
• 6:11
  - What becomes possible when the
• 6:13
  - suites of agents are working together across an
• 6:16
  - entire enterprise?
• 6:18
  - Yeah. Let's let's take that as, like we
• 6:19
  - call that the kinda lost to follow-up orchestrator.
• 6:22
  - You know, that was one list. By the
• 6:24
  - way, of those 250
• 6:26
  - folks who did this, scheduled it, a 180
• 6:29
  - showed up. We found nine significant findings.
• 6:33
  - In fact, there's probably gonna be three cancers
• 6:35
  - found out of that.
• 6:37
  - Oh, I know there was one where they
• 6:38
  - just had to they caught it so early,
• 6:40
  - they were able to just do a resection
• 6:42
  - and didn't need to actually do, more,
• 6:46
  - invasive,
• 6:47
  - and,
• 6:49
  - you know, chemo and other things. So it
• 6:50
  - was great, that we caught it so early.
• 6:52
  - So, I mean, you know, we're this is
• 6:54
  - delivering real good. But
• 6:56
  - our goal isn't just to do it for
• 6:58
  - CT
• 6:59
  - lung cancer screenings. We should do it for
• 7:01
  - everything.
• 7:02
  - Right? Lost a follow-up on colonoscopies.
• 7:05
  - Lost a follow-up on
• 7:06
  - mammograms. You know, some of those are care
• 7:08
  - gaps, but also lost a follow-up
• 7:10
  - on many there's many things that patients don't
• 7:13
  - get done. And now some of that isn't
• 7:15
  - just about
• 7:17
  - calling the patient and persuading them and convincing
• 7:19
  - him to do it. You know, that that
• 7:21
  - would be fine. We have an orchestrator above
• 7:23
  - of, like, five. But now think about this.
• 7:25
  - What if the patient has two things that
• 7:26
  - they have to do? You don't wanna call
• 7:28
  - them twice with two different agents. You wanna
• 7:30
  - avoid the abrasion and call them once.
• 7:33
  - Second, what if,
• 7:34
  - the order is expired?
• 7:37
  - Now you wanna follow-up with the doctor and
• 7:39
  - say, you know, it expired, but I don't
• 7:40
  - think it expired because you meant for it
• 7:42
  - to expire. It just expired because the guy
• 7:44
  - didn't get it done in so long. Do
• 7:45
  - you really think that mister Shaw still needs
• 7:47
  - to get that done?
• 7:49
  - And we could then activate and get a,
• 7:53
  - additional kind of just refresh of the order.
• 7:56
  - And then on top of that, there's another
• 7:58
  - agent that we will build that'll keep track
• 8:00
  - of all of your capacity.
• 8:02
  - Okay. We're not gonna call the lung cancer
• 8:04
  - folks anymore this week. We're gonna wait a
• 8:06
  - couple weeks.
• 8:08
  - Why? Because there's no slots.
• 8:10
  - Mhmm. Mhmm. You're you're full. So there's no
• 8:12
  - point in calling. We're not gonna be able
• 8:14
  - to find any slots.
• 8:15
  - And so but, oh, but our our CT
• 8:18
  - machine or our other machine, our MRI is
• 8:20
  - available, but not our CT or vice versa,
• 8:23
  - or our bone density machine is available.
• 8:26
  - Right? Our DXA scan's available. So let's go
• 8:29
  - and follow-up with all of the women who
• 8:31
  - had had hip fractures who we now wanna
• 8:33
  - make sure they've come back in and got
• 8:34
  - another bone density scan.
• 8:36
  - In fact, we're doing this use case for
• 8:37
  - a customer of ours right now,
• 8:40
  - going after,
• 8:41
  - low bone density in women in particular
• 8:44
  - and trying to encourage them to get a
• 8:45
  - bone density scan. And so so imagine now
• 8:48
  - you have a capacity
• 8:50
  - or agent combined with each of the individual
• 8:53
  - loss to follow-up agents combined with the consolidation
• 8:56
  - agent that's putting it all together,
• 8:58
  - to avoid abrasion. And
• 9:01
  - and then, obviously, you have a scheduling agent
• 9:02
  - that'll do all of the actual scheduling of
• 9:04
  - the thing, plus one that follow-up with the
• 9:07
  - actual physician if we need to change the
• 9:09
  - order in some way or get it refreshed.
• 9:11
  - And so
• 9:12
  - that is an orchestrator.
• 9:14
  - We're trying to get to the outcome.
• 9:17
  - And and this is work that is much
• 9:18
  - more
• 9:20
  - complex, conditional. It's it's multipart. It's open to
• 9:23
  - interpretation
• 9:24
  - in some ways like you just highlighted.
• 9:26
  - So thanks for those examples. I think it's
• 9:28
  - really helpful to start to understand how this
• 9:30
  - differs from the agents that health systems might
• 9:32
  - be used to in current day.
• 9:35
  - Another element of all this work and and
• 9:37
  - scaling AI across an organization, it just naturally
• 9:39
  - raises questions about governance, about trust. Hippocratic
• 9:43
  - AI's Polaris uses a primary model alongside more
• 9:46
  - than 30 supervising models that are validated by
• 9:49
  - thousands of clinicians. So this is clearly something
• 9:52
  - you and your company take quite seriously.
• 9:55
  - I guess, Munjal, at a broader level, when
• 9:57
  - you look around in the environment health care
• 9:59
  - leaders are operating within today, what is a
• 10:02
  - a common misconception or something that we might
• 10:04
  - overlook about AI safety that warrants a closer
• 10:07
  - look?
• 10:09
  - So I said two things on that. One
• 10:10
  - is, you know, trust is is,
• 10:13
  - compounding.
• 10:14
  - You know? So we didn't ask everybody to
• 10:16
  - trust our orchestrators that obviously are a level
• 10:19
  - of abstraction
• 10:20
  - and independence that's that's a little bit more,
• 10:23
  - right, than you've seen before.
• 10:25
  - We first said trust our agent.
• 10:28
  - Now that you trust our agents, now you
• 10:30
  - can trust when we combine our agents.
• 10:32
  - So it's almost like I'm not gonna give
• 10:34
  - you the molecule without first you trusting every
• 10:36
  - atom that's going in the molecule.
• 10:39
  - And so if you think about, like, we
• 10:41
  - we and then and then we told you
• 10:43
  - how we built the atoms themselves. We're like,
• 10:45
  - look. We tested them. We did this. We
• 10:46
  - did that. So I I think that you
• 10:48
  - couldn't have gotten here till now.
• 10:50
  - You had to start with, alright. Here's how
• 10:52
  - we took the LOM. Here's how we train
• 10:54
  - the LOM. Here's how we then built it
• 10:56
  - into an agent. Here's how we validated the
• 10:58
  - agent. Here's how we now run a bunch
• 11:00
  - of you know, I think we've now done
• 11:02
  - we've done this 250,000,000
• 11:04
  - patient interactions without one safety incident. Because we
• 11:07
  - do escalate to humans when necessary, we've escalated
• 11:10
  - to humans 38,000
• 11:12
  - times.
• 11:13
  - Wow.
• 11:14
  - So we know what happens when it happens,
• 11:17
  - how to ensure, you know, and we've avoided
• 11:19
  - any significant safety issues. So it's like
• 11:22
  - that's that piece. But I think that I
• 11:24
  - think when people are having conceptions about safety,
• 11:28
  - I I think it's in the abstract mostly.
• 11:31
  - It's like you've never actually
• 11:34
  - done the thing, and you're worried about its
• 11:36
  - safety or not. But once you do it,
• 11:37
  - you realize, a, it's not it's not a
• 11:39
  - worry in some things you were worried about.
• 11:42
  - And b, it is actually so you have
• 11:44
  - to be,
• 11:45
  - you know, watchful in this other area, but
• 11:46
  - you didn't know it because you've never done
• 11:48
  - it. Mhmm. Like, think about when you before
• 11:50
  - you ever went skiing,
• 11:52
  - would you have known, like, what you gotta
• 11:54
  - worry about when you ski to not get
• 11:55
  - injured? Right? Like, you know, now whenever I
• 11:58
  - ski, by the way, like, I I'm always
• 11:59
  - off the slope by, like, 01:00,
• 12:02
  - 02:00 at the moment. Right? Because I always
• 12:04
  - feel like you get injured in the last,
• 12:06
  - like, hour
• 12:07
  - of your skis thing when the snow is
• 12:09
  - slushy and it's horrible, at least here in
• 12:11
  - California. And so
• 12:13
  - but I wouldn't have known that if I
• 12:14
  - never skied. So first, I I think you
• 12:16
  - need to focus on that. In terms of
• 12:17
  - the actual things to kinda look out for,
• 12:21
  - I think there's a siren song right now
• 12:23
  - around smaller models.
• 12:26
  - And the siren song says, I can give
• 12:28
  - you a cheaper AI because I'm using smaller
• 12:31
  - models. I'm calling them SDMs
• 12:34
  - or SLMs, small language models. Okay.
• 12:37
  - That's not a thing.
• 12:39
  - The reason this was a big innovation in
• 12:40
  - the world was it's called a large language
• 12:43
  - model.
• 12:44
  - Mhmm. It's the large part that made it
• 12:46
  - smart.
• 12:48
  - Actually, very directly, the size of the model
• 12:51
  - correlates to its intelligence.
• 12:54
  - Like, everybody doesn't realize this. Like, the breakthrough
• 12:56
  - for CHAT GPT came because we crossed 30,000,000,000
• 12:59
  - parameters. What's a parameter? It's the number of
• 13:01
  - connections between the neurons and the neural network.
• 13:03
  - It's literally like the brain. The more connections
• 13:06
  - a brain has, the smarter the brain is.
• 13:09
  - So I'm like, great. You wanna downgrade to
• 13:11
  - a mouse brain?
• 13:12
  - Because that's kinda what you just did,
• 13:14
  - literally.
• 13:16
  - You literally just downgraded to a mouse brain
• 13:18
  - when you moved to an SLM. And that
• 13:19
  - may work fine,
• 13:21
  - but you know what it doesn't do? It
• 13:23
  - doesn't catch all the corner cases, and medicine
• 13:26
  - is a giant game of corner cases.
• 13:29
  - Mhmm. Mhmm. And, like, you know, like, diagnosing
• 13:31
  - UTIs in women, if you you know, we
• 13:33
  - don't do diagnosis. But if you did,
• 13:35
  - yeah. Could a little rule system get you
• 13:38
  - there eighty percent of the time? Sure.
• 13:41
  - But it's that twenty percent of the time,
• 13:43
  - five percent of the time, that's when,
• 13:46
  - you know, when the UTI is a possible
• 13:49
  - infection risk to the kidney. And, I mean,
• 13:51
  - that that that's when it gets complicated.
• 13:53
  - Yeah. And so I think that,
• 13:56
  - anyways, there's there's a,
• 13:59
  - one of the things I think people don't
• 14:00
  - realize is safety and size of model are
• 14:03
  - directly related.
• 14:04
  - Mhmm.
• 14:05
  - Mhmm. Thank you for expanding on that. I
• 14:07
  - think also just for highlighting some of the
• 14:09
  - the human biases we may bring to to
• 14:11
  - the unknown as these are now this technology
• 14:14
  - is capable of doing things that six months,
• 14:16
  - eighteen months ago just was not in existence.
• 14:19
  - So I think it's really helpful to to
• 14:20
  - keep that in mind, Munjal.
• 14:22
  - As we wind down here, in addition to
• 14:25
  - agentic orchestration,
• 14:27
  - you also rolled out what I understand to
• 14:29
  - be a self-service agent builder. Can you just
• 14:31
  - explain what this is for one second and
• 14:34
  - also what it means for the ability of
• 14:36
  - health care leaders to get to market and
• 14:38
  - production more quickly?
• 14:40
  - Yeah. So the the innovation of our agent
• 14:42
  - builder is actually not the agent builder, but
• 14:44
  - it's actually the validation.
• 14:46
  - So we developed this unique way to validate
• 14:48
  - agents that, you know, I haven't seen anybody
• 14:51
  - else do, which is we hired 7,000
• 14:53
  - now 7,750
• 14:55
  - licensed
• 14:56
  - US clinicians.
• 14:58
  - We gave them, like, you know, what we
• 14:59
  - call our gig app. They actually sign on
• 15:01
  - on their iPhone every, you know, morning, and
• 15:03
  - they see if there's tasks for them to
• 15:04
  - do, and we pay them for task. You
• 15:06
  - know? And the tasks are usually act as
• 15:08
  - a patient, call this new agent we're building,
• 15:11
  - and or this new orchestrator,
• 15:13
  - and
• 15:14
  - tell us if it's making mistakes and mark
• 15:16
  - every error. And one out of 50 times,
• 15:18
  - they got a human on the other line,
• 15:19
  - so then we got to calibrate how good
• 15:21
  - the error rate was compared to the kind
• 15:23
  - of, human error rate. We used to build
• 15:25
  - all of our models that way. They would
• 15:27
  - all get validated to this process, and we've
• 15:30
  - now done 700,000
• 15:32
  - I see almost 800,000
• 15:33
  - test calls with their simulations.
• 15:36
  - And we wouldn't release a a model until
• 15:38
  - it had been validated as as safer than
• 15:40
  - the human equivalent. We wouldn't validate we wouldn't
• 15:43
  - release a use case or an agent until
• 15:45
  - we had done that, you know, and now
• 15:47
  - we won't release an an orchestrator
• 15:48
  - until we've done that.
• 15:50
  - But we had to do that.
• 15:52
  - If you we had a little agent builder
• 15:53
  - where you could build your own agent, but
• 15:55
  - then we would, by hand, take it through
• 15:57
  - this validation system.
• 15:59
  - But now what we do is we link
• 16:01
  - the two.
• 16:02
  - So you can build your own agent.
• 16:05
  - You can send it. It'll automatically send it
• 16:08
  - for validation.
• 16:10
  - It'll validate it
• 16:12
  - and give you back the feedback. But then
• 16:13
  - we said, wait a minute. We don't even
• 16:15
  - just have to give you back the feedback.
• 16:17
  - We'll run another agent to rewrite your prompts
• 16:20
  - and your scripts based upon the feedback.
• 16:23
  - We'll send it for validation again,
• 16:26
  - and we'll do this loop for a week
• 16:27
  - without you even being involved and then say,
• 16:30
  - hey. Here's what we think is the right
• 16:31
  - one. Now you try it again and tell
• 16:33
  - us where it is, and then we'll validate
• 16:35
  - it one more time.
• 16:36
  - Wow. And so the power is really in
• 16:39
  - the combination of the agent builder feeding into
• 16:41
  - it, but then making it a continuous improving
• 16:44
  - loop that actually is somewhat autonomous.
• 16:47
  - And that's the power of this self-service is
• 16:50
  - is
• 16:50
  - you know, we probably should have come up
• 16:52
  - with a better name for it because it's
• 16:53
  - not just self-service. It's our self validating
• 16:57
  - way to kind of develop agents and and
• 16:59
  - orchestrators around that. And so, you know, I
• 17:02
  - I think that's what I'm most excited about
• 17:04
  - is is
• 17:05
  - we are now exporting the ability for you
• 17:08
  - to make safe agents, not just you to
• 17:10
  - make agents.
• 17:12
  - I'm really glad you expand on that. I
• 17:13
  - thought it was more of, like, just the
• 17:15
  - access of building an agent was gonna be
• 17:17
  - the value proposition there, but it's actually the
• 17:19
  - validation and your homegrown process of safety assurance
• 17:23
  - now being accessible
• 17:25
  - to the end user, like, in in such
• 17:26
  - a way.
• 17:27
  - Right. Exactly.
• 17:28
  - Exactly.
• 17:30
  - Manjal, you you and Hippocratic AI just continue
• 17:32
  - to be I I don't even know how
• 17:33
  - many steps ahead. I I just wanna thank
• 17:35
  - you so much for your time and your
• 17:36
  - insights today. This was a really interesting conversation.
• 17:39
  - Thanks for joining me. Yeah. Thanks for having
• 17:42
  - me, Molly. It's always a pleasure.
• 17:44
  - And to our audience, thank you so much
• 17:45
  - for tuning in. You can tune in to
• 17:47
  - more podcasts from Becker's Healthcare by visiting our
• 17:49
  - podcast page at beckershospitalreview.com.