Latest / Becker’s Healthcare Podcast / The Growing Trend of AI Agents in the Health System & What Leaders Can Do to Keep Operations Secure
Transcript
- 0:00- Hello, everyone. This is Erica Spicer Mason with
- 0:02- Becker's Healthcare. Thank you so much for tuning
- 0:04- in to the Becker's Healthcare podcast series.
- 0:08- Today, we're going to talk about the growing
- 0:10- trend of AI agents in the health system
- 0:12- and what leaders can do to keep operations
- 0:15- secure.
- 0:15- And joining me for this conversation, we have
- 0:18- Jason Elrod, a health care chief information security
- 0:21- officer and executive adviser to Elisity.
- 0:25- Jason, great to have you on the podcast.
- 0:26- Thanks for being with us today.
- 0:28- Fantastic to be here, Erica. Thanks for inviting
- 0:30- me. Oh, so glad to have you. And
- 0:33- I know AI agents is something that's becoming
- 0:36- more and more a part of health care
- 0:37- conversations, not just in the health IT space,
- 0:39- but really across the leadership board. So
- 0:42- wanted to make sure that we get ourselves
- 0:44- grounded first in what you're seeing happening here.
- 0:48- As I mentioned, we're seeing AI agents start
- 0:50- to show up inside of health systems today,
- 0:52- and I'd like to know from you what
- 0:54- that looks like operationally,
- 0:56- and how does an agent actually end up
- 0:58- with access to a health system environment?
- 1:01- Well, I think the important thing to understand
- 1:04- is that AI agent aren't necessarily
- 1:06- arriving as some large formally announced enterprise
- 1:10- deployment.
- 1:11- It's not like, hey, you know, the AI
- 1:13- cohort is here.
- 1:15- They arrive quietly.
- 1:17- You know, like a vendor adds an agentic
- 1:18- feature to a product we already use, a
- 1:20- business unit connects AI capability
- 1:22- to Microsoft three sixty five, Someone builds an
- 1:25- automation around an API. So
- 1:28- it's not quite the way that you, would
- 1:30- normally expect that. It's not something you see
- 1:32- it and you formally put it through. So
- 1:34- you do have that capability,
- 1:36- but
- 1:37- it's
- 1:38- less
- 1:40- structured.
- 1:41- Right?
- 1:42- And so so it's kinda sneaking its way
- 1:44- in. And I hate to say it that
- 1:45- way because there's
- 1:48- very
- 1:48- obvious ways that we're using AI and we
- 1:50- bring those in. But it's all those other
- 1:52- little areas that pop in. Right?
- 1:54- And from a security perspective,
- 1:56- AI is becoming much more interesting,
- 1:59- especially when it's moving from generative language
- 2:02- to exercising authority. So the difference between, like,
- 2:05- a large language model
- 2:07- and agentic AI. And that's a that's a
- 2:09- significant and very important
- 2:12- security boundary that we're seeing nowadays.
- 2:15- Yeah. Jason, I really appreciate the overview, and
- 2:18- I think you've mentioned some key points here
- 2:20- that I'm sure
- 2:21- as a CISO
- 2:23- raise some flags in your mind, you know,
- 2:25- when you think about a technology or or
- 2:27- a feature arriving quietly.
- 2:29- And in a sense, you know, feeling like
- 2:31- it's almost something that can sneak in under
- 2:33- the radar into a health systems operations.
- 2:35- So that kind of leads me to want
- 2:37- to touch on governance here.
- 2:39- I'm curious, you know, when we're thinking about
- 2:42- who owns an AI agent, who approves it,
- 2:44- who can turn it off,
- 2:46- you know, are are those things that
- 2:49- you advise organizations to take a look at?
- 2:52- Is it enough, or where do you think
- 2:53- security leaders should start?
- 2:56- I'll say governance
- 2:57- absolutely matters.
- 2:59- And it can't be governance
- 3:01- theater. It has the actual operational governance. What
- 3:03- I mean by that
- 3:05- is,
- 3:07- it can't be like the committee
- 3:08- approved the use of AI for some use
- 3:11- cases.
- 3:13- That is too vague. That it it's dilution
- 3:16- of responsibility,
- 3:17- dilution of accountability.
- 3:19- So real governance here has to be
- 3:22- operationally
- 3:23- specific.
- 3:24- What do I mean by that? Well, you
- 3:25- have to know who authorized this, who's accountable
- 3:28- for it, what data is it gonna access,
- 3:31- how are we actually monitoring it for impact
- 3:34- that it's gonna have, who's gonna be responsible
- 3:36- for that, And then how do we actually
- 3:39- enforce
- 3:41- containment if it behaves unexpectedly
- 3:43- or how quickly can we restrict or stop
- 3:45- it? And
- 3:47- from that perspective,
- 3:49- every use case or every AI that we
- 3:51- have, either we formally
- 3:53- introduce the organization or we discover,
- 3:55- needs to have an actual
- 3:57- name, like a person, not some sort of
- 4:00- committee because that dilutes it.
- 4:02- So, you gotta answer those five questions I
- 4:04- just sort of mentioned there. First, like, you
- 4:06- know, who authorized it? Who's accountable for it?
- 4:09- What data is it accessing?
- 4:11- Who's responsible for when it goes wonky? And
- 4:14- that's a technical term, by the way, when
- 4:15- it goes wonky.
- 4:17- And,
- 4:18- what are the tools we're using to support
- 4:20- that?
- 4:21- So you kinda could there's a couple of
- 4:23- components there. So governance is not just saying,
- 4:25- yes, you can do this.
- 4:26- No. So governance really answers the question, should
- 4:29- this agent exist? So do we know it?
- 4:31- Who's responsible? What are we gonna use it
- 4:33- for? What's it gonna access?
- 4:35- Security
- 4:36- architecture
- 4:37- needs to go along with that
- 4:39- and that answers how dangerous can this agent
- 4:42- become.
- 4:44- So the distinction there, governance absolutely needed. It
- 4:47- needs to be not governance theater, it needs
- 4:49- to be operational governance.
- 4:51- And governance in its essence tells an agent
- 4:54- what it should do.
- 4:56- You always have to link that with good
- 4:58- security architecture because architecture determines what it can
- 5:02- do, and that's a very important distinction.
- 5:06- Yeah. And I love this term that you're
- 5:09- using, Jason, the idea of governance theater.
- 5:12- Something I haven't heard before, but I've heard
- 5:14- so many leaders speak to the heart of
- 5:16- that issue.
- 5:17- You know, essentially saying that these governance
- 5:20- processes and protocols and account accountability
- 5:23- point people, it can't be you know, there
- 5:25- has to be backing behind it. It has
- 5:27- to be meaningful. So
- 5:28- appreciate you outlining those specific areas for listeners.
- 5:32- And I wanna take a step back here
- 5:33- too and go back to this idea of
- 5:36- when the AI agent enters a health system.
- 5:39- Health systems are incredibly complex environments in most
- 5:42- cases.
- 5:43- So how what can an AI agent potentially
- 5:46- reach inside of a hospital that might surprise
- 5:49- someone who's maybe coming from traditional corporate IT?
- 5:53- A hospital is an extraordinary
- 5:56- technology environment
- 5:58- because it's simultaneously
- 5:59- it's an enterprise, it's a clinical environment, but
- 6:02- it's also an industrial environment and increasingly
- 6:05- an IOT environment. We're gonna call it MIOT,
- 6:07- medical internet of things. So most corporate environments
- 6:12- don't have a CT scanner, an infusion pump,
- 6:14- a nurse call system,
- 6:15- and a building management system, and an EHR,
- 6:18- and a financial application operating in the same
- 6:22- ecosystem.
- 6:23- So it's
- 6:24- really interesting. So I think a lot of
- 6:26- folks don't necessarily take that into consideration. It's
- 6:28- a very blended environment.
- 6:30- So hospital isn't just an IT environment with
- 6:32- patients in it.
- 6:34- It's really
- 6:35- a clinical operating environment
- 6:37- powered by technology.
- 6:40- And we end up with a very interesting
- 6:42- space there because we can have some of
- 6:44- the most fantastic
- 6:46- sci fi latest technology in there, like surgical
- 6:49- robots as an example.
- 6:51- Next is some of the most
- 6:53- archaic
- 6:54- legacy
- 6:55- IT stuff,
- 6:56- you've ever seen. You did you know, you
- 6:58- saw it at Make Me Be in the
- 6:59- way back machine or inside of a museum.
- 7:01- You're like, wow. So I've got this really
- 7:03- old technology, this legacy in place
- 7:06- right next to things that are super sci
- 7:08- fi cutting edge. So you end up having
- 7:10- a little bit of a paradox
- 7:12- in the,
- 7:14- IT environment itself as well.
- 7:16- It makes a lot of sense as you
- 7:17- were walking us through that example, Jason. I'm
- 7:20- I'm thinking about the surgical robot, as you
- 7:22- mentioned, maybe
- 7:24- a room away from a fax machine, which
- 7:26- we know a lot of providers still use.
- 7:28- Yeah. It is it's crazy. Right? Yeah. Yeah.
- 7:31- Such a,
- 7:32- contradiction, a dynamic environment for sure.
- 7:36- So if there is risk around
- 7:38- what an agent can reach and also do
- 7:41- within this complex system,
- 7:43- how should health care leaders be thinking differently
- 7:45- about concepts like identity, permissions,
- 7:49- segmentation, and even limiting blast radius. Can you
- 7:52- shed some light on those?
- 7:54- Well, definitely. We we should begin by treating
- 7:56- agents as nonhuman identities
- 7:58- with potentially
- 7:59- significant privileges.
- 8:02- What I mean by that is the same
- 8:04- zero trust principles we apply to people and
- 8:06- devices
- 8:07- needs to extend to agents.
- 8:09- We don't just simply trust an agent simply
- 8:11- because it's internal, because a vendor built it,
- 8:13- or just because somebody approved it.
- 8:16- You give it minimum
- 8:17- necessary to perform its job just like an
- 8:19- individual. So this is why it's essential
- 8:22- that we give
- 8:23- AI, basically
- 8:25- agents, their own identity.
- 8:28- That way we can assign least privilege, basically
- 8:30- the minimum permissions necessary.
- 8:33- Segmentation
- 8:34- is intently
- 8:35- important here, so only in the environments required
- 8:38- for its function.
- 8:40- And I mentioned identity. You wanna have that
- 8:42- distinct identity with them. So I don't wanna
- 8:45- hide autonomous activity behind, like, a broad service
- 8:49- account. We can have, like, the AI identity
- 8:52- and all I all AIs get that one
- 8:54- sort of record or identity. We have to
- 8:56- give them individual ones just like people.
- 8:58- We have to be able to monitor it.
- 9:00- Right? You know, we gotta be able to
- 9:01- understand, like, what are they doing to the
- 9:04- best extent that we can.
- 9:05- And we talk about segmentation. Right? Containment.
- 9:09- You don't
- 9:10- secure
- 9:12- autonomy
- 9:13- by assuming
- 9:14- perfect behavior.
- 9:16- You secure it by limiting the consequences
- 9:19- of imperfect behavior.
- 9:22- And honestly, complex systems fail.
- 9:25- And credentials get compromised, applications malfunctions.
- 9:28- Humans make mistakes. And agents will sometimes behave
- 9:31- in ways we did not anticipate.
- 9:34- Then you're looking at some of the, you
- 9:36- know, frontier models from some of the big
- 9:38- providers out there where they did not anticipate
- 9:40- them breaking out and then going hacking another
- 9:42- company out there.
- 9:43- So you can see that there is going
- 9:45- to be a spot where something doesn't act
- 9:47- like you thought it would act.
- 9:50- That's where the segmentation comes in, especially identity
- 9:52- based,
- 9:53- segmentation. You wanna be able to help you.
- 9:55- Whether it's a human or nonhuman identity,
- 9:58- you wanna say, okay, I've authorized you to
- 10:00- do this.
- 10:01- And so when I've authorized you to do
- 10:03- something and you get into that system to
- 10:05- do it, you know, I can't really
- 10:07- control
- 10:08- so much.
- 10:10- The damage can be done through that explicit
- 10:12- authorization.
- 10:13- But what I can do is I can
- 10:14- segment that to just that system. You can't
- 10:17- take that authorization and then expand it across
- 10:19- systems that are not involved in that idea
- 10:22- of minimum necessary.
- 10:24- So again, I mean, you don't you don't
- 10:26- secure autonomy by assuming perfect behavior because nothing's
- 10:29- perfect and it will fail. You secure it
- 10:32- by limiting the consequences
- 10:34- of when it breaks bad or that imperfect
- 10:36- behavior.
- 10:38- Mhmm.
- 10:39- Yeah. So it's sounds like segmentation is essential.
- 10:43- And Credible. I'm wondering, Jason yeah.
- 10:46- And I'm wondering,
- 10:48- even though this concept,
- 10:50- it doesn't sound new, but it it does
- 10:52- sound potentially challenging for organizations to scale or
- 10:55- to execute it at scale. So I'm curious
- 10:58- what your view is on that and how
- 11:00- you're seeing organizations
- 11:01- make meaningful progress in achieving segmentation.
- 11:05- In the past, the technology was the problem.
- 11:08- You couldn't really do it at scale. So
- 11:10- but but today, the technology usually isn't the
- 11:12- hardest part.
- 11:13- The organizations
- 11:14- that succeed
- 11:16- succeed in this space, actually stop treating segmentation
- 11:19- as this giant network project and start treating
- 11:21- it as a risk reduction program. So you
- 11:23- can't say, I need to go in and
- 11:24- do it everywhere all at once.
- 11:27- You know, trying to trying to redesign an
- 11:29- entire health system network at once is enormously
- 11:31- difficult and just thinking about that can cause
- 11:34- pause or stop it, the work of it.
- 11:37- A better approach is to understand the environment,
- 11:39- identify those meaningful boundaries like we just talked
- 11:42- about a little bit there, and then progressively
- 11:45- reduce unnecessary
- 11:47- connectivity. And what I mean by that, again,
- 11:49- reduce the scope.
- 11:51- Segmentation isn't about building walls everywhere.
- 11:54- It's about removing
- 11:55- pathways that have no reason to exist.
- 11:58- That's the idea in the past you did
- 12:00- this over provisioning. You had everything go and
- 12:02- just
- 12:03- I'm thinking back in the way back machine.
- 12:04- You just put it on the flat network
- 12:06- and you gave it advent prevent credentials and
- 12:08- you were just able to do your work.
- 12:10- Well, nowadays that that's obviously a recipe for
- 12:12- disaster and obviously not where you wanna go.
- 12:15- But most
- 12:17- health care or health IT
- 12:18- ecosystems
- 12:19- didn't grow up
- 12:22- with a real coherent plan.
- 12:25- So they're not homogeneous,
- 12:27- heterogeneous. Right? They're they're they're completely diverse. I
- 12:30- mentioned before, old technology,
- 12:32- cutting edge technology
- 12:33- right next to each other. So the first
- 12:35- thing you need to do is you need
- 12:36- to start visibility. Right? You need to know
- 12:38- what you have. Understand what the devices are.
- 12:41- What's it communicating with? Why does that communication
- 12:43- exist? Because sometimes we had those the communications
- 12:46- were there,
- 12:47- for a reason in the past, but they
- 12:48- don't necessarily need to be there now.
- 12:50- And
- 12:51- what happens if that communication stops? So getting
- 12:54- that higher fidelity about what's really going on
- 12:56- in your network, extremely important.
- 12:59- Then
- 13:01- move deliberately,
- 13:02- not
- 13:04- holistically.
- 13:05- And what I mean there is you go
- 13:07- from visibility to understanding to policy to enforcement.
- 13:10- That's sort of the path you have to
- 13:12- do. Like you know what I got,
- 13:13- what what it should be doing,
- 13:15- what's a rational policy now based on that,
- 13:17- and then how do I enforce it? That's
- 13:20- the that's the macro of how you do
- 13:21- it. But don't do it everywhere all at
- 13:24- once.
- 13:25- What you have to do is you, hey,
- 13:26- hey, what are the highest impact items? So
- 13:28- and then go into that and you you're
- 13:30- gonna go ahead and do it incrementally
- 13:32- in that fashion.
- 13:34- And since we're talking about AI agents,
- 13:37- you wanna look at that from, again, the
- 13:38- identity. This agent performed a function
- 13:41- in this environment
- 13:43- and has no reason to communicate outside that
- 13:46- boundary. So like, oh, I might use it
- 13:48- for some for something else later. Great. That's
- 13:51- a future Us problem and a future Us
- 13:53- opportunity.
- 13:54- Don't
- 13:56- build in that
- 13:58- wide boundary.
- 14:00- Build in the constrained boundary.
- 14:03- Because, again, I mentioned it's about it's not
- 14:05- about building walls everywhere. It's about
- 14:09- removing the things that have no reason to
- 14:11- exist.
- 14:13- And you don't really know that if you
- 14:14- don't have that visibility. You don't understand the
- 14:16- identity. You don't understand the communications
- 14:18- and then the context in which that needs
- 14:20- to operate.
- 14:21- Mhmm. That helps? It it helps a lot,
- 14:23- Jason. It almost reminds me of the concept
- 14:26- of kind of harm reduction,
- 14:28- but everything you're saying
- 14:30- is so clear.
- 14:31- And I'm curious too. You know, you mentioned
- 14:33- that you advise leaders oftentimes
- 14:36- if they're wondering where to start, you advise
- 14:38- the highest impact use cases, you know, not
- 14:40- taking everything on all at once to achieve
- 14:43- this environment.
- 14:45- But say a health system leader who's listening
- 14:47- to this right now has a short amount
- 14:48- of time and essentially no new budget to
- 14:52- get started down this path. What are the
- 14:54- first two or three things that you would
- 14:57- tell them to do to reduce AI agent
- 14:59- risk?
- 15:00- Before you buy another AI security product,
- 15:03- understand the identities, the permissions, and the controls
- 15:07- you already have.
- 15:08- That's key.
- 15:10- So
- 15:11- when you find the agents,
- 15:13- whether you again, you intended them to be
- 15:15- there or you just kind of were gifted
- 15:17- them because of shadow IT or shadow AI.
- 15:20- Right? You find them, understand where they are.
- 15:22- So inventory them wherever
- 15:24- they are and the agentic and wherever agentic
- 15:27- features exist, which vendors have added them, which
- 15:29- is the sneaky part getting in the organization.
- 15:31- You might have had something in your organization
- 15:33- for a long time and all of a
- 15:34- sudden the next released, boom. Now you have
- 15:37- an AI agent inside of it. So understand
- 15:40- that. What automations exist?
- 15:42- Which APIs they use? How are they talking
- 15:45- back and forth?
- 15:46- And I'll go back to that initial
- 15:48- identity.
- 15:50- What identities they operate under? So if it
- 15:52- says they can't operate under
- 15:54- my identities because I created it. They have
- 15:56- to have their own identities. And when you
- 15:58- discover agentic activities in your organization
- 16:00- that are operating
- 16:02- hidden or underneath
- 16:03- the umbrella of somebody else's identity, start there.
- 16:06- Start separating those. So again, yeah, before you
- 16:08- buy another AI security product,
- 16:11- understand the boring basics here. Understand the identities,
- 16:14- the permissions, and the controls you already have.
- 16:18- And I'll I'll one more.
- 16:20- Anytime you implement an AI,
- 16:22- always ask this critical question or have this
- 16:25- on that list.
- 16:27- How would I shut it down right now?
- 16:30- What's my exit criteria from this?
- 16:34- See,
- 16:35- that key questionnaires is not how I got
- 16:37- in the environment, how I started using it.
- 16:39- It's how do I stop it? How do
- 16:41- I shut it down if it starts to
- 16:43- break bad quickly?
- 16:45- Mhmm. Because nobody knows that answer.
- 16:48- Honestly, that's something you can fix without a
- 16:51- multimillion dollar project.
- 16:52- Just go through your system and say, how
- 16:54- can I turn this off? How can I
- 16:55- shut it down immediately
- 16:56- if I needed to?
- 16:59- Yeah.
- 17:00- It's a really
- 17:02- it's such a simple but powerful
- 17:03- concept, Jason. How would I shut it down
- 17:05- right now?
- 17:06- An essential question.
- 17:08- So I I really appreciate you raising that
- 17:10- for our listeners.
- 17:11- And, also, I I thought that your pragmatic,
- 17:15- recommendation to just understand the boring basics, quote,
- 17:18- unquote, you know, that that's another
- 17:20- essential piece of this.
- 17:22- Jason, it's been such a great conversation. I
- 17:24- know we only have a few more minutes
- 17:25- together, but I wanna give our our listeners
- 17:28- just
- 17:29- kind of a forward looking view of
- 17:31- where this approach that we've talked today, whether
- 17:34- that's segmentations,
- 17:35- creating, you know, constrained or contained environments,
- 17:39- Where might that approach stop helping, and what
- 17:42- risks would you leave listeners with to consider
- 17:45- even after they've achieved the constrained access and
- 17:48- segmentation?
- 17:50- Segmentation is critical. Right? We we've already mentioned
- 17:52- that. Lease privilege, again, also helps dramatically reduce
- 17:55- risk. So segmentation on lease privilege dramatically reduces
- 17:58- risk. But what they don't do is they
- 18:00- don't make an agent trustworthy.
- 18:02- They limit the consequences when something goes wrong.
- 18:06- There are still entire categories of AI risk
- 18:09- that must be addressed
- 18:10- separately.
- 18:12- Here's the thing. The objective isn't to make
- 18:14- AI incapable of failure.
- 18:16- It's gonna fail.
- 18:18- I mean, because that's that's completely unrealistic.
- 18:20- The objective
- 18:21- is to make failure observable,
- 18:24- containable,
- 18:25- and recoverable.
- 18:27- So I think that's where the focus needs
- 18:29- to be. There's no single control for AI
- 18:31- risk. We need governance, identity, architecture, security testing,
- 18:34- all that stuff. Human oversight, you know, how
- 18:36- do I turn it off?
- 18:37- Segmentation, special values, it creates a boundary around
- 18:40- the failure. And that I think is the
- 18:41- most
- 18:42- important
- 18:43- because you're still gonna have those risks. If
- 18:45- I authorize
- 18:47- an agent to get into
- 18:48- our EHR as an example to do things,
- 18:51- I've already authorized it. I can't that no
- 18:52- amount of segmentation is gonna help that because
- 18:54- I've segmented it down just you can access
- 18:56- the EHR. So what's it doing inside the
- 18:58- EHR? Right?
- 19:00- These are
- 19:01- risks that remain. So your authorization risks, goes
- 19:04- back to that least privilege,
- 19:06- those risks are still there. What do you
- 19:07- authorize it for? How do you identify it?
- 19:09- And, again, that could that question I asked
- 19:11- there,
- 19:12- how do I stop it? How can I
- 19:14- stop it or constrain it if it does
- 19:16- break bad,
- 19:17- and has impact patient care or organizational operations?
- 19:21- So
- 19:22- you can only solve so much.
- 19:25- Yeah. Jason, I I just wanna thank you
- 19:27- for
- 19:28- everything you've shared in this conversation, a lot
- 19:30- of
- 19:31- actionable
- 19:32- steps and considerations that you've left with listeners
- 19:35- today,
- 19:36- And just wanna thank you for raising attention
- 19:39- and shedding light on a a topic that's
- 19:40- very important as we see greater adoption of
- 19:43- of AI agents. So thank you again for
- 19:45- spending the time with Becker's today.
- 19:47- Oh, it's been my pleasure, Erica.
- 19:49- And we'd also like to thank our podcast
- 19:51- sponsor for today, Elisity.
- 19:53- Listeners, be sure to tune in to more
- 19:55- podcasts from Becker's Healthcare by visiting our podcast
- 19:58- page at beckershospitalreview.com.