Latest / Becker’s Healthcare Podcast / The Architecture of Trust: How Governance and Accreditation Power Enterprise AI

Transcript

The Architecture of Trust: How Governance and Accreditation Power Enterprise AI · Becker’s Healthcare Podcast

• 0:00
  Hello, everyone. This is Jacob Emerson with the
• 0:02
  Becker's Healthcare podcast.
• 0:04
  Thanks so much for tuning in today. We're
• 0:06
  we're here to talk about the architecture of
• 0:08
  trust and how governance and accreditation power enterprise
• 0:11
  AI. So to dive into that topic, I'm
• 0:14
  joined today by doctor Sean Griffin, who serves
• 0:17
  as the CEO of Eurac. His career spans
• 0:20
  clinical practice, medical group leadership, and innovation in
• 0:23
  care delivery.
• 0:25
  And after years of practice, doctor Griffin steps
• 0:27
  into the CEO role to broaden his impact,
• 0:29
  bringing his experience beyond the exam room to
• 0:32
  help advance quality, accountability, and innovation across the
• 0:35
  country and the world. At URAQ, he focuses
• 0:38
  on strengthening and evolving accreditation programs, influencing policy
• 0:43
  in in and building a culture that supports
• 0:45
  both high standards and meaningful growth for his
• 0:48
  team. Doctor Griffin, welcome to the podcast. Thank
• 0:51
  you very much. Looking forward to it. Likewise.
• 0:53
  And we appreciate you being here and being
• 0:54
  here on-site with us this week here in
• 0:56
  Chicago.
• 0:57
  I, I wanted to get started, Sean, by
• 0:59
  talking with you a little bit about, the
• 1:01
  widespread
• 1:02
  adoption of artificial intelligence.
• 1:05
  That being said, only about 17% of health
• 1:07
  systems report having a mature
• 1:09
  AI governance structure.
• 1:11
  So as we see organizations move from isolated
• 1:15
  deployments to enterprise wide AI,
• 1:18
  what does a scalable
• 1:20
  governance infrastructure
• 1:21
  look like to you? Well, I think when
• 1:23
  you're talking about AI and you're talking about
• 1:24
  governance, you know, you could even argue what
• 1:26
  is a mature governance structure right now because
• 1:29
  it continues to evolve.
• 1:31
  When we developed our program and we brought
• 1:33
  in our advisory group, what we talked about
• 1:35
  was what is the most successful sort of
• 1:38
  support structure,
• 1:39
  especially when it comes to trust. AI right
• 1:41
  now is getting so much press in so
• 1:43
  many different areas. When it comes to health
• 1:45
  care, the biggest gap we have isn't the
• 1:47
  technical aspects. It's more sort of the trust
• 1:50
  and making sure that it's implemented in the
• 1:51
  right way.
• 1:52
  We built our program because we were very
• 1:54
  concerned about protecting patients and providers. And so
• 1:57
  when it comes to our governance structure,
• 1:59
  there's the classic sort of governance structure that
• 2:01
  you have in health care where you have
• 2:03
  technical leadership and you have clinical leadership. One
• 2:05
  of the things that we added into our
• 2:06
  framework was ethical leadership.
• 2:09
  We're very concerned about ethics being looked at
• 2:12
  from the very beginning,
• 2:13
  because if you build a program and you
• 2:15
  later on go, oh, we should have thought
• 2:16
  about ethics, that's a little late for it
• 2:18
  to be showing up.
• 2:19
  One of the nice things about our program
• 2:21
  is that our standards are available for free
• 2:23
  out on the web. People can just go
• 2:24
  look at our standards and see what we
• 2:26
  talk about. But the the clinical, the technical,
• 2:29
  the ethical leadership is really important for governance.
• 2:32
  Also, you're gonna have all the change management,
• 2:34
  all of the training that needs to be
• 2:35
  done that comes into the governance structure. But
• 2:38
  also, I think what we're seeing is we're
• 2:39
  seeing this go from being sort of a
• 2:40
  departmental hobby into being a key part of
• 2:43
  infrastructure. Definitely. There's so many pressures going on
• 2:45
  right now in health care that people are
• 2:47
  looking for opportunities and AI presents those opportunities,
• 2:50
  but you have to have an understanding of
• 2:52
  the risk that's involved.
• 2:54
  When I talk to people in Washington, DC,
• 2:55
  they're very concerned about the risks
• 2:58
  and and how people are approaching it. And
• 3:00
  that's one of the reasons why we think
• 3:01
  that our accreditation
• 3:03
  helps people to sort of go by best
• 3:04
  practices.
• 3:05
  And coming from a name that people know
• 3:07
  at Eurac for thirty five years of being
• 3:09
  a health care creditor,
• 3:11
  we think that our guidelines
• 3:13
  are great for now, but we also recognize
• 3:15
  that things are changing. You know, here at
• 3:17
  the Becker's conference, I'm going to all of
• 3:19
  these sessions where people are talking about AI
• 3:21
  and it's evolving. And we will evolve with
• 3:24
  those,
• 3:24
  evolve our standards as best practices change. And,
• 3:28
  Sean, in that pursuit of ethical leadership in
• 3:30
  this space, what role do you think accreditation
• 3:33
  plays in helping the organizations here on-site but
• 3:35
  across the industry
• 3:37
  build that infrastructure.
• 3:39
  Well, I think when you're talking about accreditation,
• 3:40
  I I I have a lot of people
• 3:42
  say, what is accreditation? They sort of don't
• 3:44
  understand what we're talking about. And the best
• 3:46
  analogy that I have is in your finances,
• 3:48
  you have an independent financial auditor who comes
• 3:50
  in and checks your books. I think of
• 3:52
  accreditation as being a quality auditor who comes
• 3:54
  in from the outside and actually checks your
• 3:56
  quality and make sure you're doing things the
• 3:57
  right way.
• 3:59
  AI is has so many facets and so
• 4:01
  many different meanings to different people. For us,
• 4:03
  the accreditation was actually built because we felt
• 4:06
  like there was that need for those protections.
• 4:09
  And and we believe that having someone who's
• 4:11
  independent
• 4:12
  provide that oversight is really important.
• 4:14
  You you really can't have the the foxes
• 4:16
  guarding the chickens in in this case. And
• 4:18
  so having an independent group who's already trusted
• 4:20
  in health care, I mean, we have multiple
• 4:22
  government deemed programs and things like that. We
• 4:24
  just think that that's a an a further
• 4:26
  step in our support for innovation and technology
• 4:29
  in health care. Yeah. And so Eurac has
• 4:31
  created separate accreditation pathways for AI developers and
• 4:35
  the health systems themselves that deploy these tools.
• 4:38
  As we see organizations scale AI and then,
• 4:41
  of course, manage multiple vendor relationships all at
• 4:44
  once, how do you think that the health
• 4:46
  care organizations listening in, how should they be
• 4:49
  thinking about holding vendors to these same standards
• 4:52
  that they're holding themselves to that we're talking
• 4:54
  about right now? And then how does accreditation
• 4:57
  shape that conversation?
• 4:58
  Well, when we went to build our program,
• 5:00
  we really were thinking that we're gonna build
• 5:01
  one program, and it was gonna be focused
• 5:03
  on users for sort of the clinical deployment
• 5:05
  of AI because that's really sort of where
• 5:07
  the risk
• 5:08
  is is coming into play is is how
• 5:10
  it's affecting care.
• 5:12
  And when we got together our advisory group,
• 5:13
  we had a fantastic advisory group. We had
• 5:15
  over 70 different organizations who wanted to serve
• 5:18
  on our
• 5:19
  advisory group. We ended up with 29, and
• 5:21
  we had everything from tech companies to, pharmacy
• 5:24
  companies to,
• 5:26
  practicing physicians to health systems to academic medical
• 5:29
  centers. And when we got them all together
• 5:31
  and we said, we're gonna build this program
• 5:31
  for the users of AI, the deployers of
• 5:31
  AI, they said, yes. You know, this program
• 5:32
  for the users of AI, the deployers of
• 5:35
  AI, they said, yes. You need to do
• 5:36
  that. We need that sort of guidance, but
• 5:38
  we're really concerned about the developers.
• 5:41
  The developers,
• 5:42
  the the funny thing about technology is a
• 5:45
  developer can look exactly the same whether they're,
• 5:47
  you know, 300 people in Silicon Valley or
• 5:50
  six guys in a garage.
• 5:52
  And and the problem is is that you're
• 5:54
  putting your trust on an outside entity to
• 5:57
  deliver something that's so important for care.
• 6:00
  And I I say, you know, it's it's
• 6:01
  really strange when you think about it, but
• 6:02
  right now, there's actually more regulation on a
• 6:04
  hospital cafeteria
• 6:06
  than there is on their AI program. Yep.
• 6:08
  And so so building these two separate paths
• 6:10
  really sort of provides the guidance for what
• 6:12
  you should look at for developers.
• 6:14
  And we've gotten very positive feedback. A lot
• 6:16
  of health care organizations have said, you know,
• 6:18
  we're gonna start writing into our contracts that
• 6:20
  they need to be accredited as a developer
• 6:23
  before we even consider deploying them into our
• 6:25
  clinical scenarios. Mhmm. And that's that's our intent
• 6:28
  is we really wanna be sort of that
• 6:29
  that
• 6:30
  house good housekeeping seal of approval or whatever
• 6:33
  the current analogy is for that type of
• 6:34
  thing,
• 6:35
  to say that we've checked behind the scenes.
• 6:38
  A developer, you know, if you're gonna use
• 6:39
  a developer as a cornerstone for care delivery,
• 6:42
  you wanna make sure that they're not gonna
• 6:43
  go out of business next week. Yeah. You
• 6:45
  know, you you train all of your people.
• 6:46
  You you rework your workflows, and then all
• 6:49
  of a sudden, you know, they get bought
• 6:50
  out by somebody and they they disappear or
• 6:52
  they evaporate. That's not the reliability that you
• 6:54
  need. There's almost a supply chain issue of
• 6:57
  reliability that comes into those developers. Sure. Sure.
• 7:00
  And to expand there on your point, scaling,
• 7:03
  the AI enterprise wide means then that the
• 7:05
  clinicians, the administrators,
• 7:07
  compliance teams, the IT leaders, they're all gonna
• 7:10
  be engaging with these tools in different ways.
• 7:13
  So beyond that accreditation framework, how can health
• 7:16
  systems build a culture where
• 7:18
  trust in AI is embedded across the entire
• 7:21
  organization?
• 7:22
  I think there has to be a real,
• 7:24
  transparency and honesty
• 7:26
  that when we look at technology implementations in
• 7:28
  health care, I mean, the the biggest recent
• 7:30
  technology implementation in health care was probably the
• 7:32
  the EHRs, the electronic medical records. Sure. And
• 7:35
  it's funny that one of the biggest uses
• 7:36
  of AI right now in clinical settings
• 7:39
  is to relieve the burden of that last
• 7:41
  implementation.
• 7:42
  It's the ambient listening. It's the documentation. It's
• 7:44
  it's those types of things, and that's because
• 7:46
  we made physicians and and providers do too
• 7:49
  many clicks at the keyboard and and care
• 7:51
  got got sort of compromised. And so when
• 7:53
  we look at these things and we see
• 7:55
  sort of how this is playing out, you
• 7:57
  know, scale,
• 7:58
  very often you're gonna have pilots.
• 8:00
  We I've heard some speakers here at the
• 8:01
  conference where they talk about, you know, get
• 8:03
  get stuck in pilots. Pilots need to have
• 8:05
  a a goal. You need to be solving
• 8:07
  a problem. You need to have an end
• 8:09
  date for it. You need to make a
• 8:10
  decision. You can't just pilot yourselves the rest
• 8:13
  the next five years or or that sort
• 8:14
  of thing. So we really talk about how,
• 8:16
  you know, you wanna have the leadership brought
• 8:18
  together. You wanna understand the risk involved in
• 8:20
  every single tool. Make sure it was designed
• 8:23
  for the work that you're going to have
• 8:24
  it do. You don't wanna take something that
• 8:25
  was designed for pediatric x rays and use
• 8:28
  it for adults. Mhmm. You're introducing those risks.
• 8:30
  And then you wanna have a plan before
• 8:32
  you implement on how you're gonna train and
• 8:34
  support, get feedback for problems, because the AI
• 8:37
  introduces things like hallucinations
• 8:39
  and drift and and those types of concerns.
• 8:42
  And and I think we all know if
• 8:43
  you look at technology
• 8:45
  that, when when a vendor tells you their
• 8:47
  quality,
• 8:48
  it's always a different number than what you
• 8:50
  actually find in the real world. It's sort
• 8:52
  of the your mileage may vary when you
• 8:54
  get into your actual organization.
• 8:56
  And so I think you have to, you
• 8:57
  know, make sure that your users can get
• 8:59
  the feedback,
• 9:00
  especially when it comes to things like patient
• 9:02
  safety.
• 9:03
  I mean, it's one thing to use AI
• 9:04
  to to determine your shopping list for your
• 9:06
  groceries. It's another thing to determine the chemotherapy
• 9:09
  for grandma. Yeah. And so we we really
• 9:11
  have to have those risks sort of spelled
• 9:12
  out ahead of time because that's gonna be
• 9:14
  one of the keys for trust from the
• 9:16
  providers, but also from the patients.
• 9:18
  Patients don't understand this, and they see it
• 9:20
  as another tool coming in. And they see
• 9:23
  the horror stories that make the evening news.
• 9:24
  Yep. And so having some independent organization come
• 9:28
  in and say, this is built right. This
• 9:30
  is safe. This is reliable.
• 9:32
  We really think it's important for both the
• 9:33
  provider trust and the patient trust. Absolutely. And
• 9:36
  so as you said, Sean, there's more regulations
• 9:38
  applied right now to the hospital cafeteria than
• 9:40
  there are to the use of AI inside
• 9:42
  the walls of the hospital.
• 9:43
  So
• 9:44
  in that current policy environment, especially one where
• 9:47
  the states have now taken the lead here
• 9:49
  versus the federal government,
• 9:51
  what is the strategic case for pursuing accreditation
• 9:53
  now before
• 9:55
  these regular type regulatory requirements exist?
• 9:58
  And
• 9:59
  how do you think that positions organizations more
• 10:01
  effectively as the regulation trickles out and evolves?
• 10:05
  I think when you think about accreditation,
• 10:07
  recognize that accreditation is being built by people
• 10:11
  who are dealing with providers on a regular
• 10:13
  basis.
• 10:14
  It's very doubtful
• 10:15
  that when the legislators
• 10:18
  start writing laws and rules about how this
• 10:20
  is going to be, that they're going to
• 10:22
  necessarily adopt a 100% of the best practices
• 10:25
  that people would expect in in doing this
• 10:27
  work. Accreditation has always been complementary to regulation,
• 10:30
  not as a replacement for regulation.
• 10:32
  And and if you want to if you
• 10:34
  want to understand the risk that's involved here,
• 10:36
  there's already lawsuits that are coming out,
• 10:39
  around risk associated with AI, decisions that are
• 10:42
  made through AI. You know, whether you follow
• 10:44
  it or whether you didn't follow it, there's
• 10:45
  a risk involved there. So if I were
• 10:47
  running a compliance team at a big health
• 10:49
  care organizations
• 10:50
  and I had an opportunity
• 10:52
  to have a trusted independent accreditation to show
• 10:55
  that we're following best practices,
• 10:57
  not just saying we're following best practices, but
• 10:59
  somebody actually came in and audited us. I
• 11:01
  think that's where you start thinking about this
• 11:03
  as having a role for the early adopters.
• 11:06
  When we launched our program, we have over
• 11:08
  I think we're up to around 15 organizations
• 11:10
  who've already signed up to be going through
• 11:12
  this accreditation
• 11:13
  because they say they wanna show that they're
• 11:15
  not just doing the bare minimum trying to
• 11:17
  slide by until the regulations land.
• 11:19
  I've talked with over half the states in
• 11:22
  The United States who are very interested in
• 11:23
  what accreditation can do for them because they're
• 11:26
  not necessarily equipped
• 11:28
  to patrol the streets here in the wild
• 11:30
  West that we have now in health care
• 11:31
  AI.
• 11:33
  Before we go, Sean, you are being listened
• 11:35
  to right now by hospital and health care
• 11:37
  leaders all across the country. What else do
• 11:39
  you wanna share with them?
• 11:42
  When I sat in their seats, when I
• 11:44
  I've been at a small community hospital, I've
• 11:45
  been in a large medical system in in
• 11:48
  an urban area, you know, this is a
• 11:50
  chance where something you can do for free.
• 11:53
  You can log into yourac.org
• 11:55
  website. You can look at our standards at
• 11:57
  a glance. They're absolutely free. They're out there.
• 12:00
  It talks about what we look at in
• 12:02
  our program.
• 12:03
  You can buy our standards. You don't even
• 12:04
  have to talk to us. You can go
• 12:06
  in and you can actually see sort of
• 12:07
  what we look at, how we look at
• 12:09
  risk assessments, how we look at ethical leadership,
• 12:11
  how we look at developers,
• 12:13
  and you can bake that into your process
• 12:15
  at no ongoing cost to you. But I
• 12:17
  think that we have to understand that AI
• 12:19
  offers a lot of promise, but there's some
• 12:21
  risk. And so all our program is designed
• 12:24
  is to be guardrails, not to be closed
• 12:26
  streets.
• 12:27
  I've made the analogy before. If you're gonna
• 12:29
  race in Formula one, you still wear a
• 12:31
  seat belt.
• 12:32
  And if you wanna go fast, you need
• 12:34
  some sort of safety involved.
• 12:36
  And there's gonna be bad things that are
• 12:37
  gonna happen because bad things always happen. But
• 12:40
  showing that you actually have an independent auditor
• 12:42
  come in and check your program, to look
• 12:44
  out for conflicts of interest, to look out
• 12:46
  that you've weighed the risk, that you have
• 12:48
  a plan,
• 12:49
  I think is really important from a systemist
• 12:51
  standpoint
• 12:52
  because nobody wants to have something hurt a
• 12:54
  patient. Nobody wants to lose the trust of
• 12:56
  their providers.
• 12:57
  And having that sort of independent group who's
• 12:59
  been around for three and a half decades
• 13:01
  say that these are the rules right now,
• 13:03
  but also we're gonna continue to evolve those
• 13:05
  to keep your patients and your providers as
• 13:07
  safe as possible.
• 13:08
  I think it's a win win situation for
• 13:10
  people to just at least consider it and
• 13:12
  look at it. There's tons of best practices,
• 13:14
  and there's probably 45 different best practices out
• 13:16
  there. But, honestly, to say you're following best
• 13:19
  practices is like a New Year's resolution.
• 13:21
  I'm gonna work out more. I'm gonna eat
• 13:23
  healthier, but then February rolls around and you
• 13:25
  forget about it. Accreditation is about you're not
• 13:27
  just saying you're doing it, but someone's checking
• 13:29
  to make sure you are doing it. Yeah.
• 13:31
  I think that's a great place to leave
• 13:33
  things. Doctor Griffin, thank you for taking the
• 13:35
  time to be with us today and for
• 13:37
  sharing your expertise with our listeners. We really
• 13:39
  appreciate it. Thank you very much. And to
• 13:41
  our listeners, if you'd like to listen to
• 13:43
  more podcasts from Becker's Healthcare, you can visit
• 13:45
  beckershospitalreview.com.