Latest / Becker’s Payer Issues Podcast / Premera Blue Cross’ In-House AI Chatbot Boosts Customer Service Efficiency

Transcript

Premera Blue Cross’ In-House AI Chatbot Boosts Customer Service Efficiency · Becker’s Payer Issues Podcast

• 0:00
  Hello, everyone. This is Jacob Emerson with the
• 0:02
  Becker's Payer Issues podcast.
• 0:04
  Thrilled today to be joined by doctor Nathan
• 0:06
  Kronk, who is the manager of AI engineering
• 0:09
  and operations
• 0:10
  at Premera Blue Cross. Nathan, thanks so much
• 0:12
  for taking the time to be with me
• 0:14
  on the podcast today. My pleasure. Thanks for
• 0:16
  having me. Yeah. And before we dive into
• 0:18
  everything we wanna talk with you about in
• 0:19
  a special new project that's been happening at
• 0:22
  Premera, can you tell us a little bit
• 0:24
  more about yourself and what it is that
• 0:26
  you do at the company?
• 0:28
  Sure. Yeah.
• 0:30
  So I have a my background is in
• 0:32
  academia. You know, I I I was researching
• 0:34
  AI,
• 0:35
  you know, back in 2016 for my PhD,
• 0:37
  and,
• 0:38
  it kinda sort of taken off, making some
• 0:40
  big splash out in the world. And I
• 0:41
  thought, you know, maybe I'd like to get
• 0:43
  out of the, the the rubber room, the
• 0:45
  ivory tower of the the the crazy space
• 0:48
  of the academics and go make an impact
• 0:50
  with some of this stuff. And health care
• 0:51
  seemed like a really,
• 0:53
  you know, it transcends race, gender, ethnicity, politics.
• 0:56
  Everybody needs health, so it seemed like a
• 0:57
  nice place to go try and make a
• 0:59
  contribution. So
• 1:00
  I joined, Premera, you know, about three years
• 1:02
  ago,
• 1:03
  and I've had the really enjoying opportunity like,
• 1:06
  the enjoyable opportunity to help kind of encourage
• 1:09
  and engender this this, adoption of AI technologies
• 1:12
  by I go do, like, AI roadshows around
• 1:14
  the company, presentations,
• 1:15
  you know, on jailbreak attacks, you know, trying
• 1:18
  to help people become familiar with, the capabilities
• 1:21
  of this technology, building prototypes to show the
• 1:23
  art of the possible.
• 1:24
  It's been a really fun role that I've
• 1:26
  had had here at the company.
• 1:28
  But since then, we've actually moved from, you
• 1:30
  know, roadshows in education into production. We've got
• 1:33
  some pretty impressive,
• 1:34
  governance standards in place. We've got, you know,
• 1:36
  an AI security gateway that's,
• 1:39
  pretty cutting edge and some real Gen AI
• 1:41
  products in production making difference in the health
• 1:43
  care space. So,
• 1:44
  that's a bit about me.
• 1:46
  Fantastic. Well, like I said, we really appreciate
• 1:48
  you taking the time to be with us
• 1:50
  today.
• 1:51
  And let's talk about some of those generative
• 1:53
  AI products that have been happening at the
• 1:55
  company.
• 1:56
  There's a new generative AI chatbot called Alice.
• 2:00
  So can you walk us through what exactly
• 2:02
  that is, how it was developed, and some
• 2:04
  of the timeline from concept to the actual
• 2:07
  rollouts? Ultimately, can you give us that sense
• 2:09
  of of scale in terms of what what's
• 2:12
  happening with this at the company?
• 2:14
  Yeah. There are some really important details,
• 2:17
  to make to make these timelines make sense.
• 2:19
  Right? So
• 2:20
  about two and a half years ago, we
• 2:22
  saw
• 2:23
  I mean, you know, my myself and my
• 2:25
  team, we came from academia. So we kinda
• 2:27
  knew what the cutting edge tech was and
• 2:29
  what the art of the possible was. And
• 2:30
  we saw this agentic
• 2:32
  future coming, or at least we believed that
• 2:34
  it was coming, and we started,
• 2:36
  building an infrastructure very early on. So some
• 2:38
  of the timelines and things I'll share with
• 2:40
  you sound pretty quick, but I wanna, you
• 2:42
  know, couch them in the the reality, which
• 2:44
  is that we started building an AI infrastructure,
• 2:46
  for the company that was reusable and reproducible
• 2:49
  about two years ago.
• 2:50
  So but the the reality is is the
• 2:52
  CSR team, you know, they had, the Dynamics
• 2:55
  three sixty five tool,
• 2:57
  advertised to them as a nice tool to
• 2:59
  help support the CSRs in finding information quicker.
• 3:02
  And they they went and tried it out,
• 3:04
  and it unfortunately,
• 3:06
  right, very early on, these these generalist AI
• 3:09
  solutions that try and, like, boil the ocean
• 3:11
  for everyone,
• 3:12
  they they can they can work, but not
• 3:15
  fantastically. Right? I mean, when you try and
• 3:17
  have a generalized,
• 3:19
  you know, the vector database, right, which basically
• 3:21
  takes any type of data of any form
• 3:23
  and chunks it, partitions it, makes it searchable
• 3:26
  for your AI agents,
• 3:28
  you're losing a lot of domain expertise that
• 3:30
  can be exploited,
• 3:32
  by people who really know what the application
• 3:34
  is is for. So that was where we
• 3:36
  came in. They they came to us and
• 3:38
  said, hey. We're trying this Dynamics thing. It
• 3:39
  isn't quite working. Why why not? And we
• 3:42
  said, well, it's not because there's a lot
• 3:44
  of idiosyncratic domain expertise that needs to be
• 3:47
  baked into the chunking parse processes and and
• 3:50
  how the text is formatted and how the
• 3:51
  agent reasons over it. Let's let's show you.
• 3:54
  And they said, okay. So, honestly, it took
• 3:56
  us about a week to make a prototype
• 3:58
  given that we had the infrastructure and everything
• 4:00
  in place. And we had a, a reasoning
• 4:03
  agent that was built on a technology called
• 4:05
  the LLM compiler,
• 4:06
  and it searched through thousands of methods and
• 4:09
  procedures and would give you answers, would reason
• 4:11
  over them and give you answers. And we
• 4:12
  said, this is the kind of thing, that
• 4:14
  we can do. And they said, great. Let's
• 4:16
  let's build this out and go into production
• 4:18
  with it. So,
• 4:19
  it took us about three months to get
• 4:21
  into,
• 4:23
  product like, the UAT phase. So that was
• 4:25
  when we rolled it out to a couple
• 4:26
  of different lines of business to start experimenting
• 4:28
  in a small in some small ways. And
• 4:30
  then over the past about two to three
• 4:32
  months, we've been slowly rolling it out to
• 4:34
  more and more, and, you know, we're up
• 4:35
  to hundreds of CSRs that are now using
• 4:37
  using this tool to help them find methods
• 4:39
  and procedures information, more quickly to help members
• 4:42
  and and people who call in.
• 4:43
  Wow. No. I mean, that's amazing. So a
• 4:45
  lot of time went into this, but in
• 4:47
  terms of the infrastructure, but then it sounds
• 4:49
  like it really a lot of it came
• 4:50
  together quickly, once you started developing that prototype.
• 4:53
  You mentioned, you know, this is for customer
• 4:56
  service operations.
• 4:57
  So can you give us a sense of
• 4:59
  some of the impact you've seen so far,
• 5:00
  what you've heard on the ground,
• 5:02
  from from people actually using this tool?
• 5:05
  Yeah. That's, that's some of the most rewarding
• 5:07
  part of it here. I mean, you know,
• 5:09
  we are we love the AI and the
• 5:10
  building stuff, but that's you know, hearing the
• 5:12
  impact on the front lines has been great.
• 5:14
  Yeah. There's there's so many quotes that we've
• 5:16
  heard. Firstly, you know, just allowing the CSRs
• 5:19
  to hear that the members are happier, like,
• 5:22
  on average because they can get information faster
• 5:25
  so that they don't sit there and get,
• 5:26
  you know, frustrated on the phone. Well, please
• 5:28
  give me a minute to go look this
• 5:29
  up. And, you know, they're they're calling in
• 5:32
  on their lunch breaks or or maybe, you
• 5:34
  know, in a break in work. They don't
• 5:36
  have a lot of time. That that's a
• 5:37
  bit of a luxury. Right? So,
• 5:39
  for the CSRs to be able to get
• 5:41
  them answers faster has been,
• 5:43
  rewarding both for the members and then also
• 5:45
  the CSRs to give them more job satisfaction,
• 5:47
  which is which has been pretty great.
• 5:49
  The the leaders are also quite happy because
• 5:51
  we're seeing, trending down in call handle times,
• 5:54
  which is,
• 5:55
  you know, as as leaders are are well
• 5:57
  aware, that's a pretty important metric for,
• 6:00
  success in call centers. So average handle time
• 6:02
  has been,
• 6:03
  trending down as more and more adoption of
• 6:05
  the the tool is used.
• 6:07
  And in addition,
• 6:08
  something else that's pretty neat is we have,
• 6:11
  the c the TSSs themselves, which are the
• 6:13
  technical support specialists that help CSRs when when
• 6:16
  calls get, tricky.
• 6:18
  Usually, they're inundated with questions.
• 6:20
  But now that the CSRs
• 6:22
  are able to go to Alice for some
• 6:25
  first level triaging of of challenging questions, the
• 6:28
  TSSs now have a bit more time to
• 6:30
  go actually improve the documentation
• 6:32
  that Alice is using to give the CSRs
• 6:34
  the answers, which has been great because, I
• 6:36
  mean, there's thousands of documents and they're they've
• 6:38
  been maintained by humans. So as you know,
• 6:40
  there's there's, of course, gonna be a few
• 6:42
  errors and mistakes in them. So Alice has
• 6:44
  been surfacing those where maybe Alice gives a
• 6:46
  quote, unquote wrong answer. And they go, well,
• 6:49
  Alice made a mistake, and they dig in.
• 6:50
  And they're like, no. Actually, our our policies
• 6:52
  or things have have evolved or changed over
• 6:54
  time, so now we can update this. And
• 6:56
  it's just making everything better where the documentation
• 6:58
  is more aligned more quickly, CSRs get better
• 7:01
  answers more quickly, and the leaders have been
• 7:03
  pretty pleased with it too. So, it's been
• 7:05
  a a pretty rewarding, turnaround to see how
• 7:08
  people have been adopting it.
• 7:09
  Absolutely. So, really, this sounds like a win
• 7:11
  win dynamic both from the member and the
• 7:13
  internal
• 7:14
  perspective, especially with that average handle time trending
• 7:17
  down, the technical support burden, being lifted so
• 7:19
  they can focus on other things. For the
• 7:22
  other health plan leaders that are listening in,
• 7:24
  Nathan,
• 7:25
  from that operational and that ROI perspective, I
• 7:27
  know you touched on a few metrics here.
• 7:29
  Are are there any lessons you would share
• 7:30
  in terms of scaling an in house AI
• 7:33
  solution, that could that they could apply to,
• 7:36
  their companies all over the country?
• 7:38
  Yeah. That's a that's a really important question,
• 7:40
  and I think we're at a very critical
• 7:42
  point where these decisions need to be made
• 7:44
  now and they need to meet be made
• 7:45
  correctly. Right?
• 7:46
  So
• 7:47
  everybody's got this build versus buy idea, and
• 7:50
  it seems attractive. Right? Well, why would I
• 7:53
  try and get an in house AI engineering
• 7:55
  team to build these things out when
• 7:57
  pretty much every company out there now is,
• 8:00
  you know, advertising
• 8:01
  AI AI everything. You can buy AI powered
• 8:03
  pizza on the way home. Right? Well, of
• 8:06
  course, you know, everybody's gonna advertise that because
• 8:08
  that's the new attractive hot topic that people,
• 8:11
  you know, wanna wanna market. But what I
• 8:13
  would encourage leaders to acknowledge is the following.
• 8:16
  Most of businesses
• 8:19
  is institutional knowledge. Right? Your your real intellectual
• 8:22
  property and value is in
• 8:25
  oftentimes, your data, of course, but all the
• 8:27
  knowledge and expertise
• 8:28
  that your staff and your employees bring, all
• 8:30
  the processes and and procedures that you've distilled
• 8:34
  over decades
• 8:35
  of of honing
• 8:36
  all this this knowledge and expertise. So
• 8:39
  if you can build
• 8:41
  AI agents internally that crystallize
• 8:44
  all of that expertise that you've owned over
• 8:46
  time,
• 8:47
  you're basically
• 8:48
  locking down that intellectual property.
• 8:50
  And that's now an asset for your your
• 8:53
  enterprise,
• 8:54
  And you can evolve it, grow it over
• 8:56
  time, allow those agents to cooperate with other
• 8:58
  agents to solve more complex problems as we
• 9:00
  start emerging,
• 9:02
  evolving towards these emerging, like, multi agent systems.
• 9:06
  So
• 9:07
  my advice is early on, if possible, if
• 9:10
  it makes sense for you, really do try
• 9:12
  to to to think through if you can
• 9:14
  try to build your own custom,
• 9:16
  agents or at a minimum ensure that you
• 9:18
  have full IP over the agents that you
• 9:20
  are trying to build internally. Yeah. That that's
• 9:22
  my advice. Sure. It makes a lot of
• 9:24
  sense, and I I think that's great advice
• 9:26
  for our audience as well. The the last
• 9:28
  thing I wanted to ask you, Nathan, is
• 9:29
  how you see AI enabled customer service tools
• 9:33
  like the one you've built. How is that
• 9:35
  gonna continue to shape Premera if you look
• 9:37
  forward the next three to five years? I
• 9:39
  know this is very, quickly evolving technology, and
• 9:42
  that is a long time frame. But,
• 9:44
  what's your predictions here for for both your
• 9:46
  company but also for the wider industry?
• 9:49
  Yeah. I mean, I can I can tell
• 9:51
  you? Right? So, I mean, we
• 9:52
  have across all of our organizations, right, cybersecurity,
• 9:56
  IT, corporate data and analytics,
• 9:58
  you know, we've been coming together, and we've
• 9:59
  got a core group of people
• 10:01
  that have been making, you know, our enterprise
• 10:03
  AI architecture and trying to ensure that our
• 10:06
  we're future proofing our adoption
• 10:09
  of AI and where we're scaling towards. So
• 10:10
  the main thing to acknowledge is that, you
• 10:13
  know, health health insurance
• 10:15
  I mean, it's it's a math problem at
• 10:17
  at the end of the day. Right? So
• 10:19
  first and foremost is you gotta solve the
• 10:21
  math problem. Computers are better at solving that
• 10:23
  math problem, and we wanna acknowledge that that
• 10:25
  is the core capability that we want, you
• 10:27
  know, technology to support in. But then also
• 10:29
  there's the human element. I mean, health is
• 10:31
  really important, and we need to be able
• 10:33
  to reach out to people, be sensitive,
• 10:35
  and be able to connect with people
• 10:37
  as well. So what we're thinking is that
• 10:40
  this insurance is basically a process driven industry.
• 10:43
  So the vast majority
• 10:45
  of the the math problems that are being
• 10:47
  solved and the input and processing of documents
• 10:48
  and output of
• 10:51
  documents. This these are things that AI agents
• 10:53
  can help automate. And what we would then
• 10:55
  be able to do is to leverage all
• 10:57
  of the human expertise to to really touch
• 10:59
  on the human side of things and make
• 11:00
  sure that all the members,
• 11:02
  are really feeling cared for or that they
• 11:04
  have the support that they really need, so
• 11:06
  we can automate the laborious slow sort of
• 11:09
  procedural factory line type work, and allow humans
• 11:13
  to connect more on the health element, which
• 11:15
  is, you know, I think at the end
• 11:16
  of the day what what we really want.
• 11:17
  That that that's one of the things that
• 11:19
  is gonna separate the health industry from others
• 11:21
  in AI adoption is that it's very personal
• 11:23
  and it's very, very important. And so how
• 11:26
  AI is integrated there is gonna be, a
• 11:28
  pretty important
• 11:29
  thing that these companies need to think through
• 11:31
  carefully.
• 11:32
  Wonderful. Well, Nathan, I wanna thank you so
• 11:34
  much for taking the time to chat with
• 11:36
  us and with our audience and for sharing
• 11:38
  our insights with all the leaders listening in.
• 11:40
  We we really appreciate it. It's my pleasure.
• 11:43
  Thank you for having me. Yeah. And and
• 11:44
  to our listeners, if you'd like to listen
• 11:46
  to more podcasts from Becker's Health Care, you
• 11:48
  can visit beckershospitalreview.com.