Latest / Agentic Conversations (formally mlops.community) / AI Agents in Healthcare?

Transcript

AI Agents in Healthcare? · Agentic Conversations (formally mlops.community)

0:00
All right, so at the beginning of April I went to New York for
0:03
the MCP Dev Summit, and while I was there I got the chance to
0:07
record a few podcasts with attendees that were at the
0:11
event. This is one of those
0:12
conversations. Hope you enjoy.
0:14
How do you make sure that your agents behave the right way?
0:18
It is learning from the queries you're you're giving, then it
0:21
starts making its own interpretation.
0:23
So we try to eliminate that by restricting the amount of tools.
0:27
So say there's non compliant patient.
0:29
So that's that's tricky because. And I don't even put you on the
0:32
spot. No, this is like, it's like a
0:33
huge problem because like, what's good for them?
0:36
And that's what they're doing. When a an agent acts on
0:41
following a specific workflow that was designed, who takes the
0:46
liability? Does the patients, does the
0:48
system that's creating that that that's hosting the agents takes
0:52
the liability or does it provide who take that liability?
0:55
So that's the line where it's like with the industry still
0:59
needs to define exactly. Hi, I'm Alex Alkiver with the
1:14
Agentic AI Foundation and I'm here today with Kingsley
1:17
Matekebu. Is that correct that the right
1:18
way? That's correct.
1:19
We are at MCP Dev Summit North America in New York City.
1:23
Kingsley's company, Heel ID is exploring novel use cases for
1:27
MCP and a gentic AI in the healthcare system.
1:30
Kingsley, welcome. Thank you.
1:32
Tell me a little bit about Heel ID and what you're thinking
1:35
about doing with with the technology there.
1:38
Cool. Yeah, well, I think in
1:39
healthcare there was a lot of comes into multiple, the
1:43
multiple use cases are on it, you know, from your provider,
1:47
you know, their workflow, the clinical workflow, the actual
1:50
patients on how they actually interact with their medical
1:54
records. And everything in mind,
1:57
everything that's currently developed, used has a patient in
2:01
in you know, in points of view where after a hospital is
2:05
discharged, doctors usually provide some kind of with the
2:09
summary of their visits, which is then given to the patient.
2:12
And the patient has to kind of like find a way to kind of, you
2:15
know, digest that information and kind of follow through on on
2:18
on the I recommendation that was provided by clinicians.
2:22
And what Hill ID does is Hill ID makes the connection between not
2:26
just the patients and the and the doctors, but also a lot of
2:31
the intermediary like, you know, your caregivers, your family
2:36
network. And one thing we find out is
2:38
like a lot of those out through the visits, especially for
2:41
patients with chronic condition, the person that actually keeps
2:45
the most burden of putting that pieces together from the
2:49
clinical records perspective from the that a clinician has
2:53
being from the symptoms the patient are experiencing.
2:56
And you know, the caregiver that's actually observing in
2:59
between is a family. So they tend to end up holding
3:02
that burgeon of like piercing all this fragmented data and
3:07
trying to make sense of what is going on.
3:09
And even they also have their own point of view into like how
3:13
the whole process can be more efficient.
3:18
And that's where MCP comes in. And MCP kind of like looks at
3:22
this from an agentic workflow perspective where, you know,
3:26
each of those individuals, like the patients, the provider and
3:31
the caregiver and the family all have their own agent accessing
3:35
the same data and doing the piece meal between what they
3:39
need to use, how they need to use the data and what they need
3:41
to see. So give me a.
3:42
Like a hypothetical. Say someone has open heart
3:45
surgery, they're returned to comb.
3:48
They probably have a like 20-5 pages of care instructions.
3:51
There's an occupational therapist, there's pharmacy
3:54
because they have a whole bunch of medicines that they have to
3:56
take after surgery. There's the physical therapist,
3:58
there's the nurses who still need to be informed about blood
4:01
pressure and wool kneeling. And then there's the, the, the,
4:04
the surgeon who cares about things.
4:06
And then there's, you know, the caregiver, the spouse or
4:09
whatever. Tell me how that would all work
4:11
together. Yeah.
4:12
So I think from the fundamental layers is the data layer.
4:17
So in how we kind of like, you know, segmented our actually
4:21
four different pillar where we have the data layer and we have
4:25
the role data in the role form like constructors from where we
4:28
have we take in that medical record that was prescribed by
4:32
the doctor, press that information into a data data
4:37
store. The data is just dumb data, no
4:40
logic, nothing built in. It's often paper out of the
4:43
hospital and they give you a piece of like a stack of paper.
4:46
Exactly, exactly. And I and I think what our
4:48
system does, like we started with personal information into
4:51
a, a structured form and just, but with no actual logic,
4:55
nothing built in. And on top of that, that's where
4:57
we actually set a context permission on top of the data
5:01
layer that identified authenticated users, what access
5:06
they have to what. So in the youth, in the example
5:08
you gave of your provider that's making the medication request,
5:15
they have a specific workflow and specific permission that
5:18
they need to do on you that particular patient's medical
5:21
record. And we map that.
5:25
On top of that, we lay on top the MCP and the MCP server has
5:30
its own set of permission and on what tools it has access to,
5:34
what specific permissions able to do for that given workflow.
5:39
And on top of that MCP, then we have the agents.
5:42
Now the agent has more focused on the reason, the capability.
5:47
And so the the doctor has a specific set of workflow, but
5:54
it's not able to actually visualize the data in a
5:56
different workflow of because they don't have active, although
6:01
they have the same database, but are not able to actually see
6:04
this particular records. An example would be like workers
6:07
around like mental health, if they're not in mental health
6:11
specialist, so only workers that's pertaining to their
6:14
permission, that's what they're able to have access to and make
6:18
the OR make this request. And it's also time bound.
6:20
So we're able to set time bound on each of this different
6:23
workflow, how long are they able to live and and and to what
6:27
extent of permission. It's much extent of which
6:29
they're able to have. So I, I have a long list of
6:31
questions which of course I, I, I basically dropped your talk
6:34
description into Claude and said, you know, prompt me what,
6:37
what should I ask easily. So why do you say MCP is the
6:42
right architectural use case for choice for this use case?
6:48
Yeah. So initially the initial
6:49
architecture was API REST based architecture where.
6:53
We rest based. You didn't even think of graphs,
6:55
you were thinking REST OK. So I mean, but how will you know
6:59
without NCP, what how do I consider look like will be you
7:02
making a request to a specific endpoint and say, you know, I
7:05
want to update this record. Then you lay it on top of an
7:08
agent that aid agent kind of queries that that was based Uri
7:14
and say it says, OK, this is the this is the result that returns
7:18
it back. I think from a in order to
7:23
enforce compliance data restrictions, it requires a lot
7:28
of encoded a lot of logic, not just in the in the agent level,
7:33
we also the data level and also the security level and which
7:37
makes it very complex. And you have to hold a whole
7:40
list of permissions. For example, if someone is a
7:43
caregiver and they're all they want, the information they
7:46
really want to say is as my parents taking their medication
7:51
versus the doctor wants to see a little bit more than just have
7:55
to take the medication. I want to know how much dose
7:58
have they taken and as a patient concerning if the medication has
8:04
been prescribed conflict with a medication you're already
8:07
taking. So this this kind of like
8:09
complex workflow makes it very difficult to implement on AI and
8:15
also there's a huge risk of violating a lot of policy there.
8:22
But I think MCP makes it cleanly be able to set specific rules
8:27
and permission in MCP server that segregates the amount of
8:33
permission they're able to access in the data layer.
8:37
And by doing so quickly, easily, we're able to kind of audit
8:43
every single step an agent is taken.
8:47
So the agents being able to access a particular record I'd
8:52
have there, there are four different checkpoints that I
8:54
have to go through before they be able to prove that, you know,
8:57
that permission is requested. But also we're able to see that
8:59
log in that in that traceability and which is easier for
9:04
compliance and easier for you for you to be able to
9:06
demonstrate not just compliance, but also all the disability for
9:09
Hippo and other providers access because that's something they
9:13
really care about. Just being able to say, you
9:15
know, this particular person access this record.
9:20
And it was not just an agent like just doing this thing,
9:23
doing random snooping on into into my database.
9:26
So being able to prove that into in a, in a, in a, in a health
9:31
setting is something that is, is, is is critical for a lot of
9:34
organizations operating in a work regulated space.
9:37
So, so like for example, I've heard from and my doctor
9:42
friends, working with the Emrs is very challenging because you
9:46
literally get permission fatigue.
9:47
It's like Claude code, except like every 5 seconds you accept
9:49
this, you acknowledge that or whatever.
9:51
Does this help potentially streamline some of the
9:54
navigations of all of these very complex regulations under the
9:58
covers that you see in Emrs? Yes, I think if it's abducted at
10:03
a wild scale, there's a potential ability to be able to
10:08
create that segregation later and, and, and the audit which is
10:11
which is mentioned. So I think there's all this
10:15
question about like liability. So like who is liable for?
10:21
Being a being HIPAA is essentially like, that's what
10:23
it's all about, my villains. Exactly.
10:25
So I think yes, it's simplifies it.
10:28
But again, a, a, a proper, I guess we haven't seen a actually
10:34
applied into a large scale system, but still being able to
10:38
recognize like, you know, when a, an agent acts on following a
10:44
specific workflow that was designed, who takes the
10:49
liability? Does the patients, does the
10:51
system that's creating that that that's hosting the agents takes
10:55
the liability or does it provide who takes that liability?
10:58
So that's the line where it's like with the industry still
11:02
needs to define exactly. Well, you probably saw some guys
11:05
out of Stanford and somebody else just launched an insurance
11:09
company for AI agency. Did you see that?
11:12
No, I'm curious now. They will ensure specific AI
11:14
agents. You know, I think they're
11:17
initially starting with 11 labs for voice agents.
11:20
But I mean, like the, the whole goal is like we will underwrite
11:23
your agent if you, if you can demonstrate sufficiently that
11:26
you have the guardrails in place anyway, just you know, I, I can
11:29
send you a link to that later. Yeah, that'll be cool.
11:31
So what healthcare integration problems specific because we
11:35
talked a little bit about what you're doing and also YMCP, but,
11:38
and we talked a bit about healthcare, but what healthcare
11:40
integration problems specifically?
11:42
Do you think MCP is better for me?
11:44
You talked a bit about HIPAA, but let's let's sort of take it
11:48
up a level almost because, you know, like Healthcare is its own
11:51
unique beast and, you know, strip away the, the jargon to
11:54
just like normal problems that people think about that they get
11:57
solved by that. Yeah.
11:58
So I think a good knowledge to think about is you can think of
12:03
it as MCP as kind of like a like a bouncer.
12:07
In a nightclub. Good analogy.
12:10
So like each and and the agents will be more like, you know, a
12:14
guest coming into into the into the club.
12:17
MCP acts as a rather than just tagging the guests as, as you
12:25
know, a member and checking up Avi Bill is MCP is actual
12:30
reorganizing the whole club for that one particular guest.
12:34
So when one guest comes in the whole, it's a whole different
12:37
club setup versus another guest. And that's how in when you put
12:42
that in the in the hospital setting is how a patient
12:47
interacts with their actual record.
12:50
It's a lot different from how a provider would interact with
12:54
with the record. So things doctors care about.
12:57
It's more of like I might prescribe any medication that
13:00
conflict with another medicine that the patient is taking.
13:04
A lot of times it's just a guesswork problem.
13:06
Provider also struggle with is, is the medication that the
13:12
hospital that was prescribed, has it been exhausted?
13:16
Is the patient actually adhering to the medication?
13:19
So that's another things that they do care about.
13:22
And as a family member, you want to see if there is, if your,
13:28
your parents are involved, they're actually improving and
13:32
indicate that they are receiving and which is a whole different
13:35
experience of what you want to do.
13:37
And, and also the workflow You also, when you think about this,
13:40
like how do I reach out to my parents if I notice something is
13:43
concerning? So, but also from the patient
13:46
standpoint, you also want us to know if there's a correlation
13:49
between the medication you're taking and your, your, your
13:53
heart rate. Am I, you know, is that is, that
13:56
is my blood pressure increasing or, you know, when I'm taking
13:59
some medication, this is actually working.
14:02
I'm, is that just dropping my sleep patterns?
14:05
So all that insightful information is how each
14:08
different stakeholder has a different way of interacting
14:13
with the same particular data. And MCP provides that
14:18
abstraction and allows us to be able to customize different
14:21
agent for different workflow and customize different reasoning
14:25
capability and patterns on how they should approach the same
14:28
data and the information that is is forwarded and the action also
14:33
that is taken. So.
14:34
So like real world example, because a lot of medicine that's
14:38
exactly just describing now, which is super interesting, is
14:42
extensibility out into I mean, I think Johnson and Johnson
14:44
actually calls it like real world data and Alex or something
14:47
like that where, you know, for example, someone that has had a
14:51
a Series, A history of arrhythmia, you know, their
14:54
Apple Watch is actually pretty decent as a first line
14:56
indicator. Yeah.
14:57
So how do you then extend into additional data sources of the
15:01
MCP, you know, is that the right way to do it or sort of explain
15:04
a little bit how that might? Work so the example example you
15:08
you you you gave us you experienced that symptoms the
15:12
the first person is allowed also the patient they will receive a
15:15
notification in the phone and also if someone within the care
15:18
network, you know like the family or someone that is like,
15:22
you know authorized to be I say authorized caregiver.
15:25
I notified like you know your your parents just experienced
15:29
sudden training that wouldn't you.
15:31
Now the question is the agent is strange like what is the next
15:35
step into that in this in this area one a lot of the physician
15:40
how critical it is so we have to train what is the severity is
15:45
this something that needs to be addressed right away should you
15:49
call a a ambulance care or it's not some it's it's something
15:54
that you could be scheduled. So this are the kind of areas
15:57
where we we have to kind of really tread quickly and and
16:01
it's also we're working with doctors actually decide what is
16:03
the right threshold to be able to set and the given condition
16:06
given their past history. So that's kind of that's still
16:09
lately where we're kind of like really working with providers to
16:12
understand, you know, a, a, a given patient's condition, what
16:15
area, what condition is considered severe and what
16:19
actions should the system take versus what you shouldn't take.
16:22
And that's how we're how we're currently solving that issue.
16:25
And how do you bake the context in at that level?
16:27
So like as an example, because this is actually someone I know
16:30
has done this. If you're, if you're, if you
16:33
worry a lot about arrhythmia, there's actually an app that
16:37
will you push a button on your on your on your wrist and you'll
16:40
watch and it will send it to a, an EKG review team.
16:45
I think it's, it's remote somewhere in the Philippines or
16:47
something like that. And it'll a real time read, you
16:49
know, on like, OK, does this look bad?
16:51
How much what do you really look like?
16:53
And, and send you back a report. Now it's actually like an AI is
16:56
doing the read. So essentially you can create
16:58
these workflows where, oh, OK, this looks not so bad.
17:01
I'll send it there for contacts or we'll send it there for
17:04
contacts. And then if it looks bad,
17:05
automate like kick it up to. So how do you like inject
17:08
context to create these fluid? Workflow yeah, exactly.
17:10
And I think that's a good use case where, you know, where we
17:14
send the AI does the review and scan and then what would just
17:17
look at like, you know, the first time, maybe indication,
17:20
maybe it could be a false positive.
17:22
But if we start to see a pattern, then the next action
17:25
will be we provide the provider of the actually the patient, the
17:29
authorities like, you know, this has repeated for the past three,
17:35
three times in the past two months.
17:39
We recommend scheduling a apartment.
17:41
Would you like to to to schedule and now the agents that does the
17:45
scheduling, which is separate from the agent that's don't do
17:48
send the alerts. Actually it makes the schedule
17:51
and workers provided look into the availability and actually
17:54
put the appointment for the patient to come in for an actual
17:58
screening with the with the primary care.
18:00
But also another workflow we were to look into is like even
18:03
in in advance of actually the doctor's visits, the doctor
18:06
already has a summary of what is going on.
18:09
They're able to look at things that like what is the sleep
18:12
pattern and also be able to rule out other things in advance of
18:17
their new meetings. So they're they're finding out
18:19
that able to be more efficient in providing care and also they
18:22
able to provide the quality of care is also improvement because
18:25
they have a little bit more holistic information rather than
18:28
just the information that we see during the actual visit.
18:31
So let me ask like a harder question and this and actually I
18:34
think this is quite relevant. So say there's non compliant
18:38
patient and the data shows, you know that because I have a lot
18:43
of doctor friends who are they they work in, you know, in, in
18:45
the in public health clinics. That's a huge problem with a
18:50
large part. Obviously, you know, like with
18:54
MCP, it's the perfect vehicle to run the agents.
18:57
How do we think about that, like permissioning in those?
19:00
I mean, I know this is like out on the edge, but I'm just kind
19:02
of curious. OK.
19:04
So let me make sure I'm saying you mean like a a patient that
19:07
they've been received a recommendation from the doctor,
19:10
but they're not adhering services that.
19:12
Exactly. Yeah.
19:13
So that's that's tricky because. And I don't even put you on the
19:16
spot. No, this is like, it's like a
19:18
huge problem because like, there's what's good for them and
19:20
there's what they're doing. Yeah, yeah.
19:21
Yeah. But I, I the way we structure is
19:23
like we leave to this with a design principle one the patient
19:28
are in charge of that decision. So whatever suggestion whether a
19:33
provider can recommend something?
19:35
Or could a patient like have their caregiver basically like a
19:37
full award, like OK, my caregiver can see this and can
19:41
understand whether to forward this to doctors or whatever and.
19:43
Yeah. That permission we it, it
19:46
depends on the way we keep we separated caregivers into two
19:50
different. So that's caregivers as
19:52
actually, you know, one to have one legal right, but also
19:56
they've been given permission. So someone like a minor for
19:59
example, that you know they're still under the care of their
20:01
parents. Like in California actually it's
20:03
a huge problem because like if your kids in college and they're
20:05
over 18, you can no longer like see their health records even
20:09
though they often you need to see their health records.
20:12
Yeah, yeah. So, yeah.
20:13
So it's trigger. So I think, but there's also
20:15
that, that one piece, but it's also a caregiver that you kind
20:17
of assigned as you know, someone to maybe you to look after you
20:23
and some of them are maybe there could also be a paid caregiver.
20:26
So that's how we separated this piece and I forgot what's the
20:29
the original question you asked there.
20:32
I'm sorry, I know it was essentially like these.
20:35
There are these cases where you know, we saw we we see it where
20:41
the the patient or or a person that would be potentially
20:46
benefit from the agentic system may not want, you know, their
20:51
non compliant patients by I said, which is a large.
20:53
Large problem, yeah. So, yeah, exactly.
20:54
I mean, the ultimate thing is that we, we leave the decision
20:58
up to either the patients to kind of like, you know, make the
21:02
decision whatever they want to do about their life, but also
21:06
the caregiver that are like required authorized caregivers
21:11
that are authorized to make that decision.
21:13
So in the case of like a minor that falls into the legal
21:16
category. The main things like we, we're,
21:18
we're very focused on, we did not want the system to breach
21:23
any way to go through your safety requirements.
21:28
So that's the, the core design principle.
21:31
And also the second thing I also look into is also we also want
21:35
patients to feel, you know, have control over the health.
21:40
And that's something we see in doing our research.
21:43
Everyone wants autonomy over the health.
21:45
They want to be able to make decisions and sometimes even
21:49
they might some recommendations that's presented, they might not
21:53
even want to disclose. So a classic example was even
21:57
with my dad, he's one of our, our awesome.
22:02
And, you know, we're having a meeting with the doctor and he's
22:05
like, doctor, yeah. So I've been, you know, taking
22:09
this medication and, you know, for the only reason my my blood
22:11
pressure is still trending high. But I'm like, the doctors look
22:16
at you also exercising. And it's like, yeah, I'm
22:18
exercising. But, you know, you could look at
22:20
the data and you're like, OK, well, the data is sure that
22:22
you're not really working. You're not putting enough steps.
22:24
Yeah, so but so but the data is like, is that sort of true?
22:29
That could kind of be accurate. And sometimes the doctor might
22:31
know, has that information. They didn't know that this
22:35
person is not exercising. But regardless, they needs to
22:39
kind of be able to work with them in, in, in, in that
22:42
capabilities and say, you know, well, maybe next time you
22:44
should, you should exercise a little bit more.
22:46
And you see that also, even in some settings where, you know,
22:50
someone goes to the dancers and the dancers like, oh, are you
22:52
blushing any flushing daily? And you're like, yeah, I'm
22:54
flushing daily. And you know, the dancers can
22:56
actually clearly see a lot of plaque and they could tell that
22:59
you're not flushing daily, but they give the encouragement to
23:03
adhere to the processes. But regardless, it falls down to
23:06
the actual patients in taking the ownership and actually doing
23:10
the recommendations. And where our classroom provides
23:14
kind of provides that accountability and that audit
23:18
ability to be able to say, yes, this patient is actually full of
23:22
my recommendations and they're not seeing an improvement from
23:25
the provider perspective. So they can actually assess if
23:29
if the recommendations they're providing is the right
23:31
recommendation. So like for GLP prescription?
23:34
Or something like that. Or.
23:35
They're adhering the best practices exactly, and they're
23:37
exercising or whatever. Yeah, and it's interesting
23:40
'cause it's almost like an intermediate.
23:42
So like, for example, I mean, it gives you these intermediate
23:44
options that weren't there before.
23:46
I mean some of this verification, but also like, you
23:48
know, if there's a non compliant patient, you could have an
23:51
option that's like, OK, I'm going to tell their spouse to
23:55
talk to them about what's going on with this.
23:58
You know, we don't disclose, but you do some you push or you tell
24:01
the doctor, you know, without actually violating privacy,
24:04
which is like, you know, that wasn't an option before.
24:06
They couldn't even see that. That's true.
24:08
That's true. So you're able to kind of noisy
24:09
family in the right way to come. Back here, right, Right.
24:11
Yeah, Yeah, that's true. So, So which parts of your work
24:14
flows are deterministic, which are, which are, you know, rule
24:18
based versus model driven, right?
24:21
And how do you determine which part should be which?
24:24
Gotcha. Yeah, that's a good question.
24:27
So for I think even what I mentioned earlier, anything that
24:31
is regulatory and anything that is clinical and like life saving
24:39
is very deterministic. So binary, binary, it's not
24:42
like, so we do not want to be in a, a process that violates any
24:48
government regulations. And the way we, we structures
24:53
and NCB allows us to do that is like, you know, from the data
24:56
layer where we kind of like don't embed any logic in the
24:59
data layer. The access layer is where we
25:01
kind of like set our permissions, right?
25:03
Then on top of that, we layer on top of the MCP layer.
25:07
So the MCP only use specific tools that it has permission to
25:12
do so and those tools are directly assigned to a specific
25:16
set of data. So a given example will be like,
25:19
you know, if someone has a high blood pressure, I've been
25:23
trending high for a while. We don't say, you know, you're
25:27
at risk of a heart attack. We kind of like share that
25:29
information with their provider and the provider comes into the
25:33
workflow and determine is this severe enough for me to request
25:39
for the patients to schedule an appointment.
25:42
And if the provider identifies that their behaviour is is
25:49
concerning, then the patients they provide themselves have to
25:53
actually approve of the agents to actually make that that
25:57
decision actually requested for the for the patients to come in
26:01
for an appointment. So the providers are also able
26:04
to set that's criticality. So that's how we're able to make
26:07
that distinction is where it's like where is clinical start
26:11
that to the provider have them make the decision of what the
26:13
patient should do things that are regulatory with is a strict
26:19
for for error like no, you know, we just.
26:22
That's that's absolutely prescriptive and.
26:25
Exactly and no access to the data given that that particular
26:30
workflow and they're done, they'll work for the agent.
26:33
Also doesn't have even the the the visibility to be able to
26:37
access it they they even wanted to.
26:39
So I mean, we've seen it. This is where agents, you know,
26:43
they take on a life of their own.
26:45
And you probably saw like what happened with Meta with the open
26:47
call instance that got got loose on, on the Meta security heads
26:51
e-mail, you know, which was then she posted about it was
26:55
fascinating. How do you make sure that your
26:58
agents behave the right way? You know, how do you keep them
27:01
on the leash? And it'd be obviously
27:03
sandboxing, but like in, in terms of how you're building
27:06
this? That is, that's a good question.
27:08
And I think the, the four layers segregation layer is something
27:14
that would, we've seen, we've seen good results on where you
27:19
know, what I mentioned before, the whole API based contacts,
27:23
like when the AI is making a lot of all this calls and making all
27:27
this query, the AI is learning from the queries you're you're
27:30
given, then it starts making its own interpretation.
27:33
So we try to eliminate that by restricting the amount of tools
27:38
the each particular agent we assigned.
27:40
So your provider agent, the, the, the patient's workflow to
27:47
kind of like its own sets of permission behavior.
27:52
And we'll also do something that is a little bit different is
27:55
where we kind of like create like a meta data of of sorts for
28:00
each of the agents behavior that kind of records their behavior
28:03
and where it's kind of correlated into to identify
28:07
there's some drift in you're. Looking for anomaly or exactly
28:10
and I'm curious like in the same vein.
28:13
So one of the magic that I've seen, at least with LMS in this
28:17
area is that it can put patients almost on par with physicians
28:22
and having conversations. Obviously they are, you know,
28:25
humans are, you know, they're they're doctors, but they're
28:27
also they're human, you know, So I got an example.
28:29
I know from, from someone's life in, in my, you know, around me
28:33
was they looked in ChatGPT were like, Hey, I'm, you know, my
28:39
blood Sugar's not going down, you know, is that related to my
28:42
statin And chat GP actually like explain to them that the statin
28:45
they were on was a good statin, but they could switch to a
28:47
different one that might actually, you know, not impact
28:50
A1C as much. And then so they had the
28:52
conversation with their provider and they switched.
28:54
It helped. And it wasn't like the provider
28:55
didn't know. It was just more the provider
28:57
had so many things going on that that's not what they think
28:59
about. How do you like use MCP to
29:03
inject from the models? And I'm assuming you're using a
29:06
couple different models, you know, that type of context and
29:09
also take advantage of the way MCP works to, you know, to
29:14
create that sort of capability. I'm not sure if that's what you
29:17
do at all, but yeah, so. We're not doing that, I think,
29:23
but I mean, I think that has a lot of like risk and I think
29:27
we're seeing that. I think was it in Utah of where
29:31
they kind of created a lot of regulations around the use of AI
29:34
in the medical? Yeah.
29:35
Oh no, they're they're getting pretty.
29:36
Serious. Yeah, yeah, yeah.
29:37
So it. It yeah.
29:39
So how we, we, we kind of said that is like the agent is more
29:44
of like the agent, the different agent has different workflow,
29:48
different personality and the agent that interacts with the
29:51
patient is more like a coach and less of a clinical decision
29:57
maker. And it's more like looking at,
29:59
OK, based on the recommendation that your doctor prescribed you
30:04
should do XY and Z. So what we did is like we always
30:08
assign each of the action or the recommendations the agents is
30:14
providing specific to a particular either clinical notes
30:18
that the doctor actually created or like you know, the
30:21
barometrics information that we received from the patients,
30:24
wearable information to be able to say, you know, this is where
30:27
this is the information came from.
30:29
So we could say stuff like last Tuesday in your meeting with the
30:34
doctor, doctor recommended to the AB and C, you're able to
30:38
look into that and see the Aurora data for exactly the
30:42
notes that you captured and what the doctor's recommendation.
30:45
So you have to kind of identify this some information drift.
30:49
So we're that's how we're kind of our workflow kind of works.
30:51
But from the clinical side, it's a whole different workflow.
30:55
And we have a more medically trained model that is able to
30:58
kind of provide the doctors like, you know, this is some
31:00
information you were seeing based on, you know, the
31:04
patient's data and just more, some more information for what
31:09
the providers kind of consider. I, I doing our research, we
31:14
noticed that some provided interested in knowing some of
31:17
that information because they also don't want to be ahead of
31:21
like, you know, what's the latest training and
31:23
understanding of like in a particular disease and
31:25
infection. But there's some provider that
31:28
are very skeptical about AI giving them recommendation.
31:32
So we're that's kind of how the clinical workflow usually exist
31:36
is where we are providing information to the clinicians
31:39
and clinicians kind of making their judgement of whether
31:41
that's something they should consider or like validates
31:44
versus information going to the patient.
31:46
That or that use case you presented about a patient using
31:49
ChatGPT to kind of read a health summary.
31:52
What kind of? Failure modes have you seen and
31:54
how have you mitigated that? I think.
31:57
That's a good question. They also the, the, the 1st is a
32:00
scenario where you know, the physical patient has like a
32:05
relationship drift with their family or the caregiver and all
32:10
of a sudden they're like, hey, well, I want to kind of restrict
32:13
the access. I don't want him to be able to
32:15
kind of like to check on me anymore.
32:19
And I think that area is like initially it's kind of like.
32:24
Although they have access. But the information, maybe
32:27
they'll restrict the access, but the information is still
32:30
transmitting. So they everything we see some
32:34
form of the response that is on that's not completed.
32:38
So what we did is to kind of like treat the permission as
32:45
almost like a shut off switch where as soon as the request
32:49
permission is revoked instantly, it returns a for for for error.
32:54
So it doesn't show any partial information or personally
32:57
generated information. It's just they requested returns
33:00
like I'm sorry, we can't the the agent is not working on, you
33:04
know, yeah, so that's how we interrupt very brought like.
33:08
So we try to do that quiet time on the permission layer where we
33:12
the the patient has that almost kill switch to the permission.
33:16
So that's that's the first one. And I think we also see it's
33:23
also kind of like the whole liability question comes into
33:26
play of like who takes liability on, you know, if issued does
33:30
occurring in production setting. So, and this are the areas where
33:34
I think, you know, we're design our own like, you know, action
33:39
action level where, you know, we're able to assign providers
33:44
action specifically to the agent's behaviour.
33:47
So we have to say if it a medical professional recommend a
33:52
medication for patients to be taken, the medical and you know,
33:57
the agent needs to be idea strictly to what the
33:59
recommendation was provided by professional.
34:02
And so they better adopt that. As you know, I mean, they're
34:05
able to stick that way liability if there's a drift in that, but
34:09
also if there was a drift into the work accommodation that the
34:13
provider provider, but the agents made a mistake, then
34:16
obviously the agents take that liability.
34:19
So that's how we're kind of just segregated.
34:20
So just some of the challenges where we're still address and
34:23
also even a big industry type, you know challenge are are.
34:28
You also layering additional like A to a or designing so that
34:33
your agents can interact with other agents too.
34:36
And what else are you putting are you thinking about on top of
34:38
MCP to extend the functionality? That's interesting we.
34:42
Have we haven't really exploited?
34:43
I think our focus right now is just trying to get the agents
34:49
consent, providing consistent results and we're we're seeing
34:54
progress in that, in that, in that, in that area where it's
34:56
like, you know, we're able to provide that, you know, that
34:58
traceability within in audit. So yeah, but maybe in the future
35:03
and I had I think that's probably where the the industry
35:05
is heading towards where agent will be in Taco, all the agents
35:09
there was the agent that your agents goes into the providers
35:13
agents and does whole bunch of other workflow kicks all the
35:16
workflow processes. So that's something that I could
35:18
see, you know, the industry going into it.
35:21
But I think in the health of the regulated industry in general, I
35:24
think it will be much slower. Of course.
35:28
Yeah, much slower. Yeah.
35:29
I mean, the hospitals still use paper.
35:31
Oh, no, I know. Discharge.
35:33
Instruction I. Was not joking.
35:35
Because I've seen discharge instruction so.
35:38
I think that is a long ways and I think in the past couple of
35:42
years there have been a lot of eases on the the restrictions
35:45
are on data. So I'm seeing a little bit more
35:47
innovation in that space and a lot of start-ups are creative
35:50
solutions such as the challenge. And I think there's some more
35:54
features. I think, you know, the MCP
35:56
community in general, I think can can invest in especially for
36:01
over regulated industry and it can be applied not just in
36:04
healthcare or in finance or or other or legal other areas that
36:08
requires a lot of strict enforcement of data.
36:10
So, So what? We'll let's.
36:11
Round it out of Matt point in that specific question, which is
36:15
if you had a wish list for the technical committee of here's
36:19
the things I would love to see at MCP to help me or help other
36:23
people in the healthcare sector better use it as a, you know, as
36:27
as a, as a platform for genetic AI and building the highly
36:31
functional app. What what would you say?
36:32
Like what's my wish list so. The first one is that I don't
36:37
know why where to put it, but that's being able to kind of
36:40
identify on, on your MCP server the enforcement traceability.
36:48
So right now like what we're doing is we're created, we
36:52
created the United measure, we created a, a static caching
36:57
later that that addresses each agents, what specific tools
37:02
they're using and how they access it.
37:03
Right now there's no, I haven't seen any particularly MCP server
37:07
that actually has that also already embedded.
37:09
So there's like no hotel? Spans or anything like that
37:12
exactly. So there's.
37:13
Always this weird work around like for example like if a given
37:16
agent you only need it to be live for maybe two weeks.
37:22
It's not something you can kind of like enforce on the NCP layer
37:25
where you can able to shut up the access on the time bound.
37:28
I said how, you know, some of the things we do, we do as a
37:32
work around. It's like created that caching
37:34
layer of of logs that kind of like creates that audit of like,
37:37
you know, based on this setting that you're right into this into
37:41
this log, it has this time bound and we kind of make and I
37:45
request that I sort of that service.
37:48
So if that was something that was embedded by default in a lot
37:52
of MCP servers, I think that would be something that I will
37:56
help in the regulated industry perspective and I think will be
37:59
more with a little bit more adoption in that in that area.
38:02
Anything else that? Jumps to minder what I think.
38:05
Another thing is just more around the old security related
38:10
guardrails that prevents, you know, do how prevent agents from
38:16
communicating with each other and giving them solid data
38:19
access. So this I mean, we have a lot of
38:22
you know, I mentioned the three latest stacks of how we kind of
38:25
address that. That's true that that issue, but
38:28
decided two things I think I think MCP needs to be really
38:33
more robust than to be able to attract more enterprise level
38:37
implementation. Very cool.
38:38
Well. Thank you so much for coming and
38:40
sharing all this. I learned a lot and hopefully I
38:44
will be able to use your ID in the world.
38:46
Not hopefully not. I don't like you're the sick
38:48
people, but if I have to, you know I will be able to use your
38:51
ID in the future. Hope you have a good summit and
38:55
hope to see you at another event.
38:56
Thank you. Thank you.
38:57
I appreciate. You interviewing me and just
38:59
give me the opportunity to share what we're doing and also being
39:01
part of this community of just a lot of, you know, 8 engineers
39:05
just building and just line the, the nest level of, you know,
39:09
Agentic, our deployment, which is really excited to just be
39:13
part of. So thank you for what you do and
39:15
I, I really enjoyed this conversation and just giving us
39:19
the opportunity to just present what we don't.
39:21
Thanks. Thanks Weasley.