Latest / Pediatrics Now: Cases Updates and Discussions for the Busy Pediatric Practitioner / AI in Medicine: Ready for a Texas-Sized Leap Forward or Not?

Transcript

AI in Medicine: Ready for a Texas-Sized Leap Forward or Not? · Pediatrics Now: Cases Updates and Discussions for the Busy Pediatric Practitioner

0:00
So the light bulb is a disruptive innovation. So we're in a room filled with light.
0:05
In the old days, maybe when people were going along that river on horses,
0:10
candles provided light.
0:11
So if we just improved the candle, we would have never gotten to the light bulb.
0:17
I'm Holly Wehmint, and this is Pediatrics Now, cases, updates,
0:21
and discussions for the busy pediatric practitioner.
0:25
Click on the link in this podcast for free credit that may include CME,
0:29
MOC, or ethics credit, depending on our topic or podcast.
0:33
It's a new year, and this week we have a new episode on something that's very new in medicine, AI.
0:42
You may not know this, but UTHealth San Antonio has a dual degree program that's
0:48
an MD and a master's in AI. Down the road, stay tuned, I plan to interview the
0:55
doctor who teaches that program.
0:57
For this AI episode, I attended the Neonatology Fellowship Conference in downtown
1:03
San Antonio recently, and I'm bringing you this talk by Dr.
1:08
James Berry. It was such an honor to hear what he has to say with his special interest in AI,
1:14
and he's a professor of pediatrics and medical director of neonatology at University
1:20
of Colorado Neonatal Intensive Care Unit.
1:23
You might hear him refer to Dr. Kaiser Aziz.
1:27
He also has a special interest in AI, and he's an assistant professor of pediatrics
1:32
and general internal medicine at Johns Hopkins.
1:36
I've interviewed him previously for the podcast, and you can find that in this
1:40
Pediatrics Now news feed.
1:42
The other person he mentions in this fascinating talk is Dr. Alvaro Morera.
1:46
He is a neonatologist here at the University of Texas Health Science Center.
1:51
He works in the NICU at University Health's new Women's and Children's Hospital.
1:57
I have kind of a probably an academic career that you wouldn't want to follow.
2:01
So you guys up front here, you wouldn't want to follow what I've done because
2:05
I've kind of navigated to things that have interested me over time.
2:09
So to begin with, I was a basic science researcher and studied heart development
2:14
and how how it was affected by IUGR, became a medical director.
2:18
And what I'll start talking about is kind of how I ended up navigating to AI
2:24
and how maybe we should be thinking about AI today.
2:28
The next 10 years in medicine is going to change pretty significantly.
2:35
And so you're entering a really exciting time into healthcare because in a decade,
2:40
it's going to look completely different than it does today.
2:43
Hopefully better, but potentially not. So we have to make sure that you guys
2:47
are all involved in that.
2:49
I don't really have any disclosures to make for this talk.
2:52
I'm not going to talk about anything that I'm financially supported for.
2:57
I will bring up product names or company names.
3:00
I have no relationship with them and get no money, but I'll just use them as examples.
3:05
I was a site investigator for an aerosolized surfactant study.
3:09
I was a co-founder of Neomind AI, which there's two in the audience here tonight
3:14
that are also co-founders of that,
3:16
which is an AI collaborative learning group that we want neonatologists and
3:22
pediatricians to use as a resource for understanding how we can apply AI safely,
3:28
effectively, ethically and unbiased in health care.
3:31
And I used to be the District 8 representative on the Executive Council for
3:35
the Section of Neonatal Perinatal Medicine for the AAP, but my six years are up.
3:40
So I'll talk a little bit about AI tonight, really kind of where it is,
3:45
maybe a little bit why I'm involved in it,
3:47
some kind of studies that relate to us in our field, and then how you all should
3:54
kind of be thinking about getting involved or why you should get involved.
3:58
So probably one of my most important career moves is I became medical director
4:04
of one of our NICUs back in 2007.
4:07
And what you have to know about me is I'm kind of an early or late majority
4:12
adopter of new technology.
4:14
So if you really knew me, you would say that I would be the very last person
4:18
to be standing up here and talking about AI.
4:20
Because that's the newest, shiniest innovation that people are really excited about.
4:26
Normally, I'm the one who says, hey, we can't be involved in that.
4:30
We have to make sure that that's safe and effective.
4:32
The thing that I learned about myself as a medical director is I like to fix
4:36
things. I like to fix hospital systems and processes.
4:39
And I'm always on the lookout for like, what is the next best thing that's going
4:45
to improve care in our NICU?
4:47
Most of the time, it's been approaches to care, not always like an intervention.
4:51
Probably the thing that we have learned in the last 10 years in neonatology
4:57
to improve outcomes was to actually not do something.
5:00
Do you guys know what that might be? That not do something is as a neonatology fellow or student?
5:06
It's to delay clamping the umbilical cord.
5:09
That's probably been the most significant thing that we've done in neonatology
5:13
in the last 10 years to improve patients' outcomes.
5:15
So we actually decided we weren't going to do something. So much different than
5:20
you think about innovation.
5:21
But in 2019, I started hearing hospital leaders talk about AI and how it was
5:26
going to solve all of our problems and how it was going to help us improve our
5:30
care and support the care that we're providing to our patients.
5:33
All I could really think about AI was, did you guys ever see that movie iRobot
5:37
with Will Smith and Sonny was a robot that helped save the world from all the evil robots?
5:42
So that was 2021. And I decided that probably isn't what my healthcare leaders
5:47
were talking about. I don't think they were thinking about Sunny and iRobot.
5:50
So I really started jumping in and trying to learn about AI.
5:54
I don't have computer science background. I'm not an engineer.
5:58
I'm not a data scientist. But I decided I was going to learn because all these
6:02
healthcare leaders were telling me that it was going to be important.
6:05
And I have to say today, when I jumped in, I was skeptical.
6:08
Everybody said it's going to solve all of our problems. I don't think it's going
6:12
to solve all of them, but it's going to solve some of them.
6:14
So I'm no longer a skeptic. In November 2022, ChatGPT blew up on the scene,
6:20
and the world kind of took notice.
6:23
It took five days for ChatGPT to get to one million users online,
6:27
much faster than all the other social media platforms.
6:32
It took two months to get to 100 million users, much faster than TikTok,
6:36
Instagram, and others. So it really took the world by storm.
6:39
In healthcare, we've actually had multiple things kind of take us by storm over the years.
6:45
So it got me thinking about what are some big changes that we've seen in healthcare
6:50
that have really had important impact on our care.
6:54
The penicillin discovered in 1928 by Alexander Fleming.
6:58
Pretty remarkable. The global mortality rate has declined dramatically because
7:03
of the discovery of penicillin, the mass production of it, and then the mass
7:07
production of many other antimicrobials.
7:09
So global mortality rates are decreased because of the discovery of penicillin.
7:16
Anybody heard of thalidomide before? Are you guys using it in your perinatal care areas? No.
7:22
Why not? It was a medication that was used for morning sickness.
7:26
Turns out it actually caused limb malformations in babies. So it was a bad idea,
7:31
but it was an innovative approach at the time.
7:34
How about this, the light bulb? So the fact that they figured out that shooting
7:39
electricity through tungsten filaments causes bright light kind of changed our world.
7:44
So it really disrupted. If we had just improved candles, we wouldn't really
7:48
have the light that we have today.
7:50
So what I want you guys to think about today is AI.
7:53
Is it going to be our next penicillin?
7:55
Is it going to be our next thalidomide?
7:58
Or is it going to be our next lipo?
8:00
And I'd have to say I want you guys to kind of clear your mind a little bit
8:03
here and kind of go on an open mind journey with me for a moment.
8:07
I had to do the same thing. So I'm going to ask you to do the same thing if you haven't already.
8:11
So I want you to take out the word can't, won't, shouldn't, couldn't from your minds right now.
8:17
So this is a slide that I've been talking off of for probably about a year and
8:22
a half. And I've been telling people that AI will be coming in our electronic
8:28
medical record systems. And most of the time, people kind of said, yeah, whatever.
8:32
So ChatGPT is a household name. Epic or Microsoft invested in OpenAI, $10 billion.
8:40
They have a 49% position in OpenAI. OpenAI is who created ChatGPT.
8:46
So, Microsoft is partnered with Epic, which is in 65% of the healthcare systems in the country.
8:52
So, by way of connecting those dots, ChatGPT is going to be in Epic at some point in time.
8:57
So, this is a publication just from last week that said that Microsoft and Epic
9:03
are even deepening their partnership.
9:05
They're going to start using ambient technology for nursing documentation.
9:08
So, listening to nurses talk with patients and they'll be able to create documentation.
9:14
So three weeks ago, Epic had their annual user group meeting.
9:18
So where they get together at their big Wisconsin campus and have a meeting
9:22
and talk about what they're working on.
9:23
And so they unveiled that they have 100 applications plus of AI in their ENSAR system.
9:30
And it just depends on when you start getting the switch flipped at your institution.
9:36
So Kaiser, if he has Epic back there, he's sitting there thinking,
9:39
yeah, what are we going to switch on, right?
9:41
So I'm not going to give you a list of 100 things, but here's some examples of what's available.
9:46
Medical necessity criteria, extracting cancer staging data from clinical notes,
9:53
AI text assistant that can create documentation for you.
9:56
It can go through patient messages and summarize them.
9:59
It can predict transplant success and complications. It can draft denial and clinical summaries.
10:06
So these are just kind of a few examples.
10:09
This is a figure that's quite busy, but this figure actually includes companies
10:14
that are involved in applying generative AI in healthcare.
10:18
So there's a lot on here, right? There's different categories.
10:21
So there's different companies that are working on drug discovery,
10:24
clinical trials, care navigation, remote patient monitoring,
10:28
note-taking, medical coding, revenue cycle management, recruiting, prior authorization.
10:34
So this already exists out there today.
10:37
There's now 950 FDA-approved AI-enabled devices in the United States.
10:42
75% or so of them are in radiology.
10:46
There's 1% to 3% in pediatrics.
10:48
So right now, pediatrics is being left behind in kind of the AI race going forward.
10:55
So the next thing is, can you use these devices and get paid?
10:58
It just so happens there's now CPT codes that you can use to get paid for using them.
11:03
This is a recent publication looking at U.S.
11:06
Claims data, and the database that they use included about 40% of the claims
11:11
in the entire United States.
11:13
And this bottom part of the figure,
11:15
going from 1 to 16, shows back in 2018 was the first CPT code used.
11:22
Now, as of March 2023, there are 16 CPT codes that are available associated
11:27
with these technologies in clinical care.
11:30
And the top part of the figure just shows that it's increasing claims per month,
11:35
especially in the adult world of coronary artery disease and diabetic phreatenopathy.
11:40
Last week, the Nobel Prizes were awarded.
11:44
Four of the people that won these prizes were associated with AI.
11:49
So Jeffrey Hinton and John Hotfield won the Nobel Prize for physics,
11:54
and they won it for their work in kind of foundational modeling of artificial neural networks.
12:01
So really what's underlying the big breakthroughs of AI today.
12:05
And then Demis Hassabis, I'm not sure if I'm saying his name correctly.
12:09
He's the co-founder and CEO of Google DeepMind and John Jumper,
12:12
just won the Nobel Prize in chemistry for their work on protein structure prediction, AlphaFold3.
12:19
And what alpha-fold 3 can do is it can predict the 3D structure of proteins
12:23
based on amino acid sequences.
12:25
But it goes beyond that, and it helps identify the function of those potential proteins.
12:31
And you can identify binding sites and mechanisms of disease-causing proteins.
12:36
So the pharmaceutical industry has certainly taken notice of this.
12:41
In 2022, they spent about a billion dollars on AI products.
12:45
Models. It's estimated that they're going to spend about $22 billion in 2027
12:51
using AI and pharmaceutical development.
12:54
And that's because in drug development, they can decrease the time from drug
12:59
discovery to manufacturing production and having it in the marketplace by about 50%.
13:04
And the bottom right there is all the different pharmaceutical companies that
13:09
are involved, all the big name ones that you would think of.
13:11
And most of them are using the AlphaFold 3 approach to develop drugs and molecules.
13:18
So I hope I've at least convinced you that AI is here. If you guys didn't know
13:22
it, it's here. It's not going anywhere.
13:24
And the reason I go around and give talks like this is that I want neonatologists,
13:29
pediatricians, people involved in perinatal care to understand we have to be
13:34
involved and be part of this process.
13:38
Right now, AI and Chachi PT is that the hype of peaked inflation expectations.
13:45
And this is kind of a Gartner hype cycle, which is something that's become somewhat famous.
13:52
But basically, there's an innovation or trigger that then creates some huge
13:58
expectations and excitement. So large language models were created.
14:02
ChatGPT has really inflamed the expectations, and
14:06
it's not going to be able to do everything that's promised and we'll realize
14:10
that over time and we'll end up understanding how to use it over time when i
14:15
talk with clinicians right now about ai many of them it's kind of a spectrum
14:20
view of things many of them fear ai and they feel that ai will destroy care as we know it,
14:26
others feel like it's they're over exuberant and say it's going to solve all
14:30
of our problems with most things it's going to probably fall somewhere in between.
14:34
So it seems like AI is new, right? That's kind of the new hot topic.
14:39
Well, AI, the term, has been around since the 1950s.
14:42
So a Stanford professor at a Dartmouth conference first coined the term in the 1950s.
14:48
Machine learning has been around since the 1990s.
14:51
Deep learning has been around since the 2010s. And the new kid on the block,
14:55
generative AI, has been around since now the 2020s.
14:58
So artificial intelligence and Kaiser is probably going to give you a different
15:01
definition tomorrow, I think.
15:04
But it's a branch of computer science that focuses on creating systems in the
15:08
form of algorithms and software that's capable of performing human intelligence-like tasks.
15:14
So learning, problem-solving, perception, language, understanding, and decision-making.
15:20
If you talk with people who kind of understand the big picture of AI.
15:25
There's kind of three main types of AI.
15:28
Narrow, artificial, general intelligence, and superintelligence. artificial
15:32
intelligence right now we're only working in
15:35
the narrow area it's the only type that really exists today
15:38
so it has a defined task it's trained by humans so
15:42
things like siri or self-driving cars the
15:45
next step and depending on who you listen to sam altman
15:48
who is the ceo of open ai says within a few thousand days we're going to be
15:54
with artificial general intelligence there's other people that say that won't
15:58
happen for like a decade but artificial general intelligence It's able to accomplish
16:03
a new task using training from other kind of training sets and models.
16:09
So it can do something different. So talking about like a self-driving car,
16:12
what the self-driving car could understand is here on Market Street,
16:16
there's a protest and it knows that.
16:18
So it's driving you, but it's going to drive you around the protest because
16:22
it's in connection with the Internet and news and so forth.
16:25
So it's actually going to make some decisions on its own and take you away from
16:28
the protest. And then super AI is kind of the scary stuff that we think about,
16:33
that machines start thinking on their own.
16:35
They'll have their own emotions, beliefs, and needs.
16:38
And that's only theoretical at this point in time.
16:41
So now I want to ask you guys a question. Have any of you ever used AI in healthcare?
16:47
So I want to see a raise of hands. Everybody has to say, okay, right?
16:52
So I'm saying about 50%. That's pretty good.
16:55
So EKG interpretation is a form of AI. You guys probably haven't thought about
17:00
it that way before, though, right?
17:02
If you actually, if you really look into the technology of EKGs,
17:05
it's pretty fascinating. So this paper that's at the bottom is,
17:08
I found, pretty fascinating.
17:10
But black squiggly lines that are going over red-gritted paper,
17:15
we've told computers to give us information about, so may I come up?
17:23
But so we've, we, not me, but others have actually created rules that these
17:29
computers follow to give us information about these black squiggly lines going across.
17:34
So we get information about heart structure, heart injury, electrolyte abnormalities,
17:38
medication toxicities, conduction abnormalities.
17:42
So we get that information from these black lines going across a piece of paper,
17:46
which is pretty remarkable. I think EKGs are a great example of how we should
17:51
think about AI in healthcare in general.
17:53
So when you guys get an EKG, do you trust the automatic interpretations?
17:58
No. Because you know sometimes they're wrong, especially in neonatology.
18:02
A lot of times the EKG interpretations are either adult or pediatric and not
18:08
really applicable to newborns.
18:10
The other thing, I think, how we think about AI in EKG in similar fashions is
18:15
you've been trained to understand when do you use an EKG and when to get it.
18:19
You understand the limitations of EKGs and you know how to read EKGs.
18:25
So that, I think, is a fair way to think about AI.
18:29
We need to be involved in understanding how to interpret AI in much the same way that we do EKGs.
18:36
They help us, but we don't completely trust them. So I always heard a lot about
18:40
healthcare data being biased.
18:42
In AI, that's going to be a big issue.
18:45
And so I really started trying to understand what that meant.
18:50
And so EKG is yet another great example of inherent kind of bias that's built into our technology.
18:56
So if you're a pediatrician and you're getting EKG on a 16-year-old in your
19:02
clinic, would you get the same results back if your patient were white or if
19:06
they were black? Turns out, no.
19:09
So there's an increased false positivity rate for left-intricular hypertrophy
19:15
in black individuals with EKGs.
19:17
This is a study out of Stanford using convolutional neural networks.
19:22
And what their study question was, can we look at 12 lead EKGs?
19:26
So they looked at 320,000 of them.
19:28
And can we predict heart failure in five years from those 12 EDKGs?
19:33
And it turns out this is a receiver-operator characteristic curve.
19:37
Most of those lines are kind of in the top in this picture up here.
19:42
One's down here by itself, and that's an orange one. So the majority of race
19:47
or ethnic groups that they looked at kind of were about 0.8.
19:51
So the sensitivity and specificity of the test, the algorithm that they used
19:55
was actually pretty good.
19:56
Except for the orange, and orange
19:58
actually turns out that it's young black patients, especially women.
20:02
So if you think about when EKGs were developed, they were probably developed
20:07
on a patient population that looked a lot like me, an old white guy,
20:11
not a very diverse population to be training models on.
20:15
We already know this exists for pulse oximetry.
20:18
Pulse oximetry with darker pigmented skin, it doesn't work as well as light.
20:23
So I want you to think about that, that we're kind of surrounded by data or
20:28
bias with a lot of the tools that we use on a day-to-day basis.
20:32
And this is important because AI is really amplifying the ability to get important
20:38
information from some of these tools.
20:40
So this is EKG that now gives information about race, gender,
20:44
age, LB dysfunction, heart injury, prediction of future heart failure,
20:49
atrial fibrillation. We have it on our wrist.
20:52
There's AI-enabled stethoscopes. There's mobile single-lead ECGs that are being used in telemedicine.
20:58
These are being used for screening, diagnosis, prognosis, treatment,
21:01
selection, and monitoring.
21:03
So one of our tools is getting amplified by AI to be used in this capacity.
21:08
If we continue just to use it, it's going to amplify the bias that exists as
21:13
well. So we need to account for that.
21:16
So EKGs and pediatrics, have we started looking at those?
21:19
So yes. So this is actually a deep learning model that took EKGs and determined
21:25
if they could detect if a patient had congenital heart disease.
21:30
So what I have circled down here is the sensitivity and specificity for their
21:34
AI model was greater than 90% sensitive and specific for pulmonary atresia,
21:41
single ventricle, tetralogy of flow,
21:43
AVSD, and double outlet right ventricle.
21:45
And this is just from a 12-lead AG.
21:48
This is another study looking at EKG to see if you could detect left ventricular
21:54
dysfunction. and they took 100,000 EKGs that were paired up with echoes to see
22:00
if they could detect LB dysfunction.
22:02
And from this model, the deep learning model had a 97% negative predictive value for LB dysfunction.
22:11
You're all probably sitting in this room and saying, well, I don't really care
22:14
about that. I have echo tech that can come down and do an echo on my baby and
22:18
have a pediatric cardiologist read it within a couple hours and I have information.
22:23
I'm sure you're a lot like us in Colorado. If you go three hours north,
22:28
two hours west, two hours east, three hours south, you're in areas and hospitals
22:33
that don't have pediatric echotechs.
22:36
You don't have pediatric cardiologists. You have snowstorms that shut down transportation.
22:41
So oftentimes, our community pediatricians and family medicine doctors are having
22:46
to manage these patients to 12 hours, 16 hours, sometimes on their own.
22:50
And just imagine if they could use a 12-week EKG and get important information
22:55
about does the baby have congenital heart disease, yes or no?
22:58
Does the baby have LV dysfunction, yes or no? Just think of kind of how we could
23:03
actually improve care, especially in areas where we have limited access.
23:08
And that's where I think a lot of the power of AI will come from.
23:11
So talking a little bit more about physiological monitoring,
23:15
ECGs, EEGs, pulse oximetry.
23:17
I think you probably all have heard of the HERO monitoring system out of UVA.
23:22
There's been quite a few publications. It's probably the most well-studied form
23:25
of AI that we can use in neonatology.
23:30
And what it looks at is heart rate variability changes that are associated with
23:35
different morbidities in babies, in particular sepsis and NEC.
23:39
And they, UVA, have shown that they've decreased their mortality rate by 20% by intervening sooner.
23:47
So, these models detect changes in heart rate variability patterns that us at
23:52
the bedside cannot, but the machines can, and then they alert the clinicians to.
23:57
Ediometry is another platform that's being developed using continuous physiological
24:02
monitoring data streams.
24:05
And they have an oxygenation index, hemodynamic index, ventilation index.
24:10
So things that actually can help us improve care. They're pretty early.
24:15
There's another platform out there called Sickbay. Do any of you guys have this
24:19
in your NICUs or institutions?
24:21
Ediometry, Sickbay, anything like that? Hero Monitor?
24:26
Us either, by the way. our etiometry is being used in our cardiac intensive
24:30
care unit, but not in our NICU, even though I'm trying to get it there.
24:34
So imagine if you're a neurologist having to look at all these squiggly lines
24:39
going across and you had to detect seizures.
24:42
Is there a better way to detect seizures? So the issues with looking at EEGs,
24:48
fly EEG signal interpretation takes tremendous expertise.
24:52
Even just a regular run-of-the-mill pediatric neurologist can't read an EEG,
24:56
so it's super specialized.
24:57
Oftentimes, these EEGs have artifacts that are hard to discriminate from seizures.
25:03
And about 25% of the time, even the neurology experts can't agree if it's a seizure or not.
25:09
So there's significant inter-rater reliability when it comes to seizure detection EEG.
25:15
So this is a recent study where the investigators use sonification of an EEG
25:20
to help recognize seizures,
25:23
and so what they're able to do is take one hour of an EEG sample and condense
25:27
it down to five seconds of the sound.
25:30
Turns out the human is much better at recognizing sound changes in frequency
25:35
than we are at visualizing something.
25:38
So they took this characteristic of our human senses and used an AI approach.
25:43
And what they found is with this model, they had an 89% sensitivity and 78%
25:48
specificity for seizure detection using sonification.
25:52
So pretty interesting, a different approach. How about for predicting outcomes?
25:57
We have babies that have HIE. We use MRIs.
26:01
We kind of use their EEGs. We use their neuro exam to give us some information
26:07
about how they're going to do.
26:09
This was a recent publication using a machine learning model that used the amplitude
26:16
integrated EEG information.
26:18
So they took 80 full-term babies that had moderate to severe HIE.
26:22
Put them on amplitude integrated EEGs and had their AI model characterize the
26:30
brain state activity and recovery after the brain injury and categorize them
26:35
and then compare them to outcomes for the patients,
26:38
both their MRI findings and also their clinical outcomes.
26:41
And what they found, so BSN is the brain state of the newborn.
26:46
That's the indices that they developed in the AI model. And they followed over
26:50
three days for these patients.
26:52
And what you'll see at the top is green.
26:55
Green is favorable outcomes. The babies did well. And you can see most of them
27:00
are kind of up at the top, so above 50.
27:03
Cerebral palsy is down below. Cerebral palsy with epilepsy and then died,
27:07
their BSN scores were a lot lower.
27:10
So they use this model to help predict outcomes for patients.
27:15
And so this is an example of how ai can
27:18
augment what we're doing and support our care and maybe
27:21
give us some better information about predictions but now i'm gonna talk a little
27:26
bit about the issues with ai so i've at least hopefully i've shown you that
27:32
ai is here it's blowing up around us even if we don't necessarily know it to
27:36
medicine has some applications there's potential bias.
27:42
But there's four main issues that I would like to talk a little bit about,
27:45
too, is that garbage in versus garbage out. You've heard that a lot.
27:49
It's estimated that about 70% of health care data is usable.
27:54
30% probably isn't because it's mislabeled, missing, or labeled incorrectly.
28:00
So that means about 70% of data that we can even use. I've already highlighted
28:05
some of the bias that probably exists within it.
28:08
Accountability is going to be a big issue. So when one of these models cause
28:12
harm to a patient, and it will, who's going to be accountable for that harm?
28:17
Is it AI developer? Is it the hospital or healthcare system that purchased the
28:21
model? Is it the physician that used it?
28:24
Is it the patient or caregiver who went and bought it off the shelf?
28:28
So those answers are really evolving.
28:32
And then for us in the room, explainability. I'm not going to use something
28:36
in patient care if I can't understand how it works and how it generates its answers.
28:41
That's a huge issue. Many of the times, even the folks who develop the AI models
28:47
can't even really explain to you how it generates the output.
28:51
So explainability is a big issue. And then trustworthiness. Is the output that's
28:55
created accurate? Is it correct?
28:57
Is the data that we're feeding it private and secure?
29:01
Those are big issues. I see some bigger issues. I see that data scientists are offering big promises.
29:07
They think us in healthcare are kind of stupid in a way.
29:12
They think that we haven't solved problems that should have been solved for
29:14
quite some time. So they think that our healthcare problems are quite simple.
29:19
So they're underestimating the complexity of medical care.
29:23
Big tech is sinking big money into it. And then healthcare, we have big concerns.
29:28
And I'm kind of concerned what I'm seeing too. And that's why I go on top of groups like you all.
29:33
When I look at these publications, and there's a lot. So there's now a New England
29:37
Journal of Medicine AI, if you didn't know that.
29:39
And just this week, JAMA started their own AI issue.
29:43
This is a publication that was pretty noteworthy. Basically,
29:47
this group said that they had created a model that could act like a primary
29:50
care physician or nurse in a clinic.
29:54
30 authors are on the paper. 29 of them are data scientists and one is a physician.
30:00
There's no ethicist. There's no educators. There's no patients,
30:04
no caregivers, no administrators.
30:07
It's data scientists and one physician who's creating this for us.
30:11
Things have changed fast in this world. And so this figure on the bottom right
30:16
shows in 2018 to 2019 is when GPT-2 was created. It acted like a preschooler.
30:23
So this is kind of open AI timeline.
30:25
It could put one or two words together to create a sentence.
30:29
That's about all it could do.
30:30
A couple of short years later from that, GPT-3 was created and it had more like
30:36
a grade schooler who could put sentences together to make paragraphs and maybe
30:39
write a page. Then came GPT-4s, what we have today, that acts like a smart high school student.
30:45
And the author of this, I don't know if you guys have read this,
30:50
but Situational Awareness by Leopold Aschenbrenner is a pretty fantastic kind
30:55
of explanation about the evolution of large language models.
30:58
But he had predicted that there would be like a research AI that would come
31:03
out sometime 2025 to 2028.
31:07
It turns out there's a publication from last month in Archive,
31:11
which is kind of where AI publications come out.
31:14
And the title is The AI Scientist Towards a Fully Automated Open-Ended Scientific Discovery Model.
31:20
And I'm going to read off exactly what it says that this model does.
31:24
It generates novel research ideas, writes code, executes experiments,
31:28
visualizes results, describes its finding by writing a full scientific paper,
31:33
then runs a simulated review process for the evaluation of the paper.
31:37
And it's going to act like the human scientific community. And it costs $15.
31:42
So you guys don't have to do science anymore. So we have a model that's going
31:46
to do it all for us, right? Right.
31:48
So my point is things are moving fast, and I see big issues with it.
31:54
I see a lot in the data scientists world.
31:56
Big tech AI developers, government are taking notice and being involved.
32:00
Us on the other side, physicians, scientists, patients, healthcare staff,
32:04
administrators, educators, clinicians, ethicists, we're kind of being left out.
32:09
This needs to be a continuous collaborative learning model if we're going to
32:12
take full use of this powerful tool.
32:15
So that's part of the reason I'm here tonight. I want you all to become at least
32:20
AI conversational, take notice that it's here and be involved with the one-day
32:25
hope of being AI literate like Kaiser or Alvaro back there.
32:28
I'm still not. I'm still working on it. So you as a doctor, a therapist,
32:32
or nurse know a lot more about healthcare than these folks that are developing
32:36
healthcare AI at the moment.
32:39
So AI is a big umbrella term that captures machine learning.
32:43
You think that you've heard of before linear regression logistic
32:47
regression goes down to neural networks deep
32:50
learning and the new kid on the block generative ai so large
32:53
language models i think might have the first to
32:56
have the widest impact within health care it has
32:59
the ability to generate and understand text models are trained
33:02
on huge amounts of data in their prediction machines and
33:06
they're built on the transformer architecture and they have self-attention so
33:09
they actually understand contest so in
33:12
the far right up there is gpt4 and this is showing
33:15
what it was trained on a huge corporate so basically it's trained on the entire
33:20
internet and almost all of the human written text has been used to train these
33:24
models and what that means is these models can reach out to pretty much all
33:28
of a human written text and act as your resource when you're asking your questions For GPT-4.
33:35
It cost $100 million to train their model.
33:44
So you can see why these big tech companies want to get some money back.
33:49
So you might be asking yourself, well, do we even need AI then because it's
33:53
dangerous, it's not trustworthy, it's not accurate, it's going to amplify bias?
33:57
The answer is we do. Medical data is exploding. We're in a data tsunami.
34:02
Medical information doubles about monthly. We're surrounded by data.
34:07
We need to be able to connect the dots so that our data becomes information,
34:11
wisdom, and insight so we can improve care delivered to our patients.
34:15
When we look at our ability to analyze data, we've always been kind of down
34:21
on the bottom left descriptive.
34:23
Have a baby who's sick, who's acting ill, they're hypotensive.
34:26
We now know that's sepsis. We can now diagnose it because of blood cultures.
34:31
We don't do a good job of predicting, even though we're starting to get to that.
34:35
There's a lot of predictive sepsis models that are being developed out there,
34:38
mainly in the adult world.
34:40
But we certainly haven't got to prescriptive.
34:42
So once you have the information, how are you going to change your care for
34:46
your patient? And that's where we're evolving to, and AI will help us get there.
34:51
So we're surrounded by EHRs. We're kind of slaves to them, unfortunately.
34:56
95% of the data that exists within EHRs we don't even use. 3% to 5% we use.
35:03
But it's a big issue because we're just surrounded by data that's sitting there.
35:08
And for us, it's a lot of clicks that we have to get to different pages.
35:12
AI is great for pattern recognition. So as humans have created languages that
35:16
we put into words, sentences,
35:18
paragraphs, chapters, into books, and we create libraries so we have wisdom
35:22
and knowledge and resources that we can go to,
35:26
we're not so good at the part where we have to put together the English language
35:31
and the Chinese language, but things like AI can do that.
35:35
So this will be my last kind of technical part before I'll get into how should
35:39
you use some of these large language models.
35:42
Machine learning is broken down into two main types, supervised learning and
35:46
unsupervised learning.
35:47
Supervised learning, we feed it labeled data so it realizes what we're giving.
35:52
So as an example, we'll give it 1,000 retinal images, 100 of which have retina,
35:57
but the 900 of which it doesn't.
35:59
Over time, it begins to recognize what's ROP versus what isn't.
36:03
So that's an approach to using machine learning.
36:06
Another approach, and a lot of you guys in this room are a lot more sophisticated
36:09
than me and machine learning and deep learning.
36:13
So you can correct me at any time if I'm wrong.
36:15
But unsupervised learning, we feed it data, and it then recognizes patterns
36:21
within the data that us humans haven't been able to recognize.
36:24
And I think it will help us with disease processes like NEC and BPD.
36:29
This is a recent study out of the University of Florida where they used a unsupervised
36:34
machine learning model and looked at 183 babies that had intestinal injury from 2013 to 2019.
36:42
And they took 50 different data points, some of them maternal,
36:45
some perinatal, some of them neonatal, and then identified five different phenotypes
36:50
within those 183 babies.
36:53
And so I think in healthcare and neonatology in particular, things like NEC
36:59
and BPD, we still don't really understand the different phenotypes that exist.
37:03
And if we're really going to
37:04
improve care, we have to understand our disease better than we do today.
37:08
And i think this is an approach and i wanted to call out a couple is antonio here tonight.
37:15
So Alvaro is, go ahead and stand up, Alvaro.
37:23
So I don't know if you guys know this. You guys probably all know Alvaro.
37:27
He is one of the brightest minds in this field right now.
37:31
So the fact that you guys have him in San Antonio, it's a blessing that I don't
37:36
know if you guys understand what you have right now.
37:39
He is a prize. And so he's working on BPD and trying to help us understand the
37:46
pathology and phenotypes that exist within BPD.
37:50
But I could count on my hand like some of the brightest people in neonatology
37:54
pediatrics in this field, and two of them are sitting right next to each other.
38:00
Kaiser, who will be speaking later, is definitely in that realm.
38:03
But we're lucky to have people like them in
38:06
our field that are going to help lead us into this
38:09
and help us understand how to use some of these tools so thanks alvaro and kaiser
38:15
so now i'm going to switch gears and this is going to be how do you use large
38:19
language models how many of you guys have chat gpt gemini or clod an account
38:25
so raise your hand how many of you guys have,
38:27
that's a little better than average usually i get 30 to 50 percent of the audience
38:31
will raise their hand and say, I've used a few things. How many of you have used it this week?
38:37
Ooh, that's better than that. How many of you used it today?
38:41
Nice. You guys are outstanding. I love that.
38:45
So I'm only going to be talking to the other half of you that haven't really
38:49
then started using it. Maybe you guys can give me insight on how to use it better.
38:53
So I give workshops on how to use some of these models and how to prompt them
38:58
to get better results what I'm going
39:00
to talk about it's really high level probably boring to at least have,
39:04
But when I'm thinking about these models, I think that you have to think about
39:08
the output that you're generating.
39:09
Is that output going to be helpful? Is it going to be honest?
39:13
Is it going to be harmless?
39:15
So I've been to San Antonio once before. I could have used Chachi P to help
39:20
me decide what I should do along the Ribble Walk today.
39:24
And the results that it gives me, probably not life-threatening to me unless
39:28
it tells me to go scuba diving or something in the river.
39:32
But really it's so that you can ask chat gpt like
39:35
give me a itinerary of what i should do in san antonio when
39:38
i'm here that's pretty safe but if i'm asking
39:41
it i have a 24 week baby who's now six days old who has these five symptoms
39:46
what should i do or what should i be thinking about there's no way we should
39:50
be using it for that so that's where it kind of clicks into that category when
39:54
it gives you the output you should treat it as if it's a first-year medical
39:57
student you're going to question a lot.
39:59
What you'll hear from a lot of people in the AI world, they'll tell you to trust
40:04
the output and then verify.
40:06
I think for healthcare, that's wrong. I think we have to verify.
40:09
We have to verify again, use, and then maybe trust.
40:13
But I think we have a long way to go before we just kind of jump in and start using it.
40:17
And the first draft that you get should always be just kind of considered your
40:22
first draft, just the same way that we would write a manuscript.
40:25
You have to go through many revisions to make it better in the output that you want.
40:30
So here's Jim's rules for AI using large language models.
40:34
I feel like anything you type into them is now in the public menu.
40:39
So don't put any PHI, anything about yourself private that you don't want the world to know about.
40:44
Don't blindly trust the output. Don't use it for clinical care and have like
40:48
an 80 to 20% rule when I use it About 80% of what it generates.
40:53
I would have already done on my own or already knew 20% of it's new Of that
40:58
20% though 10 is great Fantastic a new angle something I would have never thought
41:03
about before really additive what I was doing And then 10% is completely wrong
41:07
and made up and fictitious.
41:09
So you still have to be somewhat of a subject matter expert to use this and
41:14
understand the output and understand what's reliable and what is it.
41:18
So when I use it, I use it to brainstorm, to learn, to be creative,
41:21
and to edit, or kind of my main things, at least for tonight,
41:25
how I use it, sort of to brainstorm.
41:28
So I was kind of put in charge of trying to improve clinic throughput for a
41:33
different pediatric subspecialty clinic.
41:36
And so I used it to help me brainstorm. And then it used something called theory
41:39
of constraints, which I know about.
41:41
It helps you identify where the constraint in the system is.
41:46
But so it helps build kind of information in your thought about it.
41:50
This was actually an email.
41:52
So one of our nurse managers knows that I use ChatGPT all the time.
41:57
And she said, geez, I got this email and I need to respond to it. And it's long.
42:02
She said, can you help me? I said, OK, come on. let's try
42:05
to do that and but what i told chachi beauty
42:08
to do is please create an optimistic and positive message
42:11
in response to this email so it did
42:14
it said thank you for the valuable discussion this week's meeting it's
42:17
clear that we have critical decisions ahead and then down at the bottom thank
42:21
you for your dedication and leadership so i've also used it this way and i don't
42:25
know if others have but to you probably should use it from this do you ever
42:29
get emails do you ever get emails from people that are just,
42:34
you know, what's coming.
42:35
It's like three pages long. It's ranting and raving about something that they're upset about.
42:40
You can have ChatGPT create an email back to them and you can put in there,
42:44
please make this positive, optimistic, but make it three pages and like two.
42:49
So they have to read through three pages of an email, but you don't send it
42:53
like a nasty one back. You can make it be nice and positive and send it back
42:58
to them. And so I do that quite frequently.
43:02
You can actually have it read pdf articles and summarize
43:05
findings and so i only do this for
43:08
open access articles i don't do it for ones that are copyright protected
43:11
but you can put the pdf in there and say please provide five highlights from
43:16
this pdf article and it does a pretty good job so instead of taking maybe 20
43:21
minutes to read something you can get important information with a matter of
43:24
like a minute you can also have a look at figures and say please explain to me figure five
43:30
So a lot of times I'll be reading something, especially in the AI world,
43:33
where they have all these transformer architecture things that I don't really understand.
43:37
And I'll say, can you explain this figure to me? And most of the time it will
43:41
explain the figure better than the authors did in the manuscript, quite honestly.
43:45
And you can also fill it in output. So I asked it to say, can you explain this
43:49
figure at an eighth grade level? And it did pretty good.
43:52
And I said, can you explain it at a college level? so you
43:55
can change the output both language kind
43:59
of jargon and then understandable level
44:02
by how you talk to it and then
44:05
this was i used i applied for a higher level job within our health care system
44:11
and i downloaded the job description my cover letter and my cv and i had it
44:17
analyze what i had written for my cover letter and had to look at my cv and
44:21
then It gave me some insight,
44:23
and it said, oh, you're pretty good, whatever.
44:26
But then it said, tailor your expertise to the role's focus,
44:29
and you're kind of missing this, so make sure you emphasize X, Y, and Z.
44:33
And so you can use it in that regard. I didn't get the job, so maybe it wasn't that good.
44:38
I'm not sure. But so you can use it for things in that manner.
44:43
And actually, I talked with Kaiser tonight that Colorado has passed an AI bill,
44:48
and it's really about consumer protection.
44:51
And I think their definition of AI is horrible, but I actually had to have ChatGPT
44:56
read through the AI bill for me and summarize because I don't understand like
44:59
lawyer or whatever it's written in legislation lingo.
45:03
And it helped me do that. You can use it for poem writing, image creation.
45:08
So I've been married to my beautiful, lovely, fantastic wife for 27 years.
45:13
So a few months ago, I wrote her a poem about how, you know,
45:19
it's been great to be married to her for 27 years.
45:21
You guys, some people are probably thinking that is off.
45:25
OK, so should I go to the grocery store or the nearest Hallmark store and go get a card off the shelf?
45:31
Or the fact that I actually put personal things in there and told my wife how
45:36
great she's been for 27 years.
45:38
She's a great mother, a great clinician.
45:41
She's the backbone to our family. and it
45:43
created a poem that had that in there so is it
45:46
a little better that the personal part came from me versus
45:49
hallmark so i get a lot of
45:52
snickers when i say that but i she knew it too by
45:54
the way when i sent it to her she's like yeah chat gpt wrote this huh i said
45:59
yeah but look at it it's mean and she did so when you're using these and i don't
46:05
know if we're going to have to do this forever maybe in the next couple years
46:08
it'll change but right now when you're using chat GPT, you do kind of three things in a prompt.
46:13
You assign it a role, you assign it a task, and you tell it what output it should come in.
46:17
And it can do tons of different outputs, PowerPoints, and I'll show you some of that.
46:21
So I told you I don't use it for clinical care.
46:24
I do kind of test it almost on a weekly basis. I take pictures of things in
46:29
the unit that don't have any private health information data on it.
46:34
And I see if it recognizes things. So three months ago it wouldn't have recognized
46:37
this so i took pictures and students and residents and fellows what do we have
46:43
here this is a left lateral z cubitus x-ray there's something here,
46:48
then maybe there's a lucency what do we have on this film.
46:52
Yeah. Yeah. So pneumoperitoneum. I just told it to analyze the image and it
46:58
told me about the gastro-2 pneumatosis. It's a bowel gas pattern,
47:02
but it missed the pneumatosis.
47:05
I mean, the pneumoperitoneum.
47:06
So then I gave it the same images, but I prompted it differently.
47:10
I said, act as an expert in neonatal critical care and radiology when evaluating the images.
47:15
Then I gave a context. I will be asking you a series of questions based on the
47:19
medical image. They're from a newborn with feeding intolerance,
47:22
blood and stool, hemodynamic instability.
47:25
So for us, it's kind of a slam dunk, right?
47:27
We know this baby probably has NEC. And then please provide answers in short
47:31
paragraphs not to exceed five sentences at a level that a medical student could understand.
47:36
So then it looked at it, and even kind of before it looked at the image,
47:40
it said, based on your description, I have a high suspicion of NEC,
47:43
particularly concerned for complications like pneumoperitoneum.
47:47
So it was able to look at the image differently and get the right answer.
47:52
So I don't use this for clinical care, but I can tell you like six months ago, it could not do this.
47:56
So it is learning on interpretation of images better.
48:02
So it also talked about the diagnosis, but then I didn't tell it to,
48:06
but it started giving me recommendations on immediate surgical consultation, supportive care.
48:12
And I said, okay, these parents are PhD scientists, and I have to talk with
48:16
them about what NEC and pneumoperitoneum means. So can you help me explain it to them?
48:22
And so here's the output, and it was actually pretty good at a level that probably
48:26
PhD parents could understand.
48:28
And even at the end, it added a few kind of compassionate things to it.
48:33
It said positive aspects to keep in mind.
48:36
There was an early detection, which we know probably doesn't change the outcome
48:39
in NEC see that much, but ChatGPT said it did, said, don't worry.
48:43
You have a team of specialists taking care of your baby who do this all the time.
48:47
So it actually tried to throw a compassionate spin.
48:50
So I said, all right, same scenario, but create an explanation that a 16-year-old
48:56
mother with developmental disabilities who has a limited ability to comprehend
49:00
how ill her baby is and explained it at a fourth grade level.
49:05
So it did. It said, your baby is very, very sick. Your baby's belly is swollen
49:09
and there's a hole in their intestines.
49:11
Uses words like tummy, baby get better and so forth. So you can see how you can change the output.
49:16
I asked it to create an image of a cartoon and
49:19
this is what it came up with i don't think it's very good so there
49:22
is maybe a hole in the stomach i think not really
49:26
sure there's a baby that looks pretty happy doesn't really look
49:29
that sick to me so i've done it for other times too so this was an example i've
49:34
used of a five year old who had kawasaki's disease and i asked chat gpt to create
49:38
a cartoon that would help the five-year-old feel better and this one at the
49:42
left it kind of goes through a series of kind of pictures about what's happened
49:46
to the kiddo with Kawasaki's disease.
49:48
And a lot of times when you do pediatrics, they do superhero stuff.
49:52
And I do think kids like superheroes.
49:53
So it's kind of a nice thing to do.
49:56
As a medical student, as a fellow, I think that these models are going to disrupt medical education.
50:03
You now have the capability. I have an individual tutor that's in your pocket
50:08
24-7. You can ask it questions.
50:11
NYU is already doing this. They have a model built within their EHR system.
50:16
So if you're a medical student in their institution, you admitted a patient
50:20
with diabetic ketoacidosis, it'll look at your HMP and say you missed these
50:24
three elements, so add these.
50:26
And also say, by the way, in your last in-service training exam,
50:30
you missed three questions on VKA.
50:33
Here's five questions to make you think about that again, and here's three articles
50:37
for you to read to help you understand that disease process better.
50:40
And that's an email that they get by 8 a.m. the next day.
50:44
So there's a capability of changing the way that we teach. All right,
50:49
medical students, what's this?
50:52
Hemoglobin oxygen dissociation curve so picture and
50:56
i took and said what's this i told it
50:58
to analyze the image and it knew what it was and
51:01
actually broke it down into very important physiological considerations about
51:05
what that image was and i said please create five multiple choice questions
51:10
based on this image to help me better understand this and i also said use chain
51:14
of thought reasoning so any of you guys use that as a prompt chain of thought
51:18
reasoning if you have it, you should.
51:20
And the reason that is, especially for somebody who's learning,
51:23
it tells the large language model to generate an output that goes through its logic.
51:28
Like, how did you get to the answer?
51:30
And so this, it was arterial oxygen content question, gave an answer.
51:36
But then down here, it gives an explanation. It shows you the formula that it
51:39
used and came up with the answer.
51:42
So if you're learning about something, that's a powerful thing to do is always
51:45
ask it to use chain of thought reasoning. So you can then understand how it
51:49
comes up with its answer.
51:50
You can have it create PowerPoint presentations. So I did for this. They come up as generic.
51:56
And I have to say, it could save some time. Then the other thing is,
52:00
I said, please create a clinical scenario of a 10-year-old with asthma.
52:04
Anemia, new pneumonia, and a new oxygen requirement.
52:09
And explain to me how understanding this hemoglobin oxygen dissociation curve
52:14
will help me take better care of her.
52:16
Please create a clinical scenario that would kind of tie all those things together.
52:20
And it did. I don't show you anything here, but it did pretty good.
52:23
It even created vital signs and symptoms for the patient, laboratory results,
52:27
and then really went through it kind of step-by-step.
52:30
About six months ago, our unit was looking at new ventilators.
52:33
I wasn't familiar with all the different numbers.
52:37
So I took a picture of it and said, analyze this image and explain to me the
52:41
difference between VTE and VTL, which I knew.
52:43
And it gave me the right answer. But something it did give me,
52:46
it said, oh, your tidal volume's three mils per kilo.
52:49
And if this is a premature baby, you're using a lung protective strategy.
52:53
And I didn't ask it that. I just said, analyze the image.
52:56
So pretty remarkable. I'm almost done, so we'll kind of move on.
53:00
But I wanted to make sure that you kind of understood that there's a lot of power in these models.
53:06
And I think it should change the way that we do literature reviews today.
53:10
There's something called Research Rabbit that uses medical subject headings.
53:15
It uses NIH grant funding.
53:17
And each one of these circles is kind of a lab group.
53:20
And it allows you to figure out which labs are working together,
53:23
maybe where there's an area people aren't investigating if it's your topic of interest.
53:27
Open knowledge maps is based on PubMed, but each one of these circles,
53:31
so if you put in retinopathy of prematurity,
53:34
each one of these circles will be a subtopic that you can click on,
53:37
and then it shows you all the different manuscripts, and it will even show you
53:41
how many times they've been referenced.
53:43
So you can say, oh, the one that's been referenced 78 times is probably the
53:46
most important, I should read that one.
53:48
And then open evidence is kind
53:50
of the first model that's been created on Elsevier's medical publications.
53:54
So you chat with it like you would chat GPT, but it's been trained on medical
53:59
and scientific publications.
54:01
And a recent one that I've become aware of is called Glass Health that my wife
54:06
has been kind of using in her practice just to see how it works.
54:09
And it gives a good differential diagnosis. It helps start creating a progress note for you.
54:14
So Open Evidence was the first AI to score above 90% of the USMLE.
54:19
You can ask it questions that are controversial. Do red blood cell transfusions cause NEC?
54:25
So who believes red cell transfusions cause NEC?
54:29
Anybody? Well, this one at least gave a good answer.
54:32
It said the relationship between red blood cell transfusion and NEC is complex.
54:36
Oh, good. Because of this.
54:38
And then it actually gives you a list of the articles that you click on.
54:41
It goes right to the PubMed.
54:43
So we're here today in kind of the evolution of AI and healthcare,
54:48
but it's evolving rapidly.
54:50
We're going to move quickly to the end in our lifetimes and maybe even the next
54:55
10 years that these models are going to have huge capability.
54:59
And it's due to the fact that we have the ability to have high computational
55:04
levels now and also cloud-based storage data and other kind of GPUs and other things.
55:11
But I want, remember I talked to you about penicillin, thalidomide, or the light bulb.
55:15
I don't know what AI is going to be in healthcare. It has the potential of being many, any of those.
55:21
But there's no way we're going to figure it out without people like you in the
55:25
room actually being involved in using it, evaluating, analyzing it,
55:29
providing feedback when it causes errors.
55:31
So we really need us to be involved, and I don't see that happening.
55:35
And so some resources for you all, kind of where I got started is in the American
55:40
Board of AI and Medicine. I took a certificate course by somebody by the name
55:44
of Anthony Chang, who's a pediatric cardiologist.
55:47
But he's really set up an infrastructure for people interested in learning about AI in health care.
55:52
And it's pretty remarkable. So that's a great resource.
55:56
And the Neomind AI, Kaiser, Alvaro, and myself and others have created something
56:03
called Neomind AI, which right now is a non-revenue generating learning collaborative.
56:09
Really focused on bringing people together that want to leverage AI in healthcare
56:13
and make sure that we do so safely, effectively, ethically, and unbiased,
56:18
and make sure that improves our patient care.
56:21
And so this is kind of our founding group. We're from around the country and
56:25
really have diverse skill sets.
56:27
And like I told you guys before, I'm not a computer scientist.
56:30
I'm not a data scientist. I'm just somebody trying to figure out how to think
56:33
about this from kind of wide angle lens.
56:36
And I see a lot of positives and I also see some potential negatives.
56:41
So I think AI stands for augmented intelligence. It's going to improve what we do if we let it.
56:47
It's not going to replace any of you anytime soon.
56:50
So thanks for kind of going with me on that journey tonight.
56:54
And at any time I'm willing to talk and I post a lot on LinkedIn.
56:58
So I don't know if any of you guys follow LinkedIn but I follow a lot of smart
57:01
people on there I've learned a lot from through the years and yeah.
57:12
I'm happy to answer any questions, anything technical. I'm just going to point to the table back there.
57:21
I was thinking about future and long-term AI and the struggles that healthcare
57:28
provided right now based on it comes to other people,
57:32
thinking that they have a safe healthcare and making a lot of big decisions
57:36
around healthcare probability of knowledge and how AI could influence that for the force.
57:43
No, I think you're right. So I think in general, I'll stand on my soapbox a
57:49
little bit. It isn't even necessarily AI.
57:51
I think we as physicians, it wasn't us probably, it was kind of our foremothers
57:56
and forefathers who came before us, decided that they weren't going to be involved
58:00
in the business side of healthcare.
58:01
And they just wanted to be doing the medicine and let the business people run medicine.
58:06
And we're kind of stuck with that now, that we should have always been involved
58:11
in it because there's different priorities a lot of times for businesses.
58:15
Even nonprofits are there to generate profits. And so profits,
58:20
unfortunately, a lot of times come before patient care.
58:24
And certainly we noticed it during COVID and beyond that there's a lot of staff reduction.
58:30
So Texas children's right now is reducing their staff
58:33
and people aren't working less hard they're working harder
58:36
and so it's it's hard i just
58:39
think that overall us physicians let things slip
58:42
away and that's kind of where i'm here too so how many of us were involved in
58:46
the development of our ehr system that we use probably zero right i want to
58:52
see tell me how were you actually in the room creating like epic you said this
58:57
is how it's going to work or did you actually kind of modify what already existed for your institution.
59:02
So were you in the planning room with Judy Faulkner when she was creating Epic in the 90s? Okay.
59:09
Yeah. Well, so most of the time, what I would say is we were not involved in
59:13
it. We were handed this platform to say, you're going to use this.
59:17
You might change things over time and make it get better. How many of you think
59:22
EHRs work really good for the NICU?
59:25
I haven't met anybody who says, oh, this is fantastic. I think there's some good things from it.
59:30
Good things being that it's helped with imaging. You can look at chest x-rays
59:34
over time. You can look at lab data.
59:36
That's better, but you sure have to dig through a lot of stuff to get other information.
59:41
So it hasn't really improved our care. And my point being is we weren't involved in EHRs.
59:46
AI is going to be EHRs on steroids.
59:50
And if we're not involved in that, it's going to go down pathways that we don't want to go down.
59:55
So we need smart, well-intended people that are putting patients and caregivers
1:00:00
first and understanding how it will affect them.
1:00:03
So as an example, there was some Bay Area health care system that started using
1:00:09
some of these large language models.
1:00:12
And one was to create staffing models for their unit.
1:00:16
And you can imagine who's paying for that, right? So the hospital administrators
1:00:22
are paying for that model.
1:00:23
So they're going to tell the model they want there to be stringent or lower
1:00:28
resource staffing models versus high.
1:00:30
And then nurses used to answer a call line, but a chat bot started answering
1:00:36
their call line and displaced them from doing that.
1:00:39
So they actually started protesting in the Bay Area as nurses against AI.
1:00:43
And my guess would be is that that health care system didn't involve nurses
1:00:47
and actually the development of that AI. And that's what needs to be from the
1:00:51
very beginning, that these are being created and developed.
1:00:54
We, who are going to be using it, need to be involved. And right now,
1:00:58
I'm not seeing that very often.
1:01:00
So you bring up a great point. I wish I could say it was different.
1:01:04
All upon that, do you think it would be wise for us to start,
1:01:08
not unless we're going to get a nice background of it, at least all care providers,
1:01:12
but definitely start seeing, do you think it's wise for us to start meeting our budget?
1:01:17
Just that came well
1:01:21
there's different ways so one is so everybody
1:01:24
in this room should have raised their hand when I said have you
1:01:28
used Chachi PT Gemini Clot every single one of us should have raised their hand
1:01:31
because that's going to be one of the kind of platforms that are being used
1:01:36
and it's going to be widespread because it's publicly available it's free in
1:01:40
some cases and guess who else is going to be using it your patients and their families,
1:01:46
And there you think Dr. Google was the issue.
1:01:49
Dr. ChatGPT is a whole nother level of issue.
1:01:53
And you're going to have to figure out as a clinician how you're going to handle
1:01:56
that when a mother or a father comes up to you and says, you know,
1:01:59
I use ChatGPT and I downloaded my son's lab results,
1:02:05
x-ray result interpretation. I think you're missing something.
1:02:09
What's your response to that going to be? That wasn't trained on any medical
1:02:13
literature, so it doesn't know anything. I think I probably showed you that it knows something.
1:02:18
And so we have to start figuring out how to actually work with this kind of technology.
1:02:23
And the first step is just using it. And then the second is I do think that
1:02:28
many health care systems are not going to be able to create their own.
1:02:32
There's going to be some systems that can because they have a lot of money and
1:02:35
they'll invest in it. So the Hopkins of the world may be Stanford's,
1:02:40
Cleveland Clinic, Intermountain Health, others that have a lot of resources,
1:02:44
but most healthcare systems.
1:02:46
They're not going to have the money to be able to invest in the IT infrastructure,
1:02:51
the data infrastructure, and the day-to-day managements of AI.
1:02:54
So they're going to be buying stuff off the shelf, which is going to be a big
1:02:57
issue because it may not work for their patient population or the way their clinicians work.
1:03:02
So that's why you have to be knocking on doors. So if your institution is turning
1:03:07
on AI, be it ambient intelligence where there's no documentation that's being
1:03:13
created, you should make sure that there's clinicians involved.
1:03:17
And if you're not, people should be knocking on the CMIO's door,
1:03:20
the chief medical officer's door, and say, we shouldn't be using this unless
1:03:23
we have a way to feed back when it's causing problems and refine it over time.
1:03:28
So we have to be involved. And it's not great because we speak different languages.
1:03:34
The data scientists speak a different world.
1:03:37
They really do think that healthcare is easy and we just haven't figured it out along the way.
1:03:42
And they don't really understand the clinical side of things.
1:03:45
And I still see that they're working in silos most of the time.
1:03:48
Most campuses, the data scientists are off over here.
1:03:52
Clinicians are still off in our own world. And we're not really kind of hybridizing
1:03:57
and working together and we need to.
1:03:58
And I think it will be us that has to go knocking on the door.
1:04:01
From a technical aspect, so I've gotten an MBA and I did it so I could speak
1:04:04
with healthcare administrators, MBA is not very hard.
1:04:08
Data science is hard. I'm trying to understand what that is.
1:04:12
So unless you have some background in it, you're not going to,
1:04:15
so I could go and get an MBA and not have background in business,
1:04:18
what they teach and the foundational information, but that is not rock.
1:04:24
Data science and AI is kind of rocket science because it's a whole new language
1:04:28
and it's really sophisticated and the people teaching you a lot of times are
1:04:32
not very good at teaching you because they think way up here and they don't
1:04:36
even remember what words they don't know.
1:04:38
So Halvoro and Kaiser often use words when we're in our NeoMind AI meetings
1:04:43
that I have to go look up afterwards because I don't even know what they're talking about.
1:04:49
But I think we have to be involved and my approach is to make people get interested about it.
1:04:57
And because there's a lot of potential benefit.
1:05:00
Thank you so much for listening to that talk. Don't forget to click on the link
1:05:05
in this podcast for free credit that may include CME, MOC, or ethics credit,
1:05:10
depending on our topic or podcast.
1:05:13
Our podcast is brought to you by the Department of Pediatrics at UT Health San
1:05:18
Antonio and University Health's new Women's and Children's Hospital.
1:05:24
We also have Pediatrics Now for Parents. That's meant to be one less thing you
1:05:30
have to say in the exam room.
1:05:32
With the exception of talks or grand rounds, almost everything you hear on here,
1:05:37
we will have a version on Pediatrics Now for Parents.
1:05:41
The website for that is PediatricsNowforParents.com. It's also found wherever you get your podcasts.
1:05:49
I'm Holly Wehment. I'll see you soon.