Latest / AI and Healthcare / The AI Doctor Debate: Autonomous AI in Healthcare with Zeke Emanuel, MD, PhD

Transcript

The AI Doctor Debate: Autonomous AI in Healthcare with Zeke Emanuel, MD, PhD · AI and Healthcare

• 0:00
  The source of the disagreement is about this issue of autonomous AI
• 0:03
  without case-by-case physician
• 0:05
  oversight.
• 0:06
  That's what autonomous means.
• 0:08
  Can
• 0:08
  AI manage a patient without a doc
• 0:10
  overseeing in some way?
• 0:12
  Humans aren't perfect.
• 0:13
  We make mistakes, right?
• 0:14
  Let's just be blunt about it.
• 0:16
  AI will make mistakes too.
• 0:18
  We've certainly not claimed that AI is going to be perfect.
• 0:21
  You know, Waymo
• 0:22
  Cars, they're not perfect.
• 0:24
  They have accidents, they make mistakes, they hit people, unfortunately.
• 0:28
  They just happen to be comparatively
• 0:30
  way safer than
• 0:32
  Human drivers.
• 0:33
  And the same thing is gonna be true in medicine, that autonomous AI
• 0:36
  is gonna be better consistently than humans.
• 0:39
  That doesn't mean they're gonna be
• 0:41
  Perfect.
• 0:55
  If you're in healthcare at all, you probably heard about Dr. Zeke Emmanuel's article in JAMA
• 1:00
  where he talked about autonomous agents and AI replacing physicians.
• 1:05
  If any part of you believes that AI could replace what doctors do today, if it's even possible,
• 1:11
  then one may argue it is your responsibility to make sure we are prepared for that future,
• 1:17
  whether or not you agree with it.
• 1:19
  That is why I'm very excited about our guest today, Dr. Zeke Emmanuel,
• 1:23
  who is an esteemed professor, a doctor himself, might I add, an oncologist, a bioethicist.
• 1:29
  And he has started this conversation because he deeply believes that autonomous agents will
• 1:35
  replace a lot of the things that physicians have to do today.
• 1:38
  With that said, welcome to AI and Healthcare Podcast.
• 1:40
  My name is Dr. Sanjay Janaeja.
• 1:42
  I'm a hematologist and medical oncologist, and I cannot wait to get into this.
• 1:46
  Zeke, thank you so much for being here.
• 1:48
  Great.
• 1:48
  Lovely to be here.
• 1:50
  So Zeke, I'm curious, where's the first place that you think AI is going to kind of replace
• 1:56
  what a physician does and why do you believe that?
• 1:59
  Uh, managing the side effects of chemotherapy.
• 2:02
  So if you think about where AI is likely to be best and least problematic, that is have
• 2:13
  the fewest errors, it's places where we have great guidelines, great algorithms for how
• 2:20
  to manage patients.
• 2:22
  In the regular clinical scenario, that turns out to be things like managing chronic illness,
• 2:28
  like hypertension or diabetes or hypercholesterolemia, where you have very good guidelines, you know,
• 2:34
  if then or, you know, ask a question, you know, is a blood pressure over 120 over 80,
• 2:41
  then, you know, do this.
• 2:42
  If you've tried this and it hasn't worked, you know, add another second line agent, one
• 2:48
  of the following.
• 2:50
  And I think in regular, quote unquote, non-oncological medicine, my own view is that the management
• 2:57
  of the big three, hypertension, diabetes, and hypercholesterolemia are places where
• 3:04
  we as physicians do a terrible job.
• 3:07
  And I think that's one of the first places we will have deployment of autonomous AI in
• 3:16
  medicine at scale.
• 3:18
  In oncology, the similar place is the management of the very common side effects of chemotherapy,
• 3:27
  whether nausea, vomiting, diarrhea, insomnia, et cetera.
• 3:32
  And I think in those cases, we have a lot of good guidelines with a lot of if-then statements
• 3:39
  and specific treatments that are the preferred interventions.
• 3:45
  And so I think that is likely to be the place where we're going to actually have autonomous
• 3:53
  AI, and in some proportion of cases, it's gone further than we would want.
• 3:59
  And the autonomous AI will say, "Well, you have to come to the physician's office.
• 4:04
  You should go to the emergency room."
• 4:07
  That, you know, if we've managed it right and made all the arrangements, that should
• 4:11
  be a very small number of cases.
• 4:15
  You know, maybe 10 percent of cases come to the office and, you know, a few percentage
• 4:19
  go to the emergency room.
• 4:21
  That means that, you know, 90 percent can be managed autonomously at home by an AI agent.
• 4:29
  And that would, you know, reduce emergency room visits and probably reduce hospitalizations,
• 4:34
  be good for patients and save money.
• 4:36
  So that's my number one spot.
• 4:39
  I love what you said of at home, manage at home.
• 4:42
  And the reason I do is because when you are evaluating a toxicity, especially as, you
• 4:47
  know, kind of nuance some of these toxicities are getting with these newer drugs and agents,
• 4:52
  you got to get an optometry exam and all of this stuff.
• 4:55
  We have to remember the way it is currently is your bottleneck is a 15-minute visit to
• 5:01
  discuss all things related to your cancer, its treatments, the vacation you're taking,
• 5:07
  and a, you know, thorough assessment of all of these side effects that are potentially
• 5:11
  happening and addressing at an earlier, early enough toxicity level to hopefully not, you
• 5:16
  know, be in a situation where you can't do therapy at all.
• 5:18
  So you have just introduced this concept of, hey, here is one solution of many of the
• 5:24
  different parts or elements of the health care system, which, you know, very well and
• 5:28
  I've written on that are limited and haven't had a solution that in between the visit.
• 5:33
  That's like meaning that you're not limited to that 15 minutes.
• 5:35
  And the way it is currently, it's just literally my wife is also an oncologist.
• 5:39
  We just get calls on our phone.
• 5:40
  And that's only if you have met the threshold to call a physician at 10 p.m., which her
• 5:45
  and I can tell you would probably be very, for whatever reason, probably unjustifiably
• 5:50
  like scared to do.
• 5:51
  So you eliminate a lot of that stuff, that human element of disruption.
• 5:55
  And is this important enough?
• 5:56
  People are dying of their cancer.
• 5:58
  I have nausea with no, you know, with improved scans, for example.
• 6:01
  I'm going to flip the question a little bit on you.
• 6:04
  If there is one element that A.I. or autonomic agents or A.I. can handle in cancer care that
• 6:16
  a physician should not override, where one could argue it's potentially like an ethical
• 6:22
  thing to say, I don't know, je ne sais quoi.
• 6:24
  I just disagree with this A.I. in this situation.
• 6:27
  What is one thing where you would argue potentially that that is something that that we should
• 6:32
  say probably won't exist?
• 6:34
  Where a human should not get in between A.I.
• 6:37
  Well, look, let's get to the situation of when do you think A.I. is superior to a human?
• 6:44
  Yes, yes.
• 6:46
  I think we have to put on the table, and this maybe has been lost in the discussion.
• 6:53
  Humans aren't perfect, OK?
• 6:55
  We are not perfect.
• 6:57
  We make mistakes, right?
• 6:58
  Let's just be blunt about it.
• 7:00
  We make mistakes on diagnosis there.
• 7:02
  It's estimated that between 5 and 15 percent of all diagnoses, not just cancer diagnoses,
• 7:07
  are in error.
• 7:09
  We make mistakes, as we've mentioned, in managing hypertension and managing diabetes.
• 7:14
  We make mistakes in ordering unnecessary tests.
• 7:20
  There's a lot of data about unnecessary PET scans, et cetera.
• 7:24
  So humans are prone to mistakes.
• 7:26
  A.I. will make mistakes, too.
• 7:29
  I don't know that we've ever shown that A.I. is or we've certainly not claimed that A.I.
• 7:35
  is going to be perfect.
• 7:36
  You know, Waymo cars, they're not perfect.
• 7:39
  They make they have accidents.
• 7:41
  They make mistakes.
• 7:42
  They hit people, unfortunately.
• 7:44
  They just happen to be comparatively way safer than human drivers.
• 7:49
  And the same thing is going to be true in medicine.
• 7:51
  That autonomous A.I. is going to be better consistently than humans.
• 7:57
  That doesn't mean they're going to be perfect.
• 7:59
  Now, here's the conundrum that the better and better they get that, you know, right
• 8:04
  90 percent of the time, right 95 percent of the time, 98 percent, 99 percent, the better
• 8:08
  they are, the worse our intervention are, human intervention are, because we will introduce
• 8:16
  more errors than we will catch on A.I.
• 8:19
  And so I think the dilemma is going to be, you know, are we confident that they've just
• 8:25
  gotten either the diagnosis of this patient wrong or that they have misapplied the therapeutic
• 8:31
  regimen that's supposed to happen, whether, you know, starting with neoadjuvant chemo,
• 8:38
  then surgery, then adjuvant chemo, whatever the sequence is.
• 8:44
  The latter, where we have a good guideline because it's a common cancer, it's really
• 8:52
  going to be unlikely that you're going to have A.I.
• 8:55
  screw it up.
• 8:57
  What you're going to have A.I.
• 8:59
  screw up is more likely to be the situation where you've got a very unusual cancer and
• 9:06
  there's not a clear guideline driven treatment or there are multiple treatments and none
• 9:12
  of them have been shown to be either great or superior to the other.
• 9:17
  And there, you know, yeah, humans might be better at using judgment in those contexts.
• 9:24
  You know, all of us have managed some of those patients, you know, whether it's appendiceal
• 9:29
  cancers or, you know, some other cancer that's rare as hen's teeth.
• 9:36
  So I think that's the situation where human judgment overriding A.I. may come into place.
• 9:42
  Yeah, I think, you know, if I could pick one part of that specifically, the part where
• 9:49
  data lacks is really something that that makes sense to me.
• 9:52
  Literally today, I have a kind of a higher profile patient that has an ATM mutation and
• 9:59
  apparently very dense breasts, but has to get an annual MRI.
• 10:03
  This is a pathologic ATM area.
• 10:05
  And so I went to CHAT-GPT just to double check myself.
• 10:07
  My wife does most of the breast cancer.
• 10:09
  And, you know, it said when I put in the specific mutation, it was like, whoa, this one.
• 10:13
  They're like, I don't know.
• 10:14
  I can't tell you on bilateral mastectomy if that improves survival or mortality versus,
• 10:19
  you know, an annual MRI.
• 10:22
  I can't tell you if it's this high risk one.
• 10:23
  And so it really just came down to a matter of judgment.
• 10:26
  I was like, look, you have increased density, which does mean you have an increased chance
• 10:29
  of breast cancer just inherently like in addition to this ATM mutation.
• 10:33
  Same time, you know, MRIs aren't aren't nearly as fallible to density as a mammogram can
• 10:40
  be, et cetera.
• 10:41
  But it's a perfect example of what you just said, where there was just this application
• 10:45
  of reason that that was able to hopefully, you know, make her feel better.
• 10:50
  Well, I think actually you point out something good, which is there's also this issue of,
• 10:57
  you know, you've got a lot of value preferences in there.
• 11:00
  How much risk are you, you know, repeated MRIs and willing to live with the risk and
• 11:05
  the uncertainty versus, you know, I just want the cancer out of me.
• 11:10
  And you hear both from patients.
• 11:11
  And that's a real value judgment.
• 11:14
  It's a real lifestyle preference judgment.
• 11:17
  And it seems to me those are ones, at least at the moment, that, you know, it's not clear
• 11:24
  that A.I. is superior to people or appropriately necessarily factors in the human element.
• 11:31
  I'm not saying it can't be.
• 11:32
  What I am saying is that, you know, that's probably a place where we ought to take a
• 11:37
  beat and say, you know, it's probably best that I discuss this with the patient.
• 11:43
  Now, you're right.
• 11:44
  At some point, we might have enough data and the data may be pretty determinative.
• 11:49
  Yeah. And it started with her just saying the statement from the the MDA-FPP that saw
• 11:55
  her was there is no evidence that bilateral mastectomy helps survival, you know, compared
• 12:01
  to just getting the screening.
• 12:02
  And she's like, this makes no sense to me.
• 12:03
  And I was like, look, realize that is not saying that your chance of getting breast
• 12:07
  cancer is way reduced.
• 12:08
  Obviously, you take away 99 percent of your breast tissue.
• 12:11
  So it was a nice, you know, a nice lecture or series on kind of that statement of evidence.
• 12:16
  So you laid out beautifully, you know, this idea of of the spectrum of error.
• 12:22
  I heard a great quote.
• 12:24
  Kevin Schulman was the one that quoted this individual.
• 12:26
  And I can't remember who said it, but he was like, I do think, you know, there's this
• 12:31
  idea of if nothing else on the subject of error, I can help us not just be the best
• 12:37
  version of ourself, but even just the good slash average version of ourself.
• 12:43
  So, you know, like depending on what our sleep is, depending on what potentially our
• 12:46
  wearables say, which I know, you know, a lot of them are trash, as we saw recently on
• 12:51
  one of popular one as far as its REM sleep evaluation.
• 12:55
  My point is, if there is any kind of effect the next day or the time of day or, you know,
• 13:01
  proximity to caffeine use, could something just kind of be a hedge against those dips
• 13:06
  or naders in our processing and fallibility?
• 13:09
  And if nothing else, I would hope that the argument is pretty uniform that and that we
• 13:13
  have the humility to say that would be nice.
• 13:16
  Well, I think Bob Wachter puts it well.
• 13:18
  You know, he says when I walk on the ward floors, you know, I feel like I have multiple
• 13:24
  consultants in my pocket.
• 13:25
  I can just pull up open evidence and consult with them and, you know, maybe reassure
• 13:31
  that my judgment is right or, you know, find a perspective that I hadn't anticipated or
• 13:36
  fill in a blank that I can't remember at that point.
• 13:41
  And I think that's right.
• 13:44
  You know, that is sort of AI as decision support.
• 13:48
  I think it's the minimal use of AI, as it were.
• 13:51
  I think that probably is a kind of consultation that is widely endorsed by all people in the
• 13:59
  medical field.
• 14:01
  And that, you know, my disagreement with the AMA, say, or whomever, I don't think that's
• 14:07
  not the source of it, the source of the disagreement is about this issue of autonomous AI without
• 14:12
  case by case physician oversight.
• 14:14
  That's what autonomous means.
• 14:16
  Can AI manage a patient without a doc overseeing in some way?
• 14:22
  And that I think is a that's not about making me my average self.
• 14:28
  It is about the AI substituting for a doc.
• 14:33
  Now, let's remember, you know, in this country, one of the big complaints at the moment is
• 14:40
  I can't get an appointment or my next appointment is two months from now and I have an urgent
• 14:45
  problem.
• 14:48
  And I think the first widespread use of AI is likely to be in the case of places where
• 14:57
  we either have physician shortages, you know, 140 million Americans live in counties without
• 15:04
  adequate numbers of PCPs.
• 15:07
  And we all know a number of oncologists in rural communities and other communities is
• 15:13
  too low.
• 15:15
  You know, I was once many years ago did a survey of doctors and found out that there
• 15:20
  were three oncologists or something like that in North Dakota and that they flew around
• 15:24
  to give chemotherapy.
• 15:25
  And I'm like, whoa, that's not enough.
• 15:28
  And I think in those scenarios, AI is going to augment what not augment each individual
• 15:35
  doctor.
• 15:36
  It's going to augment the supply of doctors and give patients the ability to get the consultations
• 15:43
  they need.
• 15:44
  And, you know, oncology is one area.
• 15:46
  I will tell you the area I'm most sort of focused on in some ways is, you know, pediatrics.
• 15:54
  We are losing pediatricians a lot and pediatric specialists have always been in shortage as
• 16:00
  far as I can tell.
• 16:02
  Don't pay a lot.
• 16:03
  Don't pay anywhere near what specialists for adult medicine do.
• 16:08
  And the delays and the wait times are huge.
• 16:13
  And so I think that's an area where you're likely to see AI come in, not displace a single
• 16:19
  health care clinician today and really help patients.
• 16:23
  Plus, as you say, there is this convenience factor, which I think the medical profession
• 16:30
  probably doesn't fully appreciate and hasn't spent enough time thinking about how they
• 16:34
  do their shopping and how much convenience is the determinant of, you know, am I doing
• 16:41
  Amazon?
• 16:41
  Am I going out to the grocery?
• 16:42
  Am I going out to the bookstore?
• 16:44
  Am I going out to buy a pair of shoes or am I ordering online?
• 16:48
  You know, convenience plays a huge role and let's face it, doctors are not convenient.
• 16:54
  Not at all.
• 16:55
  I mean, I literally find myself paying out of pocket for, you know, now everything is
• 17:00
  kind of DTC or direct to patient.
• 17:02
  And I argued the other day on a panel that it's just because of knowing what you're in
• 17:09
  for as far as what the cost is.
• 17:10
  You know, like it's I don't it's a black box if I go in for my shoulder pain that I've
• 17:15
  had for four months now and I can't go to the gym.
• 17:17
  And I was like, but I have no idea what it would come out at, like with the ultrasound
• 17:20
  with whatever they were going to do.
• 17:21
  So I was like, I'd rather just pay for it out of pocket.
• 17:23
  To that same point, one of the things that I hear and, you know, a lot of people often
• 17:28
  talk about digital imaging as one of the first things where I may be quite valuable.
• 17:34
  It's demonstrated things in mammograms, on picking up lesions, as well as on pathology
• 17:38
  slides.
• 17:39
  So we're talking about things where you go to a radiology center or get pathology for
• 17:44
  either cancer or non-cancer stuff.
• 17:46
  And when you talk about this, and we had Anand from Emory who did a great job of explaining
• 17:50
  kind of some of the traps.
• 17:51
  And he even used this example of of like a husky versus, you know, like a wolf or whatever
• 17:57
  is pulling a sled.
• 17:58
  And really the AI just noticed the snow in the background or not.
• 18:00
  And it was highly accurate.
• 18:02
  So like you need someone to think about those things.
• 18:04
  My question to you is, when I hear, well, you always need someone liable, like at the
• 18:09
  end of the day, that's kind of some people's answer on why the doctor isn't going anywhere
• 18:13
  other than everything else.
• 18:16
  At the end of the day, there are going to be things that are replaced.
• 18:18
  Let me remind everyone that my, you know, program director told me that when you're
• 18:25
  doing a CBC, a complete blood count, that he would look and he would have to count a
• 18:29
  lot of times, like kind of approximate on a certain magnification and do the calculation
• 18:32
  of what the platelet count is or whatever the matter.
• 18:35
  Suddenly, we're comfortable with automating a lot of that, right?
• 18:38
  Who does that fall on at the end of the day?
• 18:39
  It's a value.
• 18:41
  People get transfused or not transfused because of something that has been replaced by someone
• 18:45
  that was a human.
• 18:47
  My question is, where do you think if there is a day when other tasks will be replaced,
• 18:54
  where does this whole liability thing fall?
• 18:55
  Which I think is a weak argument personally, but.
• 18:57
  Well, look, there are four barriers to widespread deployment of AI.
• 19:06
  Licensure is the first one.
• 19:08
  You don't want out of the box Check GPT or out of the box Claude or Google Gemini doing
• 19:15
  medicine.
• 19:16
  That is a mistake.
• 19:17
  And when people say, oh, we tested, you know, Check GPT.
• 19:21
  Well, that's just the wrong test.
• 19:23
  And you should throw out that paper and they should never be published.
• 19:27
  You need AI that is trained, has guardrails, has harnesses so that people can be assured,
• 19:34
  you know, just like doctors are trained and see a lot of cases that they have the right
• 19:40
  focus.
• 19:41
  That's the first thing.
• 19:42
  And when that happens, we haven't established a guideline.
• 19:45
  Now, we made a proposal in JAMA, I don't know, a couple or three months ago about that.
• 19:51
  And I think I think I think we have the right proposal, but you've got to pass a board exam
• 19:59
  above the median of all doctors.
• 20:01
  You've got to have observed, carefully assessed interactions in the scope of practice that
• 20:09
  is being proposed.
• 20:09
  And then there's a bunch of other you have to repeat it.
• 20:13
  Every new version has to also go through this process.
• 20:16
  The second is liability, as you mentioned.
• 20:19
  And, you know, this licensure model gives you a liability framework.
• 20:25
  You get license for this.
• 20:27
  That's where the liability falls.
• 20:30
  You if the AI practices that, you know, the program for oncology and now suddenly practicing
• 20:36
  endocrinology, wrong, liable for any mistake.
• 20:41
  And who's liable?
• 20:42
  Well, I think the hierarchy is pretty clear.
• 20:44
  The developer should be liable for lots of reasons, one of which is they have the deep
• 20:49
  pockets.
• 20:51
  They've created this thing with the maybe inadequate guardrails.
• 20:56
  And you want them to be cautious, not like they're in the wild west.
• 21:00
  And the last thing is you want them to learn of all the problems so that they can actually
• 21:05
  update their programs to do better and avoid problems that are elicited.
• 21:11
  And they will not learn about those problems if they're not in the line of fire on for
• 21:17
  those problems.
• 21:18
  So I actually think liability is not a problem if we get the right legislation.
• 21:24
  But you got to get legislation.
• 21:25
  And right now we don't have a uniform national legislation.
• 21:29
  And by the way, I think it should be national because there is no way you're going to say
• 21:34
  AI works in Oklahoma but is not certified to work in Nebraska.
• 21:40
  I mean, that just makes no sense.
• 21:42
  So they're going to work across borders.
• 21:45
  And we ought to just realize that and make the licensure and the liability the same uniform
• 21:50
  policies.
• 21:51
  The third one, of course, is reimbursement.
• 21:54
  What you ideally -- well, we definitely, definitely, definitely do not want to have reimbursement
• 22:00
  by every patient fee for service because then every patient will get AI just so the doc
• 22:05
  can bill.
• 22:07
  You want to have the doc -- excuse me, incentivized for doing the right thing, like 80% of patients
• 22:14
  with hypertension have it under control, and incentivized to use autonomous AI to get to
• 22:20
  that goal and given a big bonus so that they can both deploy the AI and have an incentive
• 22:26
  to.
• 22:28
  So I think -- I also think the reimbursement, you know, we're not there, but I think it's
• 22:33
  pretty clear how to do it.
• 22:35
  The last problem is deployment and getting it into the workflow and making sure it works
• 22:43
  smoothly in the workflow.
• 22:45
  My initial hypothesis -- and this is just a hypothesis -- is that the reason we have
• 22:50
  not gone gangbusters in radiology with autonomous AI is the workflow issue, none of the other
• 22:57
  things.
• 22:58
  And I think that's a critical problem.
• 23:02
  It's going to be a critical problem.
• 23:04
  I don't think it's going to, in the long run, impede the deployment of AI.
• 23:09
  It might delay it, but it won't prevent it.
• 23:13
  And why might it delay it?
• 23:15
  Well, you know, you might slow things down because you're not being paid.
• 23:19
  And ultimately, I think it's going to save enough money that it's just going to be rolled
• 23:26
  out, and it really is going to be about the money unless doctors embrace this.
• 23:32
  Now, radiologists haven't embraced AI.
• 23:34
  We now have, I think, pretty good data that, you know, AI can do most mammograms.
• 23:41
  And I think you're going to see that deployment coming.
• 23:43
  And I think you're right, you know.
• 23:46
  I was early trained just at the borderline when we read the slides and then the machines
• 23:52
  read the slides, and, you know, there's a great example of convenience.
• 23:58
  A hell of a lot more convenient to have the machine read it, even if it's off 5%, than
• 24:03
  have me spend half an hour, you know, prepping the slides and then reading them.
• 24:09
  First of all, I make mistakes that way.
• 24:11
  And second of all, that ain't the best use of my 30 minutes.
• 24:15
  So I think we're going to have the same thing in healthcare.
• 24:21
  And I think it, again, the issue is how do we deploy it and how do we make it so it doesn't
• 24:27
  add time to the doctor's day, but makes him or her more efficient.
• 24:31
  I love it.
• 24:32
  And, you know, one thing that is similar to what you're saying, one thing that Bertalan
• 24:38
  Mesko, the medical futurist, we had him on in his new book where he talks about futurist
• 24:43
  planning, and it's futurist with a plural.
• 24:45
  Like the way I describe it, you know, in my lectures is it doesn't make a good football
• 24:49
  coach is not just trying to bet or gamble on what the other team's one play is.
• 24:54
  Like you can computer about three to four plays and the most likely the one you want
• 24:58
  it to be, et cetera, et cetera.
• 25:00
  I'm curious what happens 10 years from now when you're getting, you know, we have this
• 25:05
  bimodal distribution in oncology where there's quite a few kind of, I want to say old, but
• 25:09
  older doctors and younger doctors.
• 25:11
  What happens when the younger ones come and don't have necessarily the experience to say
• 25:17
  something doesn't look or feel right about this AI model?
• 25:19
  Are we going to have doctors training to just understand stuff, you know, book wise to be
• 25:25
  able to kind of guide it?
• 25:26
  How do you see that playing out?
• 25:27
  I actually think this is a huge and important question that unless we accept that there's
• 25:34
  going to be autonomous AI and that autonomous AI is going to take over a lot of doc work,
• 25:39
  we don't confront as boldly as you put it.
• 25:44
  It's a question of de-skilling and de-experiencing, right?
• 25:48
  How does someone become an expert?
• 25:50
  Well, they see thousands and thousands of cases and they can pick out which, you know,
• 25:55
  issue doesn't quite fit the pattern and therefore needs further investigation.
• 26:00
  And then, oh yeah, this one looks like one I saw a long time ago or whatever.
• 26:05
  That kind of expertise does require a lot of experience.
• 26:10
  And if we take away a lot of that experience by deploying AI on the easy cases, it may
• 26:19
  become harder for someone to become an expert, maybe impossible for someone to become an
• 26:24
  expert.
• 26:25
  And then we lose something that we all recognize is incredibly valuable.
• 26:30
  And similarly, if the average doc becomes de-skilled, which happens quite rapidly by
• 26:36
  not seeing that many cases, then what are we going to do?
• 26:41
  And I think this is an issue of how do you get people to have enough experience and maintain
• 26:46
  skills in an era of AI, you know, to flip it around, to take it away from medicine in
• 26:50
  terms of something else, right?
• 26:53
  The legal profession, you know, this is going to hit way earlier than medicine.
• 26:58
  You know, scanning cases and summarizing them is a lot easier than summarizing medical
• 27:04
  experiences.
• 27:05
  And, you know, lots of firms are like, well, we don't need as many 1Ls.
• 27:10
  We can get just a few.
• 27:12
  They will have much less experience, each one of them.
• 27:15
  And then where is the wise lawyer who's seen it all in this area of law going to come from
• 27:23
  if you don't train those people and if you've just substituted AI?
• 27:28
  It's not unique to medicine.
• 27:29
  It's going to be common wherever we end up deploying AI.
• 27:34
  And as a society, we have thought way too little.
• 27:39
  I was about to say a nanosecond.
• 27:41
  I haven't thought -- I haven't heard anyone come up with a reasonable hypothesis as to
• 27:47
  how that skilling gets maintained and trained up.
• 27:53
  You know, if you're not struggling, you're just giving it to AI, we're going to lose
• 27:57
  those skills and we're going to lose that ability and those connections and that, you
• 28:02
  know, unique voice that people have.
• 28:04
  And I think that's one of the real risks we have with AI that we have not -- certainly
• 28:11
  haven't solved and I'm not even sure we have, you know, three or four plausible ways that
• 28:16
  we might approach it and prevent that problem.
• 28:20
  And until we accept that AI can do a lot of these things and recognize it, we're probably
• 28:27
  not going to find the right solution to this very, very important problem of de-skilling.
• 28:34
  You're the first person that really highlighted what has been a true fear of mine.
• 28:39
  And it's the lift and labor and struggle necessary for dopaminergic reward that somehow I
• 28:46
  believe conditions us to be really good in the setting of, you know, where we have to be
• 28:52
  resilient in the future, right?
• 28:53
  I mean, we have to drive to the library, open Britannica, like go to the section
• 28:58
  alphabetically, all that just to start a project.
• 29:01
  If the lift becomes, you know, super low in all of the ways that it wasn't in the past,
• 29:07
  we need to -- and you're huge on this -- we need to proactively and prospectively ensure
• 29:12
  that that stuff is held up.
• 29:14
  One of my friends suggested doing what's called 4-H club.
• 29:18
  She was like, you don't understand the lift it takes to groom and take care of and feed
• 29:24
  this like sheep or whatever you're showing is the thing when you compete all for a day
• 29:28
  at the end of three months.
• 29:29
  And she goes, it teaches such good discipline because you work for the entire summer for
• 29:34
  one day's reward.
• 29:35
  You need to even do something like that to potentially at least offload the other things
• 29:40
  that otherwise took labor.
• 29:41
  I think it's such an interesting one.
• 29:42
  I -- when I sent my children off to college, I told them, you know, one of the things that's
• 29:47
  going to happen at college is you're going to suffer.
• 29:50
  You're going to suffer with a project.
• 29:52
  You're going to suffer, you know, with figuring out what you want to do with your life.
• 29:56
  You're going to suffer, you know, writing an essay and not getting it right.
• 30:00
  That suffering is an essential element in learning and in growing.
• 30:05
  And if we short-circuit it, we are actually doing our students and also ourselves a no
• 30:15
  favor at all.
• 30:16
  And one of the problems is we have as a society and as people, we become used to a lot of
• 30:23
  ways of short-circuiting that kind of frustration and suffering and hard work necessary.
• 30:30
  And I think that is one of the most important elements of growing.
• 30:37
  And we can't take it away.
• 30:39
  And I am worried, and I think this is a very legitimate worry, that AI is just going to
• 30:44
  be too easy.
• 30:46
  That doesn't say you don't get educated in AI.
• 30:48
  You have to because, you know, it's going to be in society all around you, and you can't
• 30:55
  be a Luddite in that way.
• 30:57
  You do, however, it has to have its right lane.
• 31:01
  And in the education process, that is, I think, much more minimal than maybe later in life
• 31:08
  once you have a lot of these skills.
• 31:10
  But you got to get the skills, and you can only get the skills by suffering.
• 31:13
  That's exactly right.
• 31:14
  I mean, the example you gave about legal, my friend, their judge, they work in the First
• 31:21
  Circuit Court of Appeals, and they were just like, their judge is never around.
• 31:26
  They're apparently doing a lot of, you know, equivalent of CME, I don't know what it's
• 31:28
  called in judge language.
• 31:30
  But like, I just want something to talk out.
• 31:32
  And I was like, well, let's just get on chat to BT.
• 31:34
  And we did the verbal.
• 31:35
  And it was unbelievable how much this seeming wisdom was there.
• 31:40
  Well, you know, there was one statute here.
• 31:42
  This was one case.
• 31:43
  And that's what experience does.
• 31:45
  And I think you ask a great question.
• 31:47
  Where do we get that from?
• 31:48
  And another case is House, right?
• 31:49
  Like the TV show we watched growing up.
• 31:51
  I mean, he just said, well, this case, I saw this, I saw that.
• 31:54
  So we need a solution for that.
• 31:57
  In the similar manner, and I have my opinion, I wonder if it's the same as yours.
• 32:02
  What is a skill or two that is just imperative that a physician or what at least we think
• 32:10
  of as a physician today and possibly in the future should have?
• 32:14
  Like, what is one that is just not, you know, something that should be even considered to
• 32:20
  be replaced for?
• 32:21
  Let me just back up a little bit.
• 32:23
  The thing that started me on this path, approximately, I mean, about 10 or 15 years ago, I met Vinod
• 32:29
  Koslow and he started me on the path in some ways by telling me that by 2035, you know,
• 32:35
  80% of what Dr. Zhu AI is going to do.
• 32:37
  And I've said bullshit, you know, it's wrong.
• 32:40
  He was right.
• 32:41
  I was wrong.
• 32:43
  But what approximately got me on was Bob Wachter's A Giant Leap.
• 32:47
  And I read the book.
• 32:48
  He wanted me to blurb the book.
• 32:49
  So I read the book and, you know, it's like, well, if you really think AI is going to advance
• 32:54
  this far, your view of what doctors are going to do at the end just didn't seem right to
• 32:59
  me.
• 32:59
  I don't think he got it right.
• 33:01
  He was thinking the hybrid model would be the best.
• 33:07
  And I've been thinking about what's left for doctors to do.
• 33:10
  We have a paper on it, which we call Wither the Doctor.
• 33:13
  It's not quite ready to submit for publication.
• 33:17
  But I do think this issue of what's likely to be left for doctors to do, you know, a
• 33:23
  lot of it, a lot of my critics are saying, well, it's all about empathy, compassion,
• 33:28
  trust.
• 33:29
  And I was like, well, you know, maybe that's true.
• 33:32
  First of all, there's not a ton of evidence.
• 33:34
  And at least some of the evidence that we cite in our paper, the JAMA paper that you
• 33:40
  reference is that AI is more empathetic than docs.
• 33:44
  At least patients perceive AI to be more empathetic than a doc.
• 33:48
  But also, it seems to me we don't want to reduce everything docs do to just empathy.
• 33:58
  As important as I think that is, it's not the only important thing.
• 34:01
  And the final thing I would say about that is, you know, we don't select docs for empathy.
• 34:06
  We select docs for organic chemistry grades, for MCAT scores, a lot of what we do in the
• 34:13
  first three clinical years is not empathy related.
• 34:17
  It's really science memorization.
• 34:19
  So I do think talking through tough value judgments is going to be important.
• 34:27
  Not clear AI can't do that.
• 34:29
  But, you know, figuring this out is one of the more important things to do today.
• 34:39
  But again, like the issue of how do we prevent these skilling, until you accept that
• 34:44
  Autonomous AI is going to take over a lot of what docs do today, you don't get to that
• 34:49
  question.
• 34:49
  So I just have such a deep appreciation for now, like for what you're doing and saying.
• 34:56
  You're not saying like, you know, I'm just betting on this bullish and I'm not being
• 35:01
  like Michael Barry or whoever is saying the stock market is going to crash every day.
• 35:05
  Like you're not doing it for fun.
• 35:07
  You're doing it because your deep research and understanding and appreciation of these
• 35:12
  things have led you to believe that these autonomous agents, and I'm definitely more
• 35:16
  on this side, will be able to effectively, and I would, you know, many would argue more
• 35:22
  responsibly, replace some of the things or many of the things that otherwise are just
• 35:28
  rote and routine and subject to fallibility.
• 35:31
  And if you believe that future, your argument and your real message is we have got to
• 35:37
  plan for that.
• 35:38
  And that is not a, that is not a medical concept.
• 35:40
  That is a human life concept.
• 35:41
  That is what we learn in business strategy when it comes to, you know, at our Stanford
• 35:45
  masters where it's like, you know, you talk about every factor that has to do with the
• 35:49
  growth of your company and what everything else may happen with everything else.
• 35:53
  It's the same thing that you think about when it comes to futures planning, whether it's
• 35:57
  Berlin and saying, look, the end of the day, you know, you want to have every version of
• 36:03
  preparedness for that most realistic future.
• 36:07
  And I am more concerned.
• 36:09
  I'm leaving this more concerned.
• 36:10
  And I was always the one that was probably the most concerned in the room.
• 36:13
  I'm pretty concerned about the way we're picking doctors that we're not saying doctors
• 36:17
  going away, but what are the properties that matter?
• 36:20
  What are the things that we can ensure have this, you know, house or judge or pick your,
• 36:26
  you know, element of wisdom component?
• 36:29
  Is it just shadowing, you know, like these AI models and seeing the outcomes and just
• 36:34
  being a patient assistant, just kind of being along their journey?
• 36:37
  And then you do enough of that and you become this giver, like the book of experience that
• 36:42
  can say, you know, I shadowed for 10 years, 10 patients with cancer.
• 36:46
  Who knows?
• 36:47
  But these conversations need to happen.
• 36:49
  And I think they're very interesting ones.
• 36:51
  And we're also going to have to make, you know, the training programs reflect the judgments
• 36:55
  we make.
• 36:56
  Right now, our medical education isn't doing anything regarding the AI era.
• 37:00
  And I think we need to, you know, really think that through.
• 37:04
  Yeah.
• 37:04
  I mean, one of my friends, a lifelong dream of going to med school, but also would require
• 37:09
  being away from two young kids.
• 37:11
  And I said, look, my only fear, I'm like, it's not about not going to med school, not
• 37:14
  being there.
• 37:15
  I'm like, if you go now, you disappear from the world for two years.
• 37:19
  And I'm like, you're learning things in a memorization capacity that very likely in
• 37:23
  the two years that you're out will not be within your umbrella of responsibilities.
• 37:27
  And I'm like, all that happens is you miss out on slowly getting better, like a new sport
• 37:32
  in the way the world works.
• 37:33
  And I was like, so please just delay.
• 37:35
  I'm just as you delay it and not miss from your kids because there's going to be a lot
• 37:38
  of dust settling, you know, over time.
• 37:41
  And to answer my own question, I think, you know, if I had to, if I had to pick one where
• 37:46
  a physician's role likely needs to be preserved, it is what you said at the end.
• 37:50
  You said the measurement of value, or sometimes the way that I describe it is like this, this
• 37:55
  quotient of suffering, this like units of suffering.
• 37:58
  And I don't mean suffering in a terrible way.
• 38:00
  I mean, even suffering, you know, in traffic, but everything has a consequence, whether
• 38:06
  it's a surgery, the recovery time, what that entails, what it entails for you as a father,
• 38:11
  all of these things, I think can potentially help with decisions.
• 38:15
  As we said about the bilateral mastectomy versus, you know, doing annual MRIs, you laid
• 38:21
  it out perfectly.
• 38:21
  You said all these things that are considerations that an LLM may be able to just say, because
• 38:25
  it's seen it enough, but is it really considering what the anxiety of potentially getting the
• 38:31
  cancer is and where you are at the age of your, you know, as a mother for your kids
• 38:36
  and all of those things.
• 38:36
  I think there's some, you know, unit of measurement that somebody's going to be responsible for
• 38:41
  that for, and we'll have to understand the field to do so.
• 38:46
  I don't know.
• 38:46
  That's just my preliminary thinking.
• 38:48
  I think this is an area we all need to do some more thinking on and talking about, because
• 38:53
  I think it's very unclear.
• 38:56
  And I think you're 100% right that, you know, maybe it is about, you know, these value
• 39:05
  assessments or, you know, these other more intangible things.
• 39:11
  Yeah.
• 39:11
  We don't have a lot of data.
• 39:13
  Yeah.
• 39:13
  I mean, at the clinical informatics management masters at Stanford, you know, we found out
• 39:20
  that bioethics is all four terms.
• 39:22
  And we kind of asked the question, we're like, what about the ones about health information
• 39:25
  exchanges and insurance?
• 39:26
  And like, now I see why Magnus is our professor all year long.
• 39:30
  And it's like, because these are questions that just with everything that comes up really
• 39:34
  need to be considered.
• 39:35
  And I had a nonmedical person in the class and they said, why are we, it's like coffee
• 39:39
  talk to me.
• 39:39
  Like, why are we really talking about this?
• 39:40
  I'm like, because at our core medicine is fundamentally like ethical and bioethical.
• 39:45
  I'm like, that's, we, we, that's just how it's come to be on what it is and something
• 39:50
  that's really hard to depart from.
• 39:51
  And if nothing else, that's our inherent responsibility.
• 39:54
  And I really thank you and admire you for guiding us and reminding us of that because
• 39:59
  it's very easy to get comfortable and kind of let the waves, you know, send you a certain
• 40:02
  direction.
• 40:03
  Great.
• 40:03
  Well, thank you.
• 40:04
  This has been wonderful.
• 40:05
  And I think a very, very revealing.