Latest / CareTalk: Healthcare. Unfiltered. / Why Minors Need Stronger AI Guardrails Than Adults

Transcript

Why Minors Need Stronger AI Guardrails Than Adults · CareTalk: Healthcare. Unfiltered.

• 0:07
  John: In this environment, um, do you see a path where C-Congress or, or a-any, any leaders in the White House are gonna actually demand the kind of transparency and clarity of the separation between the human and the machine that you're looking for?
• 0:30
  Dr. John Whyte: You know, right now the focus typically of the administration is not to have too much regulation, and there's always a debate, do you regulate before you try, or do you try before you regulate, right?
• 0:42
  Dr. John Whyte: We wanna promote innovation, but we also wanna protect against potential harm.
• 0:46
  Dr. John Whyte: So where we have focused a bit is where is there common ground on regulation?
• 0:51
  Dr. John Whyte: So for chatbots, it's particularly there for minors, right?
• 0:56
  Dr. John Whyte: They, their emotional intelligence is, is not the same as an adult.
• 1:00
  Dr. John Whyte: So we should have disclosure.
• 1:01
  Dr. John Whyte: We should have transparency.
• 1:03
  Dr. John Whyte: We should have escalation that if they're expressing suicidal ideations, that they're connected to a real-life health professional.
• 1:10
  Dr. John Whyte: I'd like to see industry come together and really adopt responsible use of AI.
• 1:18
  Dr. John Whyte: Let's work together in trying to develop benchmarks.
• 1:23
  Dr. John Whyte: Let's talk about what a medical device is and the regulatory pathway of some of these wellness apps, which I think are more along the lines of a supplement and, and could potentially be harming patients.
• 1:37
  Dr. John Whyte: That-that's what this is all about.
• 1:39
  Dr. John Whyte: This is about patient safety.
• 1:41
  David: So you've talked about the federal side and then industry getting together.
• 1:45
  David: Meanwhile, the states are doing things.
• 1:46
  David: So you mentioned, um, you know, s-suicidal ideation.
• 1:50
  David: Mm-hmm.
• 1:50
  David: I think California actually requires, uh, chatbot developers to monitor for that.
• 1:54
  David: Illinois has banned AI for therapeutic decision-making.
• 1:57
  David: Pennsylvania's has a lawsuit against Character.ai for impersonation of a licensed medical professional.
• 2:03
  David: How do the states fit in with that?
• 2:05
  David: Is federal preemption essentially required, or is, is states also a, a relevant pathway?
• 2:11
  Dr. John Whyte: Well, you alluded to there is an executive order that apparently preempts states from getting involved.
• 2:17
  Dr. John Whyte: I'm not sure if all the states have been listening to that because you've given some examples where states have gotten involved, and the challenge with that potentially is you could have 50 states with 50 different policies.
• 2:31
  Dr. John Whyte: I'm not sure that's helpful to the medical community or to the patient community.
• 2:37
  Dr. John Whyte: So we wanna work with the states, and, and states ultimately are the practice of medicine, but let's take an example of a, a pilot in Utah where there is autonomous prescribing.
• 2:50
  Dr. John Whyte: FDA could get involved and say that's a medical device subject to a different level of regulation, and then that would preempt the pilot from continuing.
• 2:59
  Dr. John Whyte: We could talk about standards in terms of licensure, and we've talked about that in JAMA, uh, Network in terms of if these products wanna say they practice medicine, they should be subject to licensure.