Latest / CareTalk: Healthcare. Unfiltered. / Why Clinical AI Safety Has to Be Proven Not Assumed

Transcript

Why Clinical AI Safety Has to Be Proven Not Assumed · CareTalk: Healthcare. Unfiltered.

• 0:06
  SPEAKER_00: The most recent interview with Greg Brockman I saw, again, he threw healthcare in is is almost like a you know potentially an enterprise strategy for hospitals that OpenAI is working on.
• 0:19
  SPEAKER_00: How do you compete in a world where the anthropics and the open AIs are have infinite money?
• 0:24
  SPEAKER_00: And they appear to, although they don't necessarily show up in most pitches and and and programs, they talk a lot about getting into healthcare.
• 0:33
  SPEAKER_00: How have you differentiated and and what's sustainable for Hippocratic in that world?
• 0:38
  SPEAKER_01: Aaron Powell We picked the hardest problem.
• 0:40
  SPEAKER_01: We said we're gonna go after clinical patient-facing conversations.
• 0:44
  SPEAKER_01: So there's three things in that clinical, patient-facing, and voice, i.e.
• 0:49
  SPEAKER_01: the conversation part.
• 0:51
  SPEAKER_01: And each of those has provided a different need.
• 0:54
  SPEAKER_01: First, on the clinical side, you've got to be the safest.
• 0:58
  SPEAKER_01: So you know, let's say I'm building voice AI for your Taco Bell drive-thru.
• 1:02
  SPEAKER_01: And let's say I'm two percent better than the open AI version, but they're half the cost.
• 1:06
  SPEAKER_01: I lose every time then.
• 1:09
  SPEAKER_01: But let's say I'm building clinical AI and I'm two percent better, and they're still half the cost.
• 1:15
  SPEAKER_01: Nobody buys it.
• 1:16
  SPEAKER_01: Because you can't tell your boss, oh, I saved half the money, but I I killed two percent more people.
• 1:22
  SPEAKER_01: It's a bad day at the office.
• 1:24
  SPEAKER_01: Um It's a bad day at the office.
• 1:26
  SPEAKER_01: So incrementalism matters in clinical safety, and we built safety system after safety system after safety system and got every last corner case and made sure they're right on every last agenda call.
• 1:39
  SPEAKER_01: And there's a l and we're the only ones that do massive output testing where we have 7,000 clinicians calling the AI.
• 1:46
  SPEAKER_01: Now they've called it over 800,000 times, marking every error it makes, and we do not ship.
• 1:51
  SPEAKER_01: And then one out of 50 times they get a human clinician on the other line and mark the errors they make, and we don't ship till we're safer than the human.
• 1:58
  SPEAKER_01: We don't use one LLM, we use 31 LLMs where the other 30 are supervising the main one to make sure it's not doing anything wrong.
• 2:05
  SPEAKER_01: Like we we built so many redundant safety systems and went after that hard problem.
• 2:11
  SPEAKER_01: And that's what healthcare wants.
• 2:13
  SPEAKER_01: It wants a system that is truly safe.
• 2:16
  SPEAKER_01: Um, there's also unique things we've done on the voice side that ensure safety there, which uh you know are more technical, but we've ensured that we can deliver the best models as fast as we can.
• 2:29
  SPEAKER_01: And the analogy I always draw here is OpenAI is like General Motors, they make all kinds of cars, they make trucks, light trucks, and they make you know convertibles and they make SUVs.
• 2:40
  SPEAKER_01: Um But despite General Motors being there, there's still room for a caterpillar.
• 2:46
  SPEAKER_01: Why?
• 2:46
  SPEAKER_01: Because caterpillar's design requirements need a level of reliability and you know, kind of my analogy to safety that that is second to none and makes no sense to put into a normal car because you'd over-engineer it and the car would cost too much.
• 3:03
  SPEAKER_01: But this is where we've realized that there are going to be segments where things like safety matter so much that you can build a specialized product.