Latest / AI Meets MedTech / AI Liability: The Legal Blind Spot
Transcript
- speaker-0: This is AI Meets MedTech Podcast with Anna Nestorova. Hi, I'm Anna Nistarova, and this is AI MitsMedTech. Today my guest is Pavel Kolmogorov from Kolmogorov Law. Pavel has become one of go-to voices on AI liability, figuring out who is accountable if something goes wrong in a context of AI. And I'm so excited to have him here today in the studio. Welcome to AI MitsMetTech. speaker-1: Thanks for having me. speaker-0: I wanna start with a very simple question. Is the law even keeping up with everything that's happening in AI? speaker-1: Well, it's interesting question And there's no a simple answer to that. That that's for sure. but generally speaking, the law is obviously behind from my point of view, and the law is keeping up the way Yeah, the runner keeps up with the run running car. So it's like very hard, very new technology. It's been there for a little while, but it's still the law is still not there. Yeah. speaker-0: I think it's hard to keep up for all of us. Would you agree with that? I know I know you explored multiple industries. I know you wrote a lot of articles about AI liability. And of course, my podcast is mostly related to healthcare. And I'll ask you most of the questions in that field. Well, feel free to use examples from other industries as well, because I think it's equally interesting how AI liability is shaping up. I read I read that in one of your articles, the doctors are now under dual risk. One is not using AI when it's needed, and another one over relying on AI too much. Can you talk about that a little bit more? Because I think it's a pressing matter for a lot of healthcare professionals right now. speaker-1: Absolutely. There's an irony in it right now, cause again, like you properly said, not using AI may be may subject doctors to liability and and overusing AI may also subject to liability. I would like to refer to one interesting case law that is just being taught in all law schools in the US, and that is a famous case which was decided by and very respectful judge Judge Lernhand. I believe it was 1930 thirty-eight, thirty-two. So what happened is two ships were delivering coal from Virginia to New York, and due to the weather, they sink. they didn't know about weather forecast, you know, some weather conditions, and they sink. So the cargo was lost. the boat owners pursuing cargo ship operators for not equipping the boats with radios. So they could listen to the radio channel and you know dock in the port to to be safe. the defense in that case was from the side of the boat operators, that we don't have to equip each boat with a radio. it was not customary in that date. some boats were buying these radios on their own and and keeping them just to be safe. when a judge decided it's not it was not a good defense. you cannot hide behind custom that you know the boat should should not have this radius. So going back to your question, the the the premise is that the doctor or medical professional can be liable for not using AI. Basically, right now the defense would be nobody uses AI as you know to diagnose patients, right? Therefore, why should I use it or why should I use it, right? but now the legal argument would be you cannot hide behind it. if you have a reliable tool. speaker-0: can you help me to understand it a little bit better? Yeah. I understood the case with the ship. they cannot say, Well, we're not obliged to use a radio, so we don't have to have them. we are okay without radios. And the judge said, No, if you have a p superior technology that can help you to increase safety, you have to use it. Is that correct? speaker-1: Kind of. if you have like you probably said, you have a t reliable tool to diagnose patients or to be used in the medical industry, you have to use it if you have a chance, if it's superior to your own knowledge or technology. speaker-0: That is really interesting. So you're saying that we will see more and more of that for healthcare professionals. Is that what you speaker-1: thinking. Exactly. Yeah. And I I believe if I I'm correct, one of the chief operating officers of the leading medical insurance company basically expressed the same concerns. Same same concerns and same statement. So it may be the case in the future. speaker-0: now let's go to the other part of it. What if now doctors are over relying on AI tools? And what if AI tool is not I don't know, is making a mistake because we know it may have some mistakes? So what happens then? speaker-1: Well again in terms of the liability, it's not there's a not no there's no straightforward answer to that, right? Who is liable? the doctor who used it or the company who you know the one of the yeah main companies that produce that tool or the coders or some other third party companies that implemented that in medical industry. So there's like so many parties that can be held liable for that. Yes. And there's no simple answer to who you can go after to would you speaker-0: You say that that actually opens up a new area of expertise for legal professionals. Now you have to dig in and figure out who is liable in such cases. speaker-1: Well, the problem is right now as we started this podcast, right? the law is behind the technology. Yeah. and we've been operating on the premise of negligence or breach of warranty, strict liability for products for a hundred years now. In fact, there's no s federal statute that would discuss AI liability now. it's all deferred to states. And some states are Better than others. For example, California right now have few bills regarding AI, and I believe it's SB eleven twenty, which says you you know the physician has to make a final decision whether deny the coverage, medical coverage, right? the actual person should be behind that decision. The AI cannot make this decisions on on its own. And there's another bill saying that all communications with patients should be at least Checked by the real person. the customers speaker-0: I have a quick question right here. you're saying right now, California is saying that doctors have to make the final decision. NAI cannot make final decisions. speaker-1: That is correct. As of January first, twenty twenty. speaker-0: We live in the future where AI can make the decision. Who's liable then? Can we even have AI being liable, or it will be the people building the AI tool who will be liable? I'm speaker-1: Exactly. That's the speaker-0: wondering, like this is a very interesting feel because like AI is an entity on its own in a way. We kind of look at it as as a as a I don't even know, is it a tool? Is it a mechanism that makes certain decisions and can help with certain decisions? But in case we give that tool the autonomy to make those decisions. Can we even hold it accountable? Or are we holding accountable people building that tool? speaker-1: Right. First of all, it's we're not there yet, right? So we don't have that this technology. A lot of legal scholars trying to come to some conclusion, whether it's a product, whether whether it's like some autonomous you know machinery that can make a decision. And like I said, there is no straightforward answer to that. So We still kind of trying to find a way to deal with it. our current framework is not capable of addressing AI liability yet. that is as of you know today anywhere in the world. So there's there was this famous AI safety bill. And funny part is maybe not that funny anymore. So A European Union came up with this directive for using AI back in I believe it was twenty twenty three. the directive became effective in twenty twenty five and then they withdrew it. So basically it's not Why? I guess just many many concerns and again it speaker-0: Can tell me more what it was about? Because I I am sure not many people are familiar with that. Can you tell me in in a very short form what speaker-1: back in 2023 there was a safety AI summit in UK. Yep. where hundred hundred plus scientists participated from thirty plus countries and they tried to come up with a framework I believe you know we started talking about AI in twenty twenty-three and then the technology was evolving. A lot of people raised concerns about that. they got together just to decide and and kind of lay out a framework for But kind of speaker-0: Framework to do what? Framework to do we should not use AI or we should speaker-1: Should how do we safely use it? Or if there's you know what the potential risks are and so on. they decide and they identify they we have risks, but we don't have to deal with it. Because they slowly slowly evolving, and it's just hard very hard to assess the actual harm of AI yet, because it just doesn't come the same minute. We have AI and it's very interesting technology in terms of how it evolves, right? So it can learn right now. Yes. And that is also, what makes it hard to to say who is liable at the end. speaker-0: Okay, so they withdrew the document that speaker-1: why? Exactly for th for that reason. They couldn't determine what's the you know, the framework would be. speaker-0: it's actually not a good thing. It's it's it's it sounds concerning to me. speaker-1: Yeah, it is. Well, again, it's just like I said, a lot of scientists and scholars have been wrestling with this question for a number of years now. Yeah. speaker-0: Okay, now let's move from the healthcare professionals to let's say hospitals. Because I want to go through healthcare professionals, hospitals, and patients side of things. one of the things I was pondering about is what if the hospital has an AI tool that is supposed to like we think it's superior to a doctor's judgment. Are they legally obliged to go and say, You have to use this tool now versus doctors saying, I don't want to use AI. I want to do everything by myself. speaker-1: Yeah. From the possible. I think we touched on it a little bit already. if you have a way and reliable tool to use, but there's a good argument to be made that you should have used it, right? If it comes to some liability. Yes. So but again we're not there yet. And partly because we don't have this reliable tool yet. even in the my industry, in the legal industry, the hallucination rate is like very high, higher than you would expect. So it just I don't r remember the exact numbers, but they over I believe for some major players over twenty percent, which is very high. yeah. And there's another concept that we came to just recently called cognitive surrender. huh. which is an interesting term that I just recently discovered myself. basically it means that people over rely on AI. any output that they they get from AI, they don't scrutinize enough. And that may be the case in legal industry, that may be the case in medical industry as well. speaker-0: how can doctors deal with that? if we now know this phenomenon exists, so what can we do now? This is AI Meets MedTech Podcast with Anna Nestorova. I'm trying to put myself in the shoes of, let's say, a doctor. And like you said, I have a patient, I diagnose something, and I used AI, a certain tool that is approved, to help me out with diagnosis. and now it was a mistake. So AI was wrong, let's put it that way. speaker-1: as of right now, like I said, the liability falls to the person who signs, whether it's legal documents or medical decisions. Right. So that's gonna be a dent physician who did s something like that, or the attorney who if if it's in in legal industry who submitted the pleading to the court. as of right now we don't have any approved tools that can make their own decisions autonomously. Right. the person they have to sign under whatever the AI output is. speaker-0: So now let's move to the patient side of things. I read something very interesting in one of your articles. It was related to how conversations with chat bots now can be subpoenaed. So the court can get access to them. And I wonder if the communication with the chat, I don't know, chat GPT, Claude, you name it, Gemini, as a patient, if I had some sort of a conversation and that can be subpoenaed, can that be used against me in court? speaker-1: What do you think? Yes. Well it's interesting topic. Again, it's been on the headlines for a while now, and I I've been advising or have been a commentator for a while now. the answer is it depends, as always, in the leak industry. speaker-0: industry for that. All depends. speaker-1: Right. Yeah. Yeah. as far as I'm aware, there's one case and it's a criminal case where one of the LLMs, I believe it was Chad Chad GPT, subpoenaed and that was used against person in a in a criminal framework. But it's not necessarily the case in the civil cases right now. So it's very hard to get it's possible, but it's very hard to get information from one of those those LLMs and and use it. practically speaking, it's still confidential. So but again it's can be subpoenaed. In technically it's evolving industry where we don't know yet. we know that the case law exists where it was done in a in a framework of criminal law. We don't know yet in a civil framework. So we'll see. I can tell you right now there is the doctor patient privilege, right? There's an attorney client privilege. Yes. that information is protected one hundred percent. So you're saying some exceptions. But be mindful when you converse with with Chat GPT or Claude regarding your health problems or legal problems can it potentially it can be spin it. It can be used against you. It's something to to keep in mind. speaker-0: That's an interesting topic because now I'm thinking about what you're saying. Do you think in the future, let's say insurance companies can go after some patients and saying, Well, you hid information from us on your health conditions, and we found the communication with Chad GPT about your pre-existing conditions, which led to certain events, and you already in some sort of a legal battle or case? Do you think that's what's gonna happen? speaker-1: I can see how it can be used against a person multiple or different kind of lawsuits, right? So in this the example you just referred to, one of those. Okay. So basically the answer would be Penashi, yes. that's one of the scenarios, but I can see how it can be used and abused in all other speaker-0: Can you give me a couple of examples or one? Well speaker-1: Again, anything if you solve insurance, health industry, let's see, anything else, 'cause basically people rely on relying on Chad GPT or other L LM system too much now, whether it be legal advice, medical industry, some some financial speaker-0: Yeah, financial area. Financial industry. Yeah, I can see that. Well, if we talk about future, what do you think is the most plausible development in AI liability in healthcare do you think we will keep relying on doctors and humans or you think we will start shifting towards liability b being placed on AI? speaker-1: as far as right now, I cannot imagine the the scenario where AI can be liable, right? So w and what exactly does it mean yeah liability of AI, right? Essentially the the owner of this LM system speaker-0: talking about the company probably company. Yeah the company. The AI or the AI tool. speaker-1: Yeah, but it's interesting question. Again, it just hard to predict what's gonna happen, but essentially it's so many people involved in this change, right? So like we previously discussed it, just developers, it's the company, it's the third party tools that were used to to implement in certain industries is it's you know data collectors, let's so to say, right? So it's so many people and I think it's gonna be something called fractional liability, right? as for example in car accidents right now, right? drivers can be liable in certain certain degrees, like sixty percent this person's you know, forty percent this person. I think something like that fractional liability will come into play. with AI as well. So it's gonna be fractioned. Let's say the developer ten percent liable in this and then yeah the the company that owns AI may be liable for some certain degree, right? 'Cause they've just created all this tools speaker-0: The software will have the part of it. Maybe the hustle will have the part of it. Right. It's actually interesting. Okay. speaker-1: yeah, fractional liability, that's the answer to it. speaker-0: Okay. Is there anything that scares you about AI? speaker-1: I wouldn't say anything scares me. I'm a proponent of technology. I like technology, but again to the degree. I don't want to give up the technology. And again, there's a a concern of bias that comes to it c comes with AI. So for example, there's this a pilot project in an illegal industry where what they do is they try to implement AI tools that would help judges to make their decisions, the orders and rulings, right? the idea is to feed in AI all the documents and case law and all the pleadings and then judge would have in front of him or or her drafted order that he or she need to look over and submit. again we come to to the question or the term of cognitive surrender where the judges scary it's scary because for the most part judges are overloaded with cases. They in you know average handle over a thousand cases. It's a huge docket, it's a lot of cases, it's a lot of work, and they most of the time don't have enough you know, time to to go over and check everything. So that's the risks of it. an AI being taught on on a Prone data can come to a wrong decision. So that's the reason speaker-0: Exactly the same as in in the medical field. I find that legal field and medical field have so many parallels because they're highly regulated and they truly decide on people's destiny and life. So you judges are making serious decisions that can affect people's lives, and of course doctors and healthcare professionals, they make decisions that affect people's lives. I find it's quite scary, like you said, if the judge looks at it and the I proposes a certain decision. What are the chances that the judge will go with that decision? It's the same as the doctor who relies on AI on the diagnostics of the patients, right? I find it's quite scary. Because yes, maybe in ninety eight percent of cases AI is correct. But in two percent of those cases, it may have I don't know, maybe had a hallucination, maybe had I don't know. speaker-1: But how about I'll be devil's advocate here? Please do, please I love For a second. look, most of the people we're human, right? So be prone to some biases. Sure. That we some of the biases unconscious even. meaning that how about if the AI properly taught, right, on on good data, it will do a better job than an average human, right? human tend to To be tired, over overworked and and so on, to be in in a bad mood, hungry. Yes. That can influence decision in some way. But AI, it can work twenty four seven, right? potentially it can be better. So who knows? Maybe it's a good thing. It's a good thing. Yeah. speaker-0: Yeah. Yeah, it would be nice to know how many mistakes are being made now without AI tools, let's say by judges or but doctors, and then compare it to one AI is. speaker-1: And I can tell you quite a few from my own experience. speaker-0: Okay. Okay. Has anything about AI in your field surprised you recently? speaker-1: yes. I think one of those platforms, LM platforms, wanted to claim a copyright on anything created using their tool, which is this interesting concept. So people using AI in different industries, healthcare, legal, some art, movie industry and so on, right? Mm-hmm. There was at some point, I think even you know, some CEO of Chat GPT or Claude said that we want to explore a concept where we will be the owners of anything created by our and speaker-0: Remember that. Yes, I do. speaker-1: it's interesting concept that basically they wanted to be an owner of the world to say of everything anything and everything created they would own, right? but now I think it it just kind of backed off a little bit with that concept because again I just your multiple questions come up, how you would enforce it, how it's actually would work. and it's interesting how it's gonna evolve in the future. So that's something very interesting. Do you think speaker-0: Like there is a possibility they can copyright it from your experience? speaker-1: It's all new. I I wouldn't be surprised if they try to implement it to to a degree. speaker-0: But if o I would ask you, would you give them right to copyright? Or you wouldn't. You would say no because it's it's human plus AI. So what what's your speaker-1: My take on it, I think it wouldn't benefit society overall 'cause people are very creative. They're trying to use it in multiple industries and that is what pushes forward the progress or development. if they implement or try to implement something like that, people would be more careful what they fit into AI because they would be scared that they'll be you know, they'll lose it potentially. speaker-0: Hm, but it could be even a good thing. Well, I guess it depends how you look at it. Like you said, it all depends. speaker-1: Right. speaker-0: Now my last question is, if you had a magic wand and you would ask AI to do something for you, anything you want, it doesn't have to be work related, it could be work related, what that would be? speaker-1: I would ask the magic AI to make peace in the world. I don't know, it's like how would you better express it, but just to make sure that people don't kill each other for no reason, so so to say, given the circumstances in a w in the world right now. speaker-0: Would you trust EI t with that task? speaker-1: Well, I would task the AI and see the output. Maybe it would offer something good. Who knows? speaker-0: Mm-hmm. Okay, well thank you so much for being here. speaker-1: Thanks for having me. speaker-0: This is AI Meets Med Tech Podcast with Anna Nestorova.