Latest / AI Tools for Practicing Lawyers / Episode 034 Jane Jetson’s Book Club and the Demise of Prompting

Transcript

Episode 034 Jane Jetson’s Book Club and the Demise of Prompting · AI Tools for Practicing Lawyers

• 0:03
  SPEAKER_01: Welcome to AI Tools for Practicing Lawyers.
• 0:07
  SPEAKER_01: Practical, no-nonsense guidance to help attorneys put AI power to work in their practice right now.
• 0:16
  SPEAKER_03: Students in 2026 cannot fathom writing a 10-page paper without ChatGPT.
• 0:24
  SPEAKER_03: But beyond feelings of nostalgia, are they wrong?
• 0:28
  SPEAKER_03: If AI can write a term paper in five minutes, it can be challenging to explain to a college student why struggling for five hours to pound out a history of the Roman Empire is the right choice.
• 0:41
  SPEAKER_03: We all know we need to keep humans in the loop, but is human judgment necessarily wiser than AI judgment?
• 0:49
  SPEAKER_03: In this episode, we meet one of AI's deepest thinkers, lawyer and professor Maura Grossman, who makes a convincing case for why culture, morals, and wisdom remain the province of humans and why a student's personal journal may be the best non-use of AI in the classroom.
• 1:10
  SPEAKER_03: Welcome to another episode of AI Tools for Practicing Lawyers.
• 1:15
  SPEAKER_03: I'm Ron Drescher.
• 1:16
  SPEAKER_03: I'm Heather Gardner.
• 1:18
  SPEAKER_03: And our guest today is Maura Grossman, one of the leading researchers working at the intersection of artificial intelligence, computer science, and the law.
• 1:30
  SPEAKER_03: Maura is a research professor in the School of Computer Science at the University of Waterloo, an adjunct professor at Osgood Hall Law School, and a visiting professor at Duke Law School.
• 1:43
  SPEAKER_03: She's also a practicing lawyer who serves as a consultant, expert, and special master in matters involving electronic discovery and legal technology.
• 1:54
  SPEAKER_03: Long before ChatGPT came along, Maura was helping establish that machine learning could perform legal document review more effectively and efficiently than traditional human review.
• 2:07
  SPEAKER_03: Today, she's studying some of the most consequential questions AI presents for our legal system: how judges should use it, how courts should respond when lawyers misuse it, whether we can trust AI-generated evidence, and whether the newest generative AI tools are actually better than the technologies they promised to replace.
• 2:30
  SPEAKER_03: Mora, welcome to AI Tools for Practicing Lawyers.
• 2:34
  SPEAKER_00: Thanks, Ron.
• 2:35
  SPEAKER_00: Thanks, Heather.
• 2:36
  SPEAKER_00: Happy to be here.
• 2:38
  SPEAKER_03: So, Maura, you very kindly sent me five of your more recent or perhaps most recent law review articles to take a look at.
• 2:47
  SPEAKER_03: And in one that hasn't even been published yet, which is The Judicious Judge's Guide to Generative Artificial Intelligence and Large Language Models, which is a title that is sufficiently long that even ChatGPT abbreviated it when it gave me my list of questions.
• 3:10
  SPEAKER_03: And it says when Gen AI is used to help draft orders or analyze evidence or filings, it must do so in a subordinated role that never displaces the judge's own reasoning.
• 3:24
  SPEAKER_03: At the same time, a complete refusal to understand or use Gen AI may undermine competence and diligence.
• 3:35
  SPEAKER_03: So you've spent a lot of time dealing with judges and AI and technology.
• 3:42
  SPEAKER_03: And what's the state of the bench in October of 2026?
• 3:47
  SPEAKER_00: Well, it depends where you're asking me about.
• 3:51
  SPEAKER_00: And really specifically, it's very different in Canada versus the U.S.
• 3:55
  SPEAKER_00: And it's different in federal court versus state court, and it's different between courts and courts.
• 4:02
  SPEAKER_00: One of the things that I've learned is this is a wild west out there.
• 4:06
  SPEAKER_00: You have in the United States at least limited guidance at the federal level.
• 4:12
  SPEAKER_00: So you have judges, some judges using it, some judges not using it.
• 4:18
  SPEAKER_00: Some of the states have said it's okay, some of the states have provided tools, other states have not.
• 4:24
  SPEAKER_00: Canada just about a week ago changed its rules from absolutely no use to some limited use.
• 4:33
  SPEAKER_00: So you've got a complete spectrum from absolutely nothing to people who are actively using it.
• 4:42
  SPEAKER_03: So the absolute no use rule, where is that coming from?
• 4:46
  SPEAKER_00: Largely fear.
• 4:49
  SPEAKER_00: When I do training of uh educational programs with judges, it's really interesting.
• 4:55
  SPEAKER_00: I do workshops sometimes and I introduce them to generative AFI.
• 5:01
  SPEAKER_00: And when they first come in and I have them log on, many of them are terrified.
• 5:06
  SPEAKER_00: They have no idea whether they're going to be downloading malware.
• 5:11
  SPEAKER_00: Can they go onto a site?
• 5:13
  SPEAKER_00: What is it doing to their computer?
• 5:15
  SPEAKER_00: They don't really understand.
• 5:16
  SPEAKER_00: And uh the first thing I have them do is look at the privacy and security settings to set and make sure that the system is not training on their data, and I show them how they can delete their conversations if they want to.
• 5:30
  SPEAKER_00: And then I have them upload an opinion they've written and interact with their opinion.
• 5:35
  SPEAKER_00: And by the end of an hour, they now realize this is just a tool.
• 5:41
  SPEAKER_00: It's sort of like it's better than Word, and it's better than a lot of other things, and it's kind of cool.
• 5:48
  SPEAKER_00: But but they're frightened at first because they've heard hallucinations, they've heard security incidents, and so many things.
• 5:56
  SPEAKER_00: So I think part of it is fear and conservatism and not wanting to be on the front page of a newspaper.
• 6:05
  SPEAKER_03: You know, we started the podcast in February of 26.
• 6:09
  SPEAKER_03: And uh here we are, nine months later.
• 6:12
  SPEAKER_03: And it feels to me, maybe because I just started paying attention really closely back over the winter, that AI adoption in legal circles is accelerating rapidly.
• 6:26
  SPEAKER_03: Are you seeing that as well?
• 6:28
  SPEAKER_00: I think that's true.
• 6:30
  SPEAKER_00: We've seen not only specialized tools, legal technology like Harvey and Lagora, but now you're starting to see.
• 6:38
  SPEAKER_00: I saw today Claude now has new AI plugins.
• 6:43
  SPEAKER_00: And so you're starting to see the big technology companies realize that this is a considerable market and start to develop legal-oriented tools.
• 6:54
  SPEAKER_02: I have a really interesting question about the the AI hallucinated citation issue.
• 6:59
  SPEAKER_02: Attorneys have been submitting inaccurate citations, making factual mistakes, misrepresenting authorities for generations.
• 7:06
  SPEAKER_02: AI did not invent this problem.
• 7:09
  SPEAKER_02: It's just introduced a new way to make them.
• 7:12
  SPEAKER_02: So, shouldn't the standard really be focused more on the accuracy of the final work product and not how it was produced overall?
• 7:19
  SPEAKER_02: I think that it's in the news because of technology.
• 7:22
  SPEAKER_02: If they had put in the news all of these things happening with attorneys doing them live years and years before, I think the public would understand that it's not a new issue.
• 7:32
  SPEAKER_00: I agree with you.
• 7:33
  SPEAKER_00: And Judge Paul Grimm and I came out very early when the hallucination problem started, and you started to see courts issuing all these standing orders saying you must disclose or you can't use this in my court.
• 7:47
  SPEAKER_00: And we said, whoa, whoa, whoa, wait, time out.
• 7:50
  SPEAKER_00: That's not a good idea.
• 7:52
  SPEAKER_00: The it doesn't really matter how somebody gets to what they get to, what they it's what they file that matters and that they need to confirm that, whether it's been done by a human or an algorithm.
• 8:04
  SPEAKER_00: And there are plenty of rules, including Rule 11 of the federal rules, that require you to submit things that, you know, not to submit things that are uh false or incorrect.
• 8:16
  SPEAKER_00: So yes, I think what's a little bit different here is that these tools come across as confident and authoritative that people don't feel they need to check them the same way as other sources.
• 8:31
  SPEAKER_03: AI does talk a good game.
• 8:32
  SPEAKER_02: And and yeah, and a junior associate or a paralegal that drafted and gave, you know, or an attorney who m maybe miscited it it just wasn't caught, you know, it also wasn't caught, but no one was raising cane about it back then.
• 8:46
  SPEAKER_02: No one was all upset about it back then.
• 8:47
  SPEAKER_02: Now it's this is terrible, don't use AI.
• 8:50
  SPEAKER_02: No one ever said stop using your junior associates, you know.
• 8:53
  SPEAKER_00: No, you're right, you're right about that.
• 8:55
  SPEAKER_00: And and it it particularly harms people who are not represented who really get the benefit uh of these tools rather than writing on a yellow, scribbling on a yellow legal pad, pages and pages of things that don't make sense, they can actually file something that looks reasonably close to what a court expects.
• 9:16
  SPEAKER_03: When I was a brand new lawyer, uh the first job I ever got, uh my boss asked me to write a research memo for him, which I did.
• 9:25
  SPEAKER_03: This was in the 1980s, and his secretary typed it up for me and made it all pretty, and I gave it to him and I'm dying for his comments, and he said it was nicely typed.
• 9:39
  SPEAKER_03: Yeah, right.
• 9:40
  SPEAKER_03: And I was wondering, pro se litigants are using AI in courtrooms, and and like you say, they're putting out really polished-looking work product, but is it only nicely typed, or is it really substantively valuable, or is it all over the place?
• 10:00
  SPEAKER_00: So I I've heard mixed reports from judges.
• 10:04
  SPEAKER_00: Some judges are saying the volume has increased, and the volume of motions in a particular matter is increased because somebody, why not throw spaghetti at the wall and see what sticks?
• 10:17
  SPEAKER_00: I'll make every single motion I possibly can.
• 10:20
  SPEAKER_00: Some judges have said they're getting longer and longer submissions, so that now they're at a point where they may have to put page limits or other things.
• 10:30
  SPEAKER_00: And then I've heard other courts say they're not really, they've seen an improvement in the quality of what's submitted to them, and and they're at least they know what what somebody is trying to claim or assert.
• 10:42
  SPEAKER_00: So you get mixed views about whether this is uh a blessing or not.
• 10:48
  SPEAKER_03: Should there be a different standard for hallucinated citations from pro se litigants than from licensed attorneys?
• 10:57
  SPEAKER_00: I have taken that position in the in a paper.
• 11:01
  SPEAKER_00: I don't know if it's one of the ones I sent to you called Utopia or Dystopia, where we said the self-represented litigants don't have access to Westlaw, Lexis, and some of the other tools.
• 11:17
  SPEAKER_00: So they're they're stuck with the internet and what they can find on Google or or a free site.
• 11:24
  SPEAKER_00: And maybe they can't check as well as an attorney.
• 11:29
  SPEAKER_00: So, yes, cut them some slack.
• 11:32
  SPEAKER_00: Of course, if they keep repeating the same uh faux pas, that's a problem.
• 11:37
  SPEAKER_00: But, you know, y we said in this paper, cut them a little bit more slack, at least initially, because they don't have access and they don't have legal training and and they may not be doing it intentionally.
• 11:49
  SPEAKER_03: You know, I've seen you argue in your in your articles for a more relaxed, punitive standard relative to lawyers disseminating hallucinated citations in briefs.
• 12:04
  SPEAKER_03: And then there are judges that feel like they are personally offended by hallucinations in briefs.
• 12:13
  SPEAKER_03: Where's the happy medium?
• 12:15
  SPEAKER_00: Well, the funny thing is that some of the sanctions have gotten very severe.
• 12:21
  SPEAKER_00: Referrals for disciplinary proceedings, people's licenses taken away.
• 12:26
  SPEAKER_00: And until recently, we really weren't seeing a decrease in the number of hallucinations.
• 12:32
  SPEAKER_00: So when Judge Grimm and I wrote our first paper saying, don't be so quick to outlaw this or to require disclosure, we thought this is an educational issue that will be cleaned up in a matter of time.
• 12:47
  SPEAKER_00: We were surprised that it wasn't corrected as quickly as as we had anticipated or would have liked.
• 12:55
  SPEAKER_00: But I'm not sure that these very harsh penalties help.
• 13:00
  SPEAKER_00: I fortunately I was working on a case with a judge where I'm a special master, and there was a hallucination in in a brief that was submitted to us.
• 13:09
  SPEAKER_00: And I was proud that my judge brought it up.
• 13:12
  SPEAKER_00: She said, we couldn't find this case.
• 13:15
  SPEAKER_00: I don't want to know how it happened.
• 13:17
  SPEAKER_00: I just don't want it to happen again.
• 13:19
  SPEAKER_00: And a warning was sufficient, and I think people were very appreciative of that, rather than, like you say, somebody taking it as a personal affront and issuing a$10,000 or a$15,000 fine and referring them for disciplinary actions.
• 13:37
  SPEAKER_00: Some of it, yes, it's sloppy work, but it's not necessarily intentional.
• 13:42
  SPEAKER_00: It's it's small lawyers for small firms who are very pressed for time, who don't understand that these tools hallucinate.
• 13:51
  SPEAKER_03: So from the from the chambers point of view, should there be a uniform attitude and guidelines towards judicial use of AI?
• 14:02
  SPEAKER_03: Or should it just stay in the Wild West where it is now, where every judge kind of gets to decide how their chambers uses that tool?
• 14:12
  SPEAKER_00: I think the judges are crying out for some guidance.
• 14:16
  SPEAKER_00: Uh, and that's part of why we wrote the Judicious Judges Guide.
• 14:20
  SPEAKER_00: We had written a paper that was published in the Sedona Conference in journal in 2025 that was at a 30,000-foot level of general principles.
• 14:32
  SPEAKER_00: And everybody was asking us, but we need more specific.
• 14:35
  SPEAKER_00: We need to know where to go to shut off training in our tool.
• 14:40
  SPEAKER_00: We need examples of what good prompts look like.
• 14:44
  SPEAKER_00: And that's why we really focused on this paper.
• 14:48
  SPEAKER_00: So I think judges really do want the guidance.
• 14:51
  SPEAKER_00: They don't want the guidance that says, no, you can't use it at all, but they want to understand what's safe and what's not safe.
• 14:59
  SPEAKER_03: Is anybody out there creating knowledge-based materials for the judiciary as to how to use AI responsibly and effectively?
• 15:10
  SPEAKER_00: I know the Federal Judicial Center created a toolkit.
• 15:14
  SPEAKER_00: It's not public, but it is available to federal judges online.
• 15:18
  SPEAKER_00: I was a consultant on that matter and helped them with content.
• 15:22
  SPEAKER_00: The National Center for State Courts and Thomson Reuters Institute have a joint project where they've done some really magnificent pieces that have guidance.
• 15:34
  SPEAKER_00: So you do have you do have people who are trying.
• 15:37
  SPEAKER_00: Maybe it's not enough, I don't know.
• 15:39
  SPEAKER_00: Maybe it's not fast enough.
• 15:41
  SPEAKER_00: And then the judges established their own group, Maritza Braswell in Colorado, Scott Schlegel in Louisiana, and Xavier Rodriguez in uh Texas created a group, I think it's called Jake, J A I C, where they get judges together monthly or so, and they and it's private, there's nobody else but judges there, and they share their experiences and what they've learned and ask questions.
• 16:08
  SPEAKER_00: So that's really good.
• 16:10
  SPEAKER_03: Have you had the opportunity to sit in on any of those sessions?
• 16:14
  SPEAKER_00: I have not.
• 16:15
  SPEAKER_03: Is that something you want to do?
• 16:18
  SPEAKER_00: I offered.
• 16:19
  SPEAKER_00: I said if you if at any point you want somebody who with technical knowledge to come in.
• 16:24
  SPEAKER_00: But I think judges do need to do their own, have their own space.
• 16:29
  SPEAKER_00: It is I do a lot of judicial education.
• 16:32
  SPEAKER_00: Uh I'm very fortunate that I'm trusted to do this.
• 16:36
  SPEAKER_00: It's a blessing, it really is.
• 16:38
  SPEAKER_00: But it is very hard to train judges in front of lawyers.
• 16:44
  SPEAKER_00: They're very, very careful about what they disclose.
• 16:47
  SPEAKER_00: They don't want to appear idiotic.
• 16:50
  SPEAKER_00: So they really do need their own place.
• 16:53
  SPEAKER_00: So I understand this completely.
• 16:55
  SPEAKER_00: Uh I'm supportive of it.
• 16:57
  SPEAKER_00: And I said if if you ever have technical questions or want somebody to come in to answer technical questions, I'm available.
• 17:04
  SPEAKER_00: But so far they haven't called on me, which is fine.
• 17:07
  SPEAKER_02: So if if judges, if there are judges that are refusing to use or engage with AI, and we know that it's being used widely, right?
• 17:15
  SPEAKER_02: By litigants, by attorneys, by everyone else in basically the life cycle, does this become a judicial competence issue if they can't look at those cases with AI knowledge?
• 17:27
  SPEAKER_00: I think they need to understand the technology.
• 17:30
  SPEAKER_00: I don't know that they necessarily need to use it.
• 17:33
  SPEAKER_00: Okay.
• 17:34
  SPEAKER_00: But you certainly need to understand its capabilities.
• 17:38
  SPEAKER_00: Look, evidence is going to be AI generated, and you need to understand what questions to ask.
• 17:44
  SPEAKER_00: And in order to sanction on one of these cases, I think you need to understand the technology.
• 17:51
  SPEAKER_00: I gave a keynote to the Georgia Bench and Bar conference recently.
• 17:57
  SPEAKER_00: And I was very surprised when three judges who were on the judicial panel said that there was no protection for anything a lawyer put into a generative AI tool because it was like social media was like screaming it out onto the internet.
• 18:16
  SPEAKER_00: And I raised my hand and I don't want to be rude or anything, but I said, well, maybe you need to understand a little bit more of the technology, because under that theory, every search an attorney does on Google and every document that an attorney has ever put in the cloud, storage, well, all of it is going to a third party and subject to review by third parties.
• 18:42
  SPEAKER_00: So maybe you need to like study this a little bit more.
• 18:46
  SPEAKER_00: And they said to me, Well, you keep saying we don't understand the tech, but the lawyers in front of us need to explain it properly then.
• 18:55
  SPEAKER_02: Well, if a judge isn't using AI, you say they need to understand it.
• 19:00
  SPEAKER_02: If they're not using it, can they possibly understand it?
• 19:03
  SPEAKER_00: I think it helps to try it, even if you decide later on.
• 19:09
  SPEAKER_00: I don't use a lot of Gen AI, but I have certainly played with it.
• 19:15
  SPEAKER_00: I have a very good sense of its strengths and weaknesses, but I don't use a lot in my my writing is me, and then my writing is my thinking, and I don't want to outsource that.
• 19:28
  SPEAKER_00: I want my materials to sound like me, but I know something about it.
• 19:32
  SPEAKER_00: You know, unless you're retiring in the next 15 minutes, you really do need to know something.
• 19:37
  SPEAKER_00: I understand this technology.
• 19:39
  SPEAKER_03: See, I'm following up on Heather's question, which is this is really interesting to me, because the statement judges need to understand AI carries with it a lot that needs to be unpacked.
• 19:53
  SPEAKER_03: For example, there's understanding that, yes, you have a prompt box, and that little plus sign in the lower left-hand corner allows you to upload things and connect.
• 20:03
  SPEAKER_03: That's one level of understanding.
• 20:05
  SPEAKER_03: There's a there's a whole other level of understanding as well.
• 20:08
  SPEAKER_03: There are enterprise levels, and then there's Teams, and then there's Pro, and then there's Macs, and then there's Plus, and then Gemini has AI Vertex, and then there's Google Cloud, and then there's the whole Microsoft world.
• 20:26
  SPEAKER_03: So where are we on that spectrum for the judiciary to obtain competence over their AI understanding?
• 20:35
  SPEAKER_00: So I've done programs where I've explained the difference between an enterprise model and a public-facing model, and the difference between paid tiers and non-paid tiers.
• 20:46
  SPEAKER_00: So they're getting up to speed.
• 20:48
  SPEAKER_00: I think what's critical for judges to understand is the difference that prompts can make.
• 20:55
  SPEAKER_00: So if you ask, give me a summary of the key or most important claims in this matter, that's very different than asking, give me a summary of all claims in this matter.
• 21:09
  SPEAKER_00: And an AI may decide what's key or important by how much real estate it occupies.
• 21:16
  SPEAKER_00: So this occupies three pages, that occupies one page.
• 21:19
  SPEAKER_00: Obviously, the three pages is more important, and that's not necessarily so in the law.
• 21:25
  SPEAKER_00: So those are the kinds of subtleties I think judges need to understand.
• 21:29
  SPEAKER_00: Otherwise, really their use becomes very problematic if they don't really understand what kinds of things can go wrong and how prompting can make a big difference.
• 21:42
  SPEAKER_03: Do you think there's an argument for a standard library of prompts for the judiciary relative to any particular practice area, like a PI library of prompts or a bankruptcy library, a patent law IP library?
• 22:00
  SPEAKER_03: So that judges at least have a baseline ability to dig deeper into ideas without having to re-educate themselves from the start?
• 22:09
  SPEAKER_00: In part, what we were trying to do with our judicious guide, that we provided examples of different kinds of tasks and what a decent prompt might look like.
• 22:20
  SPEAKER_00: And I'm certainly not opposed to libraries, but I don't want somebody to automatically just go and apply something.
• 22:29
  SPEAKER_00: They need to apply thought, they need to think about their own specific circumstances and understand how to generate a prompt that applies to what's in front of them.
• 22:41
  SPEAKER_00: But I think having a starting place is always helpful.
• 22:44
  SPEAKER_03: See, if I was if I was a judge, and that is a risk that nobody ever needs to fear, I I would create a whole series of clawed projects and skills for each practice area.
• 22:57
  SPEAKER_03: And you know what?
• 22:58
  SPEAKER_03: I'd have my judicial clerk just dump all the evidence into this clawed skill and give me a summary and and then from there work out.
• 23:07
  SPEAKER_03: What what would be some dangers in that kind of approach?
• 23:12
  SPEAKER_00: We don't want the AI tool making the judgments we expect of judges.
• 23:19
  SPEAKER_00: So we don't want the AI tool determining credibility or which facts are more important or less important or are legally significant.
• 23:30
  SPEAKER_00: So we want to use this.
• 23:33
  SPEAKER_00: There's a sandwich approach.
• 23:35
  SPEAKER_00: The judge needs to be here, and the judge needs to be at the end.
• 23:40
  SPEAKER_00: And then in the middle, you can use this as a tool to help you.
• 23:43
  SPEAKER_00: But you don't want the tool doing the legal thinking, the judicial thinking that a court does.
• 23:50
  SPEAKER_00: That needs to remain with the judge.
• 23:53
  SPEAKER_00: And it's a fine line, it's a gray area between, but that's the difference between putting in all the briefs and saying draft me the first draft of my opinion versus saying, here's what I'm uh here are the points I want to make in my opinion.
• 24:09
  SPEAKER_00: Give me a first version.
• 24:12
  SPEAKER_02: So do people really need prompts at this point?
• 24:14
  SPEAKER_02: AI has advanced to, I haven't prompted it.
• 24:17
  SPEAKER_02: I don't know when.
• 24:18
  SPEAKER_02: I mean, I just talk to my AI and tell it what I need.
• 24:21
  SPEAKER_02: Every attorney and every firm works differently.
• 24:24
  SPEAKER_02: They all have different requirements that they look for, different ways they want their, you know, summaries done, what they're looking for in their discovery or their cases.
• 24:35
  SPEAKER_02: Isn't it best for everyone to just collaborate with their AI?
• 24:39
  SPEAKER_02: I mean, who is really using prompts anymore?
• 24:42
  SPEAKER_02: And why?
• 24:43
  SPEAKER_02: You need to know your tool because the only way I give people a prompt at this point is to try to teach them how to use an AI tool if they really don't know where to begin.
• 24:52
  SPEAKER_02: And then I explain to them that this is your starting point, but you need to be telling it what you're looking for.
• 24:58
  SPEAKER_02: You know, taking a generic prompt that someone else wrote is not going to give you a specific result that you need for your purpose.
• 25:07
  SPEAKER_03: I I don't I don't know, Heather.
• 25:09
  SPEAKER_03: I think saying that prompts are so March 2026 is a complaint that Jane Jetson makes at her book club.
• 25:17
  SPEAKER_02: It's true though.
• 25:18
  SPEAKER_02: It is very true.
• 25:19
  SPEAKER_02: You've seen me talking to my AI.
• 25:21
  SPEAKER_02: We don't use prompts.
• 25:22
  SPEAKER_02: We talk to each other.
• 25:23
  SPEAKER_00: Those are prompts.
• 25:24
  SPEAKER_00: Those are prompts.
• 25:24
  SPEAKER_00: Those are prompts.
• 25:25
  SPEAKER_02: Right.
• 25:25
  SPEAKER_02: But I'm not copying, I'm not pasting a pre-text prompt into my AI any longer.
• 25:31
  SPEAKER_02: I haven't for a very long time.
• 25:33
  SPEAKER_02: And and it wouldn't, it would get me what it would get anyone else.
• 25:37
  SPEAKER_02: Something that I would need to really refine and hone and go back and forth with for a while.
• 25:42
  SPEAKER_02: Why not just start with telling it what you're really looking for?
• 25:45
  SPEAKER_02: Right.
• 25:46
  SPEAKER_03: So I I have a whole other question.
• 25:48
  SPEAKER_03: And this comes up all the time.
• 25:51
  SPEAKER_03: And it it's caused me to really reflect on it.
• 25:54
  SPEAKER_03: And that's the idea that human judgment is what elevates us over AI and needs to always be part of the way you interact with AI.
• 26:06
  SPEAKER_03: And that human judgment is something that AI will never have.
• 26:10
  SPEAKER_03: And when I think of that, I think, well, wait a second, where does how does one develop judgment?
• 26:17
  SPEAKER_03: Where does judgment come from?
• 26:19
  SPEAKER_03: Isn't judgment really just a collection of the data that a human perceives over the course of their life and those experiences?
• 26:28
  SPEAKER_03: And why is that better than an AI that has been fed so much more data than a human will ever be able to absorb in the course of their lifetime?
• 26:39
  SPEAKER_03: And why has that become the buzzword of how humans interact with AI?
• 26:46
  SPEAKER_00: So one one of the issues that we refer to in this context is called alignment.
• 26:51
  SPEAKER_00: So if I give an AI system an instruction and I say, do this, I want you to do this at all costs, like figure out this particular way to break into a certain system or something like that.
• 27:22
  SPEAKER_00: That's not good.
• 27:23
  SPEAKER_00: That's morally bad, or that's a problem, or or whatever.
• 27:27
  SPEAKER_00: So humans have culture, humans have morals, humans have wisdom, and we're not, most humans, at least, don't just embark on a goal without any regard for how they're going to go to get to that goal.
• 27:44
  SPEAKER_00: And I think right now that's a big difference.
• 27:47
  SPEAKER_00: Uh Bostrom always gave this example of maximizing the paperclips in the world.
• 27:53
  SPEAKER_00: And how uh an AI, if it was given the instruction, no matter what, maximize paperclips, eventually it would go to Heather's mouth and it would extract all her teeth because there might be metal in her teeth from cavity, you know, from cavity fillings.
• 28:07
  SPEAKER_00: And a human wouldn't do that, but an AI might because they're literally trying to reach the goals.
• 28:16
  SPEAKER_00: So that's what people mean when they say there needs to be a human in the loop.
• 28:20
  SPEAKER_03: All right, so let's apply that to judges, because really what's going on is judges are instructed, well, you or juries, you should only regard what's in the record.
• 28:33
  SPEAKER_03: But so much of knowledge and judgment and and morals and what you talk about are derived from a person's experience outside the record, what we like to call common sense.
• 28:46
  SPEAKER_03: How do you balance that?
• 28:48
  SPEAKER_00: Well, our assumption is if we have a diverse judiciary, that balances out overall, that you're gonna get some more conservative people, some more liberal people, and so forth.
• 29:01
  SPEAKER_00: So on balance, your your judges are are gonna be roughly at the same place.
• 29:07
  SPEAKER_00: See, the problem is when you get into things like sentencing guidelines, where if you have 0.2 ounces of cocaine or whatever, you get 11 years no matter what.
• 29:22
  SPEAKER_00: And say somebody has that because their kid is dying of cancer and they they need to pay for the treatment and they had no other option, that's a different circumstance than somebody who's perhaps just doing it so that they can buy a new Mercedes-Benz.
• 29:38
  SPEAKER_00: And when you when you have these strict rules without the human discretion, we found that the sentences and things like that are not any better.
• 29:51
  SPEAKER_00: If anything, uh they're worse because they're so rigid.
• 29:54
  SPEAKER_00: So there's a balance between having guidelines and having rules and then having the flexibility to apply discretion.
• 30:01
  SPEAKER_00: And usually we trust our judges to figure out if the person in front of me did this because they have the really sick kid or they did it because they want the Mercedes.
• 30:14
  SPEAKER_03: So you would give a downward departure for Walter White as he battles cancer to support his family.
• 30:21
  SPEAKER_00: I would consider that, yes.
• 30:23
  SPEAKER_02: And and no one thinks we should maybe, you know, look at changing legislation and laws to help those situations in the first place so that people aren't put in those positions.
• 30:33
  SPEAKER_03: All right.
• 30:34
  SPEAKER_03: So, more, you've written a great deal about e-discovery and machine learning and e-discovery.
• 30:42
  SPEAKER_03: First of all, could you could you tell our listeners what that involves and and how it pertains to the AI discussion that we're all having now?
• 30:51
  SPEAKER_00: So if you and I are in a lawsuit, discovery requires me to give you all the relevant evidence to our lawsuit considering proportionality.
• 31:03
  SPEAKER_00: In other words, if the matter's worth$1,000, I don't have to give you$10,000 worth of discovery.
• 31:09
  SPEAKER_00: So all relevant information considering proportionality, and you have to do the same for me.
• 31:16
  SPEAKER_00: But if you have two million emails in your email box, how do we find the hundred that matter that have to do with the dispute between us?
• 31:28
  SPEAKER_00: So I was very interested in this early on, and I met a spam guru named Gordon Cormac, who was at the time teaching at the University of Waterloo, and it occurred to me spam and what we call ham, which is the good stuff, is not all that different a problem from relevant evidence and not relevant evidence.
• 31:49
  SPEAKER_00: So I asked him if he wanted to work with me.
• 31:52
  SPEAKER_00: I had lots of data, he had lots of algorithms, and we started to apply this kind of machine learning techniques in the legal area.
• 32:03
  SPEAKER_00: That was back in about 2009.
• 32:05
  SPEAKER_00: In 2011, we published what has become a seminal paper that showed that these tools were actually better than lawyers and more efficient than lawyers in finding the evidence.
• 32:17
  SPEAKER_00: So that has moved forward.
• 32:19
  SPEAKER_00: I've always been very interested in evaluation and actually not just intuition, I think this might work, but actually I'm testing different methods and trying to figure out what works and what doesn't work under what circumstances.
• 32:36
  SPEAKER_03: So in one of your papers, you argued that latest generative AI technology isn't necessarily an improvement over established machine learning approaches.
• 32:49
  SPEAKER_03: What did you mean by that?
• 32:50
  SPEAKER_00: So it was a little controversial.
• 32:53
  SPEAKER_00: Some folks had done some research and published a paper that received a lot of headlines that said basically, get rid of all the machine learning and replace it with this generative AI.
• 33:08
  SPEAKER_00: So we would just ask a generative AI system, find me all the relevant data.
• 33:13
  SPEAKER_00: And we looked at their data and we looked at their results, and Gordon and I said, well, maybe not so fast, because they had compared apples and oranges.
• 33:25
  SPEAKER_00: In one circumstance, they compared using the generative AI simply to draw a cut point between what to produce and what not to produce, but nobody looked at the stuff that was being produced.
• 33:39
  SPEAKER_00: When they used the old-fashioned technology, they had it look at everything that was being produced.
• 33:46
  SPEAKER_00: And we said you can't compare the statistics from something that's just drawing a cut point to something that's actually looking at everything.
• 33:55
  SPEAKER_00: And when we went back and looked at their data, all these amazing advances of Gen AI disappeared when you made it an apples to apples comparison, when you either used the machine learning to draw a cut point, or when you had humans review the documents that the Gen AI said were above the cut point, then it looked very different.
• 34:18
  SPEAKER_00: I would say that the Gen AI was about as good as the humans, but it wasn't this miles ahead like had been presented.
• 34:25
  SPEAKER_03: How did all right, now we can get to talk about prompting?
• 34:29
  SPEAKER_03: How does the prompt and the instructions you would give to AI affect the outcome of that comparison?
• 34:37
  SPEAKER_03: I mean, is it truly apples to apples?
• 34:40
  SPEAKER_00: No, it's not.
• 34:41
  SPEAKER_00: And and in this particular case, they had done 43 iterations of the prompt by an expert.
• 34:49
  SPEAKER_00: We never got a legal expert who was very familiar with the matter.
• 34:53
  SPEAKER_00: We never got our hands on the prompts.
• 34:55
  SPEAKER_00: We did ask for them, but they said that was proprietary and confidential, so we never got our hands on it.
• 35:00
  SPEAKER_00: We would have liked to.
• 35:02
  SPEAKER_00: But it's it's very interesting.
• 35:05
  SPEAKER_00: Recent work I've done with with Gordon suggests that maybe these extraordinarily detailed legalese kind of prompts aren't any better than something short and clear and specific.
• 35:22
  SPEAKER_00: We need to do a lot more work on that because the assumption has been, and I I think Heather would agree with me, all the instructions were you need to give these very long, detailed, specific prompts.
• 35:34
  SPEAKER_00: And it the tools, at least the more recent tools, don't seem to need that.
• 35:39
  SPEAKER_00: No, no, not at all.
• 35:41
  SPEAKER_00: So this is a moving target, and and I don't know that we can conclude for once and for all the best way to do this.
• 35:50
  SPEAKER_00: I think we have to do the science.
• 35:52
  SPEAKER_02: No, what I was saying earlier is prompt engineering and prompt writing was a workaround to the limitations of the earlier models.
• 36:00
  SPEAKER_02: AI has come so far.
• 36:02
  SPEAKER_02: What we need to be teaching attorneys, judges, legal professionals is how to communicate your objective, what they are looking for as a result and responsibility.
• 36:12
  SPEAKER_02: Pasting and prompts is probably giving the most inaccurate results that anyone is getting, truthfully.
• 36:20
  SPEAKER_00: Well, they the more detail and extraneous stuff you put in there, the more likely you get hallucinations, is is what we've seen.
• 36:28
  SPEAKER_00: So keeping it short, clear, precise may be better, but that that's what science is for, is to answer these kinds of questions.
• 36:37
  SPEAKER_02: Right.
• 36:37
  SPEAKER_02: It was truly when the technology was less capable, it was really necessary.
• 36:42
  SPEAKER_02: But you don't need the 300-word magic to paste in anymore.
• 36:46
  SPEAKER_02: They they need to learn how to communicate what they're looking for, the context, and then to human-verify the results.
• 36:53
  SPEAKER_02: It it is so much far, it has come so far in three years that you know we could by the time this comes out tomorrow, there will be a whole new story about new AI technology.
• 37:04
  SPEAKER_02: I mean, it is just but prompting isn't just so last year.
• 37:09
  SPEAKER_02: It's not Judy Jetson, sorry, Magmart.
• 37:13
  SPEAKER_00: I was gonna say the one thing, Heather, that bothers me a little bit is you put two plus two equals into a calculator, every single time you do that, you will get four.
• 37:27
  SPEAKER_00: With these generative AI tools, Ron can ask a question, I can ask the exact same question, you can ask the exact same question, and because they're probabilistic rather than deterministic, we could get three different answers.
• 37:41
  SPEAKER_00: Now, they should probably be fairly similar, but that makes me a little bit uncomfortable.
• 37:47
  SPEAKER_00: I I would like if they were more reliable.
• 37:49
  SPEAKER_03: We're in season three of our podcast.
• 37:52
  SPEAKER_03: We've evolved from looking at Flintstone, Simpsons, and Jetsons to users and builders, which seems more apropos of where we are at this time in the evolution of AI.
• 38:05
  SPEAKER_03: And the idea is when we picture a user, we picture somebody who just wants to sit in front of their computer, download an app, start putting data into it, and get an outcome.
• 38:19
  SPEAKER_03: And when we think about builders, we think about people who want to sit in front of their computer and vibe code or upload the last 10 years of their practice into a frontier model so that it really understands them.
• 38:33
  SPEAKER_03: And so the question is which creates a bigger risk?
• 38:37
  SPEAKER_03: You users who don't really want to know about all the ins and outs of how the AI works, or builders who really want to get in their study and create their own AI app applications?
• 38:52
  SPEAKER_00: I think it depends what your goal is.
• 38:55
  SPEAKER_00: If your goal is to build a website and you're not interested in actually learning how to build the website, it's great.
• 39:05
  SPEAKER_00: You use one of these tools and you can get an amazing website within an hour.
• 39:11
  SPEAKER_00: And you would walk away and you wouldn't know anything more today than you knew yesterday about how to build a website.
• 39:18
  SPEAKER_00: You just tell the system, here's the color scheme I want, and here are the tabs, and so forth.
• 39:25
  SPEAKER_00: But that's very different than going to law school and coming out of law school knowing how to identify issues, to think critically, and so forth.
• 39:35
  SPEAKER_00: And there you don't want to outsource your thinking.
• 39:39
  SPEAKER_00: So I had a student who told me last year that he had never read a single case or paper in law school, that he spent three years summarizing everything through Chat GPT.
• 39:52
  SPEAKER_00: And I said, Well, how did you learn to think and analyze for yourself?
• 39:58
  SPEAKER_00: And then I had this long silence.
• 40:00
  SPEAKER_00: So it really depends what your goal is.
• 40:02
  SPEAKER_00: If you're trying to complete a task, fine.
• 40:06
  SPEAKER_00: If you're trying to maintain your thinking and your skills and so forth, then I wouldn't outsource to these tools.
• 40:15
  SPEAKER_03: You know, I was thinking about something yesterday.
• 40:19
  SPEAKER_03: I was having ChatGPT or Claude research something for me and look something up.
• 40:24
  SPEAKER_03: And I thought, you know what?
• 40:26
  SPEAKER_03: Back in the day, whatever that day was, before three years ago, I'd hire a kid, you know, out of college or out of school, to uh to help me do this research and these tasks.
• 40:40
  SPEAKER_03: And they would, first of all, they would see how how I think and how I instruct them.
• 40:46
  SPEAKER_03: And then they would kind of learn the substantive aspects of the tasks that I was giving them.
• 40:52
  SPEAKER_03: And that's how they developed and they became more productive and wiser and more competent.
• 40:59
  SPEAKER_03: But if I'm having everything done by an AI chatbot, how how do we replace that kind of interaction with a mentee?
• 41:12
  SPEAKER_00: I don't think we should be doing that in the first place.
• 41:15
  SPEAKER_00: If you look at some of the research that's been done at MIT, they've had three conditions.
• 41:21
  SPEAKER_00: One where a person writes using a large language model.
• 41:26
  SPEAKER_00: The second is where they write, but they're allowed to search Google, but they still have to do the writing themselves.
• 41:33
  SPEAKER_00: And the third is they just write and they have to do old-fashioned research.
• 41:37
  SPEAKER_00: And it is very clear that your brain is most engaged in where you are writing and least engaged or where you're using the LLM.
• 41:47
  SPEAKER_00: And if you ask people a couple of weeks later about the essay they wrote using the LLM, they don't remember it.
• 41:53
  SPEAKER_00: It hasn't come in, it hasn't been integrated in any way.
• 41:57
  SPEAKER_00: So I worry about this with my students that that is not the way to teach people how to how to think.
• 42:06
  SPEAKER_00: We we need to remember that these are tools.
• 42:09
  SPEAKER_00: They're not to replace.
• 42:12
  SPEAKER_00: It's very easy to replace all the research using one of these tools, and they do a pretty good job, especially the re the ones that reason can do a very nice job of getting you the information.
• 42:24
  SPEAKER_00: But in the end, uh you're not going to integrate it in the same way as if you had actually read primary sources and so forth.
• 42:32
  SPEAKER_03: Well, I mean, it's interesting you brought up your students, because it makes me wonder how do you address the AI problem that so many educators are facing now, where students really abandon the learning process to AI.
• 42:49
  SPEAKER_03: How do you deal with that?
• 42:50
  SPEAKER_00: So, in in the class I just finished teaching, which was to health informatics graduate students on AI and ethics, there were three components to the grade.
• 43:01
  SPEAKER_00: One third was class participation.
• 43:03
  SPEAKER_00: I run them in the class as a seminar, and people's participation, either orally or in the chat, since it's an online class, is 30%.
• 43:12
  SPEAKER_00: I have them, I used to give essays.
• 43:15
  SPEAKER_00: I no longer give essays because when you give essays, you just get Chat GPT back.
• 43:20
  SPEAKER_00: So I have them keep a personal journal for the term and write about their own reactions to things they've read, their own reactions to the news in the course of the week, their own reactions to the discussion in class, and that's 30%.
• 43:37
  SPEAKER_00: And you may not use a large language model for that at all.
• 43:44
  SPEAKER_00: Yes.
• 43:45
  SPEAKER_03: It's the R system.
• 43:46
  SPEAKER_00: And then the third component I had this particular class in invent or come up with a new app or technology for healthcare that had never been used before, and to create a website.
• 44:00
  SPEAKER_00: Showing the pros and cons of this particular application.
• 44:04
  SPEAKER_00: And for that, they may use AI freely and particularly to make the website.
• 44:11
  SPEAKER_00: And that for me captures reaches a right balance that on the ethical issues, they are required to think, but on getting help in making a website or doing imagination and developing a product or a tool, they can brainstorm using the Gen AI.
• 44:30
  SPEAKER_00: So I have a little bit of each.
• 44:33
  SPEAKER_00: So that's 30, 30, 30.
• 44:35
  SPEAKER_00: And then whatever area they're the strongest in, I give them a 10% bump for that area.
• 44:41
  SPEAKER_03: That sounds a very practical way of approaching the problem.
• 44:45
  SPEAKER_03: You know, a lot of lawyers now are complaining about their clients' use of AI in connection with the engagement.
• 44:52
  SPEAKER_03: Do you have any advice for lawyers, practicing lawyers, to help them manage the attorney-client relationship now that uh generative AI has entered the scene?
• 45:05
  SPEAKER_00: Well, number one, I think you need to have a discussion with your client because the case law has been a little bit inconsistent about how protected that work is that that client is doing if it's not directed by an attorney or it's not something in preparation for litigation, that they may end up having to produce that.
• 45:30
  SPEAKER_00: There may not be work product protection for it.
• 45:33
  SPEAKER_00: So that's one thing.
• 45:34
  SPEAKER_00: I think the other issue, which happens more with uh less sophisticated clients, is they've had a conversation with ChatGPT, who has told them they have a great case and they should get a million dollars.
• 45:48
  SPEAKER_00: And then they come to you and you say, Well, here's a problem and there's a problem, and maybe we ought to settle this for 20,000.
• 45:57
  SPEAKER_00: And they're like, No, they feel very betrayed by you, and you don't know what you're talking about because they've done the research and they've been told a completely different story.
• 46:08
  SPEAKER_00: So that's a bit of a challenge, I think, for lawyers.
• 46:11
  SPEAKER_02: I I would challenge that with if that attorney could say to their potential client, go back and give these notes to your to whatever you were talking to.
• 46:22
  SPEAKER_02: It's gonna give them, of course, yes, this may be a real, it's gonna tell them the reality.
• 46:27
  SPEAKER_02: You know what I mean?
• 46:28
  SPEAKER_02: It does that all the time.
• 46:30
  SPEAKER_02: That's that's so.
• 46:32
  SPEAKER_03: But you're saying the I you're saying the AI is going to tell the client the reality?
• 46:36
  SPEAKER_03: I think that I think that the AI may say, you're right.
• 46:38
  SPEAKER_02: Your attorney is probably giving you real life examples of cases in your area.
• 46:43
  SPEAKER_02: And let me look these up.
• 46:45
  SPEAKER_00: Well, there's actually one lawsuit where that didn't happen.
• 46:48
  SPEAKER_00: Where the test, where it said you're getting snowed by your attorney and dump him.
• 46:57
  SPEAKER_00: You know, the the you know, hopefully, if you put in additional facts, then the AI would say, oh, now taking these into consideration, I would modify my view.
• 47:08
  SPEAKER_03: You know, the last time I checked in on that case, it was early September, and they were still wrangling over the motion to dismiss.
• 47:17
  SPEAKER_03: Do you know how has it progressed beyond that point?
• 47:20
  SPEAKER_00: Do you know?
• 47:20
  SPEAKER_00: I'm I'm not sure.
• 47:21
  SPEAKER_00: I haven't uh followed it recently.
• 47:23
  SPEAKER_00: Do you know, Heather?
• 47:24
  SPEAKER_02: No, but I would say, I mean, come on, look at the position the courts are in here.
• 47:29
  SPEAKER_02: Look at the precedent that case is going to be setting.
• 47:32
  SPEAKER_02: It's an impossible case to even, I think, see a disposition on now.
• 47:37
  SPEAKER_03: Well, it's just like the cases where these uh frontier models are being sued for copyright infringement because the when they were building out the AI LLMs, they fed everything in the universe into it, and that included copyrighted materials.
• 47:57
  SPEAKER_02: Many of those have settled.
• 47:58
  SPEAKER_02: Many of those have settled.
• 47:59
  SPEAKER_03: Well, they'll they'll settle, but we've got to be able to do that.
• 48:01
  SPEAKER_03: Many of those are settling.
• 48:02
  SPEAKER_03: Do you do you think that do you think that's gonna move the needle one way or the other?
• 48:08
  SPEAKER_00: I think artists and writers probably should be compensated in some fashion.
• 48:14
  SPEAKER_00: I don't think that big tech should get 100% of the benefit of having trained on people's life creations and and work.
• 48:25
  SPEAKER_00: I think there should be some giving back in some way.
• 48:28
  SPEAKER_00: Exactly how we do that, I'm not sure, but smarter people than me could figure out how to spread some of the wealth a little bit better.
• 48:37
  SPEAKER_03: You know, this has been a fascinating conversation, and we could keep going on and on, but it is now time for the practice signal segments of our show.
• 48:49
  SPEAKER_03: And this is a segment where we read from lawyers in the wild who have questions or problems, and we consider them.
• 48:59
  SPEAKER_03: They always come from Reddit.
• 49:00
  SPEAKER_03: We consider them and whether or not AI will help them or whether AI can't help them.
• 49:06
  SPEAKER_03: And today's practice signal comes from a lawyer preparing to launch a solo litigation practice.
• 49:15
  SPEAKER_03: And he writes, I'm launching a solo litigation practice and toying with the idea of adding a basic transactional practice like business contract drafting.
• 49:27
  SPEAKER_03: So it could be like vendor contracts, employment contracts, partnership contracts, et cetera.
• 49:33
  SPEAKER_03: I have absolutely zero background on contract drafting, red flag.
• 49:40
  SPEAKER_03: So it could be like, so, and in law school, I took mostly litigation-related cases that only have litigation practice experience.
• 49:49
  SPEAKER_03: He doesn't ask, or she doesn't ask, where do I start?
• 49:53
  SPEAKER_03: You know, is this a good idea?
• 49:55
  SPEAKER_03: They ask, where do I start learning it?
• 49:58
  SPEAKER_03: Apparently they've already made the decision.
• 50:00
  SPEAKER_03: Are there platforms that provide templates?
• 50:03
  SPEAKER_03: Also, where do I start learning things related to contract drafting, like knowing what is enforceable, what is not?
• 50:10
  SPEAKER_03: Laura, what advice do you have for this lawyer?
• 50:14
  SPEAKER_00: This is a little problematic because we know our ethics rules say you can take on a matter that is outside your area of expertise, but you either need to get up to snuff or you need to associate with somebody else who's up to snuff.
• 50:31
  SPEAKER_00: And I'm not hearing either of those things in this person's question.
• 50:37
  SPEAKER_00: So I don't think it's necessarily a problem if they use the LLM to help draft leases or contracts or things like that.
• 50:48
  SPEAKER_00: But they need to have the knowledge and the skill to be able to read it afterwards and know what's a problematic uh clause and what isn't.
• 50:59
  SPEAKER_00: And if you don't know anything about corporate law, I would imagine that's a problem.
• 51:02
  SPEAKER_02: Heather, what do you think?
• 51:04
  SPEAKER_02: I I love the palm to the forehead.
• 51:06
  SPEAKER_02: Yes, it is a good reaction, but I would say my first thought as soon as you read it, Ron, was they just need to hire an associate that that does transactional law, who is, who does have that expertise to bring into their firm to handle that portion.
• 51:19
  SPEAKER_02: You know, to me that would be the answer.
• 51:21
  SPEAKER_02: But yes, of course, becoming competent, which I feel like hiring an associate who is experienced in that area would be one way to help you become competent.
• 51:30
  SPEAKER_02: It could be a mentor as well as your partner or associate.
• 51:34
  SPEAKER_03: Well, that's true.
• 51:36
  SPEAKER_03: Get help, get human help.
• 51:38
  SPEAKER_03: This is one of those areas where, oh, I'm the guy who always wants to figure out a way to get to yes.
• 51:43
  SPEAKER_03: You know, yes, if was a thing we talked about a lot earlier, but not in this case.
• 51:48
  SPEAKER_03: And if you start using AI to try to review these contracts and give you an idea and get you up to speed, open AI is not going to pay your malpractice deductible.
• 51:59
  SPEAKER_03: They're not going to give you a slime towards that, but they will very confidently give you wrong information about the contracts that you're trying to address.
• 52:11
  SPEAKER_03: So I would lean pretty strongly into Heather's advice here, which is get help, get human help.
• 52:18
  SPEAKER_02: And we talk about constantly the the need for human judgment and verification.
• 52:21
  SPEAKER_02: If you're not competent to verify the contract or the information, you you are not, you know, meeting your duty to the court or to your clients.
• 52:30
  SPEAKER_03: So, Mara, before we go, any closing thoughts about what lawyers are doing right or doing wrong about AI and what they should be thinking about?
• 52:44
  SPEAKER_00: I think the one of the most important things is not to just say, I need to go get me some AI, but to figure out where you have a problem that can be automated.
• 52:57
  SPEAKER_00: So the AI shouldn't come first.
• 52:59
  SPEAKER_00: The problem should drive the solution.
• 53:02
  SPEAKER_00: And I I see lawyers running to get AI because other people have AI, but what do you need a tool that does bankruptcy if that's not what your practice is?
• 53:13
  SPEAKER_00: So figure out where your pain points are and what could be automated better or made more efficient in your own system, and then go look for the right tool to do that.
• 53:27
  SPEAKER_03: Mau, thank you so much for being here.
• 53:30
  SPEAKER_03: I'm gonna make an Instagram clip of your face palm moments because I I really think it it covered the waterfronts.
• 53:39
  SPEAKER_03: I really appreciate your being here.
• 53:42
  SPEAKER_03: Uh, thank you for joining us.
• 53:43
  SPEAKER_03: This has been a great conversation.
• 53:44
  SPEAKER_00: Thank you, Maura.
• 53:45
  SPEAKER_00: Thank you for having me.
• 53:47
  SPEAKER_03: Well, thank you all for joining us.
• 53:49
  SPEAKER_03: This has been another episode of AI Tools for Practicing Lawyers.
• 53:53
  SPEAKER_03: We'll see you next time.
• 53:55
  SPEAKER_01: That's it for today's episode of AI Tools for Practicing Lawyers.
• 54:00
  SPEAKER_01: Thanks for listening.
• 54:01
  SPEAKER_01: We'll see you next time.