Latest / AI Tools for Practicing Lawyers / Episode 015: The Last Flintstones Lawyer

Transcript

Episode 015: The Last Flintstones Lawyer · AI Tools for Practicing Lawyers

0:03
SPEAKER_01: Welcome to AI Tools for Practicing Lawyers.
0:07
SPEAKER_01: Practical, no-nonsense guidance to help attorneys put AI power to work in their practice right now.
0:15
SPEAKER_00: Welcome back to AI Tools for Practicing Lawyers.
0:19
SPEAKER_00: Today's episode is a little different.
0:21
SPEAKER_00: Maybe it'll catch you by surprise.
0:24
SPEAKER_00: In just a few days, our co-host, Heather Gardner, will be speaking at the Maryland State Bar Association's Legal Summit in Ocean City as part of a panel entitled Ethics, Accuracy and Efficiency AI in Legal Writing.
0:43
SPEAKER_00: Joining Heather on that panel are Maryland attorneys Sean Koenig and Donna Mandel.
0:49
SPEAKER_00: The three of them will be discussing AI tools, legal drafting, client communication, verification, ethics, confidentiality, and the rapidly changing role of artificial intelligence in modern law practice.
1:05
SPEAKER_00: But rather than simply preview the PowerPoint presentation, I thought we'd use today's episode to explore some of the questions that practicing lawyers are actually asking.
1:17
SPEAKER_00: So if you're a lawyer who feels like you're being left behind, what should you do first?
1:24
SPEAKER_00: How do you build an AI-enabled practice without chasing every shiny new object?
1:29
SPEAKER_00: And how do you navigate confidentiality, ethics, client consent, and the astonishing pace of change without losing your mind?
1:39
SPEAKER_00: That's where we're headed today.
1:42
SPEAKER_00: Heather Gardner, Donald Mandel.
1:45
SPEAKER_00: Welcome to the podcast.
1:48
SPEAKER_00: Hi.
1:49
SPEAKER_02: Hi, thank you.
1:50
SPEAKER_00: Okay, so let's start with the Flintstones lawyer.
1:55
SPEAKER_00: And and and Donna, we have uh a framework to help us reach lawyers where they live.
2:05
SPEAKER_00: Uh we know that there are Flintstone lawyers and there are Simpsons lawyers and there are Jetsons level lawyers.
2:12
SPEAKER_00: So let's talk about a Flintstone lawyer.
2:15
SPEAKER_00: Uh they attend your panel, they go back to the office, and they say, Wow, I really am a Flintstones lawyer.
2:22
SPEAKER_00: Maybe they've used ChatGPT a couple of times, maybe they've never used it.
2:28
SPEAKER_00: Uh, but they don't really have a system.
2:30
SPEAKER_00: So what's the first thing that this lawyer should do on Monday morning when they get back into the office after hearing your panel?
2:40
SPEAKER_02: Oh wow.
2:41
SPEAKER_02: First thing they should do, well, if they're already using a like Lexis or Weslaw, which has AI integrated into it already for certain aspects of legal research and writing, if they're not using that, get with your Lexus Weslaw rep.
2:58
SPEAKER_02: Not that I'm I don't work for Weslis Weslaw or Lexus, full disclosure.
3:03
SPEAKER_02: Um, but they already have AI integrated into it in various forms.
3:07
SPEAKER_02: If they're not using Lexus or Weslaw and they're just like you said, I played around with Chat GPT a couple of times, stop playing around with Chat GPT, especially for legal work, because you need to figure out then what are you going to use the AI for?
3:25
SPEAKER_00: Okay.
3:25
SPEAKER_00: Is it so you so what I'm hearing you say is map out your workflows and identify your pain points.
3:35
SPEAKER_02: Yes.
3:36
unknown: All right.
3:37
SPEAKER_00: So Heather, when you're talking to lawyers in the wild, which I know you do daily, what kind of conversations are you having with them about how they maybe should get started with AI?
3:53
SPEAKER_03: So I do talk to them about not putting, of course, any information into an open AI tool, but I also encourage them to go ahead and bite the bullet and buy, for instance, the enterprise GPT, if that is, and that is the most commonly used, of course, LLM that we're seeing people play with, so to speak, especially your Flintstones level.
4:13
SPEAKER_03: I think Chat GPT is a great entry, if you will.
4:17
SPEAKER_03: It's very simple, it's fun.
4:19
SPEAKER_03: You can get the enterprise level at a very reasonable cost.
4:23
SPEAKER_03: It's measly compared to what we see in other legal AI.
4:27
SPEAKER_03: And I would say start using that for the few really starter things like email drafting and uh using using it for the things that are going to save you the tedious time first.
4:40
SPEAKER_03: Start, you know, helping build SOPs from take your consult notes and you know, do a follow-up email to my client, things like that that generally are tedious, time consuming, and now can be taken care of by a great program called Enterprise Chat GPT.
4:57
SPEAKER_00: So are you suggesting that the first thing the Flintstones lawyer does is get into the enterprise level?
5:06
SPEAKER_03: If they're going to put any information into it, yes, because you do not want to put any real information for your farmer clients into an open source tool.
5:16
SPEAKER_00: Okay.
5:16
SPEAKER_00: Now, the last episode that we did was actually a field note that I did called Tears of the Clown, uh with apologies to Smokey Robinson, uh, which was uh which was a an episode I've been wanting to do for a while because the tiers of AI tools are so confusing.
5:37
SPEAKER_00: Donna, are you seeing that?
5:39
SPEAKER_00: Are you seeing a lot of confusion about like the different tiers of AI tools when lawyers talk to you about it?
5:49
SPEAKER_02: Yes, because there's so many different versions of programs, such as Chat GPT, as Heather mentioned, Enterprise is one level, but there's so many different versions.
5:59
SPEAKER_02: And every single day you'll be get you'll get marketing emails from co-pilot for lawyers and this for lawyers and that AI tool for lawyers.
6:08
SPEAKER_02: That it really does, as you mentioned, it can be overwhelming as to where do I start?
6:14
SPEAKER_02: What do I do?
6:15
SPEAKER_02: You know, how is Claude different from chat?
6:19
SPEAKER_02: How is Harvey different from, you know, another tool uh that I might have gotten a marketing email on.
6:26
SPEAKER_00: Donna, what what should our listeners know about you?
6:31
SPEAKER_02: Well, I've been practicing law for over 25 years in Maryland.
6:36
SPEAKER_02: I've been an educator for over 20 of those 25 years.
6:40
SPEAKER_02: I'm happily retired from practice because I am focusing full-time on my teaching and my work at the community college of Baltimore County.
6:48
SPEAKER_02: Um, and I've jumped full force into AI and AI with my instruction, teaching our students, because I really believe we train students to be paralegals.
6:59
SPEAKER_02: They're going to be on the front lines of utilizing AI in the law office, of being the person to do the review, review the drafts that AI can produce to catch what lawyers are getting in trouble with every day.
7:16
SPEAKER_02: Hallucinated sites.
7:19
SPEAKER_00: Do you have students who have to take exams and write papers in order to get a grade at the end of the term?
7:27
SPEAKER_02: Yes, we shy away from exams.
7:30
SPEAKER_02: Um we're an ABA-approved program, and they focus on practical application, practical projects.
7:37
SPEAKER_02: So that's what we try to focus in our program on is not so much exams, but actually the students are doing the work that they would be doing in a law office.
7:47
SPEAKER_00: So do you have any concerns or protocols about students' misuse of AI in the academic setting?
7:59
SPEAKER_02: Yes.
8:00
SPEAKER_00: And what do you what have you implemented?
8:03
SPEAKER_02: Well, it's kind of an all of the above approach that we've implemented in terms of we have ways that we can monitor whether something is or appears to be AI generated.
8:14
SPEAKER_02: There's never a foolproof method.
8:17
SPEAKER_02: I've put my own documents that I wrote 20 plus years ago into AI, and it has said that they are AI generated.
8:24
SPEAKER_02: I said, Well, that's nice to know because I wrote it 25 years ago or so.
8:28
SPEAKER_02: But we we have things we can do to determine whether or not a student generated an assignment through AI.
8:35
SPEAKER_02: But actually, we're kind of we're we're switching our focus, similar to what Heather's going to talk about a little bit later about switching our your focus in utilizing AI.
8:42
SPEAKER_02: We're switching our focus from using AI as a drafting tool for students to more AI can draft the document.
8:52
SPEAKER_02: The student needs to be the one who can review it, check it, make sure the citations are accurate, make sure the cases say what AI said the cases say, and you're utilizing those cases correctly.
9:05
SPEAKER_02: So that's kind of where we're we're switching our focus a little bit.
9:09
SPEAKER_00: All right.
9:10
SPEAKER_00: So Heather, Donna just threw you under the bus saying that you were going to be talking about something uh uh a little bit later.
9:17
SPEAKER_00: So what's that all about?
9:19
SPEAKER_03: So I think what she means is, and um, I'm really excited for this year's panel because Donna and I were on an AI panel last year.
9:26
SPEAKER_03: I think it was just AI in the law office, and it was Ethics Meets Innovation, which was a really cool panel, very huge attendance, huge turnout, a lot of attorneys who did not even know where to start when it comes to AI.
9:41
SPEAKER_03: Last year I was surrounded by Donna and another legal educator who were both pretty anti-AI at the time.
9:48
SPEAKER_03: And I felt like the only one there that was a proponent of AI.
9:53
SPEAKER_03: Uh, we would spend meetings even sort of going back and forth about that.
9:57
SPEAKER_03: So this year, the tables have turned and people are more on board.
10:01
SPEAKER_03: And I think it's gonna be a more exciting discussion.
10:05
SPEAKER_03: But I think that what she means that I'm gonna talk about is the way AI has evolved and the way that the tech technology has advanced so rapidly that it's changing the way we need to use it now.
10:18
SPEAKER_00: How long until you think there are no more Flintstones lawyers?
10:22
SPEAKER_03: I think there are gonna be some dinosaurs that die on that hill.
10:26
SPEAKER_03: Like, you know, I I'm almost sure of that, as I'm sure you guys, you know, are shaking your head.
10:30
SPEAKER_03: That's and that's okay.
10:32
SPEAKER_03: If if attorneys don't want to utilize AI, they don't need to, but they do need to understand that their clients are utilizing it and using it.
10:40
SPEAKER_03: So are courts, so are lots of other people surrounding that case.
10:44
SPEAKER_03: OC, clerks, you name it.
10:47
SPEAKER_03: Anyone managing that case on any side of it is already using AI.
10:52
SPEAKER_03: So they can avoid it all they want.
10:54
SPEAKER_03: And if you really don't want to use it, don't.
10:57
SPEAKER_03: You know, I you and I both have talked about we know attorneys that say, I'm retiring soon, I don't need to do this.
11:03
SPEAKER_03: That's fair.
11:05
SPEAKER_00: I predict that by the end of this decade, there will be no more Flintstones level lawyers in AI.
11:13
SPEAKER_00: May even come before that.
11:15
SPEAKER_00: Look, I remember 30 years ago.
11:19
SPEAKER_00: Uh yeah, it was about 30 years ago, maybe 35 years ago, uh, the internet became a thing.
11:27
SPEAKER_00: And uh we were going to start uh uh electronic case filing.
11:32
SPEAKER_00: And there were older lawyers there complaining that now they had to have a computer.
11:37
SPEAKER_03: Look how long it took for every state to get on electronic filing.
11:41
SPEAKER_03: I remember it wasn't even that many years ago Maryland finally did in case files.
11:44
SPEAKER_00: But the point is they but the point is they all are now.
11:48
SPEAKER_03: Well, those that's a true, but I'm saying some are going to die before this decade is over.
11:54
SPEAKER_03: A lot of them they're gonna reach higher and or pass their practice on.
11:57
SPEAKER_03: They they aren't going to try to learn this technology, and that's okay.
12:01
SPEAKER_03: I believe the majority will, and I hope that they will leave the Flintstones level.
12:07
SPEAKER_03: I think it's going to be almost impossible to stay in a Flintstones level for very much longer.
12:13
SPEAKER_00: All right, so let's talk about the the life of a Simpsons lawyer who has fallen in love with Claude.
12:22
SPEAKER_00: They they're following all the trades, and maybe they dip their toe into LinkedIn and they see everybody's talking about great Claude, isn't it?
12:31
SPEAKER_00: Claude is great.
12:32
SPEAKER_00: And this lawyer is in love with Claude, and then they discover that Claude Consumer, which the pro, which they've been paying for, the$200 a year tier, is not safe for client information.
12:47
SPEAKER_00: Right.
12:48
SPEAKER_00: And they check it out and they say, whoa, it's gonna cost me$125 a month.
12:54
SPEAKER_00: I mean, my my Claude budget has gone from$200 a year up to like$1,400 a year.
13:02
SPEAKER_00: And they don't want to jump in with both feet there.
13:05
SPEAKER_00: What do you tell that lawyer?
13:09
SPEAKER_02: I don't think I don't think you have a choice.
13:10
SPEAKER_02: You know, it's putting confidential information in any AI program that is not secure, you are you're risking your license.
13:20
SPEAKER_02: And there's there's no amount of money that's worth that.
13:24
SPEAKER_02: I I personally feel.
13:27
SPEAKER_02: So I think that's a small investment because the whole point of using those secure systems is so you can have more access, more ability to do things, you know, for your clients and using a secure system, you're gonna make up that money in the efficiency that you're gonna gain in utilizing Claude.
13:47
SPEAKER_02: Um, I'm personally a chat GPT person, but I know a lot of people, I know a lot of people who are Claude fans.
13:53
SPEAKER_02: You're gonna gain in the efficiency and you're gonna gain in your revenue, you're gonna make all that back.
13:59
SPEAKER_02: So I think it's an in it's an initial outlay that you're gonna have to do to make to gain the security because it only takes one instance to get that call from bar council or the letter in the mail, if they still send letters in the mail.
14:15
SPEAKER_02: You know, that was the thing we were always told in law school.
14:18
SPEAKER_02: You never want to open up the mailbox and see a letter that has a return address that says bar council on it, because it's never a good letter.
14:25
SPEAKER_00: I was on the peer review committee for a little while.
14:28
SPEAKER_00: And so I would get those envelopes as part of a case that I was on, and I would freak out.
14:34
SPEAKER_00: Just getting those envelopes.
14:36
SPEAKER_00: So what you're saying, I totally buy into.
14:39
SPEAKER_00: All right.
14:39
SPEAKER_00: So, Heather, what's your take on the the Claude addict who who doesn't want to pay for Claude Enterprise?
14:46
SPEAKER_03: There are ways around it.
14:48
SPEAKER_03: You can you can you can keep your Claude Pro.
14:50
SPEAKER_03: You can put all of your confidential into your enterprise GPT.
14:55
SPEAKER_03: Claude, drafting for Claude is so head and shoulders above OpenAI.
14:59
SPEAKER_03: Amthropic has got, I don't even understand how the gap is so wide, but drafting, you want to use Claude.
15:08
SPEAKER_03: It is, again, head and shoulders above anything GPT is doing.
15:13
SPEAKER_03: But you don't need to enter a bunch of client confidential information into your Claude to draft a motion to compel or to draft the things that you need for your case.
15:23
SPEAKER_03: You can do all of that work in your GPT that is secure, that is Soft2 certified, that is HIPAA certified, all of those things.
15:30
SPEAKER_03: Then you can have it give you the output you need to give to Claude to draft what you need where there's no confidential client information included.
15:40
SPEAKER_00: I love the Claude as the cleanup hitter concept, where you do all your work in a more secure tool.
15:50
SPEAKER_00: And then when it comes time to doing that deliverable, you bring the bring the big guns over to Claude and you and you do Claude.
15:59
SPEAKER_00: But why isn't anybody talking about Google?
16:04
SPEAKER_00: Why isn't anybody talking about the$36 a month enterprise tier for Gemini?
16:10
SPEAKER_03: Because it's oh go ahead.
16:13
SPEAKER_03: No, no, no, Heather, why why because it it has there's no comparison.
16:18
SPEAKER_03: Why no?
16:18
SPEAKER_03: No, people are talking about it, but have you tried drafting in Gemini?
16:23
SPEAKER_03: I wouldn't, I wouldn't ever, I would never, ever suggest that you you really put a final draft of anything from Gemini other than an email.
16:33
SPEAKER_00: I'm sorry.
16:34
SPEAKER_00: So so here's where I think there's a nuance.
16:37
SPEAKER_00: And by the way, apparently if you use the word nuance in your legal writing, you get flagged for being AI.
16:44
SPEAKER_03: That's really awful for people in the bankruptcy industry because nuance has been one of our keywords for all the years.
16:50
SPEAKER_00: It's a great word.
16:52
SPEAKER_00: But I believe that the people who are trying to draft in Gemini aren't approaching the tool correctly.
17:01
SPEAKER_00: Because if you go into the Gemini chat box and you ask to draft something, you're not gonna be terribly happy with the output.
17:11
SPEAKER_00: But if you go into a Google Doc and you bring up the Gemini window, and then you start prompting there to populate that window, that Google Doc, you're gonna get a much better result.
17:29
SPEAKER_00: Okay, so you're getting ready to have the panel.
17:33
SPEAKER_00: The lights are coming up.
17:34
SPEAKER_00: All right, people are sitting in their seat.
17:36
SPEAKER_00: They can't wait to hear what you're gonna say.
17:38
SPEAKER_00: What's the what's the first thing?
17:40
SPEAKER_00: How do you open your panel?
17:42
SPEAKER_00: Donna?
17:43
SPEAKER_00: Donna opens it.
17:44
SPEAKER_00: Go for it.
17:45
SPEAKER_02: Oh, I'm gonna open the panel.
17:46
SPEAKER_03: You open it usually.
17:48
SPEAKER_02: I don't think we went through that yet for the panel this year.
17:51
SPEAKER_02: Um nominate Donna.
17:53
SPEAKER_02: You know, I haven't even thought Ron, I haven't even thought about how to open the panel.
17:56
SPEAKER_02: That's next week.
17:57
SPEAKER_02: I'm on today.
17:58
SPEAKER_02: Um give me some suggestions.
18:00
SPEAKER_02: Do I want to scare everybody first and say, let me go to our second slide and tell you that's still open?
18:05
SPEAKER_00: Here's here's all right.
18:07
SPEAKER_00: So what if you started off with a hot take, which is that everything we are about to tell you is going to be obsolete in 90 days?
18:15
SPEAKER_02: Unless honestly, that's how you should open it up because it you're right, it's changing so fast that you know, are you smarter than a fifth grader in AI?
18:24
SPEAKER_02: Say 30 days.
18:25
SPEAKER_00: I love that are you smart.
18:26
SPEAKER_00: All right.
18:26
SPEAKER_00: And then somebody raises their hand and says, then why should we sit through your panel?
18:33
SPEAKER_00: Heather, what's the answer?
18:34
SPEAKER_03: Because we are going to give them framework that will that will help them and help guide them no matter where AI technology is.
18:42
SPEAKER_02: Yeah.
18:43
SPEAKER_02: And what did we tell them last year?
18:44
SPEAKER_02: The lawyers who don't use AI will be replaced by the lawyers who do.
18:50
SPEAKER_03: I made the statement that I've made many times.
18:52
SPEAKER_03: I've made it on this show.
18:53
SPEAKER_03: I do believe that all legal professionals who are refusing to use AI or learn AI are going to be left behind eventually, if they really never get on board.
19:04
SPEAKER_03: Like you're saying you believe by the end of this decade there will be no more attorneys not using AI.
19:10
SPEAKER_03: So you believe it too, Ron.
19:12
SPEAKER_03: Obviously, if you're making that statement.
19:14
SPEAKER_03: I do.
19:14
SPEAKER_03: But no, I believe that you're either going to start learning this technology or you will eventually you may not be replaced, especially if you are a solo and you owe your own your own firm.
19:25
SPEAKER_03: No one can replace you, of course, but you're going to be at a serious disadvantage.
19:30
SPEAKER_00: Yeah.
19:31
SPEAKER_00: See, you know, I think that, and this came up, Heather, in our talk with Rich Rogers.
19:38
SPEAKER_00: I think that what one of the things that's going to happen is that Flintstone's clients will want to be with Flintstone's lawyers.
19:47
SPEAKER_00: Simpson's clients with Simpson's lawyers and Jetson's clients with Jetson's lawyers.
19:53
SPEAKER_00: The problem is AI is going to become so ubiquitous in our culture that there won't be Flintstone's clients anymore.
20:01
SPEAKER_00: And so Flintstone's lawyers will have to adjust.
20:04
SPEAKER_00: What do you think, Donna?
20:06
SPEAKER_00: You think that's a fair assessment, or you think I'm overstating my case?
20:09
SPEAKER_02: I think you have it right.
20:10
SPEAKER_02: There, there will always be certain clients who will stay with their old school lawyer and maybe keep paying the, you know, three, four, five hundred dollars an hour, you know,$2,000 to draft a complaint when AI can draft it in two minutes and you just have to review it and make sure it's accurate to get it filed.
20:31
SPEAKER_02: Uh, I think you're gonna see more people.
20:34
SPEAKER_02: We saw it years ago in the legal profession where clients got way savvy in reviewing bills.
20:40
SPEAKER_02: And gone were the days when you could send a$5,000 invoice to a client and they just cut you a check and here you go.
20:47
SPEAKER_02: What is this for?
20:49
SPEAKER_02: What is this line item for?
20:50
SPEAKER_02: What is 1.2 hours to do this?
20:52
SPEAKER_02: And who is this person working on my case if they aren't familiar with everyone in the law office who might be working on their case?
20:58
SPEAKER_02: Clients are getting more and more savvy.
21:00
SPEAKER_02: AI is making clients more and more savvy.
21:03
SPEAKER_02: Because as Heather said, your client is using AI, your expert witnesses are using AI.
21:09
SPEAKER_02: The game is getting raised.
21:12
SPEAKER_00: So now the title of your panel includes the phrase legal writing.
21:19
SPEAKER_03: That's what the panel is all about.
21:21
SPEAKER_00: Okay, so so what's what's a recommended workflow for a lawyer who's got a brief that they need to write on any particular topic?
21:33
SPEAKER_00: How how do you see that lawyer starting and managing the process?
21:40
SPEAKER_03: Um, if I were if I were in that attorney's office and I were working there, I would have them, of course, use a secure tool, but upload everything we have on that case.
21:49
SPEAKER_03: You know, every document the client has provided, the consult notes, uh all information that that you have, and tell the AI what you're looking to accomplish.
21:59
SPEAKER_03: Get a good Outline to start and go from there.
22:02
SPEAKER_03: You know, perhaps put that into Claude to draft whatever they need to.
22:07
SPEAKER_03: Or if you want to get your first draft from GPT and then drop that into Claude, that's also a good way to go.
22:13
SPEAKER_03: But you want to give the AI as much information as you possibly can, tell it what you want it to do, give it all of the context and instruction that you can, and get your first draft and go from there.
22:26
SPEAKER_00: Uh, you know, I did a field note some time ago based upon a Reddit thread that I really enjoyed.
22:36
SPEAKER_00: It was big law associates talking about how they use AI to draft legal pleadings.
22:47
SPEAKER_00: And the takeaway that I had was that none of them were endorsing, have AI give you a first draft, a complete first draft, and then go back.
22:58
SPEAKER_00: They were more advocating for do a couple of paragraphs at a time.
23:05
SPEAKER_00: Make sure those paragraphs are logical and make sense, and then move from there.
23:10
SPEAKER_00: Donna, what do you think about that approach?
23:13
SPEAKER_02: That's one approach to do it, to kind of train your AI program.
23:18
SPEAKER_02: Um, it's similar to if you want AI to do a document review.
23:21
SPEAKER_02: You give it a couple of documents to review, see if the review is accurate before you give it a thousand pages of discovery and say, review it.
23:31
SPEAKER_02: You know, you give it 10 pages, 20 pages, review this, give me a print, give me a summary of it, make sure it's accurate, and then, you know, feed it more information as you go along.
23:43
SPEAKER_02: So that's one way to do it.
23:44
SPEAKER_02: I also like Heather's way of give give me a first draft and let me see if the first draft is is accurate or not, and then go from there with ref refining it, fine-tuning it, making corrections.
23:58
SPEAKER_03: If you've just gotten the case, you're it's it's gonna be a really, really first draft anyway.
24:03
SPEAKER_03: You know, you're talking about if they just took a client, just took on their case.
24:07
SPEAKER_03: We know that's, you know, you haven't even gotten to discovery at that point.
24:11
SPEAKER_03: You have a lot of context still to even gain yourself before you could prepare, uh depending on what what type of motion.
24:20
SPEAKER_00: Well, you can't talk about legal writing and AI without talking about the hallucination issue.
24:30
SPEAKER_00: First question Are we in a crisis or is it overblown?
24:36
SPEAKER_00: What do you think, Donna?
24:38
SPEAKER_02: I don't know if it's I don't know if it's a crisis.
24:41
SPEAKER_02: I don't know if it's overblown.
24:43
SPEAKER_02: I really think it's I hate to use this term, but it's a little bit of laziness.
24:50
SPEAKER_02: I would tell people you need to treat AI like a first-year associate.
24:54
SPEAKER_02: No lawyer is going to take a document that a first-year associate drafted, not review it and submit it to the court.
25:03
SPEAKER_02: No lawyer should ever do that.
25:05
SPEAKER_02: You would we wouldn't, you know, that's just not that's not ethical.
25:09
SPEAKER_02: It's not professional.
25:11
SPEAKER_02: So I think these hallucinations can all be caught by someone putting eyes on that document, going into Westlaw, Lexis, Fast Case, whatever you use, and making sure the case is real, the citation is accurate, and the case stands for what the AI said it stood for, that you're using it appropriately.
25:33
SPEAKER_02: And the fact that we're getting these kinds of cases coming down that you know, we see on, you know, every couple of weeks in the news about an attorney being sanctioned or whatnot, um, including the recent case that went recent, last fall in Maryland, the case where the attorney had the appellate argument.
25:50
SPEAKER_02: Um, I use that case all the time with my students, and I say the court said it in its first page of its opinion, but for the use of AI, this would have been an unpublished case.
26:01
SPEAKER_02: And that speaks volumes.
26:03
SPEAKER_02: Sting.
26:04
SPEAKER_00: So so you think the hallucination verification process should be fairly easy?
26:11
SPEAKER_02: It has to be.
26:12
SPEAKER_02: It's eyes.
26:13
SPEAKER_02: You can ask AI to give you the citations.
26:15
SPEAKER_02: You can say, give me the citations to what you give me the hyperlinks.
26:18
SPEAKER_02: Yeah.
26:19
SPEAKER_02: Give me the hyperlinks, give me the pinpoints.
26:21
SPEAKER_02: But you you can't know that it's accurate until you check it.
26:25
SPEAKER_02: It that's where humans will never lose out to AI because you have to have a human eyes on it before it goes in, before it gets submitted to the court.
26:35
SPEAKER_02: It goes back to the old days when we would shepardize real quickly before we submitted a brief to the court or a motion just to make sure nothing changed with those cases we were using.
26:45
SPEAKER_00: We did another field note.
26:48
SPEAKER_00: 21 ways that AI can hallucinate in your legal brief.
26:53
SPEAKER_00: And I discussed this with Claude.
26:58
SPEAKER_00: Uh, in fact, Claude put out the the 21 ways after much brainstorming between Claude and ChatGPT.
27:06
SPEAKER_00: And Claude came out with a new model that supposedly is going to fix a whole bunch of the hallucinations.
27:15
SPEAKER_00: So I fed Claude the 21 ways that Claude had given me 30 days earlier.
27:24
SPEAKER_00: And I said, okay, how many of these ways is your new model going to fix?
27:30
SPEAKER_00: And Claude said, out of the 21, it's going to fix about 14.
27:36
SPEAKER_00: All right, but seven, which is but seven are really hard.
27:42
SPEAKER_00: For example, AI will change the standard of review on something where you might say the standard of review should be uh clear and convincing, but in fact, uh the AI will say preponderance of the evidence.
28:03
SPEAKER_00: So it will make that very, very subtle mistake.
28:09
SPEAKER_00: I don't think it's so easy to pick those up.
28:12
SPEAKER_00: What do you think?
28:14
SPEAKER_03: I mean, I think that hallucinations are what everyone constantly talks about, and those are not even the biggest risk.
28:20
SPEAKER_03: I think that the subtle mistakes are the much larger risk when you you can often get them to give you real citations, but then when it's quoting it, it's giving you inaccurate quotes of those citations.
28:32
SPEAKER_03: So some attorneys are checking the citations, seeing, oh, it really exists.
28:36
SPEAKER_03: This case really exists.
28:37
SPEAKER_03: But you have to check much deeper than that.
28:41
SPEAKER_03: Hallucinated citations are easy to catch with human review, and they really shouldn't be seen as a crisis or something that's that's going terribly wrong.
28:50
SPEAKER_03: These are just things that are going to happen.
28:54
SPEAKER_00: I I think it's very interesting that that the legal field and and especially big law are really pushing AI for legal writing.
29:05
SPEAKER_00: And the courts are stepping back and saying, whoa, wait a second, you're feeding me all of these polluted briefs.
29:14
SPEAKER_00: Where is it all gonna land?
29:17
SPEAKER_02: I think the courts are gonna step in.
29:19
SPEAKER_02: Um I'm surprised the Maryland Supreme Court hasn't stepped in yet to provide more guidance in the rules on the utilization of AI.
29:27
SPEAKER_02: Uh, because I can tell you from the judges I've spoken to, they're getting very frustrated.
29:33
SPEAKER_02: One, with the flood of AI-generated stuff, especially from pro se litigants.
29:38
SPEAKER_02: Sure.
29:39
SPEAKER_02: Um, but two, the briefs that motions that are inaccurate.
29:44
SPEAKER_02: Not again, some hallucinations, but also situations where, yes, the case does exist, but it doesn't stand for the proposition you you put in your motion because you didn't really read the case.
29:57
SPEAKER_02: You don't really know what the case is about.
30:00
SPEAKER_02: And they have to respond to it, their clerks have to respond to it, and it's taking a lot of time in having to deal with that on the side of the judiciary.
30:10
SPEAKER_02: And I think the judges are gonna put their foot down and say, no, we need to have some, you know, more uh stringent guidelines and the rules about the utilization of artificial intelligence, more so than just saying a human looked at it.
30:26
SPEAKER_02: Yes, I certify a human looked at it.
30:29
SPEAKER_00: I I read somewhere a source that I thought knew what they were talking about, that up to 33% of all cases discussed by AI in legal briefs contain one or other hallucinations.
30:48
SPEAKER_00: That's a lot.
30:50
SPEAKER_00: And I'm not I'm not sure.
30:53
SPEAKER_00: Yeah, I'm not we are in early years.
30:56
SPEAKER_00: We're in the early years.
30:58
SPEAKER_00: I think someday the hallucination problem will get fixed.
31:01
SPEAKER_00: I think it'll have to be fixed.
31:03
SPEAKER_03: You can you can help that yourself now.
31:05
SPEAKER_03: You can give your AI guardrails, you can give your AI constraints that will help reduce drastically hallucinated citations and inaccurate information.
31:17
SPEAKER_00: You know what, Heather, I'm gonna let that be the last word on hallucinations and legal AI because it is time for us to jump into our practice signal segment.
31:30
SPEAKER_00: And this is a segment where we scour social media to find Tales of Woe by lawyers in the wild.
31:41
SPEAKER_00: Uh, and we as a panel try to figure out whether or not AI can really help this person deal with their problem.
31:51
SPEAKER_00: So here, this one comes out of a Facebook group.
31:54
SPEAKER_00: Actually, I think it came out of the Maryland Facebook group.
31:57
SPEAKER_00: And someone wrote, can somebody help me figure out a new occupation outside of workers' compensation practice?
32:04
SPEAKER_00: Doesn't even have to be a lawyer position.
32:07
SPEAKER_00: I mean, is that something you think that a person could use AI to help them change their area of expertise?
32:22
SPEAKER_03: In some ways.
32:23
SPEAKER_03: There are a lot of ways you could.
32:25
SPEAKER_00: What how how would you do it, Heather?
32:28
SPEAKER_03: I mean, you could use your AI to help you find the certifications, classes, education that you might need to gain that expertise or to get a new job and learn that expertise.
32:40
SPEAKER_03: Uh, can you learn a lot from AI?
32:42
SPEAKER_03: Absolutely.
32:43
SPEAKER_03: Can it make you an expert in something?
32:45
SPEAKER_03: I I'm not sure about that.
32:47
SPEAKER_03: You still need practical real life skills and you don't, you know, using that knowledge in the field in order to change your expertise.
32:55
SPEAKER_00: So, Donna, are you seeing lawyers who are embracing AI because they're excited about technology, or are they embracing AI because they're exhausted and looking for a better way to practice law, or just because they feel like they gotta do it because everybody else is doing it?
33:12
SPEAKER_02: I think it's all three.
33:14
SPEAKER_02: You're gonna have some lawyers who are 100% in on the tech, and this is great, and I'm gonna use Claude and Harvey and Claude Code, and I'm gonna create, you know, ways that we can uh make the practice more efficient and economize.
33:29
SPEAKER_02: And then you've got some lawyers who are like, I don't have a choice.
33:32
SPEAKER_02: I have to figure it out because the law firm down the hall is using it and they've figured it out.
33:38
SPEAKER_02: And I've got to make myself more efficient so I can compete.
33:42
SPEAKER_02: Especially, you know, in some of the industries where you have a kind of a glut of lawyers in certain practice areas, you've got to be able to stand out.
33:52
SPEAKER_00: On that note, let's switch to our final segment, which is the Flintstones Simpsons Jetsons segment, where we try to give some sort of tips or information to help them elevate their practice.
34:08
SPEAKER_00: And so I came up this week with more resources.
34:13
SPEAKER_00: And in fact, since we're talking about legal writing, I thought for Flintstones, you should follow Dan Block, who I really want to have as a guest on this show because he is a real AI uh negative evangelist.
34:33
SPEAKER_00: Every day I'm seeing him post about more of the dangers of hallucinations in AI.
34:38
SPEAKER_00: And I think there's value in Flintstone's lawyers not being swept up in the AI tide uh without really understanding what they're looking at.
34:49
SPEAKER_00: Uh for Simpsons, I recommend you follow Ruben Hasid, who Heather has been trying to get me to look at and who I actually have been looking at.
34:59
SPEAKER_00: He's a Claude evangelist who has very user-friendly Claude tips.
35:04
SPEAKER_00: Uh, I think he's he's a great resource for Simpsons level lawyers.
35:09
SPEAKER_00: And for Jetsons, I say, you know what?
35:13
SPEAKER_00: Follow Rich Rogers, who was our last guest, because he uh my mind was blown at the end of that episode.
35:21
SPEAKER_00: And I think he's really got a lot to say about bringing systems into your practice.
35:28
SPEAKER_00: Uh Donna, any thoughts about that?
35:30
SPEAKER_02: To tell the truth, we have our own in-house AI person, an attorney who works for the college.
35:35
SPEAKER_02: Uh, I follow him because he is Claude Code to the max.
35:39
SPEAKER_02: And he comes into my office probably weekly with, I found a new thing we can do.
35:44
SPEAKER_02: I found a new way we're going to use Claude Code.
35:46
SPEAKER_02: I found a new thing.
35:47
SPEAKER_02: You know, I created this over the weekend.
35:49
SPEAKER_02: I created that.
35:50
SPEAKER_02: He's my guru.
35:51
SPEAKER_00: I love that.
35:52
SPEAKER_00: That sounds great.
35:53
SPEAKER_00: In-house guru.
35:55
SPEAKER_00: In-house guru.
35:56
SPEAKER_00: Why not?
35:56
SPEAKER_00: Heather, what do you think?
35:58
SPEAKER_03: The people you've recommended are outstanding.
36:00
SPEAKER_03: Uh, the Flintstones and Simpsons uh level, those are very easily digestible.
36:05
SPEAKER_03: Like you said, they're very accessible.
36:06
SPEAKER_03: They do take a lot of the fluff out and just give you very practical tips.
36:10
SPEAKER_03: And the Jetsons person you suggested, Rich Rogers, of course, is the person that in 2022 and 23 taught me about AI, taught me, gave me my earliest lessons on with AI, taught me to train my AI, all of those things.
36:23
SPEAKER_03: He is an absolute expert.
36:25
SPEAKER_03: But I want to say for today, for Flintstone Simpsons, Jetsons is the future is not learning to prompt or to talk to your AI.
36:34
SPEAKER_03: It's learning how to think and collaborate with your AI.
36:37
SPEAKER_03: Prompting got us from Flintstones to Simpsons, but learning to talk to it with context and judgment is what's going to take you to the Jetsons level.
36:47
SPEAKER_00: You know what?
36:48
SPEAKER_00: We're going to close on that statement.
36:51
SPEAKER_00: What day in time is your panel?
36:55
SPEAKER_03: It's Wednesday, the 10th, I believe, at 1:30.
36:59
SPEAKER_03: And we are in room 5-6.
37:02
SPEAKER_00: All right, wonderful.
37:04
SPEAKER_00: That is the Maryland State Bar Association's Legal Summit panel ethics, accuracy, and efficiency, AI and Legal Writing.
37:13
SPEAKER_00: Donna, thank you so much for being here.
37:16
SPEAKER_00: We got a lot out of it.
37:17
SPEAKER_00: It was great.
37:18
SPEAKER_00: And Heather, you I know you'll be back next week.
37:20
SPEAKER_00: Good luck with the panel.
37:22
SPEAKER_00: This is Ron Drescher for AI Tools for Practicing Lawyers.
37:26
SPEAKER_00: We'll see you next time.
37:29
SPEAKER_01: That's it for today's episode of AI Tools for Practicing Lawyers.
37:33
SPEAKER_01: Thanks for listening.
37:34
SPEAKER_01: We'll see you next time.