Latest / AI Tools for Practicing Lawyers / Field Notes: Confession of an AI Hallucinator
Transcript
- 0:03SPEAKER_00: Welcome to AI Tools for Practicing Lawyers.
- 0:07SPEAKER_00: Practical, no-nonsense guidance to help attorneys put AI power to work in their practice right now.
- 0:16SPEAKER_01: Hello and welcome to a field note episode of AI Tools for Practicing Lawyers.
- 0:23SPEAKER_01: I'm delivering this while Heather is on a much deserved vacation.
- 0:27SPEAKER_01: She'll be back next week and we'll record a whole new episode.
- 0:32SPEAKER_01: But today I do, I have a confession.
- 0:35SPEAKER_01: It's it's 2022, and this amazing tool has just hit the scene.
- 0:41SPEAKER_01: It's called ChatGPT.
- 0:43SPEAKER_01: And supposedly it could do all these amazing things for free.
- 0:48SPEAKER_01: So of course, I download it.
- 0:51SPEAKER_01: I'm an early adopter in most technologies.
- 0:54SPEAKER_01: I take a look at it, not quite sure what to do with it.
- 0:57SPEAKER_01: And one day, a uh I get hired by a client who owns a home with his spouse, and they're owned tenants by the entireties.
- 1:06SPEAKER_01: They're a little nervous that they might lose the house in the bankruptcy that we're going to file for them.
- 1:12SPEAKER_01: So I explain, no, it's tenants by the entireties.
- 1:15SPEAKER_01: As long as you and your wife don't share creditors, you'll be okay.
- 1:20SPEAKER_01: And he asked me to put together uh a little memo that says that, which is, you know, a reasonable thing for a client to do.
- 1:27SPEAKER_01: But even though it is reasonable, I kind of don't really feel like doing it because it's just one more thing that I have to add to the top of my pile of things I got to do.
- 1:37SPEAKER_01: So I said, wait a second, supposedly, you know, ChatGPD is good at this kind of thing.
- 1:43SPEAKER_01: So I enter a prompt.
- 1:45SPEAKER_01: I didn't even know what a prompt was in 2022, and I said, write me uh a, you know, a memo about tenants by the entireties in Maryland.
- 1:55SPEAKER_01: And seconds later, it spits out this memo.
- 2:00SPEAKER_01: And it is beautifully written.
- 2:02SPEAKER_01: It's I could not have done a better job.
- 2:05SPEAKER_01: And there are these great citations that support the doctrine.
- 2:10SPEAKER_01: And I look at the cases, and I think, you know, listen, I know tenants by the entirety's cases in Maryland.
- 2:17SPEAKER_01: I've been doing this for 35 years in Maryland.
- 2:20SPEAKER_01: So I thought, okay, you know what, I'll just send it along because this is a low-stakes kind of project, just so that the client has a little bit of a comfort level.
- 2:32SPEAKER_01: But I did have that little naggling feeling that I need to really look at these cases.
- 2:39SPEAKER_01: Maybe there are new cases that I that I haven't seen before.
- 2:43SPEAKER_01: So I look for the cases, I can't find the cases.
- 2:47SPEAKER_01: And I'd already sent the client this memo.
- 2:50SPEAKER_01: So I very quickly rewrite the memo just to add the cases that I did know, which I did look up to make sure there was still good law.
- 2:59SPEAKER_01: And I sent it to the client and I said, you know, disregard the prior memo.
- 3:02SPEAKER_01: This is the real memo.
- 3:03SPEAKER_01: And that's the end of it.
- 3:05SPEAKER_01: That was the that's the whole confession.
- 3:07SPEAKER_01: I got lucky.
- 3:09SPEAKER_01: I got lucky because it was such a low-stakes matter that I wasn't gonna get into any trouble.
- 3:17SPEAKER_01: But wait a second, this amazing tool is making up cases.
- 3:22SPEAKER_01: This is crazy.
- 3:23SPEAKER_01: How's anybody ever gonna use this tool?
- 3:25SPEAKER_01: So, you know, I understand why lawyers use ChatGPT to do legal research and write briefs.
- 3:34SPEAKER_01: First of all, the output looks great.
- 3:37SPEAKER_01: You know, the it's well written, the cases are nicely interwoven in the text, and really the output looked just too good to question.
- 3:49SPEAKER_01: So if I didn't know, or at least suspect, that those tenets by the entirety's cases were wrong, you know what, I probably would have just let it go and never thought twice about it.
- 4:01SPEAKER_01: So I think that's one very reasonable reason why lawyers allow hallucinated cases to creep into their work product.
- 4:10SPEAKER_01: But there's a second problem with this whole hallucination, which is that AI doesn't always hallucinate.
- 4:20SPEAKER_01: It only sometimes hallucinates.
- 4:23SPEAKER_01: You know, you I especially like if you're using Claude or Gemini, they try to show sources, but they hallucinate too.
- 4:30SPEAKER_01: They all do, but they don't always.
- 4:32SPEAKER_01: And because they don't always, it is possible and maybe even easy to fall into a false sense of security about using these tools.
- 4:46SPEAKER_01: So so I mean, now let's think about a scenario that probably comes up all the time because you know, there's hundreds of situations where lawyers have been caught.
- 4:59SPEAKER_01: And by the way, I don't think they're caught every time, right?
- 5:03SPEAKER_01: They may not be caught if emotion's unopposed.
- 5:07SPEAKER_01: That happens all the time with lawyers, or the case settles ahead of time.
- 5:12SPEAKER_01: You know, that might happen.
- 5:14SPEAKER_01: There's plenty of reasons why you wouldn't catch a hallucinated case.
- 5:18SPEAKER_01: So you're writing a brief, it's due at midnight, it's 1150, and you finish your brief.
- 5:26SPEAKER_01: All right.
- 5:27SPEAKER_01: And now we're gonna talk about the magic word that, in my opinion, is not a magic word, and that's verification.
- 5:36SPEAKER_01: All right.
- 5:37SPEAKER_01: So you got a few minutes before midnight, and you've written your brief in Chat GPT or Claude or whichever tool you want to use, and it comes out and it's beautiful, and you say, wait a second, I better verify because they're hallucinations.
- 5:53SPEAKER_01: And here it is, by now it's 1156, and you and you run your verification process, however you do it, and you learn that these are hallucinated cases.
- 6:05SPEAKER_01: Well, you know, what are you gonna do?
- 6:07SPEAKER_01: You got you got minutes to go before your deadline.
- 6:10SPEAKER_01: So what are your options?
- 6:11SPEAKER_01: File anyway, and hope no one checks, in which case you're not gonna sleep for a few weeks, fearing that you'll be discovered and there'll be sanctions.
- 6:20SPEAKER_01: You pull the citations, maybe file a weaker brief.
- 6:23SPEAKER_01: I'll tell you what I would do at that point.
- 6:26SPEAKER_01: I would write the nastiest, uncleanest motion to extend for 24 hours, file it before the deadline, and then within the next 24 hours, I would go ahead and I would file the brief, and you'll probably be okay, but what a crappy way of dealing with that problem.
- 6:46SPEAKER_01: So, so you did your verification, but it didn't really help you.
- 6:50SPEAKER_01: It helped you avoid getting sanctions, but it didn't really help your workflow.
- 6:55SPEAKER_01: So, in my opinion, verification just isn't enough.
- 7:00SPEAKER_01: Verification is the floor, it's not the ceiling.
- 7:05SPEAKER_01: So then, all right, so courts are coming up with a different solution.
- 7:10SPEAKER_01: There, they require an attestation, some sort of little blurb that the lawyer is going to sign towards the end of the brief, where the lawyer attests that they verified all the citations and they're good citations.
- 7:26SPEAKER_01: And they do that through like some sort of administrative order.
- 7:30SPEAKER_01: They'll probably haven't gotten so far as to change the local rules, but they might.
- 7:35SPEAKER_01: But you know, plenty of lawyers aren't aware of these general administrative orders or even changes to the local rules.
- 7:43SPEAKER_01: So they go ahead and they file their brief with the bad cases.
- 7:47SPEAKER_01: And the clerk, this is done by online ECF or whatever you know, your state court online process is.
- 7:54SPEAKER_01: The court kicks it, says, no, this is a deficiency because you haven't signed the attestation.
- 8:00SPEAKER_01: So the lawyer scrambles a little bit, finds a form of attestation, you know, plugs it into the back of the brief, and then refiles.
- 8:08SPEAKER_01: And then the clerk sees that the attestation is there.
- 8:11SPEAKER_01: So the clerk says, okay, I'll accept that document for filing.
- 8:15SPEAKER_01: But the lawyer hasn't changed anything.
- 8:17SPEAKER_01: All they did was add the attestation because that was the deficiency.
- 8:20SPEAKER_01: They fixed the deficiency, and now the document is in the stream of court filings.
- 8:26SPEAKER_01: So then what happens?
- 8:28SPEAKER_01: The judge or the clerk of the court looks and sees, all right, the attestation is there.
- 8:34SPEAKER_01: They don't double check.
- 8:36SPEAKER_01: So, you know, maybe the other side doesn't double check, because that happens too, right?
- 8:42SPEAKER_01: You know, lawyers trust other lawyers' work product.
- 8:45SPEAKER_01: Or, you know, there's a string site and the string site contains some hallucinated cases.
- 8:51SPEAKER_01: You know, lawyers, they're not gonna read the string site for, especially if it's uh legal doctrine that is well known, you're not gonna check those string sites.
- 9:04SPEAKER_01: So then there are contaminated cases in the in the court flow.
- 9:11SPEAKER_01: And the attestation failed.
- 9:14SPEAKER_01: So to me, that that's not the answer.
- 9:16SPEAKER_01: And you know, at least two cases, and I'm gonna put these sites to these cases in the show notes, at least on the website, at least two cases, the judge, probably the judge's clerk, adopted the brief of the prevailing lawyer and just planted it back into the opinion.
- 9:40SPEAKER_01: And so the opinion is now contaminated with bad cases.
- 9:45SPEAKER_01: So this is what happens, and it goes up on appeal, and then the appellate courts get really upset.
- 9:51SPEAKER_01: And you know, there's there are cases where lawyers are sanctioned$5,000, which, in my opinion, is excessive.
- 10:02SPEAKER_01: I mean,$1,000 is going to make the point.
- 10:05SPEAKER_01: $5,000 because you're relying upon what's becoming a commonly misused legal tool that that are has made its way into the general public's usage.
- 10:19SPEAKER_01: I think that's excessive, but I don't know all the facts of these cases.
- 10:23SPEAKER_01: And in the Sixth Circuit case, they they they fine the lawyers$15,000, costs and the attorney's fees of the other party.
- 10:34SPEAKER_01: So that was a severe, severe sanction, and it was referred to bar counsel for disciplinary action.
- 10:43SPEAKER_01: So, you know, then you gotta another question.
- 10:46SPEAKER_01: And by the way, I like I said, I think there are thousands and thousands of these situations, most of which just haven't been caught.
- 10:53SPEAKER_01: So if you're listening to this and you're considering adding AI to your workflow, I think it would be reasonable for you to ask, why the hell am I gonna use AI if it's just gonna get me into hot water?
- 11:06SPEAKER_01: And there's a reason.
- 11:08SPEAKER_01: And this is where I get into what I call the yes if.
- 11:14SPEAKER_01: And I came up with yes if because, like a petulant child, I hate to be told no.
- 11:20SPEAKER_01: Don't tell me no.
- 11:22SPEAKER_01: You know, I want to do this thing.
- 11:24SPEAKER_01: So I came up with yes if.
- 11:26SPEAKER_01: Yes, you can do this thing if you follow the right protocols.
- 11:31SPEAKER_01: And, you know, I have this green light, yellow light, red light framework that I think helps translate the whole concept of governance for lawyers.
- 11:43SPEAKER_01: I mean, for example, let's say you're gonna use AI for client communications, kind of like what I used it for.
- 11:49SPEAKER_01: First drafts, follow-up, status updates.
- 11:53SPEAKER_01: AI is great for that.
- 11:54SPEAKER_01: AI is phenomenal at coming up with a first draft of things.
- 12:00SPEAKER_01: That's green light all the way.
- 12:02SPEAKER_01: You know, marketing and content.
- 12:04SPEAKER_01: You want to you want to create a newsletter, you want to do blogging, you want to write scripts for your videos.
- 12:10SPEAKER_01: In fact, you know, the AI tools are out there where you don't even have to film your videos, you can do them all on AI.
- 12:18SPEAKER_01: Social media posts, bio updates, you want to do a panel and you need to do a CLE paper.
- 12:24SPEAKER_01: Green light all the way.
- 12:26SPEAKER_01: Uh so you know, you want to do employee handbooks, vendor agreements, meeting summaries.
- 12:32SPEAKER_01: I just had a call with a good friend of mine who's a Flintstones level lawyer all the way, who was irritated by note-takers on um on Zoom calls.
- 12:44SPEAKER_01: And I said, why?
- 12:45SPEAKER_01: He says, I just don't like him.
- 12:46SPEAKER_01: I said, why?
- 12:48SPEAKER_01: And as it turned out, I converted him.
- 12:51SPEAKER_01: And now he's accepting and actually I think will turn out to be enthusiastic about using note-takers on Zoom calls.
- 13:00SPEAKER_01: Yeah, and you know, as long as you're being transparent, as long as everybody knows that the call is being recorded, as long as everybody knows that the note-taker is there and scribbling everything down and summarizing it and pushing out, you know, action items, it's a phenomenal tool.
- 13:17SPEAKER_01: Really, every lawyer should be using AI for note-taking.
- 13:21SPEAKER_01: Green light summarizing and organizing.
- 13:24SPEAKER_01: You know, you want to take a deposition, you know, stick the deposition in there.
- 13:29SPEAKER_01: Document review, exhibit organization, green light all the way.
- 13:34SPEAKER_01: Now, but there's other stuff, like, for example, discovery preparation, you know, draft interrogatories, request for uh admissions, write out your deposition questions.
- 13:45SPEAKER_01: It's great for that.
- 13:46SPEAKER_01: But you know, you want to double check that.
- 13:48SPEAKER_01: You don't want to just go into your deposition code where the only preparation you've done is ask AI to come up with 30 depot questions.
- 13:55SPEAKER_01: Brief drafting structure.
- 13:58SPEAKER_01: Uh, just, you know, you want to write a brief and say, you know, give me your first draft, yellow light all the way, because you want to make sure that that draft says what you want it to say.
- 14:09SPEAKER_01: But, you know, nine times out of a 10, you're gonna, you're gonna be very happy with that outcome.
- 14:15SPEAKER_01: Now, red light, case citations and filings, for sure, that's red light.
- 14:21SPEAKER_01: Red light, if you've got a consumer level version of Chat GPT, which is like the free version or the 20 bucks a month version, same thing with Claude, same thing with Gemini.
- 14:31SPEAKER_01: You know, you don't want to upload confidential client data because all three of those tools do use your inputs to train the language module for future uses.
- 14:46SPEAKER_01: That's why you probably want to go up a tier to a slightly more enterprise version.
- 14:51SPEAKER_01: I mean, it's 50 bucks a month instead of 20 bucks a month.
- 14:54SPEAKER_01: You don't want to upload client personal identifying information to a consumer-level AI tool.
- 15:01SPEAKER_01: You don't want to put social security numbers, bank statements with account numbers, and the like.
- 15:07SPEAKER_01: And you got to be careful because the truth is ChatGPT does a very nice job redacting that kind of information.
- 15:16SPEAKER_01: So you want to make sure that if you're uploading the raw data, you're uploading it to a secure platform.
- 15:23SPEAKER_01: So, yes, if there's a great things, your practice will improve if you use AI in a careful way and you do your green light, yellow light, red light analysis.
- 15:40SPEAKER_01: So, you know, I think that part of the reason why lawyers might use a consumer-level AI tool is because it's very inexpensive, especially compared to Westlaw or Lexis or Bloomberg.
- 15:58SPEAKER_01: You know, FastCase is free for most Bar Association members, but its user interface does leave something to be desired.
- 16:08SPEAKER_01: So it's possible that a lawyer might think, you know what, I can just save some money using AI instead of what are these expensive tools.
- 16:19SPEAKER_01: So that's a mistake.
- 16:21SPEAKER_01: And I'm going to talk about what is, in my opinion, a bright line rule for using this tool to do drafting.
- 16:34SPEAKER_01: And that is use the established tool first.
- 16:41SPEAKER_01: Do your research in Westlaw or in Lexis or in Bloomberg or in FastCase.
- 16:48SPEAKER_01: Get your cases, verify them ahead of time, you the old-fashioned way, right?
- 16:54SPEAKER_01: You can dip your toe in the Flintstone's water again.
- 16:57SPEAKER_01: It won't kill you.
- 16:59SPEAKER_01: Then take your verified cases, upload them into your consumer tool at this point, and tell the tool in your prompt that you want to constrain its analysis to only the authorities you are uploading and no others.
- 17:20SPEAKER_01: Because even if a tool has access like to a state law database or through Cornell or something, just because they have it available to them doesn't mean that they use it.
- 17:31SPEAKER_01: They could still hallucinate even though they have access to the proper tool.
- 17:36SPEAKER_01: And why they hallucinate is beyond the scope of this field note.
- 17:40SPEAKER_01: Just know that they do hallucinate.
- 17:43SPEAKER_01: If you do your legal research up front and you've got your citations verified, I think you'll be very happy with the first draft that the AI tool spits out.
- 17:55SPEAKER_01: Now, you're still going to have to redraft that.
- 17:57SPEAKER_01: You're still going to have to add and organize.
- 18:00SPEAKER_01: And you know what you could do is you could print it out, handwrite your own notes, 20th century style, scan them, and then upload them back into the tool, and the tool will make your handwritten revisions for you.
- 18:15SPEAKER_01: You may have to do a couple of iterations, but it will elevate your work product.
- 18:21SPEAKER_01: And to me, it's not negotiable.
- 18:24SPEAKER_01: You've got to use an established, credible legal database.
- 18:29SPEAKER_01: Either use it, or, you know, if you're going to use AI and you don't want to use this workflow, make sure that the tool you're using is connected and will draw from established legal databases.
- 18:45SPEAKER_01: There are plenty of them.
- 18:46SPEAKER_01: There's Harvey, there's Lagora.
- 18:49SPEAKER_01: There are plenty of them out there that do that.
- 18:52SPEAKER_01: But be advised that if you are going to do that, you still need to check because AI will hallucinate, even if it has an established legal database available.
- 19:06SPEAKER_01: But at least you know that you're working off an established legal database, and it will be much easier to verify the cases afterwards.
- 19:16SPEAKER_01: So this is my field note about my experience as a lawyer with hallucinated cases.
- 19:25SPEAKER_01: All right, now let's switch gears and let's talk about the practice signal segment of this field note, which is how lawyers in the wild are experiencing pain points in their practice and how AI might be able to help them.
- 19:46SPEAKER_01: This is a Facebook group message by an attorney in the bankruptcy group that I spend some time on.
- 19:58SPEAKER_01: And this attorney says, I am looking at getting back into FCRA work.
- 20:04SPEAKER_01: Anyone else doing this?
- 20:05SPEAKER_01: I would love to toss some ideas around.
- 20:08SPEAKER_01: I'll repeat my mantra, which is go ahead, post these on lawyer groups and get information from lawyers.
- 20:15SPEAKER_01: There's no downside to doing that.
- 20:17SPEAKER_01: Absolutely go ahead and do that.
- 20:19SPEAKER_01: But this is a great practice signal for using AI.
- 20:24SPEAKER_01: This lawyer is considering re-entering Fair Credit Reporting Act work and is looking for ideas and peer input.
- 20:31SPEAKER_01: So the question is, what does this practice look like now?
- 20:35SPEAKER_01: And this is a huge AI angle that's hiding in plain sight.
- 20:38SPEAKER_01: First of all, FCRA, if done right, I mean, doing the one-off case here and there is probably not going to make you a whole bunch of money and advance your practice.
- 20:49SPEAKER_01: But if you have some high volume, AI's perfect for that.
- 20:53SPEAKER_01: If it's document-heavy with credit reports and disputes and letters, AI processes those documents.
- 20:59SPEAKER_01: There's pattern recognition.
- 21:01SPEAKER_01: I mean, you're talking about fairly form type of complaints and motions because it's a very refined niche area.
- 21:11SPEAKER_01: There are repeatable processes.
- 21:14SPEAKER_01: I mean, you could absolutely do this on the Flintstones level, which is where you manually review the credit reports and all the handwritten uh documents, et cetera.
- 21:25SPEAKER_01: You could you could do this on the Simpsons level, where there's some automation, like templates or intake forms, maybe some CRM tracking.
- 21:35SPEAKER_01: You could do some outsourcing or admin support, or where this is going, which is Jetson's level all the way.
- 21:42SPEAKER_01: AI assisted credit report analysis, automated issue spotting of trade lines and inconsistency, AI generated dispute letters and complaints, intake bots that pre qualify cases instantly, pattern detection across multiple clients, mass litigation insights.
- 22:00SPEAKER_01: You know, in other words, FCRA could become a highly systematized, semi-scalable practice.
- 22:09SPEAKER_01: So my real takeaway is the post isn't just about FCRA.
- 22:14SPEAKER_01: Lawyers are looking at familiar practice areas and asking, can I rebuild this using modern tools?
- 22:21SPEAKER_01: And the answer is yes, you absolutely can, but only if you rethink the workflow, not just the law.
- 22:30SPEAKER_01: So I'm gonna use this for my Flintstones, Simpson's, Jetsons, level up tips that I like to have at the end of every episode.
- 22:41SPEAKER_01: For Flintstones, so if you've been listening to this podcast at all, hopefully, even if you self-sort into the Flintstones level, you know how to get into one of the free level tools and just type in, give me 300 words and five bullet points on how to get back into FCRA litigation in 2026.
- 23:09SPEAKER_01: That's it.
- 23:10SPEAKER_01: That's the Flintstones level.
- 23:11SPEAKER_01: And then look at the output and decide, okay, this works for me full speed ahead.
- 23:19SPEAKER_01: Now, if you're if you're on the Simpsons level, build a simple intake form.
- 23:24SPEAKER_01: Use ChatGPT or whichever tool you want to summarize a credit report and draft a dispute letter.
- 23:31SPEAKER_01: That's it.
- 23:32SPEAKER_01: That's all you're going to have ChatGPT do.
- 23:35SPEAKER_01: But it's enough to get you started on some cases.
- 23:39SPEAKER_01: Now, if you're on the Jetsons level, create a repeatable pipeline.
- 23:45SPEAKER_01: Your intake, AI analysis, issue spotting, draft the dispute and the complaints, store sample credit reports plus successful pleadings in a structured folder.
- 23:57SPEAKER_01: Use AI with strong context.
- 24:00SPEAKER_01: You know, you've got those old cases, you're getting back into the practice area.
- 24:04SPEAKER_01: Use those to generate higher quality outputs.
- 24:08SPEAKER_01: And I think that's those are great tips if you want to get back into a practice area to help AI jumpstart that process.
- 24:19SPEAKER_01: I hope you've enjoyed this episode.
- 24:21SPEAKER_01: By the way, Claude seems to think that I should be putting a call to action at the end of these episodes.
- 24:28SPEAKER_01: I haven't done it yet.
- 24:30SPEAKER_01: So to make Claude happy, I'm going to do it.
- 24:32SPEAKER_01: If you find this content or any of the other episodes useful, go ahead and share it.
- 24:38SPEAKER_01: Talk to other lawyers, other lawyers in your firm.
- 24:41SPEAKER_01: Have them listen.
- 24:43SPEAKER_01: In any event, we will see you next week.
- 24:46SPEAKER_01: Heather will be back for another episode of AI Tools for Practicing Lawyers.
- 24:53SPEAKER_00: That's it for today's episode of AI Tools for Practicing Lawyers.
- 24:58SPEAKER_00: Thanks for listening.
- 24:59SPEAKER_00: We'll see you next time.