Latest / AI Tools for Practicing Lawyers / Episode 007: Folder Mania — When AI Comes to You

Transcript

Episode 007: Folder Mania — When AI Comes to You · AI Tools for Practicing Lawyers

• 0:03
  SPEAKER_00: Welcome to AI Tools for Practicing Lawyers.
• 0:07
  SPEAKER_00: Practical, no-nonsense guidance to help attorneys put AI power to work in their practice right now.
• 0:14
  SPEAKER_01: Every AI tool claims that it can read your folders.
• 0:18
  SPEAKER_01: And we actually tested that claim because it's a massive game changer.
• 0:23
  SPEAKER_01: If you could have AI come to you, to your folders, instead of you having to upload one document at a time, that's gonna have a massive uh elevation impact on your workflow.
• 0:35
  SPEAKER_01: So we wanted to test that claim across Claude and ChatGPT and Copilot and Gemini, and the results were all over the place.
• 0:48
  SPEAKER_01: Welcome to episode 007 of AI Tools for Practicing Lawyers.
• 0:53
  SPEAKER_01: I'm Ron Drescher.
• 0:55
  SPEAKER_01: I'm Heather Gardner.
• 0:57
  SPEAKER_01: And Heather, this episode is going to be a beast because it's so important and there are so many things going on at the same time.
• 1:09
  SPEAKER_01: So let's just dive right into it.
• 1:12
  SPEAKER_01: Here's what we're trying to accomplish today.
• 1:14
  SPEAKER_01: I grabbed a folder from a more complex than usual chapter 13 debtor that had rental properties.
• 1:25
  SPEAKER_01: She and her husband are both employed.
• 1:28
  SPEAKER_01: They have multiple jobs, so there's multiple pay stubs.
• 1:32
  SPEAKER_01: We've got Zillow reports in a 48-item folder.
• 1:36
  SPEAKER_01: There are tax returns.
• 1:38
  SPEAKER_01: My lawyer notes are in there.
• 1:41
  SPEAKER_01: So the first thing we did was we created what I called the file cabinet test.
• 1:47
  SPEAKER_01: And the file cabinet test is you connect your AI with this folder, that's a cloud folder, and you say to the AI, I want you to tell me everything that's in this folder and give me a 15-word description of every document because the AI wants to fake it.
• 2:08
  SPEAKER_01: That's a thing that people don't realize.
• 2:10
  SPEAKER_01: Heather, have you come to realize that the AI kind of wants to fake it sometimes?
• 2:15
  SPEAKER_01: Absolutely.
• 2:16
  SPEAKER_01: Right?
• 2:17
  SPEAKER_01: It's like they want to see, well, what they do is they say, well, from a relevance standpoint, I'm listening to your prompt, trying to understand what you're saying.
• 2:27
  SPEAKER_01: I'm going to, I'm going to do what I can do.
• 2:29
  SPEAKER_01: I'm going to try to do it quickly because, you know, we all have the need for instant gratification, especially with these AI bots.
• 2:39
  SPEAKER_01: And so we're going to kind of rank by relevance the items that you've given us.
• 2:45
  SPEAKER_01: And of course, for lawyers, that's death.
• 2:49
  SPEAKER_01: Because we need to know that the AI tool is looking at all of the documents.
• 2:55
  SPEAKER_01: I you and I, Heather, we had a miscommunication, which actually turned out to be a great teaching moment because I sent you a zip file of the 48 documents.
• 3:09
  SPEAKER_01: And you've got, you know, you've got the Big Daddy ChatGPT enterprise version.
• 3:16
  SPEAKER_01: And what I wanted you to do, but obviously was nowhere clear enough in this, so I wanted you to unzip the file, create a folder on your drive, and connect ChatGPT to that drive.
• 3:32
  SPEAKER_01: But that's not what you did.
• 3:34
  SPEAKER_01: Tell us what you did.
• 3:35
  SPEAKER_03: I just simply gave my uh Enterprise GPT the file and told it the instructions you'd given me, and it gave me the output.
• 3:43
  SPEAKER_03: We went over, you know, we did a column, went over everything that it gave me.
• 3:47
  SPEAKER_03: It was extremely impressive.
• 3:49
  SPEAKER_03: What it was able to do asked to read and extract and everything.
• 3:52
  SPEAKER_03: It was able to basically do almost everything.
• 3:55
  SPEAKER_03: I think that the biggest, you know, takeaway you you mentioned earlier is human oversight.
• 4:01
  SPEAKER_03: How do we know?
• 4:02
  SPEAKER_03: Like you said, the lawyer needs to know that it really looked through all these arguments.
• 4:06
  SPEAKER_03: How do you know that by going over the output and knowing what you actually gave it?
• 4:12
  SPEAKER_01: So let's back up a second so that the listeners can know what we asked of the AI.
• 4:19
  SPEAKER_01: First, we asked it to do the file cabinet test, and Chat GPT on the zip drive that you uploaded did that perfectly.
• 4:27
  SPEAKER_01: Then we asked it uh to do to create a balance sheet, to create a budget, to create a preliminary means test analysis, to create uh a creditor list, and to do a pay stub analysis subtotaled by employer and employee.
• 4:51
  SPEAKER_01: And it's really important to stress that we were not testing for accuracy of this.
• 4:59
  SPEAKER_01: We we don't know that what we got was accurate because that was really kind of beyond the scope of what we were trying to do.
• 5:06
  SPEAKER_01: What we were really trying to do is see what does the AI see when we direct it to a large group of legal files.
• 5:18
  SPEAKER_01: And when you uploaded that, I was super impressed with the output you got from Chat GPT on that zip file because, and I'm getting ahead of myself.
• 5:29
  SPEAKER_01: Microsoft Copilot won't even see zip files.
• 5:33
  SPEAKER_01: Right.
• 5:34
  SPEAKER_01: You know, and we'll we'll talk about that in a minute.
• 5:36
  SPEAKER_03: So this is a visibility question, not an AI intelligence question.
• 5:40
  SPEAKER_03: Yeah, we're we're testing something vastly different than what people are usually looking for AI to do for them.
• 5:46
  SPEAKER_01: That's right.
• 5:47
  SPEAKER_01: Because it's a it's a threshold question.
• 5:50
  SPEAKER_01: We have to know is the AI seeing what we're giving it?
• 5:54
  SPEAKER_01: And that's the point of the file cabinet test.
• 5:57
  SPEAKER_01: Because if it could tell me how many fold files there are in a folder and a 15-word description of each one, then we we have a trust level.
• 6:08
  SPEAKER_03: Right.
• 6:08
  SPEAKER_03: I mean, and think about as it evolves and later when we can really trust that it understands the nuance of bankruptcy, for instance, and really looking for red flags, you know, really having it do deeper visibility work for you that can save so much time.
• 6:24
  SPEAKER_01: Right.
• 6:24
  SPEAKER_01: Or personal injury litigation, or yeah, I mean, complex civil litigation.
• 6:30
  SPEAKER_01: You know, you know, if you're going to do a uh class action lawsuit, and here's this is a tease.
• 6:35
  SPEAKER_01: We've got on the uh roadmap, if we can find the lawyer that has done this, what I call the million document file, where you have uh document production that is a literally a million documents.
• 6:52
  SPEAKER_01: Is there an AI tool that can manage this?
• 6:56
  SPEAKER_01: Because each and every one of the AI tools, when I brought this up, said, no, we can't handle that.
• 7:02
  SPEAKER_01: We can't handle that.
• 7:03
  SPEAKER_01: And one of them, you know, ChatGPT Enterprise gave a very detailed layer-by-a-layer explanation of what you'd have to do to meaningfully use AI to analyze a million-file document production.
• 7:21
  SPEAKER_01: And that's beyond the scope of this episode.
• 7:23
  SPEAKER_01: This is this is just a tease for something on the roadmap.
• 7:27
  SPEAKER_03: Yeah, it's not about size as much as it's about access for the tool.
• 7:32
  SPEAKER_03: That's right.
• 7:33
  SPEAKER_01: So ChatGPT hit a home run.
• 7:36
  SPEAKER_01: It it produced great outputs from the zip file.
• 7:41
  SPEAKER_01: But that really wasn't what we were trying to accomplish, although we learned a lot.
• 7:46
  SPEAKER_01: We learned that you know what?
• 7:48
  SPEAKER_01: If you're if you're a solo or a small firm lawyer and you want a theory for a case, and you've got a bunch of documents, you can create a zip folder and upload it to ChatGPT, you're gonna get really good output.
• 8:02
  SPEAKER_01: And that's worth knowing.
• 8:03
  SPEAKER_01: And that's that's really useful in the best tool, right?
• 8:06
  SPEAKER_03: This is the biggest thing that we're trying to do is practical tools, you know, tools that attorneys.
• 8:11
  SPEAKER_03: So it's not so much about you know the the AI, it is about which AI tool will work for which task.
• 8:18
  SPEAKER_01: Right.
• 8:18
  SPEAKER_01: And the focus is on the task.
• 8:21
  SPEAKER_01: So, so all right.
• 8:22
  SPEAKER_01: So by the way, ChatGPT failed the test when I went back and I used the enterprise license that you shared with me.
• 8:33
  SPEAKER_01: And I I was able, it took a little bit of finagling to actually connect the Google Drive to Chat GPT, but I was able to do it ultimately.
• 8:44
  SPEAKER_01: And I asked ChatGPT to do the file cabinet test, and it failed.
• 8:50
  SPEAKER_01: It failed because not only did it not read all of the documents in the folder, it brought in documents from other folders that were part of the larger collection of files in the case.
• 9:05
  SPEAKER_01: It went outside the constraints of the folder and it summarized some of those, which is scary as hell.
• 9:13
  SPEAKER_03: So frankly, doesn't I mean and doesn't that tell you because of the way I did it and then the way that you're saying the output was for that, trying to attach the GPT to your folder rather than giving it the actual zip file, tell you if you start inside the folder, you're asking the AI, look in this folder.
• 9:31
  SPEAKER_03: If you attach the AI to your own system, you're starting outside of it.
• 9:36
  SPEAKER_03: So it has a lot more, I think, room for risk, for error.
• 9:42
  SPEAKER_03: So this was a great test, you know.
• 9:45
  SPEAKER_03: And again, AI will evolve to where what you're trying to do will be nothing to it, probably six months from now.
• 9:53
  SPEAKER_01: We're gonna talk about that very concept when we talk about Gemini, because the Gemini test was really interesting.
• 10:00
  SPEAKER_01: I want to now jump to Claude.
• 10:04
  SPEAKER_01: In everything I asked it to do.
• 10:07
  SPEAKER_01: But there's a huge limitation with Claude that we're gonna get to at the end of the Claude section.
• 10:12
  SPEAKER_01: First of all, when you connect your system to Claude, Claude does not upload your documents to its system.
• 10:20
  SPEAKER_01: It's working entirely in your environment.
• 10:23
  SPEAKER_01: So if your environment is secure, it's working in a secure environment.
• 10:29
  SPEAKER_01: So it can access, you know, personal identifying information.
• 10:34
  SPEAKER_01: It could access client confidentiality information.
• 10:37
  SPEAKER_01: The documents and the information contained in the documents are not leaving your system.
• 10:43
  SPEAKER_01: And it's all within a closed system, which is hugely valuable for lawyers and law firms.
• 10:50
  SPEAKER_01: So I got a little tired.
• 10:51
  SPEAKER_01: I went, I was sitting in a Panera on uh Sunday morning looking at these documents and doing these tests.
• 10:59
  SPEAKER_01: And I so I started to redact some of the documents.
• 11:02
  SPEAKER_01: I said, I don't feel like redacting 48 documents in a folder.
• 11:06
  SPEAKER_01: So after doing that four or five, I went to Claude and I asked Claude to redact the documents.
• 11:13
  SPEAKER_01: First, I said, now tell me about the environment.
• 11:16
  SPEAKER_01: And Claude assured me that it was not going to be leaving my secure environment, which is which is a Google Drive workspace, which is intentionally created to be secure.
• 11:26
  SPEAKER_01: And so it did a bang up job redacting the documents, which, you know, we've talked about this before, Heather.
• 11:36
  SPEAKER_01: Redaction is necessary if you're going to upload exhibits as part of a you know motion for summary judgment or any kind of case filing.
• 11:44
  SPEAKER_01: You have to redo the PII.
• 11:46
  SPEAKER_01: And Claude just did a great job.
• 11:49
  SPEAKER_01: So, you know, Claude, I salute you for that.
• 11:53
  SPEAKER_01: That was that by itself was great.
• 11:55
  SPEAKER_01: Then I ran it through the file cabinet test and it handled it perfectly.
• 12:02
  SPEAKER_01: All 48 documents, 15-word description of each document.
• 12:07
  SPEAKER_01: It was great.
• 12:08
  SPEAKER_01: Then I asked it for deliverables, and it gave me an Excel spreadsheet with the four tabs, the balance sheet and the budget, and the pay stub analysis and the uh preliminary means test analysis.
• 12:23
  SPEAKER_01: And I thought that the budget was defective because I had uploaded as part of the 48 documents my client intake notes, which had a budget.
• 12:38
  SPEAKER_01: And the clawed budget did not include groceries, didn't include gas maintenance for the car.
• 12:45
  SPEAKER_01: So, okay, I mean, that's worth considering.
• 12:48
  SPEAKER_01: That's that's item one.
• 12:50
  SPEAKER_01: Item two is it timed out on me.
• 12:53
  SPEAKER_01: It it said, it said you're you run out of system resources.
• 12:58
  SPEAKER_01: It will reset at 3 p.m.
• 13:00
  SPEAKER_01: Eastern time, and it was 11 a.m.
• 13:02
  SPEAKER_01: And I needed to finish this.
• 13:04
  SPEAKER_01: I don't know if you know the song Sylvia's Mother from the 1970s.
• 13:08
  SPEAKER_01: You know, it's where you're on a payphone and you're in the middle of talking to your lover, and you know, you really need to make this next point.
• 13:16
  SPEAKER_01: And the operator says, 25 cents for the next three minutes, please.
• 13:20
  SPEAKER_01: And if you don't have a spare quarter, you got cut off.
• 13:24
  SPEAKER_01: You're done.
• 13:24
  SPEAKER_01: That conversation's done.
• 13:26
  SPEAKER_03: You had to go get changed.
• 13:27
  SPEAKER_01: That's what it was like.
• 13:28
  SPEAKER_01: That's what it was like dealing with Claude.
• 13:31
  SPEAKER_01: So that was very frustrating for me.
• 13:33
  SPEAKER_01: I shelled out another$5 so that I could finish.
• 13:36
  SPEAKER_03: Okay, I was gonna ask, what did you do?
• 13:38
  SPEAKER_03: Did you go to another tool or did you end up paying more than what your current like plan gives you?
• 13:44
  SPEAKER_03: Okay.
• 13:45
  SPEAKER_01: I I paid the$5, I was able to complete the task, and then I shut it down.
• 13:49
  SPEAKER_01: I went back after 3 p.m.
• 13:50
  SPEAKER_01: when it reset and I had full access.
• 13:53
  SPEAKER_01: But Claude was Claude was the tool to use up to that point.
• 13:59
  SPEAKER_01: But then I was really looking forward to Gemini because I mean that's sitting right inside of the Google ecosystem.
• 14:08
  SPEAKER_01: And here's an important point ChatGPT and Claude are different kinds of tools, they're like analysis tools, whereas Copilot and Gemini are really ecosystem tools.
• 14:25
  SPEAKER_01: We're gonna do an episode on AI on a budget, and you might be locked into those ecosystems and really not have room in your budget for something more.
• 14:35
  SPEAKER_01: And so you're gonna need to rely on it.
• 14:37
  SPEAKER_01: And Gemini was interesting because Gemini has it has like the classic prompt window, right?
• 14:46
  SPEAKER_01: With the little plus sign in the corner where you can upload your documents or your content, and then you can type your prompt.
• 14:53
  SPEAKER_01: And when I did that and I tried to direct it to Google, to my Google Drive document that we created for this test, it just didn't do it.
• 15:04
  SPEAKER_01: It said, Oh, there's yeah, there's 30 documents in that folder.
• 15:08
  SPEAKER_01: I said, No, they're not, there's 48 documents in that folder.
• 15:11
  SPEAKER_01: But when I went to the folder inside Google Drive, and what's nice is you're in Drive and you're looking at your folder, you've got all your your docs, and then you can open up a Gemini prompt window within that environment.
• 15:29
  SPEAKER_01: And when I did that, it was not perfect.
• 15:33
  SPEAKER_01: It found 46 documents.
• 15:36
  SPEAKER_01: But it would not tell me how many documents it had in the folder.
• 15:40
  SPEAKER_01: But except for that, it did a very nice job on the file cabinet test of the 46 documents it was able to see.
• 15:49
  SPEAKER_01: I had to manually count them.
• 15:50
  SPEAKER_01: And then it did a very nice job on the deliverables, on the balance sheet, Gemini, the budget, et cetera.
• 15:59
  SPEAKER_01: So and I have to say, all four of the environments that we tested did a pretty good job with the deliverables until we get to the Microsoft.
• 16:09
  SPEAKER_01: And that I I want I wanted to reach in and strangle Bill Gates in that moment.
• 16:14
  SPEAKER_01: Uh but but you know, Gemini, the fact that it wouldn't tell me how many documents it has in its own folder, that that was a head scratcher.
• 16:25
  SPEAKER_03: That is surprising.
• 16:26
  SPEAKER_03: Yeah.
• 16:26
  SPEAKER_03: Very I'm surprised at that output.
• 16:28
  SPEAKER_03: Uh because, like you said, it is built right in.
• 16:31
  SPEAKER_03: It's in that ecosystem.
• 16:32
  SPEAKER_03: I would think it would be the most accurate.
• 16:34
  SPEAKER_03: And if it wasn't, which, you know, obviously it wasn't, that is that's good to know because it's like anything else.
• 16:41
  SPEAKER_03: You know, attorneys don't use the same softwares, right?
• 16:44
  SPEAKER_03: We use different softwares for bankruptcy.
• 16:46
  SPEAKER_03: We use, you know, our Jubilee, our best case, our Dex Chapter for estates are using Wealth Council, you know, IP and Trademark have their own systems.
• 16:55
  SPEAKER_03: AI, the same thing.
• 16:57
  SPEAKER_03: You have to find the tool that works for what you need it to work for.
• 17:01
  SPEAKER_01: You know, we've talked about the trust gap on this podcast.
• 17:05
  SPEAKER_01: And it's important to me because big firms, they've got all these committees that can aggregate their knowledge and their time and produce meaningful research on the available tools.
• 17:20
  SPEAKER_01: As I was going through this process, I was thinking, there's no way a small firm lawyer could do what I just did because I gave up free time and I would have had to get back to billable work, to legal work.
• 17:35
  SPEAKER_01: There's just no way I could have done the in-depth analysis that we were able to do as part of this process if we were just trying to figure out what's the right tool for us to use and how to use those tools.
• 17:50
  SPEAKER_03: And that is the best point because what's interesting is all of these tools make the same promise.
• 17:56
  SPEAKER_03: They all say that they can read these files, correct?
• 17:59
  SPEAKER_03: But you were able to do a test across multiple platforms.
• 18:02
  SPEAKER_03: How many attorneys are doing that?
• 18:04
  SPEAKER_03: None.
• 18:04
  SPEAKER_03: They can't do it.
• 18:05
  SPEAKER_03: Almost none.
• 18:06
  SPEAKER_03: AI can just understand the case in all these forms.
• 18:09
  SPEAKER_03: For attorneys, almost understanding is never enough.
• 18:12
  SPEAKER_03: They have to know, you know, that human oversight that we always come back to is so vital no matter what.
• 18:19
  SPEAKER_03: And you doing that test across different platforms speaks to that maybe better than anything we've talked about so far.
• 18:26
  SPEAKER_01: You know, it just made it so clear to me that the trust gap is a real thing to try to overcome, however you can.
• 18:36
  SPEAKER_01: All right, now let's get to copilot.
• 18:39
  SPEAKER_01: I wanted to love co-pilot.
• 18:41
  SPEAKER_01: I wanted co-pilot to be great.
• 18:43
  SPEAKER_01: And one of the reasons I wanted it to be great is I know a lot of lawyers are trapped in the Microsoft ecosystem.
• 18:52
  SPEAKER_01: Somehow, in the last century, Microsoft Word surpassed WordPerfect as the dominant word processing tool for law firms.
• 19:05
  SPEAKER_01: And nobody has challenged Microsoft in that department.
• 19:09
  SPEAKER_01: And pretty much every law firm you're gonna know, they're using Microsoft Word, they're using Excel, they're using PowerPoint when needed.
• 19:19
  SPEAKER_01: Maybe they'll they don't really want to jump into Zoom, although if you do government work and you're in federal court, you need to have Zoom, but there's Microsoft Teams, which is a decent product.
• 19:31
  SPEAKER_01: So they're they're paying for it, they're paying for it license by license, and they'd like they'd like it to really work.
• 19:38
  SPEAKER_01: And so they have co-pilot.
• 19:40
  SPEAKER_01: And copilot doesn't read your folders.
• 19:46
  SPEAKER_01: It does not.
• 19:48
  SPEAKER_01: I asked it, can you read my?
• 19:49
  SPEAKER_01: It doesn't do it.
• 19:50
  SPEAKER_01: Isn't that amazing?
• 19:51
  SPEAKER_01: So I tried to upload it.
• 19:52
  SPEAKER_01: Doesn't do it.
• 19:53
  SPEAKER_01: It has an app called Notebook.
• 19:57
  SPEAKER_01: And what you have to do is you have to upload documents to the notebook, and the AI can read from the notebook.
• 20:06
  SPEAKER_03: It's each document separately, though.
• 20:08
  SPEAKER_03: You have to upload it.
• 20:09
  SPEAKER_03: Yeah, well, you could batch upload.
• 20:11
  SPEAKER_03: Yeah.
• 20:11
  SPEAKER_02: Yeah.
• 20:12
  SPEAKER_03: But still, yeah, it's still uh it is.
• 20:14
  SPEAKER_03: I think that this is such a good point because so many are uh just using the AI tools that are in their own ecosystem, so whether that's Google or Microsoft or et cetera.
• 20:24
  SPEAKER_03: But you do have to be careful that that tool may not be the right tool for what you need done.
• 20:30
  SPEAKER_03: It's not a catch-all, it's not a one size fits all, it's not a cookie cutter.
• 20:34
  SPEAKER_01: Heather, this story gets worse.
• 20:37
  SPEAKER_01: So, okay, I'm in the notebook environment and I I want to upload 48 files.
• 20:43
  SPEAKER_01: It says no, 20 file limitation in a notebook.
• 20:49
  SPEAKER_01: Okay, you know what?
• 20:51
  SPEAKER_01: I can live with that, you know, because we know that you can upload a zip file to Chat GPT.
• 20:59
  SPEAKER_01: Let's try that.
• 21:00
  SPEAKER_01: Obviously, Copilot failed the file cabinet test because it wouldn't it wouldn't even let us have our 48 folders in the environment.
• 21:10
  SPEAKER_01: It doesn't read zip files.
• 21:13
  SPEAKER_01: It won't read a zip file, it'll read a PDF file.
• 21:17
  SPEAKER_01: So, okay, all the files in that 48 file folder are PDFs.
• 21:23
  SPEAKER_01: So I combined them all into one and I uploaded it, and Copilot crashed.
• 21:30
  SPEAKER_01: Copilot could not tolerate a PDF of that size.
• 21:34
  SPEAKER_01: No problem.
• 21:35
  SPEAKER_01: So what I did was I can I created five combined PDF files and I uploaded all five and it read them.
• 21:45
  SPEAKER_01: It had trouble with the third one, so I had to re upload it and then it read it.
• 21:49
  SPEAKER_01: And it understood them and it described them.
• 21:53
  SPEAKER_01: And it did it, and you know what?
• 21:55
  SPEAKER_01: It did a pretty decent job of explaining what was in.
• 22:00
  SPEAKER_01: In all of these files.
• 22:01
  SPEAKER_03: Did it give you the?
• 22:02
  SPEAKER_01: Then I asked it for the deliverables.
• 22:04
  SPEAKER_03: Yeah.
• 22:05
  SPEAKER_01: I asked it right for the let's go through the deliverables again.
• 22:08
  SPEAKER_01: The budget, the balance sheet, the means test analysis, the paste of analysis, the creditor list.
• 22:14
  SPEAKER_01: And it spent a lot of time doing this.
• 22:18
  SPEAKER_01: And you could see its analysis.
• 22:20
  SPEAKER_01: It shows you its work as it's doing it.
• 22:22
  SPEAKER_01: It asks you a few follow-up questions, and it's doing a pretty detailed analysis of these documents.
• 22:28
  SPEAKER_01: And it says, here's your files.
• 22:31
  SPEAKER_01: But you couldn't download the files.
• 22:33
  SPEAKER_01: There was no like blue line where you could click and download it.
• 22:37
  SPEAKER_01: And it said, well, here are the three ways you could do it.
• 22:39
  SPEAKER_01: Maybe there's a problem with the client you're using.
• 22:43
  SPEAKER_01: And I said, I'm using Chrome, the most popular browser in the world.
• 22:48
  SPEAKER_01: And it just would never give me the deliverables.
• 22:52
  SPEAKER_01: And I said, well, wait a second.
• 22:54
  SPEAKER_01: Can I download Copilot for Mac so that I can then use that environment?
• 23:00
  SPEAKER_01: And it said, no, you can't read this chat in another environment.
• 23:05
  SPEAKER_01: Like every, like, like all the other tools we're using, especially Chat GPT and Claude, you know, you sign into the web browser, you sign into the desktop app, you have it on mobile.
• 23:18
  SPEAKER_01: The conversations follow you.
• 23:21
  SPEAKER_01: But that's not the case in Copilot.
• 23:23
  SPEAKER_01: And it's described as it's like a whiteboard that is trapped inside that chat.
• 23:30
  SPEAKER_01: It will not follow you from chat to chat.
• 23:33
  SPEAKER_03: That is so interesting.
• 23:35
  SPEAKER_03: So it was really basically at first, I was going to say what they should do is come with a step-by-step list.
• 23:40
  SPEAKER_03: You had to do all of these tests and trial and error to get to how do I actually get what I need, but it didn't matter.
• 23:47
  SPEAKER_03: When you finally got what you needed, you couldn't download any of it.
• 23:50
  SPEAKER_03: It wasn't really usable for you.
• 23:53
  SPEAKER_01: What it did finally was it presented in line its deliverables so that I could select, copy, and paste it elsewhere.
• 24:06
  SPEAKER_01: You know, but that's just not a workflow that's going to make any sense.
• 24:10
  SPEAKER_03: That's so 1999.
• 24:13
  SPEAKER_01: Yes, yes.
• 24:14
  SPEAKER_01: And so I had a whole conversation with ChatGPT about the copilot fail.
• 24:19
  SPEAKER_01: And ChatGPT, this was a really useful conversation because it pushed back on what I was saying.
• 24:26
  SPEAKER_01: I pushed back on ChatGPT.
• 24:28
  SPEAKER_01: And ChatGPT remains very strong for brainstorming, in my opinion.
• 24:33
  SPEAKER_01: Even though I do think Claude is better for deliverables, I think ChatGPT is great for brainstorming.
• 24:39
  SPEAKER_01: And where we landed was Copilot is going to be really good if you're trapped in the Microsoft ecosystem and you want to tell, oh, you've got a Microsoft Word document and you say, well, I want to turn this into a presentation in PowerPoint.
• 24:55
  SPEAKER_01: Copilot's going to be great for that.
• 24:57
  SPEAKER_01: Or let's say you've got a presentation in PowerPoint and you say, well, I want to take some of the data, move it to Excel, and then create some charts, move them back to PowerPoint.
• 25:10
  SPEAKER_01: Copilot's going to be great for that.
• 25:12
  SPEAKER_01: So long as you're staying in that Microsoft environment, Copilot will be strong.
• 25:18
  SPEAKER_01: And you know what?
• 25:19
  SPEAKER_01: I'm saying this now.
• 25:20
  SPEAKER_01: We're going to test this down the road in our deep dive into copilot episode.
• 25:26
  SPEAKER_01: But that's what ChatGPT kept saying to me.
• 25:29
  SPEAKER_01: And so where I landed was you need to look at what's your pain point that you're trying to resolve?
• 25:37
  SPEAKER_01: What's the bottleneck you're trying to solve for?
• 25:40
  SPEAKER_01: The bottleneck is, you know what, I want to be able to just be able to look at a folder and have a really accurate assessment for what's in that folder, Claude is the way to go.
• 25:53
  SPEAKER_01: If you want to take a have a folder and say, I want to have a theory of the case for the clients, for the judge, and for the jury, ChatGPT is going to do great for that because it will understand the relevant levels of the different files in your folder.
• 26:12
  SPEAKER_01: So you need to circle back around to what you're doing and what you're what pain point you're trying to solve.
• 26:22
  SPEAKER_01: And it's a workflow systems analysis first and then an AI tools analysis, right?
• 26:30
  SPEAKER_01: That's that's how that's where I landed.
• 26:32
  SPEAKER_03: Do you foresee?
• 26:33
  SPEAKER_03: I mean, having this conversation is making me think of something huge.
• 26:36
  SPEAKER_03: We've recently discussed more attorneys filing, you know, filings, pleadings, et cetera, with the AI hallucinated citations, which still is happening.
• 26:45
  SPEAKER_03: I recently saw, you know, another case where an Oregon attorney was uh sanctioned$10,000 for this happening again.
• 26:52
  SPEAKER_03: And I think that the biggest risk that we're discussing when we see attorneys start using AI the way we're discussing today is what we're talking about with Copilot isn't just a fail, it's a huge risk because it doesn't just give attorneys this output.
• 27:07
  SPEAKER_03: It gives them an output that looks complete, that makes them believe that it is a complete review when Copilot is not a document review tool.
• 27:17
  SPEAKER_03: It is a more search and relevance tool.
• 27:21
  SPEAKER_01: Well, all of them.
• 27:22
  SPEAKER_01: I would say the same thing for ChatGPT and Claude.
• 27:26
  SPEAKER_03: But well, but it's it's much further limited as we've seen in real practice and in these tests that you've done.
• 27:33
  SPEAKER_03: Imagine an attorney not doing the human oversight, believing that the co-pilot did review every document.
• 27:40
  SPEAKER_03: Missing documents can change an entire outcome of a case.
• 27:44
  SPEAKER_03: We know that, right?
• 27:45
  SPEAKER_03: And yes, most people know that, but in the legal field, we really know this.
• 27:49
  SPEAKER_03: So down the road, you know, this could start to come into play where we're seeing the AI hallucinations, where we're seeing attorneys utilize it for file review where things are being missed.
• 28:00
  SPEAKER_03: We could be seeing AI malpractice cases down the road.
• 28:06
  SPEAKER_01: I want to jump into our practice signal segment, which comes from my inbox, my ECF inbox, a trustee in bankruptcy that I've known for years.
• 28:18
  SPEAKER_01: Trustees do this all the time, especially in larger commercial type chapter 7 cases or chapter 11 cases.
• 28:25
  SPEAKER_01: They do a host of preference complaints.
• 28:27
  SPEAKER_01: And I saw that this trustee had filed about 40 preference complaints.
• 28:32
  SPEAKER_01: And I know, I know this trustee.
• 28:34
  SPEAKER_01: And this trustee is a Flintstones level trustee all the way.
• 28:38
  SPEAKER_01: And that's okay.
• 28:39
  SPEAKER_01: We don't judge this trustee.
• 28:41
  SPEAKER_01: The documents were filed, and then the the adversary proceedings will move forward, and the state hopefully will benefit, and everyone will go on with their lives.
• 28:52
  SPEAKER_01: But then I was thinking, how could AI elevate this process?
• 28:57
  SPEAKER_01: And one of the things we landed on was merge printing for sure.
• 29:03
  SPEAKER_01: Right?
• 29:03
  SPEAKER_01: Merge printing where you have a form document that has those fields with the squiggly little brackets in them, and then you have a database of defendant name, dollar amount, days since the petition, although that would really be part of the of the form.
• 29:20
  SPEAKER_01: But whatever things would change from form to form, and you could manually create that merge document.
• 29:27
  SPEAKER_01: Candidly, I don't think that's what this trustee did.
• 29:30
  SPEAKER_01: I think what this trustee did is had a support team member go through the file, you know, the form, and then replace everything, or a paralegal go in and replace everything document by document by document by document by document.
• 29:47
  SPEAKER_01: After the same support person or maybe a different team of support people went and looked at invoices and bank statements and correspondence between vendor and the debtor to determine if there's a new value defense potentially, or if there's an ordinary course of business defense potentially to assess that.
• 30:10
  SPEAKER_01: So certainly creating that form complaint so that the humans don't have to go in and put the little squiggly boxes in is certainly something that AI would excel at.
• 30:21
  SPEAKER_01: And then ChatGPT said to me, yes, the human would, you know, you would just create the database and then merge it into the form that the AI will create.
• 30:33
  SPEAKER_01: And I said, No, that's not what we want to do.
• 30:36
  SPEAKER_01: What we want to do is we want to take all of these bulk documents, like I said, the invoices, the emails, the check stubs, the bank statements, all of those documents, those raw documents, put them in a drive or directory and have the AI review them and create the database document that is going to get merged in.
• 30:59
  SPEAKER_01: And you know what?
• 31:00
  SPEAKER_01: ChatGPT got very excited when I said that.
• 31:04
  SPEAKER_01: If, if, if what is really just a brilliant compilation of code can get excited, it said, yes, you're right.
• 31:12
  SPEAKER_01: That is really the the next the big step.
• 31:16
  SPEAKER_01: And we talked about, and this is this is where being able to access what's in a folder, I know that what's in that folder is accurate, is so important.
• 31:24
  SPEAKER_01: This is where the AI can abbreviate a very long human intensive process and make it much more manageable for the law firm.
• 31:41
  SPEAKER_01: So that's the practice segment for today.
• 31:44
  SPEAKER_01: It will make life much easier for paralegals.
• 31:47
  SPEAKER_01: No question about that.
• 31:48
  SPEAKER_03: It excels in extracting data and organizing things and building those tables and you know, grouping things, populating forms.
• 31:57
  SPEAKER_03: It creates leverage.
• 31:59
  SPEAKER_01: Right.
• 32:00
  SPEAKER_01: It's it's a huge, it's huge benefit.
• 32:03
  SPEAKER_01: So, okay, that's that's our practice segment.
• 32:07
  SPEAKER_01: Now let's talk about Flintstones, Simpsons, Jetsons effort to level up.
• 32:13
  SPEAKER_01: And and there's actually a fantastic surprise here on the Jetsons level that that's I'm actually nerdishly a little excited about.
• 32:26
  SPEAKER_01: So on the Flintstones level, let's go back to folder mania.
• 32:31
  SPEAKER_01: Flintstones level, just upload your zip file to Chat GPT.
• 32:36
  SPEAKER_01: Just do that.
• 32:37
  SPEAKER_01: We've proven that that will work, that the that the tool can recognize everything.
• 32:42
  SPEAKER_01: Don't drive yourself crazy trying to connect the tool to your drive.
• 32:47
  SPEAKER_01: Just upload the zip file.
• 32:49
  SPEAKER_01: Make sure if if there's confidential client information or if there's PII, make sure you're uploading it into a secure environment like Enterprise.
• 33:00
  SPEAKER_01: And if you're not, if you if you're doing this on a budget using the free tool, make sure that you don't have PII.
• 33:06
  SPEAKER_01: Make sure you don't have client confidential information in there.
• 33:10
  SPEAKER_01: But the zip file to Chat GPT, I think that that's a beautiful workflow on the Flintstones level.
• 33:16
  SPEAKER_01: On the Simpsons level, go ahead and use Claude.
• 33:19
  SPEAKER_01: Check out Claude and see if you can get it to talk to your Google Drive or to whatever drive it has access to, and start run the file cabinet test yourself.
• 33:32
  SPEAKER_01: You know, tell me all the files in this folder and give me a 15-word description of it so that you can see, all right, get your feet wet in folder mania.
• 33:42
  SPEAKER_01: I think that's a really a good approach if you're on the Simpsons level.
• 33:46
  SPEAKER_01: Now, the Jetsons level, this is really, really interesting.
• 33:51
  SPEAKER_01: I've got a good friend of mine who is an attorney that uses Net documents, which is a very popular file system in law firms around the world.
• 34:02
  SPEAKER_01: And Claude on March 4th decided to announce that it has created uh an API through uh a system called they called MCP.
• 34:16
  SPEAKER_01: I don't know what MCP stands for, but now Claude can read Net Documents folders as of April 1st, 2026, which is today, no fooling.
• 34:30
  SPEAKER_01: And you know, that's an amazing, amazing step for a Jetsons lawyer who's in the Net Documents environment.
• 34:42
  SPEAKER_01: Because with Claude, the AI comes to them.
• 34:46
  SPEAKER_01: No downloads, no uploads, no manual transfers, the firm's existing permissions, ethical walls, and audit controls stay in place.
• 34:56
  SPEAKER_01: I mean, that that's that's fantastic for a Jetsons level lawyer.
• 35:01
  SPEAKER_01: And if you're a Simpsons level lawyer or even a Flintstones level lawyer and you say, I want that, stay tuned.
• 35:06
  SPEAKER_01: I mean, these are the tools.
• 35:08
  SPEAKER_01: This is what you need to know.
• 35:09
  SPEAKER_01: Claude is your answer.
• 35:12
  SPEAKER_01: And and I want to say this, Heather.
• 35:14
  SPEAKER_01: I stayed away from LinkedIn for quite a while.
• 35:16
  SPEAKER_01: And now that we're doing this podcast, I've gone back.
• 35:20
  SPEAKER_01: And to me, a lot of the LinkedIn discussions are Jetson's lawyers talking to other Jetsons lawyers and leaving Flintstones and Simpsons lawyers out of the loop, which I think is self-defeating.
• 35:32
  SPEAKER_01: And I also feel like a lot of social media about AI is performative and not informative.
• 35:40
  SPEAKER_01: So I resisted Claude for a long time because they kept saying, Claude this, Claude that, Claude the other thing, you gotta do Claude.
• 35:48
  SPEAKER_01: And whenever anybody tells me I gotta check out a tool, my first reaction is no, I don't.
• 35:52
  SPEAKER_01: But the human reaction resists.
• 35:55
  SPEAKER_01: Having gone through all of this, I I I do feel Claude is Claude is probably the best tool in the in the law firm environments.
• 36:08
  SPEAKER_03: Right.
• 36:08
  SPEAKER_03: And knowing that you have to use different tools, that Claude is not going to be the answer for everything.
• 36:12
  SPEAKER_03: You know, that is true.
• 36:14
  SPEAKER_01: Yeah.
• 36:15
  SPEAKER_01: If I wanted to write a closing argument for a murder trial, I'd be using Chat GPT.
• 36:23
  SPEAKER_01: If I wanted to write a motion to dismiss a murder trial, I'd use Claude.
• 36:29
  SPEAKER_03: Interesting.
• 36:30
  SPEAKER_03: I think that the I think the conversation with attorneys and AI is also limited because I think so many attorneys are intimidated to even join the conversation.
• 36:40
  SPEAKER_03: So I think that's why we see a lot of Jetsons level attorneys who do feel they're they're confident enough in their own knowledge and authority to speak on it publicly.
• 36:49
  SPEAKER_03: I think a lot of attorneys are just afraid to get involved in that conversation, especially in a public platform.
• 36:56
  SPEAKER_01: Well, I hope that if they're listening, they feel less afraid.
• 37:00
  SPEAKER_01: Thank you.
• 37:01
  SPEAKER_01: Heather, have a great week.
• 37:04
  SPEAKER_01: Welcome back from your trip.
• 37:06
  SPEAKER_01: And I hope your warm vacation afterglow persists for many weeks to come.
• 37:11
  SPEAKER_03: Thank you.
• 37:12
  SPEAKER_01: All right, take care.
• 37:13
  SPEAKER_01: See you next week.
• 37:14
  SPEAKER_01: Have a good one.
• 37:15
  SPEAKER_01: Bye.
• 37:16
  SPEAKER_00: That's it for today's episode of AI Tools for Practicing Lawyers.
• 37:21
  SPEAKER_00: Thanks for listening.
• 37:22
  SPEAKER_00: We'll see you next time.