Latest / AI Tools for Practicing Lawyers / AI Builds: Ground Zero

Transcript

AI Builds: Ground Zero · AI Tools for Practicing Lawyers

• 0:03
  SPEAKER_00: Welcome to AI Tools for Practicing Lawyers.
• 0:07
  SPEAKER_00: Practical, no-nonsense guidance to help attorneys put AI power to work in their practice right now.
• 0:15
  SPEAKER_01: Over the last few months, you've probably seen ads promising that you can build an AI agent in 20 minutes.
• 0:22
  SPEAKER_01: Maybe you've seen courses selling for hundreds or even thousands of dollars teaching lawyers how to create custom AI agents and workflows.
• 0:32
  SPEAKER_01: So I haven't invested in any of those, but I did decide that I wanted to know for myself how to build an AI agent and a workflow that a lawyer could actually use in their practice.
• 0:49
  SPEAKER_01: What follows is the story of how I accidentally spent way more than 12 hours building what I thought would be one of the simplest legal workflows imaginable: a motion to extend time.
• 1:03
  SPEAKER_01: Turns out that creating useful AI workflows, at least for me, was a lot harder, but also a lot more interesting than the ads suggest.
• 1:15
  SPEAKER_01: Now, you've all heard that the most important part of AI use is prompting and what you actually put into the AI.
• 1:28
  SPEAKER_01: And so there was a lot of effort put into educating the public about prompt engineering and maybe magic prompts and building prompt libraries, all very useful.
• 1:41
  SPEAKER_01: Now, what you're hearing about are agents, skills, workflows, persistent AI systems.
• 1:52
  SPEAKER_01: So in fact, I did a whole episode about that in the workflow options category on Claude for Legal and Strong Suit, which was at the end of the day really about how the movement in legal is towards completed workflows and not just standalone prompts.
• 2:16
  SPEAKER_01: So we're hearing a lot about Claude skills, then Strongsuit has workflows and repeatable systems.
• 2:26
  SPEAKER_01: We're being taught that AI can remember how to do a task.
• 2:30
  SPEAKER_01: What's fascinating is that we're watching a completely new product category emerge in real time.
• 2:38
  SPEAKER_01: I mean, persistent AI workflows barely existed a few months ago.
• 2:45
  SPEAKER_01: Now everybody wants one.
• 2:49
  SPEAKER_01: I accidentally discovered persistent workflows as part of my efforts to publish and produce this podcast.
• 3:01
  SPEAKER_01: I wasn't trying to build a legal workflow.
• 3:03
  SPEAKER_01: I was trying to solve a problem, which is that after I had an episode and a transcript for the episode, I would upload that to some AI agent or bot and ask it to give me show notes, to give me an email I can send to my list promoting the podcast, uh, or at least that particular episode.
• 3:26
  SPEAKER_01: I asked it for LinkedIn ads and the like.
• 3:29
  SPEAKER_01: I need a website blurb when I post it on my website.
• 3:34
  SPEAKER_01: I wanted to be able to create clips, so I wanted it to tell me, you know, what would make good clips.
• 3:41
  SPEAKER_01: So I created a markdown of that in Claude.
• 3:44
  SPEAKER_01: Claude walked me through it.
• 3:46
  SPEAKER_01: It was a pretty easy process.
• 3:49
  SPEAKER_01: I so what I do now is I upload the transcript, I click enter.
• 3:57
  SPEAKER_01: I don't even have to prompt it.
• 3:58
  SPEAKER_01: I just upload the transcript and then I receive all the assets in one swoop.
• 4:04
  SPEAKER_01: And you know what?
• 4:04
  SPEAKER_01: I could do that, I could do that in Gemini, I could do it in Chat GPT, I could even do it in Copilot if I wanted to.
• 4:12
  SPEAKER_01: So this was exciting to me.
• 4:14
  SPEAKER_01: I said, you know what?
• 4:15
  SPEAKER_01: This was an amazing discovery.
• 4:17
  SPEAKER_01: I need to bring this over to the legal side.
• 4:22
  SPEAKER_01: So my thought was I have bankruptcy courses that I sell.
• 4:30
  SPEAKER_01: And one of them is a very popular application called Team Accelerator.
• 4:37
  SPEAKER_01: That's, if you don't already know, it teaches bankruptcy to new lawyers and to paralegals and to support staff.
• 4:49
  SPEAKER_01: And it already teaches bankruptcy procedure and systems and some of the substantive matters of bankruptcy that would be important to a law office, but it doesn't have anything about AI.
• 5:04
  SPEAKER_01: I created it in 2022 before AI became a thing.
• 5:09
  SPEAKER_01: So I felt like I needed to add AI components to Team Accelerator.
• 5:16
  SPEAKER_01: And I thought I would create something, the easiest workflow imaginable, which were famous last words.
• 5:24
  SPEAKER_01: I decided to create a motion to extend time.
• 5:29
  SPEAKER_01: Really, it's something that comes up in most bankruptcy law offices.
• 5:35
  SPEAKER_01: I mean, unless you're extremely disciplined and have extremely disciplined clients, you're not going to have time if you file a bare bones petition, which a lot of bankruptcy lawyers frankly refuse to do for this very reason.
• 5:51
  SPEAKER_01: But if you have to stop a foreclosure sale and you don't have time to create a full package of schedules, you can ask the court for more time to file the schedules and other statements, and they'll routinely grant that.
• 6:07
  SPEAKER_01: Most bankruptcy lawyers have that form on their systems somewhere.
• 6:13
  SPEAKER_01: And I thought this would be great.
• 6:17
  SPEAKER_01: This would be a great little thing to create.
• 6:20
  SPEAKER_01: It would be a form for bankruptcy users and a process for bankruptcy lawyers to implement if they need it.
• 6:28
  SPEAKER_01: And plus, it should be a pretty easy thing to create.
• 6:33
  SPEAKER_01: So as I was preparing it, I kept in mind the airport test, which I discussed in my field note Confessions of an AI Hallucinator.
• 6:45
  SPEAKER_01: And the airport test is if after driving to the airport, parking, checking your bags, going through security, waiting at the gate, boarding, depleting, getting your luggage, and getting to your destination, if it would have been easier to drive or quicker to drive, then you should have just driven.
• 7:09
  SPEAKER_01: And to me, AI is the same thing.
• 7:11
  SPEAKER_01: If using AI is harder than just doing the task with pre-AI workflows, then AI has failed the airport test, and you should just have done that task without these fancy bells and whistles.
• 7:28
  SPEAKER_01: So what ChatGPT actually taught me was how to build one of these workflows.
• 7:38
  SPEAKER_01: And because this was my first, I asked ChatGPT not to simply build the workflow, which I may do the next time, but I wanted to understand the process, so I asked it to take me through it step by step.
• 7:54
  SPEAKER_01: And the workflow development cycle looks something like this.
• 7:58
  SPEAKER_01: Step one, define the objective.
• 8:01
  SPEAKER_01: What exactly is the workflow supposed to accomplish?
• 8:05
  SPEAKER_01: I had to answer that question.
• 8:07
  SPEAKER_01: Step two, define the user.
• 8:08
  SPEAKER_01: Who's going to run it?
• 8:09
  SPEAKER_01: Is it a lawyer?
• 8:10
  SPEAKER_01: Is it an assistant?
• 8:11
  SPEAKER_01: Is it a paralegal?
• 8:13
  SPEAKER_01: Is it any of them?
• 8:15
  SPEAKER_01: Step three, identify required inputs.
• 8:18
  SPEAKER_01: What documents are necessary?
• 8:20
  SPEAKER_01: What facts are necessary?
• 8:21
  SPEAKER_01: What should never be requested?
• 8:24
  SPEAKER_01: Step four was to define the outputs.
• 8:27
  SPEAKER_01: And I tried to keep that really simple.
• 8:29
  SPEAKER_01: I want a motion, I want a proposed order, and I want a certificate of service.
• 8:34
  SPEAKER_01: All right, then it started to get trickier.
• 8:38
  SPEAKER_01: I had to map some decision points.
• 8:40
  SPEAKER_01: What changes from time to time when you're going to use this workflow?
• 8:44
  SPEAKER_01: What's going to remain the same?
• 8:46
  SPEAKER_01: Step six: build a fact collection system.
• 8:50
  SPEAKER_01: I wanted to keep this really simple.
• 8:53
  SPEAKER_01: And I wanted it to be that a lawyer or somebody else using this workflow would have to upload as few documents and as little information as is possible to reduce user friction and to at least try to pass the airport test.
• 9:12
  SPEAKER_01: Step seven was to create reusable drafting instructions.
• 9:17
  SPEAKER_01: So those were like court captions and formatting and the federal rule and the tone and structure, which candidly was a little bit less important in this project.
• 9:28
  SPEAKER_01: Step eight was to build some quality control checks.
• 9:31
  SPEAKER_01: Are there missing dates?
• 9:32
  SPEAKER_01: Are there missing deadlines?
• 9:34
  SPEAKER_01: Am I missing the phrase to show that cause exists to extend this workflow?
• 9:41
  SPEAKER_01: I need it to be internally consistent.
• 9:43
  SPEAKER_01: All right.
• 9:44
  SPEAKER_01: Step nine was to test, to break it, to revise it, to repeat it.
• 9:49
  SPEAKER_01: Step 10 was platform optimization.
• 9:53
  SPEAKER_01: Can I make this work in Claude?
• 9:55
  SPEAKER_01: Can I make it work in Chat GPT?
• 9:57
  SPEAKER_01: Can I make it work in Gemini and Copilot?
• 10:01
  SPEAKER_01: So this was not a 20-minute project.
• 10:06
  SPEAKER_01: This was at least 12 hours of work.
• 10:11
  SPEAKER_01: Now, it's possible that it would have been easier in Claude, but I didn't want to use Claude for this project because once upon a time I had used Claude for one of these kinds of projects as I was preparing for a podcast episode.
• 10:28
  SPEAKER_01: And Claude shut me down.
• 10:30
  SPEAKER_01: I ran into usage limitations.
• 10:33
  SPEAKER_01: I had to pay an extra$5 just to finish that workflow.
• 10:38
  SPEAKER_01: So I said, I'm not going to do that.
• 10:39
  SPEAKER_01: Claude is great for the things that it's great at, but brainstorming and really working for these long processes on the pro tier, which is what I have, just didn't feel like the right choice.
• 10:54
  SPEAKER_01: All right.
• 10:55
  SPEAKER_01: So then I got ambitious.
• 10:59
  SPEAKER_01: I decided, you know what?
• 11:00
  SPEAKER_01: Let's make this an omnibus extension workflow.
• 11:04
  SPEAKER_01: Not just schedules and statements of financial affairs.
• 11:08
  SPEAKER_01: What if I want to extend the time to file an amended plan or a court order deadline or a statutory deadline or any deadline?
• 11:17
  SPEAKER_01: Because if you're building it, I'm thinking build it once, build it right, make it broad.
• 11:24
  SPEAKER_01: But as it turns out, every new option really multiplied the complexity.
• 11:30
  SPEAKER_01: But I did it, I got through it.
• 11:37
  SPEAKER_01: So the point was I wanted minimal uploads and minimal prompting.
• 11:43
  SPEAKER_01: And the test was upload the documents that I was creating, which was uh uh a markdown workflow, upload a notice of 341 meeting that I could get from any case, and then just type in the original deadline and the new deadline.
• 12:07
  SPEAKER_01: And if I left the cause element blank in the prompt, the workflow defaulted to just the usual language of accuracy requires that schedules be done completely, and so I need an extra 14 days.
• 12:26
  SPEAKER_01: And you need to just type in the deadline extension you wanted.
• 12:30
  SPEAKER_01: It was really pretty simple, right?
• 12:33
  SPEAKER_01: Two docs and about a 10-word prompt.
• 12:37
  SPEAKER_01: So this was really kind of interesting.
• 12:40
  SPEAKER_01: Claude, of course, was great.
• 12:42
  SPEAKER_01: You upload everything, Claude comes out, you could it could spit out Microsoft Word documents, very clean, pretty well formatted.
• 12:53
  SPEAKER_01: ChatGPT wouldn't produce a document, it would produce a workflow that I could just have in its kind of a separate box where I click in an icon and it copies it and I could paste it into a word processor.
• 13:08
  SPEAKER_01: And it turned out that Copilot, even though it's attached to Microsoft Word, really required the same kind of process.
• 13:17
  SPEAKER_01: And I ended up spending a long time fighting with Copilot because that made no sense to me.
• 13:22
  SPEAKER_01: This is Microsoft Word.
• 13:24
  SPEAKER_01: I should be able to put the cursor where I wanted the document to begin, prompt in co-pilot my workflow and my notice of 341 and my 10-word prompt.
• 13:38
  SPEAKER_01: Copilot would not do that.
• 13:41
  SPEAKER_01: And apologetically, it explained because, well, I've got all this legacy code and I gotta work for all these billions of lawyers or other users who already have been using Microsoft Word since the 1990s, blah, blah, blah.
• 13:57
  SPEAKER_01: Copilot, you were no better than ChatGPT, as far as I was concerned.
• 14:03
  SPEAKER_01: What surprised me and kind of delighted me was that the best outcome happened when I went into a Google Doc, a blank Google Doc, pulled up the Gemini chat box within that Google Doc, and then uploaded my two documents and my 10-word prompt.
• 14:27
  SPEAKER_01: And wow, Gemini and Google Docs put out a beautiful, more or less usable output of motion to extend with the proposed order and the certificate of service.
• 14:45
  SPEAKER_01: And that was particularly gratifying to me because I've been promoting the Gemini AI environment for lawyers on a budget who don't want to pay big money for their AI and would be happy to pay the$36 a month that Gemini is currently charging on the enterprise level to be able to know that they can have effective workflows using Gemini.
• 15:13
  SPEAKER_01: So score one for Ron, uh, but this was just a test.
• 15:21
  SPEAKER_01: Now, as I was testing, I discovered that we needed to improve the fundamental markdown workflow several times.
• 15:34
  SPEAKER_01: So I went back to ChatGPT and asked it to make a change.
• 15:38
  SPEAKER_01: And ChatGPT did it.
• 15:39
  SPEAKER_01: It made the change, it created a new section for the workflow.
• 15:43
  SPEAKER_01: It told me where the section belonged.
• 15:45
  SPEAKER_01: And then, you know, this is the first one I'm doing.
• 15:49
  SPEAKER_01: I don't know about taking down the markdown workflow and then adding something and saving it and putting it back into the AI.
• 15:59
  SPEAKER_01: This was my first one.
• 16:00
  SPEAKER_01: I didn't really know what I was doing.
• 16:02
  SPEAKER_01: So I asked ChatGPT to make the change and regenerate the entire workflow from beginning to end, which at this point it kept telling me was about nine or ten pages.
• 16:14
  SPEAKER_01: And then ChatGPT pushed back and it said, No, this is easy.
• 16:19
  SPEAKER_01: You should just copy it and paste it into your document.
• 16:22
  SPEAKER_01: And I said, Why are you pushing back?
• 16:25
  SPEAKER_01: I felt like it was a reluctant employee that didn't want to do that last draft.
• 16:32
  SPEAKER_01: But that's not what was happening.
• 16:35
  SPEAKER_01: ChatGPT explained, if I regenerate the entire workflow, I may inadvertently change something else.
• 16:41
  SPEAKER_01: It could omit a section, it could change the formatting, it could hallucinate something new.
• 16:48
  SPEAKER_01: And that didn't dawn on me.
• 16:49
  SPEAKER_01: It was the AI wasn't being lazy, it was warning me about risk.
• 16:54
  SPEAKER_01: And that made me think, you know, my wife is a doctor.
• 16:58
  SPEAKER_01: And we talk about the hospitals, and she says, don't go to the hospital if you can possibly avoid it.
• 17:05
  SPEAKER_01: And I said, To her, why?
• 17:07
  SPEAKER_01: You go to the hospital where you're sick, they do what they do, they make you better, and you leave.
• 17:13
  SPEAKER_01: She says, No, it doesn't really work that way.
• 17:16
  SPEAKER_01: First of all, doctors and nurses make mistakes.
• 17:21
  SPEAKER_01: But more so, the hospitals have a lot of infectious disease.
• 17:27
  SPEAKER_01: So you could go into the hospital with a fairly minor problem and leave with a much more serious problem because you caught something from another patient down the hall.
• 17:38
  SPEAKER_01: AI is a little bit like that.
• 17:40
  SPEAKER_01: Every pass you create on a document, you're creating a risk of distortion or drift or hallucinations or just something not being the same as you were expecting it to.
• 17:54
  SPEAKER_01: Now, you could try and prompt it every time you make a change to say, constrain it to the prior draft, and that might work.
• 18:02
  SPEAKER_01: That might not work.
• 18:03
  SPEAKER_01: Look, they're trying to get rid of hallucinations in AI legal drafting.
• 18:09
  SPEAKER_01: They haven't done it yet.
• 18:11
  SPEAKER_01: It's not an easy thing to do.
• 18:13
  SPEAKER_01: So the lesson that I learned is the more times you process a document through AI, the more opportunities you're creating for something to go wrong.
• 18:25
  SPEAKER_01: Because every system has error rates.
• 18:28
  SPEAKER_01: So my realization was that a mature AI user, which I'm certainly trying to be, and I'm trying to help my listeners be, doesn't constantly regenerate everything.
• 18:40
  SPEAKER_01: You have to isolate the problem and fix only what needs fixing.
• 18:46
  SPEAKER_01: So here's what I learned.
• 18:49
  SPEAKER_01: First of all, lesson one.
• 18:52
  SPEAKER_01: Workflows are real, they work.
• 18:54
  SPEAKER_01: At the end of the day, while the motion to extend that I created may or may not pass the airport test, depending upon your system.
• 19:04
  SPEAKER_01: Look, if you've never done a motion to extend before, this is going to pass the airport test because you're not going to have to create it from scratch.
• 19:12
  SPEAKER_01: If you want to download a PDF of a motion to extend, you could do that and then upload that to an AI bot and have them retype it with your changes.
• 19:24
  SPEAKER_01: I think that my workflow would beat that as well.
• 19:30
  SPEAKER_01: If you've done a thousand of them, and if you've done several in that case already, well, my workflow is not going to beat that.
• 19:38
  SPEAKER_01: You're better off going back, using your earlier form, and then updating it with the couple of things that you need to change and then filing it with the court.
• 19:49
  SPEAKER_01: What I always used to hate when I used forms from different cases is that I would invariably miss a change in the title or a change in one of the debtors' names somewhere, uh, or a change in a date that's maybe lodged deeper within the document.
• 20:10
  SPEAKER_01: Yeah, you do have to be really careful when you're using a form from a different case to make sure that it's consistent throughout the document.
• 20:20
  SPEAKER_01: I'm not telling you something you don't already very well know.
• 20:24
  SPEAKER_01: But the good news is that with AI, you're less likely to have that mistake.
• 20:31
  SPEAKER_01: And I have not seen that mistake at all in the multiple tests I've done with the workflow that I've created.
• 20:38
  SPEAKER_01: But certainly it's building it harder than advertised.
• 20:44
  SPEAKER_01: Testing may matter more than the prompting, and the platform matters a lot.
• 20:49
  SPEAKER_01: That's that's a very important lesson.
• 20:51
  SPEAKER_01: I did not expect that Google Docs plus Gemini was going to be the winner of this process.
• 20:59
  SPEAKER_01: And, you know, this is like I said before, a brand new category of legal product.
• 21:04
  SPEAKER_01: I haven't seen lawyers selling a la carte AI workflows.
• 21:08
  SPEAKER_01: I haven't seen a marketplace full of bankruptcy workflows.
• 21:11
  SPEAKER_01: So we're building something that didn't even exist a year ago.
• 21:17
  SPEAKER_01: So I would like to share with you what I did.
• 21:22
  SPEAKER_01: I've created a free download of what I call motion to extend light.
• 21:29
  SPEAKER_01: It's just the motion to extend time to file schedules and the statement of affairs workflow.
• 21:35
  SPEAKER_01: Uh it's a little raw.
• 21:38
  SPEAKER_01: It may not work for you as expected, but I think I think no matter what platform you use, you after you upload the the workflow and upload a 341 notice, or you know, you could take a screenshot of the docket, and that would probably do it, to help the AI know who the debtor is and who the lawyer is, and those other variables from this form.
• 22:08
  SPEAKER_01: So I put it up a landing page.
• 22:11
  SPEAKER_01: I'm going to give the uh URL of the landing page in the show notes.
• 22:16
  SPEAKER_01: I'm asking for emails partly because I'd like to keep users apprised of more of these markdown workflows that I'm creating.
• 22:28
  SPEAKER_01: Test it yourself, compare the platforms.
• 22:30
  SPEAKER_01: Uh, this will become our first public AI workflow release.
• 22:36
  SPEAKER_01: So, you know, if you've been watching YouTube videos telling you that you'll build sophisticated agents in 20 minutes using AI, maybe, maybe you will.
• 22:50
  SPEAKER_01: I'll never be the guy to say that you can't do something.
• 22:54
  SPEAKER_01: But I think the reality is is gonna be different.
• 22:58
  SPEAKER_01: Can you build a simple workflow in 20 minutes?
• 23:00
  SPEAKER_01: Sure.
• 23:01
  SPEAKER_01: The one I did that required no formatting for my podcast, that didn't take long.
• 23:06
  SPEAKER_01: It didn't take me 20 minutes, took me a couple of hours, but it was much quicker than what I did.
• 23:12
  SPEAKER_01: But can you build a robust legal workflow that passes the airport test, that works across multiple AI platforms, that minimizes user friction to produce useful legal document and serve real-world testing?
• 23:29
  SPEAKER_01: You might be spending the rest of your day on it.
• 23:31
  SPEAKER_01: And if you're anything like me, maybe into the fourth quarter of the NBA Finals too.
• 23:37
  SPEAKER_01: This has been AI Builds Ground Zero, the beginning of an experiment to find out whether lawyers can turn AI workflows into real legal products, and whether those products can actually make practice easier instead of harder.
• 23:52
  SPEAKER_01: This is Ron Dresher, and this is AI Tools for Practicing Lawyers.
• 23:57
  SPEAKER_01: We'll see you next time.
• 23:59
  SPEAKER_00: That's it for today's episode of AI Tools for Practicing Lawyers.
• 24:04
  SPEAKER_00: Thanks for listening.
• 24:05
  SPEAKER_00: We'll see you next time.