Latest / CLEs You Actually Want to Hear / Using AI Tools To Analyze and Oppose A Trial Or Appellate Brief
Transcript
- 0:04Welcome to this next exciting episode of Beverly
- 0:06Hills Bar Association's ongoing CLE events. I'm
- 0:10Mark Hogue, chair of the Beverly Hills Bar's
- 0:14new AI in the law section, which by the way,
- 0:16keep your eyes open. We'll be launching our first
- 0:18event towards the end of this month. So I'm delighted
- 0:22to introduce today's speaker, Michael Colantuno.
- 0:26So Michael is the managing shareholder at Colantuno
- 0:28Highsmith and Waitley. He's one of the of California's
- 0:32leading experts on local government revenues.
- 0:35He's argued 14 cases in California Supreme Court,
- 0:40appeared in all six of the California District
- 0:41Courts of Appeal, as well as trial courts around
- 0:45the state. And he has argued in the Ninth Circuit
- 0:48as well. He serves on the California Judicial
- 0:52Council's Appellate Advisory Committee and served
- 0:55on its Appellate Case Flow Working Group. So
- 0:59with that, I hope you're all sitting comfortably.
- 1:01Let's kick things off. Michael, take it away.
- 1:04Thank you, Mark. And while my slides are curing
- 1:07up, I should point out what wasn't in my bio.
- 1:10I'm not an AI lawyer in the sense that I advise
- 1:13participants in the AI industry. I'm not an intellectual
- 1:16property lawyer. I don't represent computer and
- 1:18software companies. AI is something I use in
- 1:21my practice and in my business. I think a lot
- 1:24about it. But ultimately, it's a sideline. My
- 1:27primary business is as an appellate advocate,
- 1:29often for local governments. So with that, let's
- 1:32dive into our topic, which is how do I use artificial
- 1:36intelligence tools in analyzing an opponent's
- 1:40brief and writing my reply to it? And this workflow
- 1:43is my workflow. is idiosyncratic, doesn't need
- 1:48to be your workflow, but it might give you some
- 1:50ideas. It works for me both in trial and appellate
- 1:53courts, and it's really the nuts and bolts of
- 1:56lawyering. How do you read an argument, how do
- 1:58you analyze an argument, and how do you put together
- 2:00your response to it? So here's my agenda for
- 2:03the hour I am to spend with you. I'm going to
- 2:06talk about my approach to reply briefs and the
- 2:09six steps along the way of reviewing my own case
- 2:11first so I remember what I'm arguing. I put together
- 2:15a top line outline of my reply, something that
- 2:18looks maybe like a table of contents. And only
- 2:20then do I review the opposing brief, so that
- 2:23I review their arguments fully reminded of what
- 2:26mine are. And then I annotate their brief, usually
- 2:29with sticky notes and a PDF. And then I build
- 2:31on my outline, collecting all of those sticky
- 2:34notes and refining it. And AI tools can help
- 2:37refine it. Then I draft, and then I edit. And
- 2:41then I finalize a lovely bureaucratic word. Also
- 2:45along the way, and at the end, I'll have some
- 2:47practical tips about particular tools in the
- 2:49marketplace where I will give you my opinions,
- 2:51which are worth exactly what you paid for them.
- 2:53You're entitled to your own opinions. And then
- 2:56I'll review the tools that I use, how I use them,
- 2:58and why. So that's our path for the day. So let's
- 3:01go ahead to the next slide. So to introduce.
- 3:10Generative AI went public, so to speak, in late
- 3:162023 when chat GPT was launched as a free tool
- 3:19on the internet. Prior to that, people in the
- 3:21industry understood the power of this technology,
- 3:24but it didn't become the cultural phenomenon
- 3:26that it now is until then. Now these tools are
- 3:29mature enough. that lawyers should be thinking
- 3:32about how to integrate AI into your workflows.
- 3:34Because your clients are going to expect it.
- 3:36Your competitors are doing it. Cost pressures
- 3:38aren't getting any better for us or our clients.
- 3:41It's just coming. There was a time when you could
- 3:43resist email. You could resist the conversion
- 3:46from WordPerfect to Word. You could resist the
- 3:49conversion from a Wang writer to a PC. All of
- 3:52those technologies are obsolete. And a law practice
- 3:54without AI, to my mind, is becoming obsolete.
- 3:59Today's question is not whether to use AI, it's
- 4:02how to use them ethically and effectively. Next
- 4:06slide. So how is AI useful for litigators in
- 4:13particular? Although I think much of what I have
- 4:15to say will apply to advisory lawyers or transactional
- 4:18lawyers too. One is it can save you time. I find
- 4:22it doesn't. I find the time I need to actually
- 4:25use the output from an AI product is as much
- 4:28as the time I saved by letting it draft for me.
- 4:31But it definitely makes my work product better
- 4:34and makes my legal research deeper. It can help
- 4:37you identify issues, arguments, and authorities.
- 4:42And of course, we have an ethical duty under
- 4:45the rules of professional conduct to be competent.
- 4:47And California is leading the 50 state bars by
- 4:51reminding us in writing and expressly that competence
- 4:54includes technology. There was an oral argument
- 4:58in the Supreme Court a few years ago. It was
- 5:00a Fourth Amendment question about what text explained
- 5:03between a police officer and his lady in his
- 5:06life who was not his wife were unreasonably searched
- 5:12within the meaning of the Fourth Amendment when
- 5:14his employer downloaded them and looked at them.
- 5:16And there was this farce of an argument between
- 5:19the lawyers on the case and the justices of the
- 5:22US Supreme Court, and which was very clear that
- 5:24none of them had the faintest idea about how
- 5:27pager technology at that time worked. In order
- 5:30to be competent, we need to know how the technology
- 5:34we're using and our clients are using works.
- 5:37Next slide. So what's the first step of my six
- 5:43-step approach? I don't like to be too swayed
- 5:47by reading my opponent's writing before I remind
- 5:50myself of my view of the case. So if I've written
- 5:52an opening brief, it's easy. I go look at the
- 5:55table of contents and get refreshed. If the opposing
- 5:59party's paper is the first paper in the case,
- 6:01there would have been something that went before.
- 6:03Demand letters, a case analysis, correspondence
- 6:06with the client. So start by reviewing the file.
- 6:09Remind yourself, ground yourself in your own
- 6:12view of the case. This allows you to reaffirm
- 6:14your client's objectives and your framing of
- 6:17the case, your story, how you think justice bends
- 6:21in your direction in this dispute. And it's intended
- 6:23to avoid letting the opponent's framing dictate
- 6:26your thinking, unless, of course, you like your
- 6:28opponent's framing. And we'll talk about that
- 6:30again in a bit. Next slide. AI can help with
- 6:35file review. AI tools like ClearBrief, which
- 6:39my firm uses, are really good at taking in a
- 6:42whole lot of electronic data and organizing it,
- 6:45summarizing it, interrogating it. So you can
- 6:49query your factual database to, for instance,
- 6:54if you're in a personal injury setting and you
- 6:58want to look for every reference in the electronic
- 7:02record. to notice to the defendant of the presence
- 7:07of the defect, you can ask ClearBrief to do that.
- 7:10And it's not doing a text search. It is using
- 7:15its ability as an AI tool to find other words
- 7:18that express the same idea that you're looking
- 7:20for. These AI tools can identify recurring themes.
- 7:24They can find factual patterns. But again, AI
- 7:28augments your work. that it does not replace
- 7:31your work, your lawyer thinking, your legal judgment
- 7:34is still at the center of your work. Next slide.
- 7:42Let's talk about our first and most important
- 7:45ethical obligation when using AI tools. If you
- 7:49put something into the prompt of a public AI
- 7:52tool like ChatGPT or Claude, you should assume
- 7:57that you are training the large language model,
- 8:01that it's absorbing your data and putting it
- 8:03in its resource, and that it could use that resource
- 8:06to produce output to others. So it's necessarily
- 8:10a breach of confidentiality to put client confidences
- 8:14in the prompt window of a mass market you didn't
- 8:19pay for it or you didn't pay very much for it
- 8:20product. So if you want to use those tools, you
- 8:23have to use them in ways that don't use client
- 8:26confidential information. That may mean being
- 8:28hypothetical, that may mean using false names.
- 8:33So sometimes I'll say, draft me an ordinance
- 8:34of a California general law city to regulate
- 8:38accessory dwelling units. I won't name the city.
- 8:42I say, draft me an ordinance for any town in
- 8:45California that complies with the following two
- 8:47statutes and does the following three things.
- 8:50None of that is compromising my client's confidence.
- 8:52It's because I didn't name my client. It could
- 8:54be any of the 483 cities. The second way to be
- 9:00consistent with your duty to protect your client's
- 9:02confidence is to use law tech tools, tools that
- 9:05are sold to you by businesses that know you're
- 9:07a lawyer and want to help you be a lawyer. Those
- 9:10tools assure you of confidentiality. You need
- 9:13to give some thought to what they're saying and
- 9:16how they're saying it. But generally, they guarantee
- 9:19your confidentiality in one of two ways. Either
- 9:22they run on your server and not in the cloud
- 9:24on their servers, or if they run it on their
- 9:27server, they delete it when you end your session.
- 9:31So for example, I use Westlaw's drafting assistant
- 9:35regularly and launch a brief, get it to critique
- 9:39the cases cited and suggest other cases. It will
- 9:43critique the quotation accuracy. It will fix
- 9:46the citation form. It will tell me about coming
- 9:49changes in the law that might affect the cases
- 9:51I'm arguing. In the end of each session, it says,
- 9:54I'm now deleting your copy. That's how it secures
- 9:57client confidences. It just doesn't keep it.
- 10:00So what are some of the tools out there that
- 10:03we're using these days? Clearbrief, which I've
- 10:05talked about, which is good at interrogating
- 10:06databases, does evaluate authorities. It does
- 10:09draft. I don't use it for those things. Logical,
- 10:13which is a tool for managing email discovery.
- 10:17That's a combination of logic and cull, culling
- 10:20all of those emails for the ones that are responsive
- 10:22to discovery, for example. Westlaw's drafting
- 10:24assistant is a big part of my practice. Westlaw's
- 10:29precision is their version of an AI enhanced
- 10:32research tool, and I use it every day. We used
- 10:35it a few minutes ago. We beta tested Lexis plus
- 10:39AI. and chose not to jump to it because they
- 10:42don't have the secondary resources that our practice
- 10:46relies on, particularly the rudder guides. And
- 10:48of course, there are many others. There's a new
- 10:50product a month. This is a marketplace you need
- 10:52to keep an eye and an ear on in order to stay
- 10:55current. Next slide. The clear brief. Clear brief
- 11:02summarizes and puts in chronological order case
- 11:05records. You can dump all of your electronic
- 11:08evidence into a ClearReef database. We have a
- 11:11paralegal do that for us. And then ask for a
- 11:13chronology. And you can ask for a chronology
- 11:15that's short and general or long and specific.
- 11:19And you can ask for it in the form of a table,
- 11:21or you can ask for it in the form of a draft
- 11:22statement of facts. And for each item in the
- 11:25chronology, there's a link that you can click
- 11:27to and jump right to the evidence. I find this
- 11:30really powerful. when getting started on a case.
- 11:34I've never used it to draft a statement of facts,
- 11:37but I do use the chronology to organize my thinking,
- 11:41to organize my team's approach, and we always
- 11:43start with an overlong one and trim it back.
- 11:46It will cross -check your citations. It will
- 11:49extract key issues. It's ideal for large records.
- 11:53In my practice, that's administrative records
- 11:55and large discovery sets. To give you an example,
- 11:58we had a CEQA case. a case under the California
- 12:00Environmental Quality Act. By the time it got
- 12:03to the Court of Appeal, there was only one issue
- 12:06left and it was a particular human health impact.
- 12:09And although the standard of care required us
- 12:11to review the 29 ,000 page administrative record,
- 12:15it was very reassuring to use ClearBrief to make
- 12:18sure we hadn't missed any references to that
- 12:20issue. Something else ClearBrief does that's
- 12:23pretty powerful, it can turn your draft brief
- 12:25into a brief that has hyperlinks. so that each
- 12:29citation to a fact, you click it and you're in
- 12:32the record at that page. Each citation to an
- 12:34authority, you click it and you're in that authority.
- 12:37And it will bundle this into something you can
- 12:39file in court, giving the court effectively an
- 12:42electronic brief. We haven't used it all that
- 12:44much. I think some of my peers in the appellate
- 12:47bar use it a bit more, use other tools as well.
- 12:50But those companies that will go hyperlink your
- 12:51briefs for you, they're going to die. This product
- 12:55and products like it are going to allow you to
- 12:57vertically integrate that function. Next slide.
- 13:04So what's step two in my six -step workflow for
- 13:08reading my opponent's work and writing my reply
- 13:10to it? I develop a top -level outline, by which
- 13:14I mean not very detailed, table contents level
- 13:16of detail. consistent with my case authority
- 13:20before I read the opposition. I may look at the
- 13:23table of contents in the opposition. I may scan
- 13:25it to get a sense of where they're going, but
- 13:27I don't study it yet. You can use AI tools for
- 13:31brainstorming. They're really good at that. So
- 13:35I'll give you an example. We got a petition for
- 13:38cert in one of our code enforcement cases for
- 13:40Humboldt County. It's in the paper today, actually,
- 13:42because we've got cert denied on Tuesday, which
- 13:44is good news for our client. outlined in some
- 13:49detail our approach to writing the answer, the
- 13:53opposition, to the petition for cert that the
- 13:55court invited. And we did that based on traditional
- 13:58research using the leading treatise for US Supreme
- 14:01Court practice. And then we talked to CHAT GPT,
- 14:05the public version of CHAT GPT, saying we're
- 14:08lawyers. We're resisting a petition for a certiorari
- 14:11to the US Supreme Court. We've made the following
- 14:14arguments about our case. What other arguments
- 14:17can you suggest that might be persuasive to the
- 14:19court based on what we've told you so far? It
- 14:23came up with two additional points that we ultimately
- 14:26decided to include, and the net result was the
- 14:29Supreme Court denied cert because our case wasn't
- 14:32a good vehicle for an issue that at least the
- 14:35conservatives on that court are very eager to
- 14:37take up. That's the kind of brainstorming that
- 14:40AI tools are really good at, giving you ideas
- 14:42for you to evaluate. And as you know, every lawyer
- 14:45knows, when you do an outline, you've got to
- 14:47organize the issues logically and persuasively.
- 14:51Next slide. You can use AI to spot issues and
- 14:57themes. So if you're in an area of the law you
- 14:59don't know well, or you have a case that's new
- 15:01on your desk and you haven't really absorbed
- 15:02all of its facts, AI is really good. at recognizing
- 15:06patterns, generating ideas. You can use it to
- 15:09test hypotheses about case strengths or weaknesses.
- 15:12So you could say, I want to argue that the landlord
- 15:14did not have notice of the defect on the staircase.
- 15:18What facts are in the record can you identify
- 15:20that will support my theory? What facts are in
- 15:23the record that will undermine my theory? What's
- 15:26the strongest argument you think I can make that
- 15:29the landlord did not have notice? What's the
- 15:31strongest argument you think my opponent can
- 15:33make that the landlord did have noticed? And
- 15:35these AI tools will answer all of those questions,
- 15:38some of them more persuasively than others. But
- 15:41you can ask these things. And of course, you'll
- 15:44see this on my slides repeatedly, you have to
- 15:46test its results with your own reasoning. You
- 15:49can't blindly take attacks generated by an AI
- 15:52tool and dump it into your brief. You know, I
- 15:56do a lot of speaking on this topic. I worked
- 15:58with Judge Carnell, who has recently retired
- 16:01from the San Francisco Superior Court, and with
- 16:03Jeff Erlich, who's an insurance coverage lawyer
- 16:05on the plaintiff's side, who chairs the AI committee
- 16:08of the California Academy of Appellate Lawyers.
- 16:10And we swap news stories about what's going on
- 16:13on this topic. And the one that came around today
- 16:15was the lawyer who got an order to show cause
- 16:18why he should not be sanctioned for filing a
- 16:21brief that included hallucinated citations. And
- 16:24in his opposition, he included hallucinated citations.
- 16:30Somebody a little unclear on the concept. Why
- 16:33is this happening? I think it's happening for
- 16:36two reasons. One is lawyers are busy and in a
- 16:39hurry, and many of us are naive about how these
- 16:42tools work. Here's the most important takeaway
- 16:45from today's presentation. All AI tools hallucinate.
- 16:52Even the ones you pay for. even the ones that
- 16:55are limited to a sandbox of authority, they make
- 16:58stuff up. Some AI tools have a temperature rating.
- 17:04The really cool ones are less creative, don't
- 17:07write quite as well, and are less likely to hallucinate.
- 17:11The warm ones are more creative, write better,
- 17:15and are more likely to hallucinate. Claude is
- 17:18warm, Chat GPT is cooled, and the Westlaw product
- 17:23is cold. So be aware of that idea that even Lexis
- 17:29plus AI and Westlaw Precision, things sold to
- 17:33you as a lawyer, by a lawyer, for legal work
- 17:36will hallucinate. The other thing that's going
- 17:40on is we all work in teams and those of us who
- 17:44are leading a team have to trust the people on
- 17:47our team to an extent. And we can't go necessarily
- 17:51read every citation for every obvious principle.
- 17:56Now, if you've got a section of your opposition
- 17:57to a demur, reciting the standard on demur, are
- 18:00you really going to go read each case that reminds
- 18:02you what the standard on demur is? Probably not.
- 18:06But if we have to assume that the people on our
- 18:11team might be using these tools and might be
- 18:14using them irresponsibly. And my name is on that
- 18:16pleading. So we've established a rule in our
- 18:19organization that the lead lawyer on every filing
- 18:23needs to confirm the citations. And we do that
- 18:27using Westlaw's quick check, which is not AI
- 18:31powered. It's traditional computer look it up
- 18:34in the database reasoning. And if there's a bad
- 18:36site in your brief, to a case that doesn't exist
- 18:39or just a typo in the volume number or the page
- 18:42number, it will tell you. So our policy is before
- 18:45it goes out the door, the lead lawyer has to
- 18:48run the quick check. Yes, it's time consuming,
- 18:50but we're betting our reputation on it. We're
- 18:52betting our licenses on it. Next slide. So when
- 18:59you're outlining a brief, You can certainly do
- 19:02it the old -fashioned way, pull out a legal pad
- 19:04and write down your thoughts or open your computer,
- 19:06pull up a word and start typing. But there are
- 19:09AI tools to assist with this as well, clear brief
- 19:11on evidence, chat GPT and CLAWD, this conceptual
- 19:15brainstorming, but remember they're not confidential
- 19:17unless you've paid for a version of them that
- 19:19is. And Westlaw Precision and its co -counsel
- 19:23tool, which we beta tested and basically decided
- 19:25wasn't sufficiently valuable to be worth paying,
- 19:28it will validate the authorities. Identify authorities
- 19:32that you didn't cite or your opponent didn't
- 19:33cite that are relevant. These are really powerful
- 19:36tools. Clearbrief has a tool for evaluating how
- 19:41well a cited case stands for the proposition
- 19:43for which it's authored, but I don't find it
- 19:46as useful as Westlaw's version of that same tool
- 19:49because Westlaw tells me in a plainer way and
- 19:53gives me other cases that establish the point
- 19:55and they're right there for me to click on. Next
- 19:58slide. So what's the third step? You've reminded
- 20:03yourself of the case, you've done your top level
- 20:06outline of your case, and you reply. Now you
- 20:09do a strategic review of the opposition. So we're
- 20:12reading it only after framed our outline. And
- 20:14what I do is I read them as PDFs, because they
- 20:17come to me that way. And I put a sticky note
- 20:19on everything in the other party's brief that
- 20:23I have anything at all to say about. I try to
- 20:27maintain focus on my narrative, how I want to
- 20:30tell this story. not my opposing council's narrative,
- 20:33unless I liked the way he or she framed it. If
- 20:35they gave something away in their framing, or
- 20:37maybe their framing is just simpler, clearer,
- 20:39and more persuasive, sometimes I'll adopt their
- 20:41framing. But when I'm done reading their brief,
- 20:44I've got a forest of sticky notes. And I collect
- 20:47all those sticky notes by putting them in the
- 20:50outline. And then I end up with an outline that's
- 20:53what I call shrubby, kind of lumpy, too long
- 20:55in some places, too short in another, and lacking
- 20:57flow. and then my work is to clean up that outline,
- 21:01to make it flow, to avoid repetition, and then
- 21:05I start to draft. So next slide. How can AI help
- 21:13you with your critical analysis of your opposing
- 21:17counsel's brief? Well, certainly you can run
- 21:21it through ClearBrief for Westlaw's drafting
- 21:23assistant to verify its authorities. I don't
- 21:26do that that much, because I often work in areas
- 21:30of the law that I know really well. And so there
- 21:33aren't going to be very many cases cited that
- 21:34I don't know. And it's easier to just look up
- 21:37those few cases than to go hunt for all of them.
- 21:41But if you're in a new area, or maybe you're
- 21:43new to the practice of the law, It could be pretty
- 21:45handy to have a copy of the brief in front of
- 21:47you with a color -coded bar for each citation
- 21:51as to whether that's a good site or not, whether
- 21:53that's a good case or not, or whether there's
- 21:55contrary authority you should be aware of. Westlaw's
- 21:58quick check evaluates your citation accuracy
- 22:01and your precedent strength. And as I said, it's
- 22:04the last thing we do before a brief goes out
- 22:05the door. Next slide. And I see that questions
- 22:10are starting to come in. I appreciate those.
- 22:12I'm going to come to those at the end of our
- 22:14presentation. Please feel free to add them now,
- 22:16however. When I review my opposing counsel's
- 22:20work, the reason I did an outline that draws
- 22:24from my side of the case is I want to identify
- 22:28things they didn't respond to. What arguments
- 22:30did I make that they just didn't respond to?
- 22:32Did they fail to rebut something? Did they concede
- 22:35any factual points? And then I argue the omissions
- 22:39as implicit concessions. But I often have a section
- 22:42at the front of my brief saying, there's less
- 22:47disagreement than may appear and lists all the
- 22:49concessions. Or fewer facts are disputed than
- 22:53one might assume. Or the following arguments
- 22:57are silently conceded, point, point, point. You
- 23:02can use the AI tools. to cross -reference statements
- 23:06in your opposing counsel's brief to the record,
- 23:09and you can use it to cross -reference your statements
- 23:11to the record. So if they make a representation
- 23:14that the landlord had noticed at 8 a .m. on July
- 23:16the 7th that the stair was defective, you could
- 23:19highlight that sentence in clear brief and say,
- 23:21go interrogate the evidence and tell me what
- 23:24authority, what facts there are for or against
- 23:28that assertion. Next slide. I tag every point
- 23:34worth your butting, I add them to my outline,
- 23:37and I include in my outline page sites to the
- 23:40opposition. The judges and their research attorneys
- 23:44want to know that your claim of what the other
- 23:49guy said or other gal said is accurate description
- 23:53of their argument. They want a page site to where
- 23:56in the brief do you think they argued that black
- 23:59is white or white is black? So I collect the
- 24:02page sites, which is easy because the sticky
- 24:04notes are on pages. And I often find I'm using
- 24:07the Contra site. So I'll say, white is white
- 24:10and has been since the beginning of time. And
- 24:12to argue the black is white is fundamental error.
- 24:15Contra, opening brief, page 12. So I use that
- 24:19Contra site not to say that one of my authorities
- 24:22has been contested, but here's where the other
- 24:24guy claims the other thing. Next slide. Step
- 24:31four in my six step process. I reviewed my file.
- 24:34I reminded myself of my case. I did my outline.
- 24:37I analyzed the other guy's brief and tagged it
- 24:39all up with sticky notes. Now I integrate my
- 24:42critique of his or her brief with the output
- 24:45of the Westlar ClearBeef tools. And I work on
- 24:48my outline to make it flow logically, cover all
- 24:51the issues. And I revise it to emphasize my narrative
- 24:55and my strategic priorities. If my opposing counsel
- 24:58devoted 8 of his 15 pages to an issue that I
- 25:00think is utterly dumb and refuted by a case site
- 25:03or two, I'm going to put a paragraph at the beginning
- 25:05of my brief that says, this claim is utterly
- 25:07dumb and here's two cases that prove it, now
- 25:09let's turn to the art of the case. How many pages
- 25:12they devote to it should not dictate how many
- 25:14pages you devote to it unless their pages were
- 25:17really effective. Next slide. So when I'm done
- 25:24with an outline, I, on projects that warrant
- 25:28it, you know, everything we do doesn't necessarily
- 25:30warrant the full treatment and the clients can't
- 25:32always afford it. But when the cost is warranted,
- 25:35I circulate it to my peers and to the clients
- 25:38to get buy -in on the outline. If there's something
- 25:41wrong with my thinking, I want to know it early.
- 25:43And I don't want to invest a lot of time in drafting,
- 25:46only to discover that the client or my peers
- 25:48don't like that drafting or are going to persuade
- 25:50me not to like it. So if you have time for this,
- 25:53this is really valuable. And when you get the
- 25:56committee approach to reviewing your work and
- 25:58general counsel wants to rewrite every darn footnote,
- 26:02you can remind them, you know, this is due in
- 26:04two days, and I did circulate an outline and
- 26:06you did sign off on it. Next slide. Fifth step
- 26:13in our journey towards a product we'll be proud
- 26:16of is the drafting itself. This is advice I give
- 26:19everybody. Everybody who I train, everybody who's
- 26:22my peer, even to my clients if they want this
- 26:24kind of advice. Always draft from an outline,
- 26:28even from emails or short documents. It may be
- 26:31three points on a notepad. It may be three points
- 26:34that you've thought through and put in your head.
- 26:36Always draft from an outline. If you just sit
- 26:39down and start typing, you're going to wander
- 26:41and repeat yourself. So outlines keep your writing
- 26:44focused and efficient, and they allow you to
- 26:46avoid wandering or repetition. Stay on topic.
- 26:50Next slide. Last step is editing and polishing.
- 26:58Good to start with somebody who's a better writer
- 27:00than I am, an English writer from the 19th century,
- 27:02Robert Graves. There's no such thing as good
- 27:05writing, only good rewriting. AI editors can
- 27:11help you improve clarity, tone, and brevity.
- 27:15So my friend Jeff, solo practitioner, his son's
- 27:18a lawyer too and sometimes helps him, had a brief
- 27:20that was over the 14 ,000 page word limit. and
- 27:23was having a bit of writer's block about getting
- 27:25it down. He fed his brief paragraph by paragraph
- 27:28in the co -counsel and asked it to rewrite it
- 27:30in a shorter, clearer tone, and used its feedback
- 27:34to reduce the length of his brief. Now, he'll
- 27:37warn you that the tools tend to cut too much
- 27:39out. They tend to overdo it in trying to simplify,
- 27:43because lawyers... often are arguing about subtle
- 27:46details and nuance, and what you might drop out
- 27:49for an executive summary for a general audience,
- 27:52you can't drop out if it's the heart of your
- 27:53case. I use word rake. Well, I'm going to put
- 27:57this differently. I make everybody who writes
- 27:59for me use word rake. It's cheap, integrates
- 28:01with word, and it will produce an orange line
- 28:04of your draft, highlighting common problems like
- 28:08passive voice, failure to maintain the right
- 28:12subject number with the verb, those sorts of
- 28:15problems. And I ask my team, before you ask me
- 28:18to invest my time in editing your work, you edit
- 28:21it using Word Break. It's like 600 bucks a seat
- 28:24license a year. It's relatively cheap as this
- 28:26kind of software goes, pretty powerful. I recommend
- 28:29it. There are AI rewriting tools as well. We're
- 28:33not really using them so much, but we do use
- 28:35Word Break. Next slide. Use Word Break for concise
- 28:41legal wording in Word. Co -counsel can be used
- 28:44to assist with summarization and simplification.
- 28:47Next slide. A word about ethics. You should use
- 28:55AI output like the work from a junior associate.
- 29:01Verify all of it. Don't assume that just because
- 29:04a computer produced it in nice type that somehow
- 29:06it's an authority. It's not. And do not delegate
- 29:10your legal reasoning to artificial intelligence
- 29:13and always maintain client confidentiality. Next
- 29:18slide. Disclosure and compliance. First, disclosure
- 29:25to courts. Courts are increasingly requiring
- 29:29disclosure of AI use. There's been more than
- 29:31200 lawyers around the country so far sanctioned
- 29:33for citing hallucinated cases. So courts want
- 29:37to know. Duke's law school has a wiki, not a
- 29:43wiki, it's a resource that they maintain, that
- 29:45lists all of the standing orders they know about
- 29:48from the courts around the country that have
- 29:51required disclosure of AI. A better rule is whenever
- 29:55you're in a new forum, an unfamiliar venue, check
- 29:58the local rules, check the standing orders, know
- 30:01what's required of you. You may well need a footnote
- 30:04in your brief saying, The research for this brief
- 30:06was supported by artificial intelligence tools.
- 30:09However, we did not use any AI generated text
- 30:12or the AI generated text produced for this brief
- 30:15was scrutinized by the lawyers who are filing
- 30:18it. There's something you may need to say. There's
- 30:22another kind of disclosure here and that is the
- 30:25rules of professional conduct require us to disclose
- 30:29to our clients any significant matter affecting
- 30:33our representation. Do you think you need to
- 30:37disclose to your clients that you use email?
- 30:40Of course not. Do you think you need to disclose
- 30:42your clients that use word? Of course not. These
- 30:45have become the bread and butter of all professionals'
- 30:48lives. But there was a time when each of these
- 30:50tools was new and to some extent controversial.
- 30:53I remember the debates about whether sending
- 30:56an email was consistent with our duty of confidentiality
- 30:58because somebody else might see it. Well, somebody
- 31:00might open your mail too. But... I think AI is
- 31:06sufficiently new that it's best to make some
- 31:09sort of disclosure. So I have a little paragraph
- 31:12in my standard form of legal services agreement
- 31:14that says, you should know that we may use artificial
- 31:18intelligence tools in this representation. If
- 31:21we do, we will take every step we can to maintain
- 31:25the confidentiality of your confidences, and
- 31:29we will review any written output from these
- 31:31products to ensure it meets our standards and
- 31:33the standards of our profession. If you have
- 31:35any questions or concerns about our AI use, please
- 31:38let me know. I've been using that language for
- 31:41about three years now, maybe two and a half.
- 31:44Only one client has ever asked and it's because
- 31:46he was in the AI space and wanted to do a little
- 31:48opposition research to see what are you using.
- 31:50So think about what you have to disclose to courts.
- 31:53Think about what you might want to disclose to
- 31:55your clients. Next slide. The buck stops with
- 32:02you. If your name is on the pleading or you sign
- 32:05the pleading, it's on you. And you can't blame
- 32:07AI any more than you can blame your secretary.
- 32:10So AI can research. I use Westlaw Precision every
- 32:14day. It can summarize. We find ClearBrief really
- 32:16helpful. It can site check. I use QuickCheck
- 32:19every week. But it can't make your decisions
- 32:23for you. You remain accountable for the accuracy
- 32:26and integrity of the work you send out on your
- 32:29letterhead. Do not trust and do verify. Next
- 32:33slide. So what's in the marketplace? I don't
- 32:39have enough knowledge to exhaustively review
- 32:42everything that's going on out there. It's just,
- 32:45I read the paper closely, I pay attention to
- 32:46the stuff, I care about it, but it's a moving
- 32:49target. So I've talked about Westlaw Precision.
- 32:52It's the upgraded version of the Westlaw research
- 32:55tool that you're probably all familiar with.
- 32:57And when you open the Westlaw window, there's
- 32:59a traditional box at the top where you can do
- 33:01an ordinary search or a Boolean search. And there's
- 33:04co -counsel down at the bottom, which I didn't
- 33:06pay for, but they still gave to me because I
- 33:08think they're learning that nobody's going to
- 33:09pay for it. And in the middle is a big window
- 33:11where you can have the kind of conversation that
- 33:14you may have had with chat GPT or Claude. So
- 33:19for instance, When I needed to produce these
- 33:25slides, I gave a paper that I'm using as the
- 33:29base of my discussion and the slide format that
- 33:32my firm uses with that logo and those colorful
- 33:35stripes to chat GPT 5 .0 and said, turn this
- 33:40into a PowerPoint presentation for my use. Please
- 33:43be detailed. I want at least 25 slides and don't
- 33:46step on my logo or my aesthetics. I didn't ask
- 33:50for pictures. To ask for pictures and it has
- 33:52to generate them will take a long time. If you
- 33:56ask it to review a clipart file, it might be
- 33:58faster. But you can use it for, you can use these
- 34:03tools. You can use Westlaw Precision in particular.
- 34:07One of the things, often you're doing some work.
- 34:10You know there's a rule of law and you need a
- 34:12citation for it. So, for example, I know for
- 34:15a fact that a local government cannot commence
- 34:18an eminent main proceeding to acquire property
- 34:21without a two -thirds vote of its legislative
- 34:24body. But exactly where in the eminent main law
- 34:28and the Code of Civil Procedure that citation
- 34:29is, is not something I carry around in my head
- 34:31because I visit that area of practice. I don't
- 34:33live in it every day. So rather than asking an
- 34:36associate to go look it up or even try to find
- 34:39it with a bully and search of the statutes in
- 34:42Westlaw, I just go to the Westlaw Precision Window
- 34:46and say, find me the section of the community
- 34:49of the eminent Maine law in the California Code
- 34:52of Civil Procedure that requires a super majority
- 34:55vote to find necessity to acquire private property.
- 34:59And it goes away and clicks for a few minutes,
- 35:01and suddenly it gives me not only the citation,
- 35:03but I can click on it and open the statute, and
- 35:06it will give me several cases applying that statute.
- 35:08Much cheaper. to my client than having an associate
- 35:11go spend an hour learning this. Much cheaper
- 35:14than having me hunt around for 10 or 15 minutes
- 35:16if it can do it in two or three. And you can
- 35:20use that Westlaw Precision Analysis to ask broader
- 35:22questions as well. So they're powerful tools.
- 35:26I recommend them. Drafting Assistant is a Westlaw
- 35:30tool that's been around a while. It uploads a
- 35:33document, evaluates the authorities, flags any
- 35:37weak ones, identifies ones you might want to
- 35:39think about citing instead. Tests how accurate
- 35:43the quotations are, can check your citation form
- 35:46and you can say I want Blue Book or I want the
- 35:48California Yellow Book. It's a powerful tool
- 35:51and I use it all the time. ChatGPT and Claude,
- 35:55we've talked about. I don't use them very much
- 35:58for legal work. I use them for marketing. I use
- 36:02them for firm management. So to give you an example,
- 36:05we added a position on our accounting team that
- 36:07we hadn't had before. I didn't have a job description
- 36:10conveniently at hand, and they needed one. So
- 36:13I said, Chad GPT, buddy, I need a job description
- 36:15for a financial services manager for a law firm
- 36:18that has these primary functions. And poof, it
- 36:21very quickly generated a kind of long and wordy
- 36:26overdone initial draft. Chad GPT, when it was
- 36:31trained, was Given more points by its initial
- 36:35critics are longer answers, more discursive,
- 36:39more chatty answers. And as a result, it's long
- 36:42-winded and a little inefficient. Claude is really
- 36:46friendly, almost to the point of being sycophantic.
- 36:49And a lot of these tools are because they were
- 36:51trained to please the people who were working
- 36:54with them. And that sycophancy can contribute
- 36:58to your trusting them to overlook their hallucinations.
- 37:03And it leads to the mental health challenges
- 37:05that particularly young people have had when
- 37:07they've struck up friendships with these AI tools.
- 37:12Logical is a tool for taking a whole lot of email
- 37:15in. And first thing it does that's most powerfully
- 37:19important is it de -duplicates. As you know,
- 37:21each email that goes out to 30 people and produces
- 37:2430 or 40 or 50 responses produces this echo chamber
- 37:27where the same email bounces around and around
- 37:29and out. Logical immediately dedupes and reduces
- 37:33the number of emails we have to look at by a
- 37:35quarter or more. And then it allows you to quickly
- 37:38flag relevant and irrelevant, hot docs and not
- 37:41hot docs, things that shouldn't be in your data
- 37:43set at all. And it allows you to collaborate
- 37:46with the rest of your team in that way. So we
- 37:48use it. on all of our cases. We used to be able
- 37:51to buy it project by project, making the pass
- 37:54-through of the cost to our clients really seamless.
- 37:56Then they got bought, and the new owner who's
- 38:00busy trying to destroy the product insisted on
- 38:03changing the economic relationship so there's
- 38:05a flat sum every month. But they give you data
- 38:08that you can use to bill through your clients.
- 38:10It's a little harder than it looks, because if
- 38:12you do just what they say, you end up over -billing
- 38:14your clients and collecting more than you're
- 38:16spending, which is unethical unless you've got
- 38:18your clients. to agree to it. So the problem
- 38:22with a lot of this whole space is when a product
- 38:24gets really good, it gets acquired. And the person
- 38:26who acquired it often destroys what's best about
- 38:29it, like the people who just bought Southwest
- 38:31Airlines. Another story. Copilot is something
- 38:36that is available to all of you if you have the
- 38:41office suite. It's pretty cheap. And what it
- 38:45does is it integrates chat GPT into the components
- 38:51of Outlook. So your email box, your word sessions,
- 38:56your PowerPoint sessions, your Excel sessions,
- 38:59all of those window suite tools can have an AI
- 39:04interface built in. It will suggest words as
- 39:07you type. It will flag grammatical problems as
- 39:09you type. It's pretty cheap. And it is the most
- 39:13commonly used AI tool in America today because
- 39:17it plugs into everybody's workspace. It's pretty
- 39:20cheap and it's pretty helpful. It's not chat
- 39:23GPT 5 .0. It doesn't have the power that these
- 39:26other products have. Some people find it insufficiently
- 39:29helpful to be worthwhile. We offered it to our
- 39:3334 folks, lawyers and support staff who have
- 39:37the Windows licenses, and only about half of
- 39:39them took it. And we've encouraged people to
- 39:41sign up and drop out whenever you want, because
- 39:43you can drop the licenses and stop spending money
- 39:45on it. And I'm going to keep an eye on it. We've
- 39:47just rolled it out. Some people swear by it.
- 39:49Some people don't use it. Your mileage may vary.
- 39:52And we talked about Clearbrief, which my firm
- 39:55does use. We spend something like $67 ,000 a
- 39:59year for seat licenses for 37 lawyers and five
- 40:04paralegals. And so far, we find it worth it.
- 40:07I suspect that Clearbrief is probably not going
- 40:10to survive. It's going to get acquired and built
- 40:14into somebody else's product at some point. And
- 40:16the reason I said that is it's really good at
- 40:18certain things, but it only does certain things.
- 40:20And there are other products like Westlaw Precision
- 40:23that will eventually do everything Clearbrief
- 40:26does and other things, too. So don't sign any
- 40:29long -term licenses. Lexus will tell you, come
- 40:33on over. We'll give you a discount. We want three
- 40:35years. Don't do it. Don't go longer than a year,
- 40:40because all of this stuff is changing. So we
- 40:42tested Lexis Plus AI. We put together a beta
- 40:45group of four lawyers and a paralegal, and we
- 40:47tested co -counsel. On the co -counsel project,
- 40:52some people loved it. A lot of people were kind
- 40:55of indifferent to it. And of the five people
- 40:57who started, two of them asked to keep their
- 40:59licenses after the test was over. And we let
- 41:01them until Westlaw acquired it and then started
- 41:05asking for a whole lot of money for it, at which
- 41:07point we let it go. But co -counsel is a generative
- 41:11AI tool that uses legal resources and your resources
- 41:16to produce text for you. We tested Lexis plus
- 41:21AI and we thought it was pretty nifty. The reason,
- 41:24and it was cheaper than Westlaw, but we would
- 41:27have had to ask everybody in my organization
- 41:29to give up all the treatises they've become comfortable
- 41:32with in the Rudder series and in a few other
- 41:34series that Westlaw has the exclusive license
- 41:37for. Lexus has competing products on all of those
- 41:41lines, but I just didn't think that the modest
- 41:45cost savings, it was like 20 or 30 grand a year
- 41:47for the whole firm and the coolness of the new
- 41:51tool. and it's apparent utility really justified
- 41:55throwing a wrench in the workflow of everybody
- 41:57on my team. I didn't want to get lynched by my
- 42:00team. There are other players out there, of course,
- 42:04RDAI is really expensive, something like a million
- 42:09dollars a year minimum licensing fee. It vectors
- 42:14your own databases. So it trains the model on
- 42:18your precedent. It can produce precedent in your
- 42:21style and in your voice. My impression is only
- 42:25the really big AMWA 100 firms use Harvey. Vlex's
- 42:30Vincent is a product that is heavy on the law
- 42:33of other nations. So those of you in a transnational
- 42:36practice may know more about this product than
- 42:38I do. And as I said, there's a new product a
- 42:41week in this space and these are just the ones
- 42:43that I've given some thought to. So rapid innovation
- 42:46in this industry requires periodic reassessment
- 42:49by those of us who consume its tools, and you
- 42:51should expect continued evolution and specialization.
- 42:56The flavor of the month right now is agentic
- 42:58AI, which is using AI to create agents that do
- 43:02things, like operating a chatbot, like making
- 43:07travel reservations for you, like taking minutes
- 43:10at meetings. Lots of agents are proliferating,
- 43:13and there are tools out there that allow you
- 43:15to create your own agents if you're at all software
- 43:18handy. So a lot of evolution, a lot of specialization
- 43:22and really niche tools are coming. My partner,
- 43:27my colleague who is our Chief Financial Officer
- 43:30is looking to automate the way that we check
- 43:36our balances in our bank statements each month.
- 43:40Reconciling the bank statements is essential,
- 43:42both to fulfill our ethical obligations on the
- 43:45client trust account, but to make sure that we're
- 43:46not getting stolen from or that we're mismanaging
- 43:48our finances. That's tedious. And if somebody
- 43:51makes a mistake in the way they code a particular
- 43:54transaction, it can take hours to figure it out.
- 43:57He's putting together an agent that will cut
- 43:59down that time for him and eventually do it for
- 44:01him. And I suspect that he'll get there. And
- 44:03then eventually somebody in the industry will
- 44:05come out with a product design just to do that.
- 44:07We'll reconcile your bank accounts for you. That's
- 44:10the kind of specialization that I think we'll
- 44:12be seeing. Next slide. Closing thoughts before
- 44:18I turn to the Q &A, and I've left 15 minutes
- 44:20for Q &A. I've said this several times, but I'll
- 44:23say it again. AI enhances but cannot replace
- 44:27legal expertise. You're still the lawyer. Your
- 44:30license is still riding on everything you file
- 44:32in court, everything you send out on your letterhead.
- 44:34So use these tools thoughtfully, but don't be
- 44:37afraid to use them. I think they're powerful,
- 44:40helpful, and your competitors are using them,
- 44:42and you should. So I end every presentation on
- 44:44every article. You may have seen my columns in
- 44:46the Daily Journal about this with, come on in,
- 44:48water's warm. And I think the next slide will
- 44:51say questions, and it does. So let's open the
- 44:55questions and see what we got. So Ralph Lamarande
- 45:01says, my understanding, which could be wrong.
- 45:03is that all the AI legal programs are relying
- 45:05on and based upon the public services such as
- 45:08Chat, GPT, Claw, Gemini, et cetera. If this is
- 45:11true, other than confidentiality issues, what
- 45:14are the main advantages in utilizing the fancy
- 45:16and pricey legal -specific AI services? It is
- 45:20true that producing a large language model or
- 45:24an LLM is hugely expensive and maintaining and
- 45:29operating one all the more so. You talk about
- 45:33the power and the water demand, and folks who
- 45:36are wanting to reactivate nuclear power plants
- 45:38to run these things, it's a huge endeavor. So
- 45:41the back office, so to speak, is going to be
- 45:44one of the three or four main large language
- 45:47models. But that's a little bit like saying,
- 45:50because all the chips in computers were at one
- 45:54time made by Intel, I don't need a computer.
- 45:58You do. You can't really make good use of these
- 46:01large language models without a front end. And
- 46:04the front ends are really different. The ChatGPT,
- 46:08WOD, and Gemini tools that you can get for free
- 46:11or for a low cost are using the entire internet
- 46:16as their information base. And they give no more
- 46:20authority to a Supreme Court opinion than your
- 46:23drunken uncle's rant. All of it's text, and it
- 46:27uses it all alike. So you'll get more controlled
- 46:31outputs relying on a, what I call a sandbox,
- 46:36a defined unit force of quality stuff if you
- 46:40use the AI specific legal services. But if you're
- 46:42not ready to buy them yet, you can still use
- 46:45these other tools. You just have to know they're
- 46:48not as powerful and that they will hallucinate,
- 46:50particularly about legal stuff. Okay, Jeff Branstetter,
- 46:55you mentioned that Westlaw Precision was cold
- 46:58on the hallucination scale. In your review of
- 47:01LexisNexis's AI tool, did you find it equally
- 47:04cold or more likely to hallucinate? I don't really
- 47:07remember, but I suspect that all of the tools
- 47:11marketed to lawyers are going to be cold because
- 47:14think about who they're selling to. If you show
- 47:16up in court on a sanctions order because of hallucination,
- 47:19are you going to be happy with your vendor? Probably
- 47:22not. So I would expect the law tech tools to
- 47:27lean on the cold side. But I'm just speculating
- 47:30here. I don't know. Daniel Smith asked, what
- 47:35product do you recommend to edit your brief for
- 47:37clarity, brevity, and simplicity? Primarily,
- 47:41I use WordRig. I know colleagues who will use
- 47:46things like Co -Counsel or ClearBrief or ChapGPT
- 47:49or Claude. It's a little awkward. You've got
- 47:52to cut it paragraph by paragraph and dump it
- 47:54into the winnow and ask for an output and then
- 47:56absorb the output. I think that's a little too
- 47:58clunky. I'd rather just use my own editing skills,
- 48:01but word break is really helpful because it gives
- 48:04you an orange line underlining everything that
- 48:06requires your attention. David Azar says, what
- 48:10blogs or news sources do you find have the best
- 48:14updates for developments in AI for legal work?
- 48:17There's two that come to mind. One is the ABA
- 48:21has a law tech newsletter that comes out every
- 48:24week. I can't get past the stories into the content
- 48:30because I'm not paying to be an ABA member. But
- 48:34just reading the headlines often sends me out
- 48:36to the general internet to go find that story.
- 48:39So the ABA's law tech tool is pretty useful.
- 48:43And then there's something that I subscribe to.
- 48:46that shows up in my personal email box, and I'm
- 48:49blanking on the name of it, so I'm going to go
- 48:51look, and I look at this periodically too. I
- 48:55signed up for the wrong one and got one that
- 48:56was aimed at coders, and I'm not a coder, so
- 48:58I sort of have to ignore that one. But there's
- 49:00another one that is pretty useful. Let's see
- 49:02if I can find it for you quickly, without wasting
- 49:05anybody's time. Byte Byte Go. I think is the
- 49:15one that's aimed at people who code. And I'm
- 49:19not quickly finding the other tool, but you might
- 49:23just go out on the net and look for a substack
- 49:29that follows. um law tech and uh and that follows
- 49:35AI. There's a bunch of them out there and you'll
- 49:37find one that's pretty useful. If you're looking
- 49:38at this more from the policy side there's a professor
- 49:40at the University of Pennsylvania's law school
- 49:42who writes a substack on this subject as well
- 49:45but that's that's a little that's not practical
- 49:47that's more um uh top level theory kind of stuff
- 49:51not quite academic. Okay how about the use of
- 49:55AI generated videos and photos and trials and
- 49:58as evidence? What if it's being used by the litigants
- 50:01and their attorneys to mislead the courts? AI
- 50:05can produce images. AI can generate videos. My
- 50:11birthday was this week. Somebody sent me a video
- 50:13of Kevin Bacon singing me Happy Birthday with
- 50:17a pig playing a harmonica. It was very funny.
- 50:19I laughed. It lasted like three minutes. And
- 50:23I'm quite sure that Kevin Bacon did not sing
- 50:25me Happy Birthday. These technologies are kind
- 50:28of amazing. They're suffusing the world around
- 50:31us, and they're going to make their way into
- 50:34court. And as Justice Streeter on the first district
- 50:40said on the panel I was on, the whole idea of
- 50:44trials is built around resolving disputes of
- 50:48fact based on reliable evidence, using a set
- 50:51of rules, arguments, and jurors to separate fact
- 50:55from fiction. authenticity is one of the fundamental
- 50:59ideas of our evidence system. And how can you
- 51:03authenticate anything in a world of deepfakes?
- 51:06I don't quite know the answer to that question.
- 51:09I think we're all gonna be struggling with that
- 51:11question. But I think we'll use the tools we
- 51:14have available today, which is to say, does it
- 51:19look plausible to you? Does it have any timestamps
- 51:23or other? Does it have a paper trail? Does it
- 51:26have a Genesis trail? Is it legit? We'll use
- 51:31the tools we've got available. I think the industry
- 51:33recognizes this problem too, and they're going
- 51:36to start using things like watermarks to authenticate
- 51:40their outputs. But yeah, non -trivial question,
- 51:46very limited answer because we're all really
- 51:48checking on it. So Jack Russo asks, is QuickCheck
- 51:52really bulletproof? I think so. The reason I
- 51:56think it's bulletproof is what am I asking it
- 51:59to do? I'm asking it to confirm that an authority
- 52:02exists and that it has not been overruled. Well,
- 52:08Shepard's has been doing that ever since it was
- 52:09called Shepard's Gummed Citations, where you
- 52:12had a list of sites that you glued into your
- 52:13paper book. When was the last time any of you
- 52:16opened a volume of shepherds? None of us do that
- 52:18anymore. We go into Westlaw and we open Keysight.
- 52:21Well, QuickCheck is just another front end for
- 52:24Keysight. And it is not AI technology. It's just
- 52:28looking things up in databases. So if I can't
- 52:32prove it's hypnotic, the judge can't either.
- 52:37The judge is using the same tool I'm using. So
- 52:39I think QuickCheck is a very It's the safety
- 52:44net that I'm prepared to rely on. So Ralph Lamarande
- 52:50says, can you compare WordRig versus Grammarly?
- 52:53No, I haven't used Grammarly, but I will tell
- 52:55you that every time I talk about WordRig, people
- 52:57in my audience bring up Grammarly. So it's got
- 52:59its fans, it must be a good tool, but I've never
- 53:01used it. Elizabeth Newton, which AI tool did
- 53:07you say you used to convert your PowerPoint template?
- 53:10and documented AI use cases into this presentation.
- 53:13Is the tool reliable? I used ChatGPT 5 .0 to
- 53:17produce these slides. And the output was a pretty
- 53:20good summary of the paper that I started with.
- 53:24It did use my format, protected my branding.
- 53:30All the text was too small. I had to increase
- 53:33it from 18 points to 30 points. It skipped a
- 53:37few points that I thought were essential. And
- 53:39it repeated itself a bit. You'll see that I repeated
- 53:44myself a bit, particularly when I was telling
- 53:46you, you've got to be the driver. You can't rely
- 53:48on AI. I left in some of the repetition for pedagogic
- 53:52reasons. I wanted to drill the point home. If
- 53:55I was writing a brief, I would be less comfortable
- 53:58with that kind of repetition. So I used Chat
- 54:00GPT 5 .0. The only reason I needed 5 .0 is I
- 54:03wanted some images and ultimately I decided the
- 54:05images were taking too long and weren't that
- 54:06helpful. So I ended up with something that Chat
- 54:09GPT for free over the internet could have produced
- 54:13for me. Blockchain tokenization, or we go back
- 54:18to silver halides. I think that's right. The
- 54:21point that Ben Williams is making is one way
- 54:24to authenticate electronic artifacts is to associate
- 54:28a blockchain with each. And of course, blockchain
- 54:30technology is this technology that's used for
- 54:33Bitcoin and for tokens of various kinds that
- 54:38I'm told, cannot be altered. The chain of provenance
- 54:43for that artifact is perpetual, perfected, and
- 54:47stored in multiple servers around the world.
- 54:49And I think Ben is quite right. That is how we're
- 54:52going to be authenticating these products going
- 54:54forward. So I see that I'm out of questions,
- 54:58but we are not quite out of time. I promised
- 54:59you an hour. I can go back to my overview slide
- 55:04and think of a few more things that I might like
- 55:06to say, or you all can type quickly. So let me
- 55:10take a breath and a drink of water and see if
- 55:11there's any questions that will appear in my
- 55:13question box. OK, we actually did have one from
- 55:18the help. desk that came through email. I do
- 55:21have one here. Any best practices for AI aided
- 55:25issue compression so the reply focuses on dispositive
- 55:30errors the panel actually cares about? Yes. I
- 55:37told you that these tools are wordy and when
- 55:40you're in brainstorming mode they can be especially
- 55:43so. I usually start with a over -inclusive product.
- 55:51I tell my associates and my colleagues, you can
- 55:54write about 10 % long, because whatever you sent
- 55:56me in just editing it, I'm going to make it 10
- 55:58% shorter. And if I think you've got arguments
- 56:01in it that I don't like, it's going to be more
- 56:03than 10%, which is why I get an outline first.
- 56:06But if they're writing from an approved outline,
- 56:08I can usually make anything anybody gives me
- 56:0910 % shorter just because I'm a good editor.
- 56:13These tools will produce too much, and you need
- 56:16to cut them back. The judgment about how to do
- 56:23that is ultimately yours. But if you've got 25
- 56:27points on the page and you only have room for
- 56:29three, you might ask ChatGPT or Clearbrief or
- 56:35Westlaw Precision of these 25 arguments, which
- 56:38three do you find most persuasive and why? I
- 56:42would not take its advice literally. but I would
- 56:45take its advice into account when evaluating
- 56:48these things. Somebody's asked if I've tried
- 56:51relativity and the answer is I have not. Sorry.
- 56:54That was Joy Morris. Jack Russo, how is your
- 56:58firm handling hallucinations and opposition briefs?
- 57:00And he's citing Rule 11 and CCP Section 128 .5.
- 57:05All I will tell you is to my knowledge, I haven't
- 57:08encountered any yet. I've occasionally come across
- 57:10significant typos. I've had people cite cases
- 57:14that really don't say what they say they say,
- 57:17but that's been part of legal practice as long
- 57:18as I've been in this business. But I've not yet
- 57:21encountered hallucinations from the other team,
- 57:24knock on wood. Kathleen March, I use Westlaw
- 57:29Precision. Haven't had any occasion where it
- 57:32invented or hallucinated a case or a statute.
- 57:35Do you have any specific instances of that happening?
- 57:37Westlaw claims it does not happen. What I would
- 57:40point you to is a study done by three law professors
- 57:43that tested a whole range of AI products and
- 57:47reported the error rate. And the error rate for
- 57:51these tools was very low, was as low as 1%. Whereas
- 57:55for ChatGPT, the internet version, the free version,
- 57:59it can be 20 or 30 percent, but it's still there.
- 58:03And Westlaw may deny it, but at least that paper
- 58:06said it's still there. And given that it's my
- 58:09license riding on it, I'm going to assume it's
- 58:13still there. I see we have two minutes. I don't
- 58:17see any open questions. Have you got any on the
- 58:20back office side there? Okay, Rachel Steinbach.
- 58:26Do the policies, protocols, and guidance that
- 58:29you provide your staff differ in any way depending
- 58:31on the level of experience? For example, do you
- 58:34allow your junior associates, first year out,
- 58:36otherwise an experienced, to use AI as a junior
- 58:38associate? I'll let anybody on my team use it.
- 58:41I will sometimes direct people on my team to
- 58:44use it. I have often sent a paralegal, who is
- 58:47particularly skilled in using clear brief, off
- 58:49to generate the initial chronology. I've often
- 58:57sent one of my mid -level colleagues who's looking
- 59:01to draft something for a city for the first time
- 59:03where we don't have good precedent. I said, why
- 59:05don't you ask CHAT GPT to draft one for you?
- 59:08Why don't you go see what an AI tool can do for
- 59:10you? It's not the end of the road. It's going
- 59:13to be a very rough first draft and you still
- 59:15have to do your legal work. But I let anybody
- 59:17use it, provided that everybody in the team knows
- 59:20they're using it. And I've told it, we have a
- 59:22written policy on AI tools. We've just updated
- 59:25it because things have changed in the last two
- 59:27years. One of the things it says is, thou shall
- 59:31not conceal that you've used it. Thou shalt report
- 59:34that you've used it. And the more senior lawyer
- 59:38is ultimately responsible for the product that
- 59:40was out the door. So Ralph Limeronde says, are
- 59:44you aware of judges now using AI for any reason?
- 59:47Hell yeah. The California judiciary has just
- 59:50published a policy about the use of AI in judicial
- 59:54administration. They have not yet touched the
- 59:57use of AI in the judicial process. There is a
- 1:00:02story you can find on the internet now, where
- 1:00:04federal judges, I think in Mississippi, were
- 1:00:07found to have issued orders that had hallucinated
- 1:00:11content in them. So judges and human too, they're
- 1:00:15capable of making the same mistakes that lawyers
- 1:00:17are. and they're doing. So I think you should
- 1:00:20assume the judge is using AI. And if you get
- 1:00:23an order that seems a little wonky to you, you
- 1:00:26might want to do a request for reconsideration
- 1:00:28and ask the judge to confirm that's what he or
- 1:00:31she really intended to say. It's 1 .31. We promised
- 1:00:36you an hour. We've gotten there. I'm willing
- 1:00:38to stick around for another minute or so if there's
- 1:00:40another question or two. Otherwise, I'll send
- 1:00:42you all back to your other more productive pursuits.
- 1:00:52This concludes today's presentation. Thank you
- 1:00:54so much, Michael, for your time and thank you
- 1:00:56so much, your audience. Have a great day, everybody.