Latest / CLEs You Actually Want to Hear / Responsible AI Use Within Law Firms: Policies, Training, and Oversight for 2026
Transcript
- 0:03Hello, everyone. Welcome to today's program,
- 0:07Responsible AI Use Within Law Firms, Policies,
- 0:10Training, and Oversight for 2026. I'm Alapoli
- 0:14Castro, Director of Education and Professional
- 0:16Development at Beverly Hills Bar Association.
- 0:18And today I'm delighted to introduce our speaker,
- 0:21Oliver Roberts. Oliver is co -director of the
- 0:25Wash U Law AI Collaborative and an adjunct professor
- 0:28of law, where he teaches at the intersection
- 0:31of artificial intelligence policy and Law. He
- 0:34is also CEO and founder of Wickard, co -head
- 0:37of the AI Practice Group at Holtzman Vogel, and
- 0:40editor -in -chief of the AI and the Law publication
- 0:43at the National Law Review. Oliver designed and
- 0:47taught the nation's first required law school
- 0:49AI course and has developed AI education programs
- 0:52both domestically and internationally. His commentary
- 0:56on AI policy and technology appears regularly
- 0:59in Bloomberg Law, National Law Review, The Hill,
- 1:02and Law .com. As a practicing attorney, he handles
- 1:05high impact litigation, including appellate work
- 1:07before the Third Circuit Court of Appeals. We're
- 1:10so excited to hear what he has to share with
- 1:12us today. Please feel free to put your questions
- 1:14to him in the Q &A box, and I'm going to turn
- 1:16it over to him. Thank you so much and thank you
- 1:23to the Beverly Hills Bar Association for having
- 1:25me. Really appreciate being back here. This is
- 1:28now my third time at the Beverly Hills Bar Association.
- 1:32Second time on AI and also did a tax presentation
- 1:35back like three years ago, because I had one
- 1:37tax case before the Third Circuit to be presented
- 1:41about that. But today's focus is all about responsible
- 1:44AI use within law firms. This is, of course,
- 1:46a very hot topic. Trying to balance the two competing
- 1:50interests of rolling out AI and being an innovative
- 1:52law firm, but also being well aware of your professional
- 1:56and ethical obligations. And those are not mutually
- 1:59exclusive. It's very important to have both.
- 2:01So I already had my biography read. I'll just
- 2:04note that I'm managing attorney at the Robert's
- 2:06legal firm, actually left and started my own
- 2:08firm a couple of months ago, focused on litigation
- 2:11and also advising. tech companies and organizations
- 2:14on AI implementation. So I probably don't need
- 2:18to go in depth and tell you how important AI
- 2:20is as a topic today. You really can't go to a
- 2:24webinar conference anywhere without someone bringing
- 2:27up AI in a legal profession. It is of course
- 2:30touching legal research. It is touching drafting,
- 2:34contract review, summarizing deposition that's
- 2:37being used widely in e -discovery. So it is all
- 2:40over the place and it's also very much leveraged
- 2:43in marketing now where law firms are adopting
- 2:46AI tools. Some law firms, candidly, are adopting
- 2:49AI tools for the sake of adopting AI tools. And
- 2:52then internally, they'll tell everyone in the
- 2:54firm, we've rolled out AI tools, and then very
- 2:56few people actually use them. And I can tell
- 2:58you that's a reality because some law firms will
- 3:00actually hire my company, Wicker, to come in
- 3:02and help show them how to actually use those
- 3:04tools. So we're very much in a weird, I'll call
- 3:07it a weird environment where it's not like the
- 3:09past where law firms would buy technology and
- 3:12tools because they see there's efficiency gains.
- 3:15We're now in a position where some law firms
- 3:16are just buying these tools so that they can
- 3:18say, we have the same tools that our competitors
- 3:21have. And so they can make the announcement and
- 3:22say, we're an AI first, AI native, technology
- 3:25first, whatever marketing term you want to throw
- 3:27out there. Firms like to be able to say that
- 3:29because it shows the market they're innovative.
- 3:31It's going to help them get clients potentially
- 3:33in proposals. But the reality is. The adoption
- 3:37rates, I don't think actually, when I say adoption
- 3:39rates, I mean actual usage of AI tools daily
- 3:42and weekly by lawyers. I don't think it actually
- 3:44matches up to what all the marketing hype says.
- 3:47So we'll dive into that a little bit more, but
- 3:49that does not discount the fact that even at
- 3:52firms that give their lawyers access to AI could
- 3:55potentially be even more problematic. If a firm
- 3:57just buys AI for the sake of buying AI, tells
- 4:00all their lawyers, here's licenses, but then
- 4:02leaves them without direction. It doesn't tell
- 4:04lawyers what can you use it for, what should
- 4:06you not use it for. You don't give them proper
- 4:08training and guidance. That is a very problematic
- 4:11scenario where you might be at more risk when
- 4:14your lawyers don't have clear direction on how
- 4:15to use these tools because your firm just adopted
- 4:18it for the sake of adopting AI. I'm going to
- 4:21give you some stats here just to round it out
- 4:23rather than me providing just anecdotal evidence
- 4:25of the AI adoption or lack thereof. This is a
- 4:28recent study that came out from Thomson Reuters.
- 4:31I always say this. When you look at a legal tech
- 4:34company releasing a survey about AI adoption,
- 4:36I think you should look at it pretty skeptically.
- 4:38So I'm gonna provide these numbers with a strong
- 4:40caveat that, again, this is a legal tech company.
- 4:42Legal tech companies have a very strong financial
- 4:45incentive to tell you, a practicing lawyer, how
- 4:48widely adopted AI is. Because then it creates
- 4:50a FOMO effect. If you see a stat that says 97
- 4:54% of lawyers now use AI, and you're an attorney
- 4:56and you're not using AI, you're gonna be like,
- 4:59oh crap, I gotta start using AI, I am falling
- 5:01behind. And at 97 % of AI adoption sounds ridiculous,
- 5:05but in September of last year, there was actually
- 5:08a group of researchers that released a published
- 5:11study that said 97 % of lawyers are using AI
- 5:14now. And it's quote, universal throughout the
- 5:16legal profession. But then you actually look
- 5:18into their study and they surveyed like 82 people
- 5:21and they found those 82 people within the researchers
- 5:24own like LinkedIn network, which is just absolutely
- 5:27ridiculous. There's over a million lawyers in
- 5:29the country. They surveyed 82. the people they
- 5:32chose heavily biased and skewed toward their
- 5:34connections, who are obviously interested in
- 5:36AI because they're in the same network, and then
- 5:38use that to extrapolate and say 97 % of lawyers
- 5:41use AI. Totally ridiculous. So that is why I'm
- 5:44heavily skeptical of all the surveys I say, all
- 5:47the adoption rates. Another one is Clio. They
- 5:49released a report in 2024 saying that 79 % of
- 5:52lawyers are using AI. They ran a study again
- 5:55this year in fall this past year, 2025. Same
- 5:59exact number. According to them, there's been
- 6:02no change, but I think this really goes to how
- 6:04I don't think these surveys are reliable. Look
- 6:06at the definitions of AI. The reality is there's
- 6:09two issues there. One, how are they defining
- 6:11AI? Who are they surveying? Are they just sending
- 6:14this survey out to their customers or are you
- 6:16already subscribed to their AI platform? Obviously,
- 6:19they're going to say they're using AI in higher
- 6:20numbers if they're only reaching out to their
- 6:23customer base. And the other concern is it also
- 6:25presupposes that all these companies are using
- 6:27the same definition of AI. presupposes that a
- 6:30lawyer even knows whether they're using AI or
- 6:32not. I would venture to guess that if you went
- 6:35and talked to a bunch of lawyers that are using
- 6:37AI, some will say, yeah, of course I am using
- 6:39Lexis protege or Westlaw as co -counsel. But
- 6:42then another lawyer might say to you, no, I'm
- 6:44not using any AI. But then you ask them, what's
- 6:47your practice area? And they'll say litigation.
- 6:49Then you can ask them, are you using Westlaw,
- 6:51like traditional legal research? And they'll
- 6:53say, yeah, of course. Well, that's powered by
- 6:56AI as well. When you make a search inquiry, on
- 6:59a legal research platform, the answers, the responses
- 7:01you get, all those cases that are produced, it's
- 7:04not just giving you random cases. It's not just
- 7:06giving you the most recent case in the database.
- 7:08It is using algorithms, which is a form of artificial
- 7:11intelligence, to surface what it thinks is the
- 7:13most relevant cases to your search inquiry. So
- 7:17yes, AI has been around for a very long time.
- 7:19So that also shows one of the weaknesses of trying
- 7:21to survey lawyers about AI use and adoption.
- 7:24Even the lawyers may not actually know whether
- 7:26they're using AI or not. So it really undermines
- 7:28the accuracy of these studies. With that very
- 7:31heavy caveat, here are some stats if you want
- 7:34to put some numbers around it. But again, look
- 7:36into the survey more yourself. I would take it
- 7:40with a grain of salt. According to this report
- 7:42from this year, there's been a doubling in generative
- 7:47AI use among professionals. And this is not just
- 7:49legal professionals, this is professionals more
- 7:51broadly. Over 80 % of current generative AI users
- 7:55use it weekly. Over 40 % say that they've been
- 7:58told explicitly to use AI or not use AI on client
- 8:02work, depending on the client, which that really
- 8:04reflects the fact that clients are making more
- 8:06informed decisions on AI, whether explicitly
- 8:09saying, yes, we need to use it for efficiency
- 8:11gains or no, do not use it. I can tell you anecdotally,
- 8:14talking to chief innovation officers, talking
- 8:16to leaders at law firms, a phenomenon happening
- 8:19right now is a client goes to you, a lawyer,
- 8:22and says, I want you to use AI so you could help
- 8:25cut my billable costs. But then the reality is,
- 8:27I don't think we're really seeing huge returns
- 8:30on that, where a lawyer may be using AI for legal
- 8:32research or legal drafting, but it's not necessarily
- 8:35being reflected in less billable hours. A lot
- 8:37of that may be due to the fact that you need
- 8:40to double check your work. you still need to
- 8:41verify. Like if I'm doing legal research, I might
- 8:44use co -counsel from Thomson Reuters. I'll be
- 8:47able to put in a search inquiry using the agentic
- 8:49search, let it run on autopilot while I do other
- 8:51things. But then when I get the result, I know
- 8:53right now I can't trust those citations 100%.
- 8:57I cannot trust the legal propositions or quotes
- 8:59appended to those citations. Therefore, I'm going
- 9:03to have to spend time still going through all
- 9:04that case law, verifying the legal propositions,
- 9:07reading the case myself. So it does save a little
- 9:09bit of time just pointing in the right direction
- 9:11and perhaps servicing the most relevant cases.
- 9:14But you're still spending a lot of time re -reviewing
- 9:17those citations. On the drafting side, may cut
- 9:20down on costs, especially contract review. You
- 9:22don't have to start from scratch. You could easily
- 9:24pinpoint provisions that may be problematic.
- 9:26So you may see more time saving there. I think
- 9:29one of the few areas that's actually seeing clear
- 9:32cut cost savings is in eDiscovery. Of course,
- 9:35you have tens of thousands of documents in eDiscovery.
- 9:38Imagine you're on a huge case. You're able to
- 9:40now cut that review time significantly. And as
- 9:42long as you have a defensible basis by which
- 9:44you're carrying out that methodology, you could
- 9:46see. savings there that you can pass on to the
- 9:50client. And then finally, very interesting stat
- 9:52here. 50 % of publicly available tools such as
- 9:55ChatGBT are still the most commonly used. Across
- 9:58all the surveys, again, take them with a grain
- 10:00of salt, but all the surveys I have seen from
- 10:02Law .com, from the ABA Legal Technology Survey,
- 10:06Thomson Reuters here, they all say general purpose
- 10:08models are the leading legal AI tools used among
- 10:12professionals. I throw in Microsoft Copilot in
- 10:14there as well. A lot of companies have access
- 10:16to it just by virtue of having access to the
- 10:18Microsoft suite. So even for lawyers, it is the
- 10:22general purpose models that are most widely used.
- 10:24That is very important to understand, especially
- 10:26as we now see in many cases, for example, Southern
- 10:30District of New York, one out in Colorado, one
- 10:32in Eastern District of Michigan, where judges
- 10:34are now making rulings on whether using a general
- 10:38purpose model like Claude, that was in the instance
- 10:40of Southern District of New York, whether using
- 10:42Claude for legal communications could waive attorney
- 10:46-client privilege, or whether it's actually can
- 10:48you claim work product doctrine to protect your
- 10:51communications with a general purpose chatbot.
- 10:53When I say general purpose, I'm talking about
- 10:55ChatGPT, Claude, and Gemini. They're general
- 10:58purpose models, meaning they're just open to
- 11:00the public. They're not tailored specifically
- 11:02for the practice of law. That is in direct contrast
- 11:04with Lexis, Westlaw, Harvey, Lagora, all these
- 11:09tools that are marketed to lawyers specifically
- 11:12that have added UI UX, that have added database
- 11:15access for the specific practice of law. So general
- 11:18purpose models are still winning in terms of
- 11:21adoption rates based on all the surveys I've
- 11:24seen. And also I'll say based on anecdotal evidence
- 11:27in my communications with firms. I'm also a writer
- 11:31at the National Law Review. We recently did some
- 11:32interviews with some smaller practitioners, some
- 11:35firms between one lawyer all the way up to 15
- 11:38lawyers, and they are also adopting general purpose
- 11:40models. The rationale for it is not the performance,
- 11:44but it's just the cost. Some cost comparisons.
- 11:47I've seen recent reports that Harvey charges
- 11:49a couple thousand dollars per seat per month,
- 11:52whereas you could go out there and get a chat
- 11:53GPT paid subscription, $20 a month. You can roll
- 11:57out enterprise license of Google Gemini for $50
- 12:00per seat per lawyer. So that's a lot cheaper
- 12:03than buying access to the legal tech specific
- 12:05tools. That of course comes with trade -offs.
- 12:08For example, Harvey has a LexisNexis tie -in
- 12:12where essentially you could access LexisNexis
- 12:14legal research cases when you do legal research.
- 12:18Thomson Reuters co -counsel, of course, has access
- 12:20to the Thomson Reuters database of cases. So
- 12:24rather than just getting a search result, you're
- 12:26actually getting a search result hyperlinked
- 12:28to real cases. So you can easily open it up and
- 12:30then find the answer. And those answers are grounded
- 12:33in real case law. Do they still hallucinate?
- 12:36Yes. And we'll get into that later on. However,
- 12:38it does so at a much lower rate and also services
- 12:40the relevant case law much quicker, as opposed
- 12:43to you going out and trying to use chat GPT or
- 12:45Claude for legal research. Good luck to you.
- 12:47because you never know whether the cases you're
- 12:50going to get are real or not. Oftentimes, it'll
- 12:52link to Justia or some other internet web page
- 12:55to show you the case. And it doesn't actually
- 12:57link you to a real reputable database like Alexis
- 13:00or Westlaw. So now you find yourself in a situation
- 13:03where you're using ChatGPT or Claude. You get
- 13:05some case results. You got to copy and paste
- 13:07the case citation, try to go find it independently
- 13:09in Westlaw or Alexis, verify it, and then even
- 13:12see if that information was accurate. there are
- 13:15improvements on the legal tech products that
- 13:18do deliver better value. But again, it's an open
- 13:20question whether that value is better vis -a
- 13:23-vis the actual cost of the seat of using that
- 13:25product in the first instance. So shifting over
- 13:28to some of the professional responsibilities
- 13:30to consider for lawyers. Imagine this, you're
- 13:33considering adopting AI tools or you already
- 13:35have. These are some of the things that should
- 13:37really be top of mind when we're talking about
- 13:39your ethical and professional obligations. Number
- 13:42one, competence. There's a requirement that's
- 13:44been around pre AI and all these rules have been
- 13:47around pre AI. Californians are currently considering
- 13:50amendments to its rules of professional conduct.
- 13:55But to my knowledge, they still haven't passed
- 13:57yet, but essentially they're not fundamentally
- 13:59changing the rules. These rules are still going
- 14:02to be. important, they're going to stay as is,
- 14:05but there's increased clarification of their
- 14:07application to modern technology and AI. So there
- 14:10might be slight changes coming down the pipeline,
- 14:12but these core rules still stay in play and are
- 14:15still extremely important. Number one, competence.
- 14:18So I'm going to jump in a bit to going more in
- 14:22depth on these, but I'm going to run through
- 14:23them quickly here. So competent, you need to
- 14:25be competent using these tools. You need to fully
- 14:27understand the benefits and risks before you
- 14:29start using them in actual client work. You need
- 14:32to maintain confidentiality. You need to know
- 14:34if I'm putting my client information into AI
- 14:38tool XYZ, is this going to maintain confidentiality?
- 14:42Or is there an instance where my client information
- 14:45might get trained on? or it might be disclosed
- 14:47to other users within my firm. That can be highly
- 14:50problematic if there's a Chinese firewall set
- 14:51in place. And there's even some instances where
- 14:55you put in data into this chatbot and that data
- 14:58could go to other users at other firms who are
- 15:00using the same product. Very rare for that to
- 15:03happen with established legal tech companies.
- 15:05But if you're working with an early stage startup
- 15:07and they don't have their stuff together, could
- 15:10be a reality that happens. Communication with
- 15:12clients. This gets down to the question of, do
- 15:14I actually need to disclose my use of AI to my
- 15:17client? Do I need to get their permission before
- 15:20I use AI? We'll jump into those questions later
- 15:22on. Supervision, this is now a super hot topic.
- 15:26Supervision, there's always been this obligation
- 15:27for you as a supervisor, you don't even have
- 15:30to be the managing attorney. If you're just a
- 15:31partner at a firm and you have associates working
- 15:33under you, you have a counsel working under you,
- 15:35some are associates, you are responsible for
- 15:38the acts that they engage in. If you're not properly
- 15:40supervising their use of AI, and your summer
- 15:42associate does some research, includes a fake
- 15:46citation, you add that to the document that you're
- 15:49about to file, say motion to dismiss, and the
- 15:52court says, hey, here's an order to show cause.
- 15:54Why should not sanction you for including a fake
- 15:56site? It is no excuse whatsoever to say, sorry,
- 15:59that was a summer associate. Although I'll actually
- 16:02caveat that because there was one case in which
- 16:05a lawyer did that and the lawyer actually had
- 16:08the summer, it was a law school intern. the law
- 16:11school intern signed on to the filing, and I
- 16:13think this was actually California, signed on
- 16:15to the filing, and the judge did not sanction
- 16:19the lawyer because the judge said the law student
- 16:21also signed on. So to avoid getting the law student
- 16:25getting in trouble as well, the judge actually
- 16:28sidestepped sanctions. That lawyer, then a couple
- 16:32months later, filed another fake citation. So
- 16:35the lawyer got off in that case but then was
- 16:37actually sanctioned because the lawyer did it
- 16:39again. but that was a little side note. The reality
- 16:44is it's not going to be an excuse for you to
- 16:45say some summer associate or someone under me
- 16:48filed that fake site, your honor, you filed the
- 16:51pleadings, you're responsible for it. Even if
- 16:53you do not file the pleadings and you don't sign
- 16:55off on them, but you are supervising that associate
- 16:58or you're supervising the team that filed that
- 17:00fake site, you could also potentially get sanctioned.
- 17:03That happened in a case involving everyone's
- 17:05favorite personal injury firm, Morgan & Morgan.
- 17:08This was February 2025. They retained local council
- 17:11in Wyoming. It was Morgan & Morgan actually doing
- 17:14all the work. The local council signed off on
- 17:16it. It was a motion in Lemonay. They included
- 17:18eight fake citations. Eight out of nine of them
- 17:20were fake. And the judge ultimately sanctioned
- 17:24the local council, sanctioned the Morgan & Morgan
- 17:26attorney, and sanctioned the supervising partner
- 17:29of the partner that actually filed the fake citations.
- 17:33So just goes to show. If you're supervising,
- 17:36you're responsible, even if you don't file the
- 17:37pleadings, sign off on them yourself. If you're
- 17:39involved in the work product and the flow and
- 17:41the supervisor capacities, you could be at risk
- 17:45of sanctions if someone on your team files fake
- 17:48sites. Candidate of the tribunal is another hot
- 17:51one. And this usually comes into play after you
- 17:53realize you've filed fake citations. There has
- 17:56been trends where lawyers have gotten lower sanctions
- 18:01if they admit to what they did. In a lot of cases,
- 18:04lawyers will try to obfuscate the situation and
- 18:07say, your honor, no, I didn't use AI or make
- 18:09up some excuse. And very rarely, if ever, do
- 18:13they get out of that. So owning up to it, being
- 18:15truthful to the court, could actually help you
- 18:18in reducing sanctions. Reasonable fees. This
- 18:21gets to the question of, what can I charge clients?
- 18:24There's a courses paradox, this billable hour
- 18:26paradox, where lawyers typically make more money
- 18:29if they spend more time and bill more hours to
- 18:31a client. Now you have this new technology. That's
- 18:35like new technology in scare quotes, because
- 18:37really AI has been around for decades now. But
- 18:41generative AI has really come onto the scene
- 18:43and changed a lot of the tools that we're using.
- 18:45So this technology, generative AI, comes onto
- 18:47the scene, promises all these efficiency gains.
- 18:49We're going to cut the amount of time. We're
- 18:51going to cut your contract drafting costs in
- 18:53half. We're going to save you all this time on
- 18:54legal research. And the lawyer thinks, well,
- 18:58doesn't that mean less billable hours? So the
- 19:00debate now is, if I use an AI tool for legal
- 19:03research or contract drafting, and it used to
- 19:05take me 10 hours to do this, but now I'm using
- 19:08Spellbook for contract review, and it takes me
- 19:11five hours, can I charge 10 hours, because that's
- 19:14what it would have taken me, or can I only charge
- 19:16five hours? We'll get into that answer to that
- 19:19question later. Unauthorized practice concerns.
- 19:22This is less so, I'd say, a concern for lawyers
- 19:25directly. This may come up in the future. Open
- 19:28AI. in a case involving NIP and life insurance,
- 19:31OpenAI, the makers of ChatGBT, they were actually
- 19:34sued for unauthorized practice of law. Essentially
- 19:36what happened was there was this litigant whose
- 19:38case basically came to an end, then she pulled
- 19:41up ChatGBT, started asking ChatGBT, like, do
- 19:43I have a case? And of course ChatGBT gaslighted
- 19:47the person and was like, yeah, you have a great
- 19:48case, bring the case, file again, filed again.
- 19:51And then the insurance companies trying to say
- 19:53ChatGBT is engaging in unauthorized practice
- 19:56of law. Will it go anywhere? I highly doubt it.
- 19:58I don't think that lawsuit will go anywhere,
- 20:00but I do think we're gonna see a future where
- 20:03these legal tech companies may move into the
- 20:05direct provision of legal services. There's a
- 20:08hot, hot claim. I don't think anyone's really
- 20:11talking about it right now. You've of course
- 20:12heard of AI native law firms like Crosby, but
- 20:15they are still law firms. They have lawyers.
- 20:18It's lawyers delivering services that are augmented
- 20:21by AI powered tools. But I could imagine a future
- 20:24where literally tools like Legora, Harvey, go
- 20:28direct and provide legal services to companies.
- 20:31I think it manifests in the form of these companies
- 20:33providing an AI chatbot directly to in -house
- 20:35counsel. And this is already happening. The only
- 20:37difference is they disclaim and say they're not
- 20:39legal services. They're not legal. They're not
- 20:43providing lawyer services. But it raises the
- 20:45question, where do you even draw the line? If
- 20:48you have a legal tech company right now that
- 20:50gives you a chatbot and gives you case research
- 20:53and now gives you analysis when you ask it a
- 20:55question like, where would we have personal jurisdiction
- 20:58over a defendant? And you search that and then
- 21:02you have chatbots that not only surface case
- 21:04law, but surface analysis and give you a final
- 21:06decision. Is it sufficient just to put a disclaimer
- 21:10at the end to say, this is not legal advice.
- 21:14And therefore that just disclaims and now that's
- 21:16not legal advice. Whereas if you don't put that
- 21:18disclaimer, it's considered legal advice. I do
- 21:20think the blind is a lot more blurred than people
- 21:23think right now. Vendor management is another
- 21:26very important thing to consider as we talk about
- 21:29responsible AI adoption of law firms. Going out
- 21:32and just adopting AI for AI's sake, firms are
- 21:35doing it, probably not the best strategy. You
- 21:37should be very much so vetting vendors, look
- 21:40out into the market, see what other companies,
- 21:43what other organizations, what law firms these
- 21:45vendors are actually working with. I hate to
- 21:47say it, I mean, I have a startup, Wicard, we're
- 21:49kind of a startup, kind of now, we're two years
- 21:51in, but... we were building legal tech in the
- 21:54beginning. And I can tell you, hate to say it,
- 21:57but I was a former tech founder, but I mean,
- 22:00early stage startups, they probably don't have
- 22:02the best technological safeguards in place. So
- 22:06will they live up to the standards and requirements
- 22:08that you need as a lawyer to ensure the confidential
- 22:11information? is not going to be trained on, can
- 22:14they really guarantee that there is encryption
- 22:16in transit and at rest? So you should very much,
- 22:19I don't want to dissuade you from working with
- 22:21early, early stage startups, but you should very
- 22:24much pay attention to what their practices are,
- 22:27because it could ultimately lead to issues for
- 22:29you as a lawyer. And for bigger firms that have
- 22:31the resources, usually have an IT team help review
- 22:34the legal tech vendor and the product you're
- 22:36being sold to and ask the right questions. Time
- 22:39permitting, we'll get to it at the very end,
- 22:41but I did put a list of questions to ask vendors
- 22:43and what you should know about an AI tool before
- 22:45you actually adopt it. That also goes hand in
- 22:47hand with cybersecurity and data governance.
- 22:50This is usually somewhere, a situation where
- 22:51you bring in an IT consultant or professional.
- 22:54Of course, smaller firms really don't have that
- 22:56luxury to do that. So in those circumstances,
- 23:00reviewing the terms and conditions, reviewing
- 23:01the privacy policy, looking at what other law
- 23:05firms that that vendor is working with is very
- 23:07important. But asking the right questions is
- 23:09extremely important no matter what size you are.
- 23:12And of course, client trust is another big consideration.
- 23:15Talk to your clients, understand are they going
- 23:18to have an issue if you're using AI? Are they
- 23:20going to want you to use AI? Are they going to
- 23:22want you to not use AI? So getting on the same
- 23:24page with their client is very important in that
- 23:26respect. I will say some clients just do not
- 23:28care. A lot of clients are just not in tune with
- 23:32all these changes. If you're representing a high
- 23:34profile tech company, of course, they're going
- 23:37to know about AI's benefits and limitations.
- 23:40But if you're like in personal injury, if you're
- 23:42in an area that's not necessarily tech forward
- 23:46in terms of the clientele, they might not even
- 23:48have an opinion on AI. But it may be important
- 23:51for you to disclose that or tell them we're going
- 23:53to use AI in your representation because you
- 23:56don't want to be in a situation where you face
- 23:57malpractice. uh issues and it probably wouldn't
- 24:00just be a malpractice claim saying oh you used
- 24:02ai therefore i'm going to sue you for malpractice
- 24:05it might be a situation where someone on your
- 24:07team files a fake site in court then the client's
- 24:10very unhappy and now they file perhaps a malpractice
- 24:14claim against you not just because you used ai
- 24:16but because you misused it and then as a double
- 24:18add -on they'll say and i never approve that
- 24:20as the client so those are some some of the things
- 24:23that consider Worst case scenario, and it's very
- 24:25easy just to talk to the client, say, hey, in
- 24:28your representation, we'll typically use these
- 24:30AI tools. They deliver better quality, better
- 24:32efficiency. We're going to review all the output
- 24:35and still use our legal judgment. But just want
- 24:37to let you know, and probably best practice,
- 24:39I do this. I put a write in my engagement letter.
- 24:42It's its own section, methods of delivering legal
- 24:46services. And I'll talk about some of the tools
- 24:48we use, some of the methods, and clearly disclose
- 24:50the limits and benefits of these AI tools. really
- 24:53doesn't hurt to add in a paragraph into your
- 24:55engagement letter just to cover. So as we shift
- 24:58to talk about more of the comprehensive framework,
- 25:00this last slide was just about talking about
- 25:02some of the professional responsibilities to
- 25:04start thinking about. We'll go more in depth
- 25:06on some of them. But when we talk about AI governance
- 25:09as a whole to take a comprehensive approach at
- 25:12your firm, here are some of the things that you
- 25:14could tangibly start implementing. Number one,
- 25:17a policy. This is very important. I know a lot
- 25:20of smaller firms will have no AI policy, especially
- 25:23if you're a solo practitioner, you might think
- 25:24to yourself, why do I need an AI policy? It's
- 25:26literally just me and what I do. But if you have
- 25:29multiple people, especially if you're managing
- 25:31summer associates, associates, even if you have
- 25:34another partner on your team, very important
- 25:37to have some policy in place. At the very bare
- 25:39minimum, it should be addressing what AI tools
- 25:43are we permitted to use at the firm, which ones
- 25:45have certain restrictions on them, and which
- 25:47ones are outright prohibited. And we'll go more
- 25:50in depth on what should actually be in the policy.
- 25:54But that's like the bare minimum. Because you
- 25:56may think, why do I need a policy? Well, if you
- 25:58don't have a policy, it's going to be the Wild
- 25:59West. People at your firm are just going to think,
- 26:01oh, I'm going to use whatever AI tool I want.
- 26:03And this is not like the old days. I'll put old
- 26:06days in scare quotes because I'm talking like
- 26:08three years ago. This is not like the old days
- 26:10where you'd have to worry about a new associate,
- 26:14a summer associate showing up to the job and
- 26:16being like, oh, I'm going to use my own legal
- 26:17research tool. and I have access to it, I'm just
- 26:20gonna use it, and I don't wanna use your firm
- 26:22tools. Like, obviously, that's really not gonna
- 26:24be a thing. Like, you have Lesslaw, you have
- 26:27Lexis, they're pretty expensive. New lawyers
- 26:29are not just gonna have their own private, personal
- 26:31accounts, and you're not gonna have to worry
- 26:33about them doing their work on their personal
- 26:36accounts. That has totally changed now. There's
- 26:39free versions of basically all of this software.
- 26:42There's free versions of ChatGBT, free version
- 26:44of Claude. A lot of people will pay, even college
- 26:46students will pay for the upgraded $20 version,
- 26:49so they're not rate limited and they get better
- 26:51output. So it's very much a reality where your
- 26:54younger associates, your summer associates, they
- 26:56have their own personal access to these tools.
- 27:00And if you as a firm have not clearly dictated
- 27:04whether that's allowed, whether you're allowed
- 27:06to use those, about what are some of the restrictions,
- 27:09about whether you have to use firm products,
- 27:11and a real scenario is this. Your firm has no
- 27:14policy on the books. you've never told your summer
- 27:16associates, you never told your younger associates,
- 27:19anything about permitted or restricted AI tools,
- 27:23I could venture to guess that your associates
- 27:26are using their own personal products for legal
- 27:29work on the job. I could almost, I don't want
- 27:32to say guarantee it, it's a little too far, but
- 27:34I could tell you they're using it. There's one
- 27:36firm I work with who had no policy on the books,
- 27:39nothing at all. And the first step I said, let's
- 27:42start with the survey. Send it as a mid -sized
- 27:44law firm, send the survey out to all of your
- 27:46lawyers and ask them what AI tools are you using
- 27:49right now for your work? Overwhelmingly majority
- 27:53were using ChatGPT. There was no approved ChatGPT
- 27:58at the firm. There's no enterprise license. So
- 28:00people were literally using their own personal
- 28:02ChatGPT for legal work. We have no idea. They
- 28:05had no idea. Was it free? Was it the paid version?
- 28:09And even in the paid version, automatically by
- 28:11default, you go into your settings, toggled on
- 28:13is train on law firm data. So this goes to show
- 28:17if you have no policy in place, that doesn't
- 28:19mean people are not using AI. It's called shadow
- 28:22AI use at work. So that is very important to
- 28:25understand. No policy is not a good approach.
- 28:28You clearly lay out what tools are permitted,
- 28:30what tools are restricted, what is absolutely
- 28:32prohibited. At the bare minimum, like almost
- 28:35guaranteed, you have to tell your lawyers you
- 28:37cannot be using your own personal AI tools. for
- 28:40work on the job that is highly problematic. So
- 28:44policy is very important. Going hand in hand
- 28:46with policy is training. Before you use the tools,
- 28:50that is ideal, before you actually adopt these
- 28:52tools, before you roll them out to your lawyers,
- 28:55you should train your lawyers on the tools. What
- 28:57are the risks? What are the benefits? What is
- 28:59AI? This all goes to the duty of competence,
- 29:021 .1, rule 1 .1, in which you actually have to
- 29:06understand the technology before you use it.
- 29:09how to properly use it, what are the limitations,
- 29:12big misconception right now. I think we're shifting
- 29:14and this is kind of is is now a more general
- 29:17understanding, but originally people knew about
- 29:19the hallucinated citation issue, but they thought
- 29:21it was just ChachiBT or Claude or Gemini, all
- 29:23the general purpose models, and then people felt
- 29:26a little bit safer when Thomson Reuters co -counsel
- 29:29and Lexus Prodigy rolled out the hyperlink feature,
- 29:32where if you're using the AI -powered agentic
- 29:34search on those platforms, you get a response
- 29:37that looks like ChachiBT, but they promise to
- 29:39hyperlink the cases and you click on them and
- 29:42they open up to real cases. So people got this
- 29:44sense of relief like, okay, I'm not doing research
- 29:48on general purpose models. So I'm using legal
- 29:50tech specific tools. These are reputable company
- 29:52is the case or hyperlink. That's great. But the
- 29:55reality is even those hyperlink cases could be
- 29:58hallucinated. And now the issue is the legal
- 30:01propositions attached to them and the quotations.
- 30:04So the scenario is this, you get a real case
- 30:07citation, you click on it, you verify the case
- 30:09citation is perfect, it's real, it exists, but
- 30:12the legal proposition it gave you, and then cited
- 30:15the case for, might be completely fabricated.
- 30:18And then purportedly, there's quotations from
- 30:20that citation, the quotes could be totally made
- 30:23up. So I know of at least five cases now, which
- 30:26lawyers have been sanctioned, and when the order
- 30:29to show cause is issued, they set to the court,
- 30:32Well, court, I use Lexus. I used Westlaw. And
- 30:35there's another legal research company out there.
- 30:37I've seen their name in a lot of these proceedings.
- 30:41It is the lawyer saying, I didn't know legal
- 30:45tech tools could hallucinate. I've used Lexus
- 30:47for many years. There's one guy in New York,
- 30:49New York Supreme Court. He said that I was using
- 30:52Lexus for years. I was a Lexus rep in law school.
- 30:55That's what he literally wrote in his affidavit.
- 30:57I never knew that this new product would hallucinate.
- 31:00because Lexis never hallucinated the traditional
- 31:02legal research database. So even legal research
- 31:06tools, even legal specific AI products hallucinate.
- 31:09You need to convey this in training sessions
- 31:11is very important. Number three, oversight. It's
- 31:15important. A lot of law firms usually midsize
- 31:17and larger. They have an AI committee, AI task
- 31:20force. They'll put usually a managing attorney
- 31:22on it. They'll put like a partner for usually
- 31:23the IP group or some group that's more tech focused,
- 31:27or they'll just put on someone at the firm who
- 31:29likes technology. At a lot of firms, it'll be
- 31:31a very small minority of people who are like,
- 31:33I love AI, I love technology, I want to get involved
- 31:36in it. And they usually head up the AI task force.
- 31:38It's also very good to put younger associates
- 31:40on that task force too. So you get a sense of
- 31:43what are younger associates thinking. I can tell
- 31:46you now, majority of law schools are still not
- 31:49properly educating students on AI. I'll give
- 31:52a shout out since this is California to Southwestern
- 31:55Law School. I know Dean Dickerson, she attended
- 31:57the last session here. She's very ahead on AI.
- 32:00I'm teaching an AI course for all their 1Ls next
- 32:03year. So I'm excited about that. So schools like
- 32:05Southwestern, they're ahead of the curve on this.
- 32:08But the overwhelming majority of law schools
- 32:11are not sufficiently training their lawyers on
- 32:14AI. And you may be thinking, who cares? Why do
- 32:16I care about law school? Well, if you're hiring
- 32:18these students as summer associates, as first
- 32:20year associates, second year associates, I could
- 32:23tell you they're not going to know everything
- 32:24about AI. And at a lot of firms, it'll be older
- 32:27partners who will just turn to younger associates
- 32:29and say, I can't figure this tool out. Can you
- 32:32do this or that for me? There's this assumption,
- 32:34this presumption that younger associates know
- 32:37about technology. And in a lot of cases, that's
- 32:40true. But these younger associates, they'll use
- 32:42like ChatGBT, or they use Claude for like assignments
- 32:45in school. But I can tell you, they're not fully
- 32:48proficient. in the limitations, the risks, and
- 32:51proper usage of legal tech specific tools. Law
- 32:54students will widely have access to Westlaw and
- 32:56Lexis. Typically, they'll focus on just using
- 32:58one or the other, but they're not well trained
- 33:01in Harvey, Legora, all these other tools. So
- 33:03you cannot just assume your new associates are
- 33:05walking in the door ready to go on day one with
- 33:08knowledge of AI tools. Verification, super important.
- 33:11It should be in a policy at a law firm. No AI
- 33:14generated work product should ever go right to
- 33:17the court. to go right to a client, there needs
- 33:20to be human review before it goes out to anyone
- 33:22else. There are some firms, there's some area,
- 33:25this is what I do through Wicard. We advise law
- 33:29firms on AI policy. What is a good AI policy?
- 33:32I'll tell you one dilemma that we've seen at
- 33:35law firms is this. Law firms want to promote
- 33:37AI adoption. They pay for the licenses. Now they
- 33:40got to prove like, this is good. They want people
- 33:42in the firm to use it. They don't want to just
- 33:44go to waste. Some firms will think to themselves,
- 33:46We want to promote AI adoption. And there have
- 33:49been studies out there showing that if someone
- 33:52tells you their work product is AI generated,
- 33:55they look down upon it. They think it's lesser
- 33:57quality. They look down on the work and the person
- 34:00using it. Therefore, if you tell everyone at
- 34:03the firm that you're personally using AI for
- 34:05your work product, people might distrust you.
- 34:07So some firms have taken the approach of, OK,
- 34:11Disclosure of AI internally might actually depress
- 34:13AI adoption rates. Therefore, our policy is going
- 34:16to be you do not need to tell anyone at the firm
- 34:18you're using AI. Just run a treat at the same
- 34:21human work product, AI generated work product.
- 34:23Yes, of course, if you're using the AI tools,
- 34:25you still need to review the final work product.
- 34:28But when you pass it off to a supervising attorney,
- 34:31a partner of counsel, you don't need to tell
- 34:33them you used AI. I think that's a very bad policy.
- 34:36Why? Because I think that just seeds distrust.
- 34:40within the firm. Now you don't know who's using
- 34:43AI, who's misusing it. And the other thing is,
- 34:47externally, there are now over 200 standing orders
- 34:52in courts around the country that say you have
- 34:55to disclose your AI use, or you have to certify
- 34:57that no AI was used. You have to certify that,
- 35:00yes, I used AI, but I checked all my sites. In
- 35:03very rare cases, one in the Northern District
- 35:05of Illinois, Judge Sharon Coleman, she prohibits
- 35:08the use of AI. for legal research and drafting
- 35:11in memorandums of law, which I think is pretty
- 35:15wild to ban it. But the point is this. Within
- 35:18your firm, you're saying you don't need to tell
- 35:20anyone else at the firm using AI. That implicates
- 35:23two very big issues. Number one, how are you
- 35:25then going to know when you go to file in court
- 35:27whether you're actually complying with the court's
- 35:29standing order requiring disclosure? Because
- 35:32let's say you're the partner, you're the one
- 35:33filing this. You don't know if your team of five
- 35:36associates No one was required to tell you if
- 35:38they used AI. So how are you going to know if
- 35:40you're accurately going to certify to the court
- 35:42that no AI was used or AI was used for this,
- 35:44but not that. Another big issue, supervisory
- 35:48role. There's a supervisory obligation on those
- 35:53in managing roles at a firm to properly manage
- 35:56their associates. And now you're telling them,
- 35:58don't tell me if you used AI. I don't think that's
- 36:00a great way to manage who's using AI properly
- 36:02and who's not. It's basically just a guessing
- 36:04game. You're hoping that everyone in the firm
- 36:06is not going to misuse AI. You may think to yourself,
- 36:08well, if we train everyone up and everyone follows
- 36:11the rules, then that's no issue at all, because
- 36:14we're not going to have that supervisory issue.
- 36:16But then look at big firms, Sullivan and Cromwell,
- 36:19just a couple of weeks ago, Southern District
- 36:21of New York Bankruptcy Court, partner had to
- 36:23file a letter with the court saying court, posing
- 36:27counsel raised this issue to us. We actually
- 36:28have a bunch of fake citations, fake quotes in
- 36:31our filing. We need to correct it. One of the
- 36:34top law firms in the world. We don't know actually
- 36:36if it was the partner who did it, but the partner
- 36:38fell in the sword. But that's one instance. Top
- 36:42law firms have these issues. So if you think
- 36:44we could trust our entire associate class, all
- 36:46of our attorneys at our firm, they're never gonna
- 36:49make a mistake. I highly doubt that. There's
- 36:51another instance. Another big firm is down in,
- 36:54I believe it was Alabama or Mississippi. A partner
- 36:57put a fake citation into a case, filed it. All
- 37:00the attorneys on the case were kicked off. That
- 37:02was a partner. We now have seven instances across
- 37:04the country of judges. Judges. Two federal judges,
- 37:09the rest were state judges. Two of these instances
- 37:11happen in California. Judges have issued orders
- 37:14with fake citations of them. In North Carolina,
- 37:18a federal prosecutor practicing for over 20 years.
- 37:21Supposedly he drafted a whole filing, accidentally
- 37:24deleted it, quickly had to redo it, used AI,
- 37:27fake quotes, fake sites. He eventually retired,
- 37:30although they fired him before he retired. and
- 37:32then he was admonished by the court back in April
- 37:35of this year, two months ago. So judges, including
- 37:39federal judges who've been on the bench for decades,
- 37:42partners at top law firms around the world, prosecutors
- 37:46who've been practicing for 20 plus years, they
- 37:48are making these mistakes. So I say this, no
- 37:52one is immune from making these errors. And of
- 37:56course, you may think to yourself, of course
- 37:57you're immune if you're carrying out the proper
- 37:59procedures and ethics and professional responsibility
- 38:01rules and you're meeting those standards. Yes,
- 38:03but I'm saying just based on your status, based
- 38:06on what school you're coming from, what firm
- 38:08you work at, people all across the legal profession
- 38:11are making these lapses in judgment. So no one
- 38:14is immune. And that just goes to the importance
- 38:17of constantly reinforcing the policy, constantly
- 38:20doing the training and verifying every step of
- 38:22the way. I'll say, when I teach law students
- 38:26on this topic, law students, their visceral reaction
- 38:28is like, this is crazy. In the US alone, we now
- 38:31have over a thousand instances of attorneys and
- 38:35pro se litigants filing fake citations in court.
- 38:38And the reaction from law students is like, how
- 38:41are attorneys so negligent on this? Like, they
- 38:43know it's an issue. You're practicing for many
- 38:45years. Like, I would never file a fake citation.
- 38:48But I explain the realities of practice. The
- 38:50reality of practice is you have a tight deadline.
- 38:53Sometimes you have to prepare a motion and literally
- 38:55a day. You have to research a lot. You're a solo
- 38:57practitioner or you have just a couple attorneys.
- 39:00You have to own your work product and you're
- 39:02rushing to meet a deadline or you're at a bigger
- 39:04firm. I was previously at two big law firms and
- 39:07I would do like small legal research tasks and
- 39:10so would other associates and eventually it would
- 39:12all be compiled into a bigger motion, a bigger
- 39:15memorandum, a bigger filing. So you have like
- 39:1810 different people potentially touching the
- 39:20document, adding to it. and that's how errors
- 39:22get introduced all it takes is one person on
- 39:24that team to be lazy or to be under a time crunch
- 39:27where they have 10 other things going on and
- 39:30they think oh i just need to look up like the
- 39:32motion to dismiss standard let me just put into
- 39:34co -counsel or protege quickly get an answer
- 39:36send it off to the team they'll put it into the
- 39:39brief and now everyone else who signed off on
- 39:41that brief they're all responsible for the error
- 39:44that you put in. So I explained a lot of students
- 39:46like, yes, it seems like crazy that there's so
- 39:49many errors here. But then when you put in the
- 39:51context of the realities of legal practice, you
- 39:53understand how it happens so often. Not excusable,
- 39:58but the realities of practice provide some further
- 40:01context for understanding why this error of filing
- 40:05hallucinated cases constantly occurs. Then vendor
- 40:08diligence. I already mentioned that on the prior
- 40:11slide. And then continuous updating, very important.
- 40:14I know big law firms talk to chief innovation
- 40:17officers at these firms. Some of them have now
- 40:19gone through seven iterations of their internal
- 40:21AI usage policy. So this is not something that's
- 40:24just static there on the books. It's very good
- 40:26to get something out there immediately, especially
- 40:28for establishing what AI tools you can or cannot
- 40:31use. But I will say there's a constant ongoing
- 40:34process. That's why it's important to put together
- 40:36that oversight, number three, that AI committee.
- 40:39You have people who are invested. That is managing
- 40:42partner, of course, partners who've been at the
- 40:44firm for a long time who care about the reputation,
- 40:46risk, and future development. And then getting
- 40:48people who are enthusiastic, like a partner who
- 40:50just loves AI. early associates who are just
- 40:53excited about the tech, getting them on to a
- 40:55committee is very important. Create that institutional
- 40:57entity because this is not an ad hoc thing where
- 41:00it's just going to be, let's put together one
- 41:02training or policy and we'll revisit it years
- 41:05from now. No, this is like every couple months,
- 41:08re -meeting, assessing what's in the market.
- 41:10evaluating vendors, that's also a task that AI
- 41:13committees have been doing internally at firms.
- 41:15So this gives you a sense of some of the core
- 41:18practical steps you could and should be taking
- 41:21within your firm. So where do we even get the
- 41:25guidance? It's a common question I get. Let's
- 41:27say we're starting on step one. Say we're starting
- 41:29on step one and two. You realize, all right,
- 41:31we gotta get a policy in place. We gotta get
- 41:34training in place. We gotta put it together an
- 41:36AI committee. Usually these all happen together.
- 41:40Usually you'll put the committee together first
- 41:42and they'll help inform the drafting of the AI
- 41:45policy and inform whether you're going to do
- 41:47the training in house or bring in someone from
- 41:49outside to do the training. But what I've seen
- 41:51as a common trend is law firms will typically
- 41:54coalesce around the ABA formal opinion 512 guidance.
- 41:57They'll look at this incorporated directly into
- 41:59their AI policy and tell their lawyers internally,
- 42:02hey, here's our AI policy. Usually they range
- 42:05between two to six pages, but they directly incorporate
- 42:07the principles from the ABA formal opinion 512.
- 42:10And they say that all lawyers who sign off on
- 42:13this policy represent that they read formal opinion
- 42:16512. I'm going to go through briefly some of
- 42:18the important elements of this ABA formal opinion.
- 42:21And I'm going to tell you why I think one aspect
- 42:23of the ABA opinion is actually plain wrong. Competence.
- 42:27Number one, I mentioned this earlier briefly,
- 42:30but putting a little bit more in depth here.
- 42:33ABA formal opinion 512 does not introduce any
- 42:36new AI specific rules. Basically what it does
- 42:39is it looks at all the existing rules, model
- 42:43rules. And for background, I think most lawyers
- 42:46probably know this, but the ABA, American Bar
- 42:49Association, biggest voluntary bar organization
- 42:51in the country. ABA formal opinions and also
- 42:54their rules are not binding directly on the states.
- 42:58They're only binding on lawyers if they're adopted
- 43:00by bar associations. So a lot of state bars will
- 43:03adopt forms of the ABA model rules. Otherwise,
- 43:06they're just model rules. Florida was out ahead
- 43:09pretty early. So is California on this topic.
- 43:12So they actually issued opinions before this
- 43:15ABA formal opinion 512, but the opinions that
- 43:18came out from state bars after this. ABA opinion
- 43:21largely track the same principles. But to my
- 43:24knowledge, not a single state bar has adopted
- 43:27an AI specific rule. That is a new rule of ethics
- 43:30or a new professional conduct rule specific to
- 43:33AI. What they do is they apply existing rules
- 43:35to the use of AI. So they'll say competence,
- 43:40for example. employers have always had this obligation
- 43:43to be competent in the representation of clients.
- 43:45They need to, quote, exercise, quote, legal knowledge,
- 43:48skill, thoroughness, and preparation reasonably
- 43:51necessary for the representation. It's in comment
- 43:541 .8 to model rule 1 .1 here that says this includes
- 44:00staying abreast of technological developments,
- 44:02including, quote, benefits and risks associated
- 44:05with the technologies. So this says nothing about
- 44:08AI. For the ABA, did was in its opinion of hit
- 44:13a formal opinion 512 it said this rule applies
- 44:17to lawyers use of generative ai because it's
- 44:20talking about technology benefits and risks so
- 44:22what the opinion says is lawyers who are going
- 44:24to use generative ai tools need to be confident
- 44:27they need to know the risks and benefits before
- 44:29they actually use those ai tools so that is essentially
- 44:33how this aba opinion and other state bar guidance
- 44:36plays out They look at existing rules in the
- 44:38books. They apply it to the context of lawyers
- 44:40using AI. There's not a single bar association
- 44:42in the country that has banned AI. They all say,
- 44:45yes, you could use it, but you need to use it
- 44:47responsibly. So confidence 1 .1, very important.
- 44:52Confidentiality is another very important one.
- 44:55This is ABA model rule 1 .6. This rule says that
- 44:59lawyers have always had an obligation to maintain
- 45:01the confidentiality of all information related
- 45:04to client representation. unless the client provides
- 45:07informed consent otherwise. This is very important
- 45:09and is directly implicated by AI tool use by
- 45:13lawyers because if you're using an AI tool like
- 45:17ChatDBT free version or Claude free, the free
- 45:21version of that as well, your information is
- 45:24getting trained on, is going to be shared with
- 45:26these developers, it will be shared with third
- 45:28party vendors, and therefore your information
- 45:31and the confidentiality thereof is being compromised
- 45:34in a violation of rule 1 .6. And within state
- 45:38bar is the applicable rule there, which follows
- 45:40the same principle. So it's very important to
- 45:42understand confidentiality. So picking a vendor
- 45:45tool is not just about which tool works the best.
- 45:48You need to fully understand how the data is
- 45:50actually being processed, whether it could potentially
- 45:52compromise confidentiality of client information.
- 45:56Shifting now to communication. This is one where
- 45:58there's been a slight shift away from the ABA
- 46:01guidance here. So under communication, ABA rule
- 46:051 .4, there's always been a duty of an attorney
- 46:08to advise clients promptly whenever there's any
- 46:11information that gives rise to important decisions
- 46:13or questions involving the client. Most simply,
- 46:17you need to advise your client on important things.
- 46:20And according to model rule 1 .4a2, a lawyer
- 46:23must quote, reasonably consult with the client
- 46:26regarding the methods to achieve the client's
- 46:29objectives. According to model rule, according
- 46:32to this ABA opinion, which is interpreting model
- 46:35rule 1 .4, lawyers actually must get a client's
- 46:39informed consent before inputting information
- 46:42relating to the representation into a generative
- 46:45AI tool. And it goes further and says, quote,
- 46:50merely adding general boilerplate provisions
- 46:53to engagement letters purporting to authorize
- 46:55the lawyer to use generative AI is not sufficient.
- 46:59So this is whenever you're putting something
- 47:00into a generative AI learning tool. as the ABA
- 47:04opinion calls it, you need to get your clients
- 47:06informed consent. And this is related to confidential
- 47:08information. So this is a pretty clear -cut rule.
- 47:12You need to get pre -permission. This is like
- 47:14you putting in client confidential information
- 47:16related to representation into Harvey or Legora.
- 47:19This is not just about general purpose models.
- 47:22According to the ABA opinion, you need to get
- 47:24your clients informed consent before you do that.
- 47:26And you can't even just put it in a boilerplate
- 47:29provision in an engagement letter. It's a pretty,
- 47:32I don't want to say onerous, but pretty operative
- 47:36obligation to now have to get on a call and explain
- 47:39to your lawyer, here are the benefits and risks
- 47:40of me using Harvey, Legora, even co -counsel,
- 47:44which is the generative AI feature in Thomson
- 47:48Reuters, and also the protege, generative AI,
- 47:52chat functions, and Lexis. Again, this opinion
- 47:57was issued in 2024, July of 2024. I think it's
- 48:00a little outdated. I mean, you really wouldn't
- 48:03call up your client four or five years ago and
- 48:04say, hey, client, do I have permission to use
- 48:07Westlaw legal research? Do I have permission
- 48:11to use Lexis legal research? You really wouldn't
- 48:14do that. It's kind of just part of the practice
- 48:15to be using the most widely accepted legal research
- 48:20tools. But according to the ABA guidance, if
- 48:22you're putting client confidential information
- 48:23into a generative AI tool, you need to get informed
- 48:26consent from your client. I mentioned there's
- 48:29a bit of a shift away from this. The Texas bar
- 48:32state bar opinion, as well as California's is
- 48:35a little bit more lenient where it says you should
- 48:37consider getting informed consent. It doesn't
- 48:40say you have to. So I do think we're shifting
- 48:43away from this requirement. I do think there's
- 48:45also kind of fact specific depending on your
- 48:48relationship with your client. So with fees here
- 48:52is 1 .5. I asked that question at the top of
- 48:54the hour. I asked. It used to take you 10 hours
- 48:57to review a contract. Now it takes you five because
- 49:00you're using Spellbook, an AI review tool. Can
- 49:04I charge my client 10 hours or can I charge them
- 49:07only five hours? The answer is five hours. You
- 49:12cannot charge them for work you did not do. Under
- 49:16ABA model rule 1 .5, you need to charge reasonable
- 49:19fees. It's always been the rule. So the ABA formal
- 49:22opinion evaluated this exact scenario that I
- 49:24just laid out to you. If AI is delivering you
- 49:26more efficiency gains, cutting into the amount
- 49:28of hours you're spending on your client's work,
- 49:31you cannot charge your client for the time it
- 49:33would have taken you. You can only charge for
- 49:35the actual time you spent on the job, even if
- 49:38AI cut into that time. Another question. I used
- 49:42AI to review contracts, took me five hours, but
- 49:45now I need to spend two hours reviewing AI's
- 49:49results. For accuracy, for final review, can
- 49:53I charge for those two hours? Yes, you can. So
- 49:56you can charge for seven hours in that circumstance.
- 49:58That is permissible because you're still doing
- 50:00work on the job, reviewing the AI output. There's
- 50:03also still efficiency gains to the client. There's
- 50:05more gray areas to this, and the ABA opinion
- 50:07recognizes this. Some of these include, one,
- 50:11let's say I'm learning how to use Spellbook,
- 50:15the contract drafting tool. I spent two hours
- 50:18learning how to use the tool before I used it
- 50:21for five hours. So it all would have taken me
- 50:2410 hours manually. Now it's five hours on the
- 50:27job, two hours prep learning how to use the tool.
- 50:30Can I charge for five hours or seven hours? You
- 50:33only charge for five. The ABA opinion says you
- 50:36cannot charge for the time you spent learning
- 50:38how to use the tool. To add another scenario.
- 50:43What if the client said to you, you need to use
- 50:45Spellbook. You are not going to use it otherwise.
- 50:47They say to you, I need you to use Spellbook.
- 50:50Now it takes you two hours to learn how to use
- 50:52the tool. Five hours into the contract, can you
- 50:54charge five hours or seven hours? The ABA opinion
- 50:58alludes to you being allowed to charge seven
- 51:00hours. If the client specifically asks you that,
- 51:03and you make it clear to the client that you're
- 51:05gonna be spending extra time learning about the
- 51:07tool, you put that in the fee arrangement upfront,
- 51:10it's in the engagement letter, in the fee arrangement,
- 51:12then that's a circumstance in which that would
- 51:14be a more permissible practice. I'll add in other
- 51:18potential scenarios. And Virginia came out with
- 51:22its own ethics opinion on this as well, saying
- 51:24that you basically incorporate into your billable
- 51:27hour rate. If you know that AI is driving more
- 51:29efficiencies, driving down your traditional business
- 51:31model, then you could increase your hourly rate
- 51:35to make up for those potential increases in quality
- 51:39and efficiency. But with that said, when you
- 51:41do that, you have to make a clear cut to your
- 51:43client to disclose that upfront that you're increasing
- 51:45your billable rates. So there are interesting
- 51:48more gray areas, but the ABA formal opinion 512
- 51:52makes it very clear. You can't just charge for
- 51:54the time you saved. All the efficiency gains
- 51:57accrue to your client. I'll note something else
- 52:01here. There's an interesting line from the ABA
- 52:04formal opinion 512. Quote, as generative AI tools
- 52:08continue to develop and become more widely available,
- 52:10it is conceivable that lawyers will eventually
- 52:13have to use them. to competently complete certain
- 52:16tasks for clients. Pretty strong statement. So
- 52:21this is from summer 2024, like two years ago.
- 52:26Imagine two years ago, someone saying to you,
- 52:28if you don't use AI, you are not competently
- 52:31serving your client and you could be a risk for
- 52:35misconduct. Pretty strong statement. I do think
- 52:37this is where we're going. We are not there yet.
- 52:39I just don't think the tools are good enough.
- 52:41I don't think they've been around for enough
- 52:43long enough for people to be required to fully
- 52:46understand the full panoply of tools and their
- 52:48limitations and benefits and risks and all that.
- 52:51But I do think we're going to see this future.
- 52:53There's a judge from the Southern District of
- 52:54New York. He also made a similar statement back
- 52:57in February this year. He was asked about whether
- 53:00it is misconduct or malpractice for a lawyer
- 53:03to be using AI tools given how often they commit
- 53:06errors. The judge replied saying no. And he thinks
- 53:10there may be a time soon where it's unethical
- 53:13to not use AI tools. And the example he gave
- 53:16was in the context of a fee dispute. And this
- 53:18is probably how I think it would arise too. You're
- 53:20not going to have a client go to you and be like,
- 53:22hey, lawyer, you didn't use AI. I'm going to
- 53:24see you for malpractice for not using AI. I think
- 53:27it'll come up in the context of a fee dispute.
- 53:29Imagine client hires you. They don't pay you.
- 53:33There's a fee dispute. The client then says,
- 53:37Going through all your billable hours here, you
- 53:38spent 20 hours reviewing this contract. I know
- 53:41there's tools out there that are commonplace
- 53:43in the legal profession. This should have taken
- 53:46you only 10 hours. Therefore, I'm not paying.
- 53:49And in fact, I think it's misconduct that you're
- 53:51charging me for 20 hours when you should have
- 53:53been using AI to cut my billables in half. So
- 53:56it's within that context of a fee dispute where
- 53:58you probably will see the first case involving
- 53:59whether it is incompetent or misconduct to not
- 54:04use AI tools. I do not think we're there yet
- 54:07where not using AI tools constitutes a breach
- 54:10of ethics or professional responsibility. But
- 54:14interesting to see that the ABA is already portending
- 54:18a conceivable future in which that happens. So
- 54:22I'm going to close out on this, probably a couple
- 54:24more minutes. I might address some questions.
- 54:26There's a lot more content to address. One hour,
- 54:30definitely not enough to go comprehensively through
- 54:31the responsible. AI implementation at a law firm.
- 54:34So I'm happy to answer questions via email after,
- 54:37and I'll possibly, time permitting, address some
- 54:40here. But I want to address a question that comes
- 54:43up pretty frequently. The ABA opinion recognizes,
- 54:48yes, you could use generative AI, but we already
- 54:53know these tools hallucinate a lot. So at what
- 55:00level can we even rely on these AI tools? Like,
- 55:03can we trust but verify? That is a phrase I've
- 55:06heard thrown around so much. Trust but verify.
- 55:10There was even a CLE given by the New York City
- 55:12Bar. I think it was the New York City State Bar
- 55:14or City Bar. I saw it. The title of the CLE was
- 55:17trust but verify. What you need to know about
- 55:19AI. That's crazy. Absolutely crazy. These are
- 55:24tools that hallucinate. And these are old stats,
- 55:262024. I put a caveat in the slide. and you have
- 55:30all the materials. You'll see this if you go
- 55:32through later slides. I put a caveat here. A
- 55:35study from 2024 says that Lexis and Westlaw hallucinate
- 55:38between 17 to 34 % of the time. Again, those
- 55:43stats are two years old. I would not trust them
- 55:45whatsoever right now. We don't know what the
- 55:47updated numbers are. These are private companies.
- 55:50It's up to them to release it. Of course, this
- 55:54study was done by Stanford. It would be cool
- 55:56if they did another study. to update it. But
- 55:59again, those are outdated numbers. I assume they're
- 56:00much lower. But the point is I shared them in
- 56:03my slide, because to prove the point, legal tech
- 56:06tools hallucinate too. So there's not a single
- 56:09AI tool out there right now that a lawyer could
- 56:11use where they 100 % know the output is accurate.
- 56:15So it is absolutely wild advice, in my opinion,
- 56:18to be telling lawyers, trust the outputs. And
- 56:22of course, there's the caveat, trust but verify.
- 56:24But how can you start from posture of trust?
- 56:26when you know these tools hallucinate. And lawyers
- 56:30have been sanctioned thousands of times. Judges
- 56:32have included fake sites. That is a terrible
- 56:34advice, in my opinion, to say trust but verify.
- 56:37And I included this in the pre -materials. The
- 56:40Mississippi State Bar issued an official ethics
- 56:42opinion, and their official guidance on AI is,
- 56:45quote, trust but verify. So even like leading
- 56:48institutions are out there telling lawyers to
- 56:51trust AI tools that we know are not accurate.
- 56:55and hallucinate potentially to the numbers between
- 57:00five to even as bad as like 35 % of the time.
- 57:03So that is not good advice. The question then
- 57:06is, how much trust can you put in them? How much
- 57:09review is required? And this leads to the practical
- 57:12question. If we need to review everything that
- 57:14comes out of an AI tool, is it really delivering
- 57:17efficiency? Or are we just gonna save time on
- 57:20the front end to lose time on the back end? Well,
- 57:23the ABA opinion gives some guidance. According
- 57:26to the ABA, if you're using generative AI tools,
- 57:28quote, the appropriate amount of independent
- 57:30verification or review required to satisfy rule
- 57:341 .1, that's the rule of competence, will necessarily
- 57:37depend on the generative AI tool and the specific
- 57:40task that it performs as part of the lawyer's
- 57:43representation of a client. That basically tells
- 57:46you nothing. And that is one of my critiques
- 57:50of basically all the AI ethics guidance out there.
- 57:52They give you these broad principles and no practical
- 57:55implementation process. So if you ask someone,
- 57:58how much review do you need to do? And let's
- 58:02say someone asked me that, and I say to them,
- 58:04well, it depends on the tool, depends on the
- 58:06task, depends on what you're doing. Is that a
- 58:09helpful answer? Certainly not. They give one
- 58:12example. For example, if a lawyer, and this is
- 58:15right from the ABA opinion, if a lawyer relies
- 58:18on a generative AI tool to review and summarize
- 58:20numerous lengthy contracts, the lawyer would
- 58:23not necessarily have to manually review the entire
- 58:27set of documents to verify the results if the
- 58:30lawyer had previously tested the accuracy of
- 58:32the tool on a smaller subset of documents by
- 58:35manually reviewing those documents, comparing
- 58:37them to the summary, comparing them to the summaries
- 58:40produced by the tool, and finding the summaries
- 58:43accurate. I think this is bad guidance again.
- 58:49Based on that example, we now have a subjective
- 58:53standard, where essentially, you're a lawyer,
- 58:57you use a tool, and based on my prior experience
- 59:00using a tool, I think it's great. Therefore,
- 59:04I don't have to rely on it, I don't have to do
- 59:05manual, I don't have to do manual review anymore.
- 59:08I tested it a bunch of times, it's never proven
- 59:10to be wrong for me personally, now I'm just gonna
- 59:14go trust this tool. Take this example, and of
- 59:19course this is the contract context, But imagine
- 59:21extrapolating this to any other area in the practice
- 59:23of law. Let's say I just did legal research on
- 59:28co -counsel on Westlaw and I'm writing a motion.
- 59:31I have a hundred citations. I use generative
- 59:34AI. I manually check the first 25 citations.
- 59:41They're all good to me. Based on the same logic
- 59:44from this small sample set of contracts, I did
- 59:48a small sample set 25%. of my 100 citations,
- 59:51based on this logic, I can then say, oh, the
- 59:53other 75 must be good to me. I don't need to
- 59:56manually review it. Let me go file in court.
- 1:00:00Heck no. You can't do that. And the fact that
- 1:00:02this sets up a subjective standard is also pretty
- 1:00:04crazy, because you could be using a generative
- 1:00:06AI tool that is totally out there. No one in
- 1:00:09the industry is using it. But if your personal
- 1:00:11experience using it has proven to be good, then
- 1:00:14you have For lack of a better word, plausible
- 1:00:17deniability to say, oh, I filed the guidance.
- 1:00:20I followed what it told me I could do. So I do
- 1:00:22not think this is good guidance at all. This
- 1:00:24also, there's so many other issues with this
- 1:00:26logic. Previous experience, these tools literally
- 1:00:29change week to week, month to month. If you're
- 1:00:32using ChatUBT, there's been like six model upgrades
- 1:00:35in the past six months. So this doesn't account
- 1:00:38for the fact that if you use a generative AI
- 1:00:40tool today and it's really good, a month from
- 1:00:42now, you can't rely on that same baseline. These
- 1:00:45models are improving significantly. Sometimes
- 1:00:47they change, and certain tasks actually deprecate
- 1:00:50in their performance. So just because the models
- 1:00:52generally get better, maybe you're using it for
- 1:00:54a one -off task, and now that process is deprecated.
- 1:00:58So I do think this guidance is not very good.
- 1:01:00I think it's misleading, and I would not necessarily
- 1:01:04recommend it. And to the point I made earlier
- 1:01:06about the Bar Association in Mississippi, definitely
- 1:01:09wouldn't follow that. Trust but verify, no way.
- 1:01:13any time any product is going to a client, going
- 1:01:16to the court, going to a managing partner who
- 1:01:19you know is probably going to rely on that information,
- 1:01:22you need to manually verify all of it. I think
- 1:01:24that's super important. In this case with the
- 1:01:27contracts, potentially there may be an argument
- 1:01:29that if those summaries are just being used for
- 1:01:31quick review to set up then further review where
- 1:01:34like your team is going to dive into the full
- 1:01:36contract so eventually they will see all the
- 1:01:38information and the summaries are just to get
- 1:01:40a high level overview that could be more acceptable
- 1:01:43for not doing a more in -depth review on just
- 1:01:45the summary because that summary is just going
- 1:01:47to inform further lawyer judgment without it
- 1:01:50going direct to counseling a client or the court.
- 1:01:52That might be an internal use that's appropriate
- 1:01:54because you know there's more verification happening
- 1:01:57down the pipeline but otherwise I don't think
- 1:02:00this is the best advice out there. So I shared
- 1:02:04all the materials with you all. There's far more
- 1:02:06to this, but I want to hit the core principles
- 1:02:08and considerations. Again, practical takeaways
- 1:02:10here. If you're just getting started with the
- 1:02:13AI adoption route, highly suggest putting together
- 1:02:15some internal committee. Get an AI policy out
- 1:02:18there on the books. That's very important. Figuring
- 1:02:20out, doing an internal accounting of what AI
- 1:02:24tools people are using in your firm. And really
- 1:02:27getting a true sense of that might be an awkward
- 1:02:29conversation to have because someone's not going
- 1:02:31to raise their hand and be like, yeah, I've been
- 1:02:33improperly using AI this whole time. Just want
- 1:02:35to let you know. No, it's, it might be have to
- 1:02:38be an awkward conversation, but you need to know
- 1:02:41how your attorneys are using or misusing AI right
- 1:02:43now to form better practices. And that goes hand
- 1:02:46in hand with also the training. So those are
- 1:02:48the three practical steps for you all. I take
- 1:02:50away from this. It's getting that team in place
- 1:02:53for your smaller firm might just be you then
- 1:02:55getting the policy in place and coupling that
- 1:02:57with training to truly be using these tools effectively,
- 1:03:00but also avoiding potential. ethical missteps,
- 1:03:04which could then turn into reputational hits.
- 1:03:06So I'm going to jump to the end here, but perhaps
- 1:03:09we'll do another session on this in the future.
- 1:03:11If you want to reach out to me, my email's here.
- 1:03:14Always happy to talk more about AI policy, training,
- 1:03:17adoption, and any other things that come to mind.
- 1:03:23All right. This concludes today's presentation.
- 1:03:25Thank you so much, Oliver, for your time and
- 1:03:27educating our members. And thank you so much
- 1:03:28to our audience for joining us today. Have a
- 1:03:30great day, everybody.