Latest / CLEs You Actually Want to Hear / Better Judgment, Better Outcomes: Eliminating Cognitive Bias in Legal Decision Making
Transcript
- 0:03Welcome to Better Judgment, Better Outcomes,
- 0:07Eliminating Cognitive Bias in Legal Decision
- 0:10-Making. I'm Lori Krip -Martin, a certified family
- 0:13law specialist and a past vice chair of the Beverly
- 0:17Hills Bar litigation section. I'm delighted to
- 0:20introduce our speakers. Scott Cohen is a shareholder
- 0:23at Engelman Berger at a certified specialist
- 0:26in business bankruptcy law with over 30 years
- 0:29of experience providing strategic results oriented
- 0:33solutions in finance, banking, bankruptcy, creditors
- 0:37rights, workouts, turnarounds, and commercial
- 0:40litigation. He represents banks, lenders, creditors,
- 0:44borrowers, landlords, guarantors, committees,
- 0:48and debtors across the full life cycle of commercial
- 0:52relationships. and disputes. A recognized leader
- 0:55in Arizona's bankruptcy law community, Scott
- 0:58is a past chair of the state bar of Arizona bankruptcy
- 1:02section and frequently serves as a expert witness
- 1:06in bankruptcy and anti -diff deficiency matters.
- 1:10Michael P. Rowland is a shareholder at Engelman
- 1:13Burger and a passionate versatile commercial
- 1:16litigator whose practice focuses on real estate
- 1:19disputes, breach of contract, ownership disputes,
- 1:23loan workouts. Chapter 11, Bankruptcies, Receiverships,
- 1:27and Appeals. As a second -generation Arizona
- 1:30native, he is a committed advocate for the Arizona
- 1:33business community, representing debtors, creditors,
- 1:36lenders, real estate developers, commercial landlords,
- 1:40and other business owners. Michael has a very
- 1:43strong record of success in state and federal
- 1:45courts, including before the Arizona Court of
- 1:49Appeals, and the Ninth Circuit bankruptcy appellate
- 1:52panel. And now let's get started. And thank you,
- 1:54Michael, for your patience. You're welcome. You
- 1:56can refer to me as the lawyer formerly known
- 1:59as Michael. The topic today, we titled Better
- 2:04Judgment, Better Outcomes, Eliminating Cognitive
- 2:07Bias, and Legal Decision -Making. concept and
- 2:11we've done this presentation a few times actually
- 2:14once to an international group of lawyers a few
- 2:16years ago and then it was published in the Arizona
- 2:19attorney magazine. The concept is one of paradigms
- 2:26by which we can conceive borrow from other fields
- 2:32and then go ahead and use those teaching lessons
- 2:37those practice pointers to reduce errors within
- 2:40the practice of law. So our goal here is to help
- 2:43you improve the quality of service to clients
- 2:45and to reduce not just malpractice, of course,
- 2:50but to improve outcomes and improve the quality
- 2:54of the representation in an applied fashion.
- 2:58So it starts out a little esoteric and theoretical.
- 3:01The story is that I was at one of my favorite
- 3:05used bookstores and I saw this book entitled
- 3:11Will Your Next Mistake Be Fatal? And as a insolvency
- 3:17lawyer where I think every mistake is already
- 3:20fatal, it felt like that title was calling to
- 3:23me and I was curious as to what the Dean of ASU's
- 3:28Business School was really writing about. And
- 3:31the concept that he had a number of years ago
- 3:35was let's look at some mishaps So they could
- 3:41be in aviation, they could be engineering mishaps
- 3:43and other mishaps, and then reverse engineer
- 3:46them to figure out what went wrong. And I thought,
- 3:49well, we could do that in law, not just in the
- 3:52concept of like a post -mortem on a case, but
- 3:57actually what could we learn from other fields
- 4:00like aviation, engineering, medicine, et cetera,
- 4:04that would help us become better lawyers. So
- 4:07this is not a presentation on how to become a
- 4:10great lawyer by reading a code or a case law
- 4:14or statute or learning the rules of evidence,
- 4:17et cetera. It's instead really what we can learn
- 4:20from other people who have perfected risk mitigation
- 4:25strategies in their own fields. So we will go
- 4:31with the quality of the decision -making as one
- 4:35of the goals. reducing the error rate and managing
- 4:37human risk for anybody's in the law firm and
- 4:40delegating, that's critical. And then the question
- 4:44is, how can we deliver better results for our
- 4:47clients? And the concern is that there is incorrect
- 4:56thinking in some respect. There's thinking that
- 5:00we have as professionals that is not a one -to
- 5:04-one relationship with factual accuracy or the
- 5:07truth, or a complete understanding of what's
- 5:11happening, and how can we get a clearer perspective
- 5:15or perception so that we can do a better job,
- 5:21do a better job at the macro level and do a better
- 5:25job case by case. This, of course, counts for
- 5:29CLE as ethics or malpractice reduction. It also
- 5:33fits within the bias reduction because the original
- 5:36conception of this wasn't just that we were going
- 5:39to reduce implicit bias, cognitive bias, but
- 5:43it was really a question about where in general
- 5:48decision making do people go wrong. So, you know,
- 5:53for example, conspiracy theories. Why are they
- 5:55held? for so long after the evidence suggests
- 5:59otherwise. And so those were the kinds of questions
- 6:02or concerns. And we thought this was low hanging
- 6:06fruit to start addressing or chipping away at
- 6:09those biases. So the disciplines we looked at,
- 6:15obviously, because it's on cognitive bias, cognitive
- 6:17psychology. And if you go through the research
- 6:21or the literature on these subjects, what you
- 6:23find is that the disciplines in which they fought
- 6:26the hardest to correct for these kinds of cognitive
- 6:31biases are the US Army as the longest track record,
- 6:35aviation and medicine. We could go on, there
- 6:39are a plethora of disciplines that have done
- 6:41an excellent job. We were looking for an interdisciplinary
- 6:45approach that we could then transplant, move
- 6:48it from their field to our field and somehow
- 6:52still maintain the integrity of that paradigm
- 6:58and get results. So we also, you'd see this at
- 7:03the end, we interviewed, I interviewed a number
- 7:05of people. So numerous surgeons in different
- 7:08surgical fields, pilots, actual researchers at
- 7:14California hospitals and so on in medical research.
- 7:20So. Hopefully that interdisciplinary approach
- 7:23will benefit you. That's the whole point of this
- 7:25exercise. The other point of the exercise is
- 7:30to validate a point my father, who was a trial
- 7:34lawyer, had, which was a long time ago, he told
- 7:38me something along the lines of this Eleanor
- 7:40Roosevelt quote, which was he was trying to tell
- 7:44me I didn't have to make every mistake and that
- 7:47I could learn from other people's mistakes. And
- 7:50I thought, well, he's probably right, but he
- 7:52kept repeating that anyway. So hopefully there
- 7:56is a takeaway here where lawyers are particularly
- 8:00bad, especially litigators, I would say, of going
- 8:04down the hall and telling you, I made this horrible
- 8:07mistake and I want you to learn from it. It's
- 8:09just not the culture generally. And so there
- 8:13is this hope that we can somehow open up the
- 8:17culture a bit. and embrace learning from our
- 8:22mistakes, which is a value and hearing in science.
- 8:28And so we tried to bring that into this approach
- 8:31as well. So here's your table of contents. Just
- 8:35as we discussed, we'll cover the first few areas.
- 8:39We'll take turns on some of them. And then maybe
- 8:41most importantly is we'll have a takeaway at
- 8:44the end as to what we can do about this so that
- 8:47we are not hostages of our current perceptions.
- 8:53And then we're gonna leave time at the end for
- 8:56Q &A. We're really hoping that it's participatory.
- 9:00In this format, it's hard to see your questions.
- 9:02Otherwise, we would have a conversation ongoing.
- 9:05I would rather have an ongoing conversation.
- 9:08But to the extent we can hold questions to the
- 9:11end, it would be appreciated. And the hope is
- 9:14also that maybe we address some of those questions
- 9:16as we go through this. So we're going to learn
- 9:20from cognitive psychology what cognitive biases
- 9:24are and we're going to see that we're looking
- 9:27at a system that is flawed and it's a you know
- 9:31what we do as lawyers is incredible and we had
- 9:35a first -year associate who was in my office
- 9:37the other day and she was saying that she's phenomenal
- 9:40at pattern recognition and I said great then
- 9:42you're going to be a good lawyer. because that
- 9:45is most of our job is at least in litigators
- 9:48you do a lot of this I can I can analogize it
- 9:52to this and I can distinguish it from that and
- 9:55that's constantly what we're doing the problem
- 9:57is when that system breaks down and we jump to
- 10:00a conclusion about it being similar to something
- 10:04else or dissimilar to something else and when
- 10:07when is our perception flawed And then our decision
- 10:12making based upon that flawed perception kind
- 10:15of spirals on us a bit. So we're going to look
- 10:18at systems of flawed perception and see if we
- 10:23can recognize them in ourselves, maybe recognize
- 10:26them in other people, of course, and then try
- 10:29to correct them. And there's an endless list
- 10:32of cognitive biases. Endless is probably too
- 10:34strong a word. But if you even go to Wikipedia,
- 10:36they do a really good job of outlining all the
- 10:39cognitive biases. There's some great books on
- 10:41this subject. There's obviously textbooks on
- 10:43this subject. I'm not suggesting that this is
- 10:46some sort of definitive explanation of cognitive
- 10:49biases by two practicing lawyers. I am suggesting
- 10:53to you that at least this is the tip of the iceberg
- 10:56and you can start going through the your cases,
- 11:02your interactions with clients, your interactions
- 11:04with co -counsel and the court sometimes and
- 11:08figure out why you're not on the same page. What
- 11:11is different about this interaction, this understanding
- 11:16that may contain some misalignment. So the first
- 11:21one is binary bias. So if you've ever heard anybody
- 11:26immediately start to put things in boxes, and
- 11:30explain to you that, well, it's either X or Y,
- 11:33it's A or B. It must be this or that. And you
- 11:38start to get the sense that there's some sort
- 11:39of polarizing going on there. That's what we
- 11:42would call binary bias. And we tend to do that
- 11:46because these are complex issues. So anybody
- 11:48who's been involved in a large case or even a
- 11:52messy small case, you get the sense that the
- 11:56client sees it as black and white. They want
- 11:58you to share their perception that it's black
- 12:00and white. This is one I recognize even in high
- 12:04school, because in high school and college debate,
- 12:06we would regularly say in response to something
- 12:08like that when presented, that that's a false
- 12:12dichotomy, or that these two things are not mutually
- 12:18exclusive. There might not even be binary in
- 12:20that respect. So look for that when listening
- 12:23to somebody or reading some presentation. Experience
- 12:28bias is we just take our own perception. We elevate
- 12:31that onto a pedestal. It really is our only way
- 12:34in the world, absent reading movies, et cetera,
- 12:38and listening to other people's stories, some
- 12:40sort of consciousness raising through shared
- 12:43experience with other people. But if we put our
- 12:46experience at the center of the universe, the
- 12:49problem is that we end up with a certain bias.
- 12:52Obviously, everything from that is purely anecdotal.
- 12:55And it's a hasty generalization because there's
- 13:00not enough data points based upon our own experience
- 13:03most of the time. Attribution bias is fascinating
- 13:06as well. This one's crazy. The idea is, in the
- 13:10literature at least, that we tend to excuse our
- 13:12own behavior. Look for this when talking to other
- 13:16people. You listen and you hear them say, well,
- 13:20I didn't do well on that exam. because I woke
- 13:23up late, I didn't eat well, I didn't sleep well.
- 13:27It's some sort of excusing of one's own behavior
- 13:30as a response to circumstances. But conversely,
- 13:35when you hear somebody else describe the character
- 13:38traits of another lawyer, for example, or client,
- 13:41what they tend to do is tend to make it inherent
- 13:45or endemic to the character or personality of
- 13:48that individual. So while we'll excuse our own
- 13:51behavior the research suggests because of circumstances
- 13:55We don't give other people that same benefit
- 13:58of the doubt So the next one is confirmation
- 14:02bias and this must be the most prevalent one
- 14:05in the practice of law You know, the problem
- 14:09is that we see certain we have a perception and
- 14:13you can't break through it and every piece of
- 14:15evidence that's presented to an individual with
- 14:18confirmation bias tends to reinforce that bias.
- 14:23There is no changing somebody's mind. I don't
- 14:26know how often anybody's gone to court, made
- 14:28an oral argument. The court may have a perception
- 14:31of it. The court's perception of it at the beginning
- 14:34of the case. Bankruptcy is much more fluid and
- 14:38dynamic, and the cases have years to them. Same
- 14:41with Michael does a lot of water law. Similar
- 14:44problem there. encased or cast, it's really hard
- 14:49to change that. And we tend to overlook or under
- 14:54emphasize evidence that challenges a particular
- 14:58belief. There's lots of great research on this
- 15:00and I commend it to anybody interested to go
- 15:03out and read it because it's absolutely impossible
- 15:05to become persuasive in any way, shape or form
- 15:08without understanding confirmation bias. Loss
- 15:14aversion and narrative fallacy. So loss aversion
- 15:17is that when they when they do the research on
- 15:19this people tend to fear losing more than they
- 15:23put weight on gains. So anybody who's done research
- 15:26on like prisoners dilemma and how we then take
- 15:30defection versus cooperation and we give a certain
- 15:33mathematical weighting, they do that same research
- 15:36for loss aversion and people are way more afraid
- 15:39of losing something than they are of actually
- 15:43what they might gain. The research, by the way,
- 15:46on this, if you look at Annie Duke's work in
- 15:48the book Quit and her podcasts and her other
- 15:51publications, Thinking in Bets is a good book
- 15:54on that subject. The problem becomes that you
- 15:57can't put a weight on what you don't know. So
- 16:00while we might say this job looks great, should
- 16:02I move firms? Should I counsel my client to tender
- 16:06their resignation and switch? We know how to
- 16:10weight what we're going to give up. We don't
- 16:12know how to weight uncertain futures. That's
- 16:15a harder scenario to calculate. Narrative fallacy
- 16:20is also fascinating, and there's a lot of Nobel
- 16:22Prize -winning work in this area. We're storytellers.
- 16:27As human beings, we're storytellers, the concept
- 16:29goes. And we love our stories, and we need them
- 16:32to make sense of the world. And what you hear
- 16:35from time to time when asking a client early
- 16:39in a case, for example, Tell me what happened.
- 16:42In your own words, what happened? And when I
- 16:45listen, I hear something that is a disconnect.
- 16:48There's a particular fact or event that's missing,
- 16:51and the client has filled that in with something
- 16:54that's faulty or they made a jump. Because we
- 16:57need our stories to make sense. It's not good
- 17:00enough to get to a point in the story and say,
- 17:02and then I didn't see who was at the intersection
- 17:04of Figueroa and Sunset. It just doesn't work.
- 17:10So we really need somebody to vet those stories
- 17:15and make sure that our narratives are not filled
- 17:18with inaccuracies. So there's another absolutely
- 17:24essential concept to cognitive psychology and
- 17:27cognitive bias, and that's the Dunning -Kruger
- 17:29effect. And anybody who's taken psychology would
- 17:32hear and see this one right away. In basic terms,
- 17:37and Adam Grant has the definition there, the
- 17:40concept is that people with limited competency
- 17:44in a particular domain Overestimate their own
- 17:47abilities. So Dunning and Krueger are two different
- 17:50psychologists and what you're looking at is research
- 17:52in which you first quiz people about certain
- 17:55subjects and then you give them another quiz
- 17:57and you ask them how confident they are in their
- 18:01perceived ability and it's crazy that those aren't
- 18:04one -to -one they do not overlap those are not
- 18:07two circles overlapping in any way at all and
- 18:10the less Qualified somebody is sometimes the
- 18:14more confident they are and we've all been in
- 18:18mediations or rooms in which you're listening
- 18:21to somebody and thinking that person doesn't
- 18:23know what they're talking about and yet they've
- 18:27Taken over the room talked in the loudest voice
- 18:30and held everybody hostage. So it's a little
- 18:33bit scary that people will extrapolate from such
- 18:37little information and We've given a few examples
- 18:43here. My favorite is the statistics. I think
- 18:46those are guys, by the way. Six percent of American
- 18:49men think they could beat a grizzly in a fight.
- 18:54There is similar research, by the way, in almost
- 18:56every field. So if you ask the average club player
- 19:02at tennis, could you return Serena Williams serve?
- 19:05You're going to hear them say overwhelmingly
- 19:07yes. In fact, the U .S. tennis Association surveyed
- 19:12folks, recreational players, 71 % thought that
- 19:16they could at least get a game off a top 250
- 19:19professional player. A similar research in the
- 19:22NBA, people just generally overestimating their
- 19:25own abilities. What Dunning -Kruger effect to
- 19:30me is a takeaway is absolute lack of accurate
- 19:36perception of one's ability and the lack of humility.
- 19:40Involved in these areas. So if we were quicker
- 19:44if our clients were quicker Somebody else was
- 19:47expert witnesses were quicker to say I just don't
- 19:51know You know, you could get those two circles
- 19:54to overlap a lot faster These cognitive biases
- 19:59show up all over the place in the practice of
- 20:01law and I'm sure that everybody listening here
- 20:03could immediately call to mind multiple experiences,
- 20:08whether it be with opposing counsel or jurors
- 20:12misinterpreting evidence you know at least in
- 20:13Arizona you get to often times get to interview
- 20:15the jurors after the trial and see what was persuasive
- 20:20to them with your clients mis -evaluating the
- 20:25case oftentimes refusing to acknowledge the significance
- 20:29of a negative fact and overweighting the facts
- 20:32that they think are in their own benefit you
- 20:35see that on the other side of the of the aisle
- 20:37as well. And these things can oftentimes get
- 20:39in the way of settlement, get in the way of the
- 20:41accurate evaluation of the case. A few examples
- 20:45where scientifically there have been studies
- 20:47done that demonstrate the ways these biases could
- 20:50show up. People really, really overestimate.
- 20:53This isn't just a legal issue, but this is everywhere.
- 20:55People really overestimate their ability to detect
- 20:57liars, to evaluate somebody based on their perception
- 21:01of them in a social setting. We just think That
- 21:05were perceptive and we get people I can tell
- 21:07that my grandmother was like this She she could
- 21:09swear that she would be able to tell within you
- 21:11know Ten seconds of meeting somebody whether
- 21:13or not they're a good person or a bad person
- 21:15trustworthy person or not Of course, she had
- 21:17no information to go on and it's like she believed
- 21:19she was a psychic but people have this sort of
- 21:21inflated evaluation of their own ability to evaluate
- 21:26others And it is completely inaccurate. There's
- 21:29a great book by Malcolm Gladwell called talking
- 21:32to strangers That among other things explores
- 21:35this particular issue and really lays out and
- 21:37give it to detail the statistics on people's
- 21:42Accuracy as lie detectors and you're terrible
- 21:44at it. We might as well flip a coin It applies,
- 21:47you know a lot of poker players think that they
- 21:51At least in the olden days. I think it's gone
- 21:53away from this a little bit, but they would spend
- 21:55a lot of time studying tells Social cues that
- 21:58they believe they could if they observe them
- 22:01carefully enough would reveal that the other
- 22:03player is bluffing or have a good, have a strong
- 22:08hand, weak hand, things like that. Poker players
- 22:10are also terrible at that. Those, they believe
- 22:13they're great at reading tales, but they're actually
- 22:15pretty bad. This is probably something that most
- 22:18listeners to this presentation will have heard
- 22:20at least once or twice before, but I would, this
- 22:22testimony is very unreliable, which is unfortunate
- 22:25for our profession because at least the litigation
- 22:30process for adjudicating disputes is historically
- 22:33based very much on eyewitness testimony. We have
- 22:37a trial, we get witnesses on the stand, juries
- 22:40are present and they watch the witness testify.
- 22:44Witness has to be there. The parties have the
- 22:47right to have witnesses be present rather than
- 22:50providing evidence through affidavit as long
- 22:53as they're available. We exclude hearsay because
- 22:58In the legal profession, we think the person
- 23:00needs to be there. Why? Because the finder of
- 23:03fact, whether it be a judge or a jury, we think
- 23:04that if they can see the person talking, they
- 23:07can evaluate their trustworthiness, whether their
- 23:09credibility is one of the primary reasons why
- 23:12we insist that witnesses be present. And all
- 23:16that is premised on the frankly very inaccurate
- 23:19belief that you can tell whether or not the testimony
- 23:23is credible or accurate, which we covered in
- 23:27the last bullet. but also because we're trusting
- 23:29that the person who's testifying that they saw
- 23:31something actually saw it. And that's really
- 23:36oftentimes not the case. There's a study, a report
- 23:39issued in 2016 by the Innocence Project that
- 23:42looked at 230 convictions that had been overturned
- 23:46by DNA testing over the civil year period. And
- 23:49they determined that of those 230 convictions,
- 23:53179 of them were based on eyewitness testimony
- 23:56where the witness claimed to have seen with own
- 23:58two eyes that the person that ended up being
- 24:01convicted was the one who committed the crime
- 24:04and DNA evidence unequivocally exonerated those
- 24:08people. That's 75 % of overturned convictions
- 24:11were based on bad eyewitness testimony and so
- 24:15that that there are biases that are woven throughout
- 24:18our preference for eyewitness testimony that
- 24:20is very problematic for a question. Inconsistent
- 24:25sentencing is another thing. I think we all kind
- 24:26of have an intuitive understanding that every
- 24:29judge is a little different Lungs, you know,
- 24:31you have a judge that's harsh one letter more
- 24:33lenient We just kind of accept this as being
- 24:35baked into our system But obviously there is
- 24:38an unjust element there. The legal system should
- 24:40be consistent and predictable Justice should
- 24:45be in some way tied to that and yet it's often
- 24:49not available to us. So This goes all the way
- 24:51back to 1974. There was a study that showed that
- 24:55they presented judges with identical hypothetical
- 24:58situations and asked them to pass sentence. And
- 25:01they gave them the exact same case file, all
- 25:03the same facts with the exact same material.
- 25:05It's all hypothetical. And the results of that
- 25:08survey show that judges passed wildly, wildly
- 25:11different sentences. There were similar studies
- 25:14done in 77 and 1981 that confirmed those results.
- 25:19There have also been studies that showed that
- 25:21a hungry judge is a less lenient judge. So a
- 25:26study of sentences passed by judges who passed
- 25:31those sentences right before the lunch break,
- 25:33so when they would have been at their hungriest,
- 25:35were much harsher than sentences passed by judges
- 25:40in similar cases. Obviously the study tried to
- 25:44equalize and account for differences in case
- 25:48character. But after the lunch break, when they
- 25:51were full and fat and happy, they were far more
- 25:53lenient. Similar issues when it comes to the
- 25:58beginning of a session, or a judicial session
- 26:01as to the end. So this is very much one of the
- 26:07many applications of cognitive bias in the law.
- 26:09And I will, at the risk of blaming this further,
- 26:12I will tell a brief story about my first memorable
- 26:14experience of cognitive bias was when I was in
- 26:17law school. participated in a sort of mock trial
- 26:21course where we took a case file we worked it
- 26:23up from beginning to end and then the end of
- 26:25the semester we did a fake trial and of course
- 26:27we had actors on the as witnesses uh we had a
- 26:30real judge who oversaw it but we had fake witnesses
- 26:33we had people off the street that were pulled
- 26:35into the act as the jurors that we presented
- 26:37our case we uh we examined the witnesses and
- 26:41then at the end of the trial the jury rendered
- 26:43his verdict and then we were able to interview
- 26:45them and ask what they found to be most important
- 26:48most persuasive And I was very disheartened,
- 26:51but we ended up winning. But the reason we won
- 26:54was not because of any fact that was actually
- 26:58relevant to the case or any legal argument that
- 27:00we made. The jurors talked about a specific part
- 27:04of the testimony, one of the witnesses, where
- 27:06she had made some comment. This is, of course,
- 27:09a pretend character talking about her pretend
- 27:11husband. It was completely irrelevant to the
- 27:14facts of the case, but it was in some way negative.
- 27:16And that led the jury to say, there was something
- 27:18wrong with her. I didn't like her because of
- 27:20that. And as a result, the remainder of the testimony,
- 27:24that witness was deemed not credible. And so
- 27:27they found against that witness's interest and
- 27:31in our favor. And it made me very, very scared
- 27:34of juries from that day forward. So I'll give
- 27:36you another example that's related to that and
- 27:40relates really to confirmation bias and the need
- 27:43to get out of that paradox. or escape that prison.
- 27:49I was local counsel to a big five at the time,
- 27:53I think, a county firm that was being sued in
- 27:56the Boston Chicken, Boston Market debacle. So
- 27:59I think Kirkland was the counsel, and we ran
- 28:04two different juries, like mock juries, and we
- 28:11presented the theme of the case to two different
- 28:13groups of potential jurors they could have been.
- 28:18And wildly different outcomes between the two
- 28:20with the exact same themes. And because you get
- 28:24to watch them deliberate for anybody who litigates
- 28:26and anybody listening to this, the concern is
- 28:31it goes off the rails rather quickly. Somebody
- 28:35had brought in their own experience, they'd worked
- 28:37for a large company, and they said, well, you
- 28:40know, it's all about the money. And that's how
- 28:43it works every time. And this is what happened
- 28:45to me at working at XYZ, no bearing at all on
- 28:50the case, and then ultimately convinced the other
- 28:54jurors in that room. The Boston Chicken Boston
- 28:59Market case was about $900 million in joint and
- 29:04several liability between large banks, law firms,
- 29:07and two different big accounting firms. But it's
- 29:11kind of scary because prior to that, all the
- 29:13lawyers going in were pretty sure they had a
- 29:16really good defense. And everybody always does
- 29:20think they have a really good case. But once
- 29:22you pitch that to lay people and find out what
- 29:26motivates them or doesn't motivate them, that
- 29:29gets you a better sense of the variation, the
- 29:35variance in the outcome. So let's start with
- 29:40the U .S. Army. When I originally did some of
- 29:45this work, my thought, Michael and I talking
- 29:47together, I assumed we were going to end up with
- 29:51medicine being the number one area. I was familiar
- 29:54with Dr. Atul Gawande's book on Checklist Manifesto,
- 29:58which is a phenomenal book and I commend everybody.
- 30:01But all of the literature and medicine seems
- 30:03to go back to, well, this is what the Army was
- 30:06doing decades earlier. So they really deserve
- 30:10the credit and you probably have to give them
- 30:12a first review if you're surveying cognitive
- 30:17biases and how to combat unintended them. So
- 30:24the Army starts by inventing the after action
- 30:27review. Think of it as a postmortem. Here's a
- 30:31good definition of it. So we have these discussions
- 30:34of an event. We look at the standards. We look
- 30:37at how this is going to help soldiers going forward,
- 30:42what happened, how to sustain strengths and improve
- 30:45on weaknesses. That's just one definition. There
- 30:49are dozens and dozens of definitions from various
- 30:52published papers on this. So what you're going
- 30:58to do, if you implement this, for example, what
- 31:01we've done, sometimes when we do it and make
- 31:04the time, that's the biggest downside to this
- 31:08is you really have to make the time, non -billable
- 31:11time, to learn and dissect what happened, what
- 31:15was our goal going into that representation,
- 31:18what was the client's goal, what was the outcome,
- 31:21what factors contributed to that outcome, what
- 31:23did we do right, and what did we do wrong? And
- 31:25I'll give you one that we did this on, and I
- 31:28don't know if you were on this case, Michael,
- 31:29but we had a case, it was a large, low -income
- 31:34housing tax credit case. It was low -income housing
- 31:37tax property in Arizona. And we won. We got the
- 31:42plan confirmed. One issue went all the way up
- 31:45on appeal. We got that resolved, went again.
- 31:51It ultimately went up on -bomb to the Ninth Circuit,
- 31:54but we had to get an original panel 2 -1 reversed
- 31:59in order to ultimately win. But what we did the
- 32:04after -action review was, We had certain evidence.
- 32:07We got into evidence at that trial. And there
- 32:11was certain evidence that got excluded. And to
- 32:14this day, it burns me. That's a decade old problem.
- 32:17But we did call a meeting after the first trial
- 32:20to say, well, what did we do right? What did
- 32:23we do wrong? What would we do differently? And
- 32:26it meant a lot to me that we would take the time
- 32:28to improve on our processes so that we won't
- 32:32make that mistake again. And it was a really
- 32:34good trial lawyer, and he couldn't get it in.
- 32:37And I think the judge actually probably did us
- 32:38a favor because it made the record better for
- 32:41us to defend on appeal. That's also sometimes
- 32:45worth knowing is when you're winning and when
- 32:48you're losing and why you're winning. So this
- 32:56again, how we're going to apply this to our practice
- 32:59is we set goals. We're going to have a strategy
- 33:02in advance, of course, for every case and every
- 33:04representation. And when we're done, We're not
- 33:08going to be the kind of lawyers who like to throw
- 33:10a party and nobody wants to clean up. Even though
- 33:15every one of us has case fatigue or deal fatigue,
- 33:19we're going to take a vacation and then we're
- 33:21going to come back into a room, the lawyers who
- 33:24worked on that, and we're going to sit down.
- 33:26We're going to have an honest, non -hierarchical,
- 33:30judgment -free conversation about that. So we're
- 33:33going to try to figure out what we did right,
- 33:35what we did wrong, and not point fingers. at
- 33:38anybody in that conversation. Here's another
- 33:43takeaway that I find fascinating from the military.
- 33:47And that's this concept of states and dates.
- 33:50So so often in litigation, for example, clients
- 33:55say, I want to do X, but how much is that going
- 33:58to cost me realistically? And how long is that
- 34:01going to take? And we do our best to triage it
- 34:04up front. We do our best to estimate it, we tell
- 34:07them we can take it incrementally, et cetera,
- 34:10et cetera, et cetera. But in Andy Duke's book,
- 34:14Quit, she does an excellent job of going over
- 34:18an example from the raid in Pakistan on Osama
- 34:25bin Laden, which was named Operation Neptune's
- 34:29Spear. And what she talks about is that Admiral
- 34:32McRaven had set out I think 162 different action
- 34:37points for that raid in advance. And that raid
- 34:42only required him, once they did the action planning,
- 34:46to look at five different command decisions.
- 34:48So five different times out of 162 events, did
- 34:52the U .S. military have to step back and say,
- 34:57okay, are we going forward or are we not? Are
- 34:59we going to do this? Are we going to do that?
- 35:02So there was that much more thought. that much
- 35:05planning. And at each time, they said, if you
- 35:08don't get this far, we're going to do something
- 35:11differently. And the way I use this in private
- 35:14practice is at the almost initial meeting, I
- 35:18say to the client, you can do this at home. You
- 35:20can do this in quiet of your own house. You could
- 35:23do it in discussion with your business partners.
- 35:26I need to know what success looks like. I want
- 35:28you to write that down on paper. I need to know
- 35:31that I met your expectations. exactly and I need
- 35:36you to know that you're not going to move them
- 35:38those goalposts on us throughout the representation
- 35:41now we might agree that we're doing better than
- 35:44we thought or worse than we thought and we're
- 35:46going to move our expectations accordingly but
- 35:48we need to at least put them on paper kind of
- 35:52like anybody who's run a 5k 10k you know half
- 35:55marathon or marathon you write down maybe on
- 35:58a piece of paper put in your shoe put it tell
- 36:00your friends kind of an accountability friend.
- 36:03I'm going to run eight -minute miles. I should
- 36:05be at this stage at this point. I'm committing
- 36:08to a 212. That's my goal for the half. Whatever
- 36:12it is, write it in advance, set that expectation,
- 36:17read on the expectations, and decide if we're
- 36:19not here by this time. So an example would be
- 36:22you tell the client, I need to know you're committed
- 36:26to this litigation. But if in six months, we
- 36:30haven't received motion for summary judgment
- 36:33rulings or teed it up and at least had it submitted
- 36:35to the court, then we're going to decide, well,
- 36:38we're going to go to mediation at that point.
- 36:40So we create these expectations that there are
- 36:43times in which we can dial it back, reassess,
- 36:47and reevaluate, and then go in a different direction.
- 36:52So pivoting to the medical industry, of course,
- 36:57in medicine, they have a similar concept to the
- 36:59action after action review, and that's the postmortem.
- 37:01which is probably the term that is way more generally
- 37:05into common parlance. But that's an institutional
- 37:08aspect of what doctors often do. They evaluate
- 37:12the treatment after it's done. And they also
- 37:14will do, you know, pre -mortems, similar after
- 37:17-action reviews throughout the treatment process.
- 37:20Some other things that we're going to talk about
- 37:21that are done in medical field is insertions
- 37:25will do check -ins with their staff. They will
- 37:28return to these pre -stated metrics that, you
- 37:33know, they'll check levels, they'll check where
- 37:35are we at this process periodically, rather than
- 37:39just forging ahead, not keeping an eye on the
- 37:41dials as well. Also empowering nurses to double
- 37:48check the doctor's work, keep them honest, force
- 37:51them to go back and check those things. That
- 37:53has a corollary in the legal practice. You want
- 37:55to empower your staff to be Active participants
- 37:58in the litigation or the representation not just
- 38:01people who you know You give them some of the
- 38:03file or you say do this do this and they're not
- 38:05expected to Do question you or otherwise contribute
- 38:08you want them to be active contributing members
- 38:11who who can hold your feet to the fire? Make
- 38:13sure that you're doing things properly And another
- 38:17thing this is one of the things that Scott just
- 38:19talked about in the military context called states
- 38:22dates where you're resetting setting criteria
- 38:28and checkpoints throughout the process. In the
- 38:30medical field, it's the same thing. You don't
- 38:33want to have to be making hard decisions or thinking
- 38:35through our decisions in the heat of surgery.
- 38:38You want to have a 10 -hour surgery. It's stressful.
- 38:42It's high constraints. If something goes wrong,
- 38:44you have to act quickly. If you are thinking
- 38:46through everything for the first time under those
- 38:48circumstances, you are far more likely to make
- 38:50a mistake. That obviously has a corollary. legal
- 38:52practice as well. You know, you don't want to
- 38:54just be responding to every pitch as it comes
- 38:57at you as if it's the first time you've seen
- 38:59one. You want to sort of have thought through
- 39:01them and established, you know, metrics for success,
- 39:05sort of pre -made some of those decisions, sort
- 39:07of minimize the number of on -the -spot decisions
- 39:10that you have to make in your practice. So thinking
- 39:14more specifically about a particular aspect of
- 39:18the industry, even routine tasks. The medical
- 39:22field is same I mean it is absolutely insane
- 39:25the things that we can do and the doctors can
- 39:28do the types of surgeries that you do they are
- 39:31incredibly complicated incredibly complicated
- 39:34even the tasks that surgeons and doctors to them
- 39:38the routine they do them every day all day every
- 39:41day they still have tons of steps and tons of
- 39:45opportunities for something to go wrong so that
- 39:47if the doctor is just doing it by memory doesn't
- 39:49have an institutional practice it's a sort of
- 39:51checklist to go through that mitigates the errors
- 39:56that can arise through memory lapses, fatigue,
- 40:00things like that. Errors can arise if not for
- 40:03those systems. So it's very easy to miss a step.
- 40:06If you're going to ask a key question, failing
- 40:08to plan properly. One particular case study here
- 40:13is central line infection. This is a very illustrative
- 40:16story and I'll try and tell this very briefly.
- 40:18The case study is a patient named Anthony DiFilippo.
- 40:22He's a 48 year old limousine driver. He went
- 40:24in for a routine surgery for a hernia. It's also
- 40:26not life threatening. But during the surgery,
- 40:30he started hemorrhaging. Tests revealed he was
- 40:33experiencing liver failure. So over the next
- 40:35few hours, as a result of that liver failure,
- 40:37each of his organ systems, including his brain,
- 40:41began to shut down. Over the next several days,
- 40:43doctors provided a variety of very complex interventions,
- 40:48treatments to keep him alive. They inserted lines
- 40:50into his wrist, jugular veins, upper chest, under
- 40:53the collarbone. And through these lines, using
- 40:56various machines that regulated his breathing,
- 40:58and he regulated intravenous fluids, regulated
- 41:01his blood composition. It was really amazing,
- 41:04magical stuff that these doctors were able to
- 41:06do to keep him alive. At some point, the main
- 41:08duct from his liver was leaking bile, so they
- 41:11inserted three drains to draw the bile out. His
- 41:14fever subsided, and things started to improve.
- 41:17But then on the 11th day, he spiked a fever,
- 41:20blood pressure sank, blood oxygen levels plummeted.
- 41:24looked like he was getting an infection but x
- 41:27-rays and CT scans couldn't identify his source
- 41:29of the infection. Fevers continued to spike and
- 41:32then he went into his heart with fibrillation.
- 41:36The doctors couldn't figure out what was wrong
- 41:38and eventually, finally, after weeks of this
- 41:41guy being in their care, they decided to test
- 41:43his lines to determine whether one of them had
- 41:46become infected. And as it turned out, all of
- 41:49them were infected. The significance of the story,
- 41:52I think it's... It really highlights the remarkable
- 41:55things that we can do in the medical profession
- 41:57But they were doing so many complex complex interventions
- 42:00to keep him alive and yet the thing that was
- 42:02causing the problem was introduced by the doctors
- 42:05themselves and Because the central line had just
- 42:10gotten infected because they didn't have proper
- 42:12steps They were observing the proper steps and
- 42:14went inserting those lines whether it was not
- 42:16washing their hands or sterilizing the area and
- 42:19Infection gets introduced and then eventually
- 42:21his whole body was getting full of this infection
- 42:23that turned out to be the cause of his problems.
- 42:28So how does this get solved? There's a Dr. Peter
- 42:31Pernobois at Johns Hopkins, he said, I'm going
- 42:33to tackle this one specific problem, central
- 42:35line infections, which is a recurring issue,
- 42:38you know, 11 % infection rate, which I mean,
- 42:42think about that, you go to the doctor, you don't
- 42:44get to the hospital, you don't expect an 11 %
- 42:46chance that you're going to get sicker because
- 42:48of an infection issues by the hospital, but that's
- 42:51what they're experiencing. He created a very
- 42:53simple five -step check list. Wash your hands,
- 42:57clean your skin, the skin of the area using an
- 43:00antiseptic, cover the area with sterile drapes,
- 43:03wear a sterile mask, hat, and gloves, and then
- 43:06put a sterile dressing over the insertion site.
- 43:09This is all basic, obvious stuff that every doctor
- 43:13knew to do, should have done, without fail. And
- 43:17yet, by creating this checklist and requiring
- 43:21doctors to follow it, which Apparently, there's
- 43:23a lot of pushback on that because doctors are
- 43:25very upset and offended that they were being
- 43:28told to do this basic stuff. I'm a doctor, I've
- 43:33been in school, I dare you give me this dummy
- 43:36proof checklist. And yet, since he was the boss,
- 43:38he was able to acquire it. And in addition to
- 43:41the checklist, he instructed nurses to observe
- 43:44the doctors for a month. And as it turned out,
- 43:47in that one month observation period, doctors
- 43:49get to at least one of those steps, at least
- 43:51one. and a third of cases. So Purnabas amped
- 43:57it up a little bit. Doctors were required to
- 43:59not only use the checklist, but he empowered
- 44:00the nurses to, in the room, stop doctors if they
- 44:04skipped the step. Now this was probably the most,
- 44:07or one of the most important parts of this change
- 44:08was a big cultural shift, usually in the room
- 44:10the doctor is the boss. They're the ones who
- 44:12make the final say, and nurses were not empowered
- 44:14to speak if they didn't have any authority. They
- 44:16were following instructions rather than, there
- 44:20was no area in which they were on equal play
- 44:21if people didn't. doctors so there was a lot
- 44:23of pressure against them to speak up so they
- 44:25wouldn't but part of us said you will have the
- 44:27backing of the administration your job now is
- 44:30to check the doctors if they skip a step you
- 44:33can stop them stop what they're doing and force
- 44:35them to go back over the next year that infection
- 44:43rate went from 11 to 0 and that held for the
- 44:46next 12 to 15 months as a result that two -year
- 44:50period This one very simple institution of a
- 44:53checklist and empowering staff, they prevented
- 44:56an estimated 43 infections, eight deaths, and
- 45:00two million in costs. That's at one hospital
- 45:02during a two -year period. Just imagine how many
- 45:05lives have been saved, how much money has been
- 45:06saved since then as it has been rolled out to
- 45:09the other hospital. We kind of talked through
- 45:15this, but using checklists in the surgical context
- 45:20produces error. You improve communication between
- 45:24staff and doctor. increases accountability, and
- 45:27it just improves outcomes. Now, how do we apply
- 45:30that to legal practice? There's a bit of, I think,
- 45:35general opposition to the idea of using checklists
- 45:37in legal practice in a lot of areas. We're kind
- 45:40of resistant to the idea that any of the cognitive
- 45:43wizardry that we employ in our practice could
- 45:45ever be reduced to a checklist. You know, what
- 45:48we're doing is magical. It's impossible. And
- 45:50yet, there are areas in which we can Institutionalize
- 45:56things, create checklists as a tool to help ourselves
- 45:59just to make sure we're going through the steps.
- 46:01We're not carrying everything in our brain. There's
- 46:03no need for us to carry everything in our brain.
- 46:05Create checklists where possible. And checklists
- 46:08can also be a tool that you use to train your
- 46:12staff to familiarize themselves with the process
- 46:14and to empower them to follow along with the
- 46:17process so they can't be an active participant.
- 46:19They can remind you, did you do this? They can
- 46:22ask meaningful questions. Are you sure about
- 46:24that? And I, you know, I rely on my staff a lot,
- 46:29and I sometimes not only do I expect them to
- 46:33chime in, I ask them, did I miss anything? And
- 46:36that is one way you'll have to think through
- 46:39how to apply this to your specific area of practice,
- 46:41the types of cases they have. There are things
- 46:43that apply to everyone, calendaring, client intake.
- 46:46There's a low -hanging fruit where checklists
- 46:48can be employed, but think through also some
- 46:51of the substantive legal areas where you might
- 46:53be able to use. a checklist and use that checklist
- 46:56to train your staff to improve the outcomes of
- 47:00your practice. Similar concept, talked about
- 47:05post -mortem retroaction reports, and just planning
- 47:10ahead at the beginning of the case thinking through
- 47:13what your goals are, the steps you're likely
- 47:15to encounter on the way to that goal, and imagine
- 47:18where could things go wrong, and then plan ways
- 47:22to prevent those possible negative obstacles
- 47:25or outcomes. I won't go through this, but again,
- 47:30I think it's very similar to the after action
- 47:31reports. Sitting down, having the hard conversation
- 47:35in a judgment free zone. The point being to find
- 47:38out where your errors are. We don't really necessarily
- 47:41have a good culture of that in the legal profession.
- 47:44We don't want to acknowledge errors or staff
- 47:46doesn't want to admit mistakes because they'll
- 47:48get in trouble. We want to flip that culture
- 47:52on its head and invite everybody to not only
- 47:56acknowledge themselves, but acknowledge to others
- 47:58in their firm when they made a mistake so that
- 48:00we can all learn from it. So I suggest that we
- 48:04skip the examples in aviation. It's 40 some minutes
- 48:08into the presentation, and we're going to run
- 48:11low on time. I suggest we make sure we've left
- 48:15enough time for Q &A and that we address taking.
- 48:19I think that's a good idea. OK. So. Skipping
- 48:23over these you can read them in your own time
- 48:25and enjoy and there's somewhat similar Concepts
- 48:29applied then to aviation to reduce and improve
- 48:33training so I could probably do it in 10 seconds.
- 48:37No, no You're gonna run out of time. Yeah, I'm
- 48:40100 % sure you're gonna run out of time. We don't
- 48:42want all this plane crash Yeah, we don't want
- 48:44to crash. Yeah takeaways, okay, so when I ask
- 48:54a neurosurgeon friend of mine what the biggest
- 48:57problem is in surgery for example again anecdotal
- 49:01he says that the problem in medicine and you
- 49:04can see this maybe in law and other fields is
- 49:07there's a hero culture and how do we combat that
- 49:11and how do we stop making all of these decisions
- 49:19in a manner that reinforces that so let's Look
- 49:24at what other fields have done to combat this
- 49:27so basic science science of science if you will
- 49:31Karl Popper's Observation that you know without
- 49:36in essence some paraphrasing without hypothesis
- 49:39testing You get confirmation bias. You see it
- 49:42everywhere every theory you've ever had just
- 49:45reinforces your outcome and in essence my takeaway
- 49:49from that is that Our bubble is our enemy and
- 49:55that the longer we spend in that bubble thinking
- 49:59X or Y or Z about our case or about a client
- 50:03or about a judge or about opposing counsel or
- 50:06about a set of facts, the longer we're unable
- 50:11to really solve that problem to the best of that,
- 50:15you know, maybe our solution's a B plus and we
- 50:18could get to an A. So here are some suggestions.
- 50:24Obtain different viewpoints. You have to ask,
- 50:27I have to ask everybody what they're thinking,
- 50:30why they're thinking that. Help me understand
- 50:32why you came up with that. Start the phone calls
- 50:36with opposing counsel in that manner. Sometimes
- 50:40listening first, talking later. And I would add
- 50:45to that, maybe you want to force yourself to
- 50:48take seriously the viewpoints that you would
- 50:49not initially want to agree with. And actually,
- 50:52that's the next one. Keep going and you'll get
- 50:55there. Yeah. So, you know, one of them is that
- 51:01acknowledging that there is a complex problem
- 51:04to be solved and that our initial reaction to
- 51:07that may not be ideal. And then we'll get to
- 51:11your suggestion just dead on. So the first thing
- 51:15to do is steal man the argument. You know, you
- 51:17have to make the opposing council's argument
- 51:20better. you have to figure out what is right
- 51:22and what is wrong with their argument. And ideological
- 51:26Turing tests sometimes it gets called. If you
- 51:28can state opposing views as clearly and persuasively
- 51:32as your opponent, then you're a long way down
- 51:36the road to being somebody who can actually attack
- 51:40those arguments and understand where they're
- 51:43coming from. Obviously admit mistakes. We've
- 51:46talked about that. The last one is emphasized
- 51:51In Ted Lasso, he credits Walt Whitman, maybe
- 51:54inaccurate, but it's be curious, not judgmental.
- 51:58The goal here is to try to figure out what it
- 52:01is we're missing and why we might be missing
- 52:03it. So let's see if we've... So these are all
- 52:09best practices, by the way. We're not suggesting
- 52:12we do them all the time. We're not suggesting
- 52:14we know anybody who's done them all the time.
- 52:17What we're suggesting to you is that... Kind
- 52:21of an experiment like we're drawing upon from
- 52:24different scientific fields I would run some
- 52:28of these through your own practice Start small
- 52:31scale and see how it changes if at all Different
- 52:36practice areas so your materials have references
- 52:39in there. It has the contributors They are not
- 52:42to be blamed for me misunderstanding The field
- 52:45of epidemiology for example, etc, etc That was
- 52:49my best interpretation of those things. So we're
- 52:55on the Q &A, and somebody had posted at 410,
- 52:58and I apologize, Gabriel, for ignoring the question
- 53:00for so long. Appellate opinions are there so
- 53:03that we can learn from the mistakes of others.
- 53:05And that's an excellent point. Very good. And
- 53:10I've been doing this more and more as just downloading
- 53:13case after case and trying to figure out, for
- 53:16example, in a... an area that we only do every
- 53:1910 years within our own profession, subspecialty.
- 53:26What's the latest development of the case law?
- 53:28What's the latest thinking on this? And try to
- 53:31figure out what we can draw from those published
- 53:34opinions and unpublished opinions on that. So
- 53:38that's an excellent point. Thank you, David.
- 53:40Any other Q &A? Do we have any other feedback?
- 53:49Answered. Can I get link to the materials? Yes.
- 53:53Beverly Hills Bar Association will provide that
- 53:56link. You can get copies. And I think that link
- 53:58was, Jose was kind enough to do that for us at
- 54:01436. So the materials are available to you through
- 54:04the CLE. You can also, we would welcome anybody
- 54:07who wanted to reach out to Scott Wright directly.
- 54:09Our emails are here. We'd be happy to talk with
- 54:11you or share this presentation with you directly.
- 54:14We both really appreciate the opportunity. to
- 54:17present all the psychological gobbledygook to
- 54:19you and try to find a way. Yeah I would also
- 54:21I would also invite any collaboration you have
- 54:25so if you have a good idea and we did not suggest
- 54:28it either because of time constraints or ignorance
- 54:30go ahead and email us and suggest whatever it
- 54:34is that's worked for you. I mean our goal is
- 54:36to deliver better legal services to clients at
- 54:39all times and if you're trying something We're
- 54:43happy to do it and give it a shot. And these
- 54:47aren't competitive in any way at all. I'm not
- 54:49suggesting practice area to practice area in
- 54:52the same market. I'm suggesting, and the one
- 54:54we did to international lawyers and domestic
- 54:56lawyers, was really an open conversation about
- 54:59what other people do from Hawaii to folks at
- 55:05Russell Cook in London. So we're trying to get
- 55:07better at this through conversation, through
- 55:11learning from others. Is that true? So any other
- 55:15in the chat? You see the chat down there? Is
- 55:17there any other? We've got three more minutes
- 55:19to answer more questions. More questions. There's
- 55:25an article that we published in the Arizona Attorney.
- 55:30It's incredibly well done. A lot of the research
- 55:32was done by, we use a lot of interns that are
- 55:37now at Stanford and Columbia and Harvard Law
- 55:41School. who have the honor to be their mock trial
- 55:46coach when they're in high school and they are
- 55:48interested in the practice of law despite being
- 55:51incredible STEM students. And so we oftentimes
- 55:54open the law firm up to them coming here, doing
- 55:58research, doing research on their senior capstone
- 56:00projects, but also helping us out. And so we've
- 56:04been fortunate to get two or three of them published,
- 56:06which helps them get into law school. And we
- 56:10obviously like AI would have to vet their work
- 56:14to make sure that it's good and we're not signing
- 56:16off on something. But they're incredibly bright
- 56:19and it's helpful to improve our thinking of certain
- 56:23subject matters is to try to explain it. It's
- 56:26a basic principle, but to try to explain it to
- 56:29somebody who doesn't know anything about it.
- 56:31I used to have a law partner, a litigation partner,
- 56:35who would gather a few legal assistants and paralegals
- 56:39up And he would pitch them different themes in
- 56:41different cases. And so Randy's his name. And
- 56:44Randy would say, well, what do you think about
- 56:46this? What's your impression of this? And introduce
- 56:48new facts into the conversation. And just get
- 56:51what a lay person, potential juror type, would
- 56:54think about it. And I thought that was really
- 56:58interesting. He loved detention. So we always
- 57:02were suspect as to why he was doing that. But
- 57:05it was a really good conversation. Because the
- 57:08average person does not come away from that conversation
- 57:12with the same perspective as somebody who's had
- 57:1430 years of trial litigation experience. And
- 57:17that's where Randy was coming in. But any suggestions
- 57:21you all have, we would take them. I'm regularly
- 57:24in Beverly Hills. I have family there. And I've
- 57:28been licensed in California for 32 years. So
- 57:31I go frequently back and forth and would be happy
- 57:34to meet for coffee at any time. somebody wanted
- 57:37to have a conversation regarding this and practice
- 57:39the law. Thank you.