Latest / a16z AI Policy Brief / How AI Is Changing the Practice of Law

Transcript

How AI Is Changing the Practice of Law · a16z AI Policy Brief

• 0:00
  SPEAKER_01: Welcome to the A16C AI Policy Brief.
• 0:03
  SPEAKER_01: I'm Matt Peralt.
• 0:05
  SPEAKER_01: AI is changing the practice of law from researching legal questions
• 0:08
  SPEAKER_01: and reviewing contracts to directing AI agents.
• 0:11
  SPEAKER_01: What could that mean for the cost and availability of legal services?
• 0:15
  SPEAKER_01: And how do we make the most of these tools while keeping lawyers accountable for the results?
• 0:19
  SPEAKER_01: Today, I'm joined by John Labar,
• 0:21
  SPEAKER_01: Chief Legal Officer at Harvey, which builds AI tools for lawyers.
• 0:25
  SPEAKER_01: We talk about how he's building an AI-native legal team,
• 0:27
  SPEAKER_01: the skills future lawyers will need,
• 0:29
  SPEAKER_01: and how policy is playing an active role in defining the future of legal work.
• 0:33
  SPEAKER_01: Along the way,
• 0:33
  SPEAKER_01: we explore a broader question,
• 0:35
  SPEAKER_01: can AI make professional expertise available to more people,
• 0:38
  SPEAKER_01: and what would it take to deliver on that promise?
• 0:40
  SPEAKER_01: Enjoy our conversation.
• 0:42
  SPEAKER_00: Harvey is very much the tool that lawyers can use.
• 0:45
  SPEAKER_00: To do their work more efficiently.
• 0:47
  SPEAKER_00: But it's not an abrogation of the duties of the lawyer.
• 0:51
  SPEAKER_00: Your job as a lawyer to practice law is gonna change
• 0:54
  SPEAKER_00: because you're gonna become sort of like an orchestrator of agents.
• 0:57
  SPEAKER_00: Think about what these tools are.
• 0:59
  SPEAKER_00: These are really not just tools,
• 1:01
  SPEAKER_00: but they're actually fundamentally changing the way that law is getting practiced.
• 1:05
  SPEAKER_00: We have a large portion of the population that really can't afford legal advice.
• 1:10
  SPEAKER_00: And I always wrestle with the question of like,
• 1:11
  SPEAKER_00: well, if they can't afford legal advice, John, welcome to the podcast.
• 1:14
  SPEAKER_00: Thanks so much.
• 1:14
  SPEAKER_00: Great to be here.
• 1:35
  [music]
• 1:42
  SPEAKER_01: So you're the chief legal officer at Harvey.
• 1:43
  SPEAKER_01: Can you tell us a little bit about what Harvey does?
• 1:46
  SPEAKER_00: So, you know, there's all these large language models out there.
• 1:50
  SPEAKER_00: There's three or four big companies that make them.
• 1:53
  SPEAKER_00: And our job is we sit in the AI application layer space.
• 1:57
  SPEAKER_00: And that means our job is to make those generative AI models useful in context of
• 2:01
  SPEAKER_00: the vertical that we serve.
• 2:03
  SPEAKER_00: We serve the legal industry.
• 2:04
  SPEAKER_00: And so our job is to make generative AI useful to lawyers.
• 2:09
  SPEAKER_01: And so what does that mean in practice?
• 2:10
  SPEAKER_01: Like what are the kinds of things that a lawyer might come to Harvey looking to find?
• 2:14
  SPEAKER_01: Yeah.
• 2:14
  SPEAKER_00: It's kind of evolved over time, right?
• 2:16
  SPEAKER_00: So I think, you know, Harvey was founded a little over four years ago.
• 2:20
  SPEAKER_00: ChatGPT hit the zeitgeist in November of 2022.
• 2:25
  SPEAKER_00: And then from that point on, it's been all anybody can really talk about, right?
• 2:28
  SPEAKER_00: But when it first started, AI was a really good answer engine, right?
• 2:32
  SPEAKER_00: And we've gone from evolving from being a really good answer engine,
• 2:37
  SPEAKER_00: you know,
• 2:37
  SPEAKER_00: tell me about data localization laws in Indonesia or something like that, right?
• 2:41
  SPEAKER_00: To being a system that can,
• 2:43
  SPEAKER_00: you know,
• 2:43
  SPEAKER_00: you could give it instructions,
• 2:45
  SPEAKER_00: do this and do that and do that and give me the output, right?
• 2:48
  SPEAKER_00: To what we're now in this like agentic era, right?
• 2:51
  SPEAKER_00: And so in the agentic era,
• 2:53
  SPEAKER_00: you can be like,
• 2:54
  SPEAKER_00: you know,
• 2:55
  SPEAKER_00: this is the outcome I want and you can In some ways,
• 2:59
  SPEAKER_00: leave it to the models to figure out how to get you to that result, right?
• 3:02
  SPEAKER_00: And so how Harvey's been used by lawyers is continually evolving.
• 3:08
  SPEAKER_00: I would say when we first started out,
• 3:09
  SPEAKER_00: like the low hanging fruit for AI and why this has been so attractive,
• 3:15
  SPEAKER_00: I think,
• 3:15
  SPEAKER_00: to lawyers is that,
• 3:18
  SPEAKER_00: you know, anytime you've got large amounts of unstructured data, right?
• 3:22
  SPEAKER_00: Turns out there's an AI tool that's going to be really useful to a lawyer in doing that.
• 3:27
  SPEAKER_00: So whether or not you're doing document review or reviewing contracts in a data room
• 3:32
  SPEAKER_00: or whatever it is,
• 3:33
  SPEAKER_00: lawyers all the time are dealing with large amounts of unstructured data.
• 3:37
  SPEAKER_00: And so that became a very early use case, I think, for AI.
• 3:40
  SPEAKER_00: And then people have gotten a little bit more comfortable with it.
• 3:43
  SPEAKER_00: They can see what it can do.
• 3:44
  SPEAKER_00: They understand what it can't do.
• 3:46
  SPEAKER_00: Hopefully they can understand what it can't do.
• 3:48
  SPEAKER_00: And we've gotten to the point now where we're building out a whole bunch of agents,
• 3:52
  SPEAKER_00: right, and agentic functionality into the platform.
• 3:55
  SPEAKER_00: And so things like contract intelligence and horizon scanning and other things
• 3:59
  SPEAKER_00: that we've got, you know, cooking at Harvey.
• 4:01
  SPEAKER_01: So I'd love to push a little more on the can and can't do line.
• 4:05
  SPEAKER_01: Obviously, the law is a licensed profession.
• 4:07
  SPEAKER_01: There's a lot of debate within the profession about how to use AI,
• 4:10
  SPEAKER_01: who's responsible for what kinds of representations or errors in their work.
• 4:15
  SPEAKER_01: And so obviously to be a product that serves the industry,
• 4:18
  SPEAKER_01: you need to be conscious of those sensitivities as well and
• 4:21
  SPEAKER_01: then reassure your customer base that you're addressing them appropriately.
• 4:24
  SPEAKER_01: How do you go about that?
• 4:26
  SPEAKER_00: You know,
• 4:26
  SPEAKER_00: at the end of the day,
• 4:28
  SPEAKER_00: Harvey is very much about,
• 4:29
  SPEAKER_00: you know, it's a tool that lawyers can use to do their work more efficiently.
• 4:35
  SPEAKER_00: But it's not an abrogation of the duties of the lawyer, right?
• 4:39
  SPEAKER_00: So when lawyers are using a tool, it's either the law firm's liability, right?
• 4:46
  SPEAKER_00: Or the in-house legal team's liability, right?
• 4:48
  SPEAKER_00: And we're just a tool that they can use to help do what they're trying to do.
• 4:54
  SPEAKER_00: There's definitely this notion that you want to build these tools the right way
• 4:58
  SPEAKER_00: for lawyers so that they can get a lot of comfort in knowing that they can use them
• 5:01
  SPEAKER_00: for legal-related tasks.
• 5:04
  SPEAKER_00: We've all heard the phrase, you know, privileged and confidential information, right?
• 5:07
  SPEAKER_00: You want to make sure that people are satisfied that Harvey is the right kind
• 5:11
  SPEAKER_00: of environment to be used by lawyers and doesn't necessarily destroy privileged
• 5:17
  SPEAKER_00: and confidential communications by virtue of just throwing it in the platform.
• 5:21
  SPEAKER_00: Those kinds of things are certainly...
• 5:26
  SPEAKER_01: I'm also curious about who within the legal community is using the tool.
• 5:30
  SPEAKER_01: So some of the use cases that you're talking about are law firm use cases,
• 5:32
  SPEAKER_01: but I assume that's actually not the entirety of your user base.
• 5:36
  SPEAKER_00: No, it's funny, right?
• 5:38
  SPEAKER_00: Like when I think about like the history of Harvey, right?
• 5:42
  SPEAKER_00: Really beginning to sell Harvey at the beginning of 2023.
• 5:46
  SPEAKER_00: And most of the customers,
• 5:49
  SPEAKER_00: but not all of the customers that we signed up in 2023 were law firms,
• 5:52
  SPEAKER_00: like forward-looking law firms who were like,
• 5:55
  SPEAKER_00: we want to get our hands dirty with this technology and see where it's going
• 5:58
  SPEAKER_00: and saw sort of the spark of interest, right?
• 6:02
  SPEAKER_00: And then we evolved into 2024.
• 6:04
  SPEAKER_00: And for most of 2024, we also started to continue to sell mostly to law firms.
• 6:09
  SPEAKER_00: I like to tongue-in-cheekly say,
• 6:10
  SPEAKER_00: like, and then in Q4 of 2024, we woke up to this really, like, keen insight.
• 6:16
  SPEAKER_00: Turns out there's lawyers that don't work at law firms that work,
• 6:19
  SPEAKER_00: you know, like, we have this, like, such a keen insight, right?
• 6:23
  SPEAKER_00: And we didn't change the product at that point,
• 6:26
  SPEAKER_00: but we did materially begin to pivot on how we sell it and to whom we sell it,
• 6:29
  SPEAKER_00: right?
• 6:30
  SPEAKER_00: And so at this point,
• 6:31
  SPEAKER_00: You know,
• 6:31
  SPEAKER_00: 40-some percent of our customer base is in-house and the remaining is law firm.
• 6:38
  SPEAKER_00: And if you look at our pipeline, it's probably the converse of that.
• 6:42
  SPEAKER_00: And it's not the converse of that because we're making a statement about who we want to sell to.
• 6:47
  SPEAKER_00: It's just math.
• 6:48
  SPEAKER_00: There are more companies that employ lawyers than there are law firms in the universe, right?
• 6:52
  SPEAKER_00: Like that's just supply.
• 6:54
  SPEAKER_01: And what about other parts of the legal ecosystem like lawyers
• 6:57
  SPEAKER_01: the Justice Department and lawyers at nonprofits and public defenders?
• 7:01
  SPEAKER_00: Yeah.
• 7:01
  SPEAKER_00: So, like, selling to government is its own special thing, right?
• 7:06
  SPEAKER_00: Which is something that we're beginning to think about how we would do in an appropriate way.
• 7:11
  SPEAKER_00: And then there's also certain regulations,
• 7:16
  SPEAKER_00: and I'm thinking specifically about the EU AI Act, right?
• 7:19
  SPEAKER_00: Which, you know, classifies different systems based on what they're purported to do, right?
• 7:25
  SPEAKER_00: Yeah.
• 7:25
  SPEAKER_00: And one of the things that can throw you from a normal risk AI system to
• 7:31
  SPEAKER_00: a high risk AI system is whether or not you're metering judgment to somebody, right?
• 7:36
  SPEAKER_00: And so we've been very careful about not trying to roll out our system to
• 7:42
  SPEAKER_00: the judiciary too quickly without appropriate guardrails in place to make sure
• 7:45
  SPEAKER_00: that they're using it the way a lawyer would use it,
• 7:47
  SPEAKER_00: which is to say to help them do their job, not to do it in lieu of doing their job.
• 7:51
  SPEAKER_01: Interesting.
• 7:51
  SPEAKER_01: I mean,
• 7:52
  SPEAKER_01: it seems like a case where the law is trying to be protective of a certain kind
• 7:55
  SPEAKER_01: of environment has actually meant they haven't been able to get access to
• 7:57
  SPEAKER_01: the technology.
• 7:58
  SPEAKER_00: I think that's right.
• 7:59
  SPEAKER_00: I think an even sadder story,
• 8:01
  SPEAKER_00: but on that same theme,
• 8:03
  SPEAKER_00: is sort of like this notion of like unequal access to justice, right?
• 8:06
  SPEAKER_00: So if you think of like in the world,
• 8:09
  SPEAKER_00: there is the unauthorized practice of law,
• 8:11
  SPEAKER_00: this rule that says only lawyers should be delivering legal advice,
• 8:15
  SPEAKER_00: just like we have rules that say only doctors should be giving medical advice,
• 8:17
  SPEAKER_00: right?
• 8:17
  SPEAKER_00: Like you want that.
• 8:19
  SPEAKER_00: It makes sense that you have that, but there's these unintended consequences.
• 8:22
  SPEAKER_00: And here in America,
• 8:23
  SPEAKER_00: especially like we have a large portion of the population that really can't afford
• 8:28
  SPEAKER_00: And I always wrestle with the question of,
• 8:31
  SPEAKER_00: like, well, if they can't afford legal advice, can we give them something?
• 8:34
  SPEAKER_00: Like, something might be better than nothing, right?
• 8:38
  SPEAKER_00: And so there's this tension, right?
• 8:39
  SPEAKER_00: Like, what happens when the unauthorized practice of law meets,
• 8:42
  SPEAKER_00: you know, unequal access to justice?
• 8:44
  SPEAKER_00: And how do we solve for that?
• 8:45
  SPEAKER_01: Because you can support lawyers doing their work,
• 8:47
  SPEAKER_01: but you can't actually provide legal advice yourself.
• 8:50
  SPEAKER_00: Under the law, correct.
• 8:51
  SPEAKER_00: But maybe one day, you know, the tools will be sufficiently good that you could do that, right?
• 8:57
  SPEAKER_01: Mm-hmm.
• 8:58
  SPEAKER_00: But, you know, Harvey's not a lawyer, right?
• 9:00
  SPEAKER_00: So it would be, you know, you could argue that that would be a violation of something.
• 9:05
  SPEAKER_00: We have to wrestle with that from a regulatory and, you know, ABA framework.
• 9:10
  SPEAKER_00: We have to figure out how to sort of,
• 9:12
  SPEAKER_00: you know,
• 9:12
  SPEAKER_00: keep up with the times and make sure that we're not disenfranchising people.
• 9:16
  SPEAKER_01: It's super interesting.
• 9:17
  SPEAKER_01: We have dealt with some of these questions at a state level
• 9:20
  SPEAKER_01: in particular around different types of state licensing.
• 9:22
  SPEAKER_01: Like it's a very big issue in mental health, a whole range of different things.
• 9:27
  SPEAKER_01: I think at least in our experience,
• 9:30
  SPEAKER_01: some of these arguments have been really difficult to make because people don't want
• 9:33
  SPEAKER_01: to say that AI could get an exemption from a license to practice
• 9:38
  SPEAKER_01: a certain profession.
• 9:38
  SPEAKER_01: But I think you're also pointing to the downside of those requirements,
• 9:42
  SPEAKER_01: which is like, yes,
• 9:43
  SPEAKER_01: we want people to know that they're getting a certain caliber of services
• 9:46
  SPEAKER_01: and we want them to have recourse if they're harmed in any way.
• 9:49
  SPEAKER_01: And we want the providers to be trained at a certain level.
• 9:51
  SPEAKER_01: And also,
• 9:52
  SPEAKER_01: if you make it harder for them to get services,
• 9:54
  SPEAKER_01: you're making it harder for them to get access to cheaper services.
• 9:57
  SPEAKER_00: Or to justice, right?
• 9:59
  SPEAKER_00: It's a fundamental issue, I think.
• 10:01
  SPEAKER_00: And some people aren't getting what I think they need.
• 10:04
  SPEAKER_00: And if technology can help solve that problem, I mean,
• 10:09
  SPEAKER_00: I think that's why a lot of us went into tech,
• 10:11
  SPEAKER_00: to try and help society solve problems with technology, right?
• 10:14
  SPEAKER_00: And so we shouldn't shy away from that challenge.
• 10:16
  SPEAKER_01: When you're making those kinds of arguments,
• 10:18
  SPEAKER_01: are you getting with the people sitting on the other side,
• 10:20
  SPEAKER_01: whether it's like it's a bar association or a state lawmaker,
• 10:23
  SPEAKER_01: are you getting head shaking or head nodding?
• 10:25
  SPEAKER_00: I mean, nobody's going to look you in the face and say, no,
• 10:29
  SPEAKER_00: you know, unequal access to justice isn't a problem or something like that, right?
• 10:33
  SPEAKER_00: So, like, nobody, like, looks you in the face and says no to that.
• 10:36
  SPEAKER_00: But I do think there's going to be this,
• 10:39
  SPEAKER_00: like, question about, like, when are we ready to try to experiment with something?
• 10:44
  SPEAKER_00: And is this generation of models good enough to start doing that?
• 10:48
  SPEAKER_00: Or do we need to wait for next generation models?
• 10:50
  SPEAKER_00: I think those questions are still being asked.
• 10:53
  SPEAKER_00: And there isn't a great answer to that.
• 10:55
  SPEAKER_01: Yeah.
• 10:55
  SPEAKER_00: One I definitely would like to see is push the envelope on.
• 10:57
  SPEAKER_01: So your role is pretty interesting because you're a chief legal officer,
• 11:00
  SPEAKER_01: not to just any company, but to a company that's offering legal services.
• 11:03
  SPEAKER_00: This is what I love most about my job, right?
• 11:05
  SPEAKER_01: Talk about that.
• 11:06
  SPEAKER_00: I've been in tech for a number of years.
• 11:07
  SPEAKER_00: I was at Google.
• 11:08
  SPEAKER_00: I was at Snowflake.
• 11:09
  SPEAKER_00: But this is the first time I've ever been at a company that's actually selling
• 11:12
  SPEAKER_00: the thing that I want to use, right?
• 11:15
  SPEAKER_00: And so I think where that shows up the most is I've always tried
• 11:20
  SPEAKER_00: to build legal teams that have a good relationship with the engineering teams,
• 11:23
  SPEAKER_00: right?
• 11:23
  SPEAKER_00: Like on the product counseling side, because you want...
• 11:25
  SPEAKER_00: Product Council to be viewed as good partners to the engineers and product people
• 11:30
  SPEAKER_00: who are building the things and not sort of like the team that you bring in at
• 11:34
  SPEAKER_00: the last moment, right?
• 11:36
  SPEAKER_00: And so it's always been important to me,
• 11:37
  SPEAKER_00: but like here at Harvey,
• 11:38
  SPEAKER_00: like it's been embraced from just the other side as well, right?
• 11:42
  SPEAKER_00: It's not just us trying to like befriend them, it's them trying to befriend us, right?
• 11:46
  SPEAKER_00: We have great engineers and product people at Harvey, but not all of them are lawyers, right?
• 11:51
  SPEAKER_00: And so- They come to us and be like, would a lawyer want this?
• 11:54
  SPEAKER_00: Is this useful to a lawyer?
• 11:56
  SPEAKER_00: And they're so creative and thoughtful about it.
• 11:58
  SPEAKER_00: But like,
• 11:59
  SPEAKER_00: we have an open dialogue and people on my team get seconded to the engineering team
• 12:03
  SPEAKER_00: for periods of time.
• 12:04
  SPEAKER_00: We have open Slack channels where obviously, you know, Uh-huh.
• 12:07
  SPEAKER_00: Meaning that the dogfooding is the practice of your team at Harvey is actually...
• 12:24
  SPEAKER_01: So it's two things, right?
• 12:28
  SPEAKER_00: It's that, right?
• 12:29
  SPEAKER_00: But it's even at a fundamental level before that,
• 12:32
  SPEAKER_00: which is to say, what features should we be building?
• 12:34
  SPEAKER_00: Like we have an opinion about that, right?
• 12:36
  SPEAKER_00: And so we offer our sometimes very strong opinions about like what should be the P0
• 12:42
  SPEAKER_00: for the product team in any given quarter.
• 12:44
  SPEAKER_00: Like what's our biggest pain point?
• 12:46
  SPEAKER_00: You know, what's going to move the needle if we can introduce a feature that does this, right?
• 12:50
  SPEAKER_00: So we do that then.
• 12:51
  SPEAKER_00: Once those features are beginning to come to fruition,
• 12:55
  SPEAKER_00: then we're like, okay,
• 12:56
  SPEAKER_00: let's start using it and giving you like sort of more granular feedback on like,
• 13:02
  SPEAKER_00: this is not quite right, or this is missing the mark, or this is spot on.
• 13:06
  SPEAKER_00: Let's, you know, jump in, like lean into that more.
• 13:09
  SPEAKER_01: So one of the main things I assume that you do as chief legal officers,
• 13:13
  SPEAKER_01: not just spending time on like heady issues about access to justice and those types
• 13:17
  SPEAKER_01: of things,
• 13:17
  SPEAKER_01: but actually thinking in a very brass tacks kind of way about structuring your team.
• 13:21
  SPEAKER_01: I assume that means something different at Harvey than it would if you were at
• 13:24
  SPEAKER_01: a non-legal company.
• 13:26
  SPEAKER_01: Does it actually change how you set up your team?
• 13:29
  SPEAKER_00: So there are two facets to this question, right?
• 13:32
  SPEAKER_00: So the first facet is just,
• 13:34
  SPEAKER_00: I consider my team,
• 13:36
  SPEAKER_00: not surprisingly,
• 13:37
  SPEAKER_00: we're an AI-forward company, but we're also an AI-forward department, right?
• 13:41
  SPEAKER_00: So a lot of what we think about is like,
• 13:42
  SPEAKER_00: hey, how can we be leaning into AI aggressively to do our jobs better, right?
• 13:48
  SPEAKER_00: So there's that aspect of it,
• 13:49
  SPEAKER_00: but then there's just like the natural complexion of a legal team, right?
• 13:53
  SPEAKER_00: And at Harvey, writ large, we've got Hundreds of attorneys, right?
• 13:59
  SPEAKER_00: But most of the attorneys actually aren't on my team, right?
• 14:01
  SPEAKER_00: I've got maybe 65, 70 people on my team, right?
• 14:06
  SPEAKER_00: We have over 200 lawyers at Harvey, right?
• 14:08
  SPEAKER_00: So what are the other lawyers doing?
• 14:09
  SPEAKER_00: The other lawyers are primarily, well,
• 14:12
  SPEAKER_00: a couple of different roles,
• 14:13
  SPEAKER_00: but one of the roles is applied legal researchers working with the research teams
• 14:16
  SPEAKER_00: on product development.
• 14:17
  SPEAKER_00: That's a smaller portion.
• 14:19
  SPEAKER_00: The much larger portion is our legal engineering group, right?
• 14:22
  SPEAKER_00: And these are all recovering lawyers, right?
• 14:24
  SPEAKER_00: Mostly from firms who've practiced law for many years.
• 14:28
  SPEAKER_00: They work sort of in a pre-sale and post-sale phase to help with
• 14:32
  SPEAKER_00: the change management that our customers need help with in sort
• 14:36
  SPEAKER_00: of adopting these tools, right?
• 14:38
  SPEAKER_00: Because if you think about what these tools are,
• 14:39
  SPEAKER_00: these are really not just tools,
• 14:42
  SPEAKER_00: but they're actually fundamentally changing the way that law is getting practiced.
• 14:47
  SPEAKER_00: And if you don't help the customer on the change management aspect of it,
• 14:51
  SPEAKER_00: how they bring that usefully into their environments...
• 14:55
  SPEAKER_00: Your customers are going to be as happy and as productive as they could be,
• 14:58
  SPEAKER_00: you know,
• 14:58
  SPEAKER_00: and you want them not to buy Harvey from you, but to renew year after year.
• 15:02
  SPEAKER_00: And so you have to help them with that change.
• 15:04
  SPEAKER_01: I assume you have like millions of examples of seeing that productivity translated
• 15:08
  SPEAKER_01: in practice,
• 15:09
  SPEAKER_01: but I'm wondering if you could just talk through a few,
• 15:10
  SPEAKER_01: either specifics or just sort of general ones about how you're seeing firms
• 15:14
  SPEAKER_01: or in-house counsel use Harvey productively.
• 15:16
  SPEAKER_00: I'll talk about my team, right?
• 15:18
  SPEAKER_00: So like, I like to describe it as like three phases of leaning in to become an AI native person.
• 15:26
  SPEAKER_00: You know, in-house team.
• 15:27
  SPEAKER_00: So I'll count like an engineer.
• 15:29
  SPEAKER_00: So stage zero is you just get them a tool and get them to start using it, right?
• 15:34
  SPEAKER_00: You're not measuring ROI.
• 15:35
  SPEAKER_00: It's too soon to measure ROI, but you get them using it.
• 15:38
  SPEAKER_00: You see, let them get familiar with these tools.
• 15:41
  SPEAKER_00: Lawyers,
• 15:41
  SPEAKER_00: generally speaking,
• 15:42
  SPEAKER_00: like if you had come to me,
• 15:45
  SPEAKER_00: it's a little bit of a tangent,
• 15:46
  SPEAKER_00: but if you had come to me five years ago and said,
• 15:50
  SPEAKER_00: We were on the cusp of this new technology.
• 15:52
  SPEAKER_00: It was gonna be fantastic.
• 15:53
  SPEAKER_00: It was gonna get embedded into all sorts of things.
• 15:56
  SPEAKER_00: And that the first group of people that would start
• 15:58
  SPEAKER_00: to use it would probably be software engineers.
• 16:00
  SPEAKER_00: I would have been like, okay, I believe that.
• 16:03
  SPEAKER_00: And then if you had told me,
• 16:04
  SPEAKER_00: and the second group would be lawyers,
• 16:06
  SPEAKER_00: I would have laughed you out of the room, right?
• 16:07
  SPEAKER_00: Like you don't normally think lawyers on the cutting edge of technology.
• 16:10
  SPEAKER_00: So going back to my department, right?
• 16:12
  SPEAKER_00: Like, so I've got lawyers.
• 16:13
  SPEAKER_00: They're all lawyers at a tech company.
• 16:15
  SPEAKER_00: They all wanted to come to Harvey.
• 16:16
  SPEAKER_00: They knew what we were about,
• 16:18
  SPEAKER_00: but you still have to get them used to the tool and show them the paradigm shift,
• 16:22
  SPEAKER_00: right?
• 16:23
  SPEAKER_00: So stage zero is just like,
• 16:25
  SPEAKER_00: Here's what AI is,
• 16:27
  SPEAKER_00: start using it,
• 16:28
  SPEAKER_00: experimenting with it,
• 16:28
  SPEAKER_00: see where it makes your life better,
• 16:30
  SPEAKER_00: see where it makes your life worse,
• 16:32
  SPEAKER_00: see what it gets right, see what it doesn't get right, and lean into that.
• 16:35
  SPEAKER_00: Then you enter phase one, and phase one is the phase where, you know, Everybody is using AI.
• 16:42
  SPEAKER_00: You even start to measure and have leaderboards and talk about it.
• 16:46
  SPEAKER_00: You have meetings where everybody shares what they're doing.
• 16:49
  SPEAKER_00: But everybody is kind of the master of their own domain.
• 16:53
  SPEAKER_00: This commercial attorney has their own three agents that they use
• 16:58
  SPEAKER_00: to make themselves more productive.
• 16:59
  SPEAKER_00: And this commercial attorney over here Doing essentially the same work
• 17:02
  SPEAKER_00: as their own different three agents.
• 17:05
  SPEAKER_00: And that's all great.
• 17:07
  SPEAKER_00: Everybody is uplifted by the fact that they're using AI and being made more efficient.
• 17:11
  SPEAKER_00: So there's nothing wrong with that.
• 17:12
  SPEAKER_00: It's just not the next level,
• 17:14
  SPEAKER_00: which I call phase two,
• 17:15
  SPEAKER_00: which is the phase that we're entering now in my department,
• 17:18
  SPEAKER_00: which is where we're trying to get sort of like
• 17:23
  SPEAKER_00: My commercial team,
• 17:24
  SPEAKER_00: instead of everybody having their own bespoke set of agents,
• 17:26
  SPEAKER_00: we now have a canonical golden set of agents,
• 17:29
  SPEAKER_00: where everybody's using the best of the best.
• 17:32
  SPEAKER_00: And we actually are leaning into our legal operations team today.
• 17:36
  SPEAKER_00: To be the team that helps us identify the right workflows and build out the agents
• 17:41
  SPEAKER_00: to do that rather than letting the lawyers do it themselves.
• 17:43
  SPEAKER_00: The lawyers can still do that themselves, right?
• 17:45
  SPEAKER_00: But to get the best of the best in everybody's hands,
• 17:48
  SPEAKER_00: you rely on the legal operations team to sort of do that.
• 17:51
  SPEAKER_00: And that's kind of where we are right now.
• 17:52
  SPEAKER_00: And it's a fascinating journey.
• 17:54
  SPEAKER_00: But once you get everybody sort of using that,
• 17:56
  SPEAKER_00: you up-level the whole team even further
• 17:58
  SPEAKER_00: because now you get not just everybody's being efficient,
• 18:00
  SPEAKER_00: but now you get consistent results across the board.
• 18:03
  SPEAKER_01: You mentioned not just how people are using AI effectively,
• 18:06
  SPEAKER_01: but you also alluded to ways in which maybe they're not using AI.
• 18:09
  SPEAKER_01: There are ways where you're saying go, go, go.
• 18:11
  SPEAKER_01: And then it sounds like people on your team are learning use cases
• 18:13
  SPEAKER_01: where maybe AI is not as appealing.
• 18:15
  SPEAKER_01: That feels to me to be Yeah.
• 18:41
  SPEAKER_01: Because I feel like this is so much a part of people's experience right now,
• 18:45
  SPEAKER_01: not just how to use AI,
• 18:46
  SPEAKER_01: but kind of the use cases where maybe it's better to sort of hold off or use cases
• 18:50
  SPEAKER_01: that aren't a good fit for the technology.
• 18:52
  SPEAKER_01: What are you learning about that in your team?
• 18:54
  SPEAKER_00: For the last 50 plus years, we've all engaged with technology in a very specific way.
• 19:00
  SPEAKER_00: By which I mean,
• 19:02
  SPEAKER_00: like if you go to your calculator and you type something in,
• 19:05
  SPEAKER_00: and so long as you don't miss hit any of the buttons and you hit enter...
• 19:09
  SPEAKER_00: With 100% certainty, you know you've gotten the correct answer.
• 19:13
  SPEAKER_00: You don't doubt, you don't even think about it.
• 19:14
  SPEAKER_00: You're like, I got that.
• 19:15
  SPEAKER_00: That's the right answer.
• 19:18
  SPEAKER_00: Here we are,
• 19:18
  SPEAKER_00: state-of-the-art technology,
• 19:20
  SPEAKER_00: and it's not a deterministic system, it's a probabilistic system, right?
• 19:23
  SPEAKER_00: And so all of a sudden,
• 19:24
  SPEAKER_00: we now have to retrain ourselves how we engage with state-of-the-art technology.
• 19:30
  SPEAKER_00: You used to say, the computer doesn't make mistakes, humans do, right?
• 19:34
  SPEAKER_00: Now computers can make mistakes, right?
• 19:36
  SPEAKER_00: And so we need to retrain ourselves and own that level of filter that you need
• 19:41
  SPEAKER_00: to apply to working with these tools.
• 19:43
  SPEAKER_00: And it really means you have to Figure out and re-engage how you interact
• 19:48
  SPEAKER_00: with technology, right?
• 19:49
  SPEAKER_00: And you have to relearn that.
• 19:50
  SPEAKER_00: And that's a super important thing to overcome, right?
• 19:52
  SPEAKER_00: Like, I don't want to be telling my kids not to use this,
• 19:55
  SPEAKER_00: but I want to make sure that my kids know how to use this responsibly, right?
• 19:58
  SPEAKER_00: And I want them to know what a hallucination is.
• 20:01
  SPEAKER_00: That's when a probabilistic system gets the probabilities wrong, right?
• 20:04
  SPEAKER_00: Like, that's exactly what it means, right?
• 20:07
  SPEAKER_00: And so it's the same thing is true in the legal industry, right?
• 20:10
  SPEAKER_00: Like you need to retrain lawyers that like, hey,
• 20:13
  SPEAKER_00: you're using technology,
• 20:14
  SPEAKER_00: but that doesn't mean that it has this like false level of precision.
• 20:17
  SPEAKER_00: I don't want to anthropomorphize anything,
• 20:20
  SPEAKER_00: but it's a little bit more like having a conversation with an associate, right?
• 20:24
  SPEAKER_00: Like as a senior attorney,
• 20:26
  SPEAKER_00: you don't necessarily trust that the associate gets everything right just
• 20:29
  SPEAKER_00: because you gave them instructions, right?
• 20:31
  SPEAKER_00: Like you bring a filter to that, right?
• 20:33
  SPEAKER_00: And you need to be able to bring that same filter here.
• 20:37
  SPEAKER_00: And, you know,
• 20:38
  SPEAKER_00: a lot of folks talk about, like, hey,
• 20:39
  SPEAKER_00: you know,
• 20:40
  SPEAKER_00: if we have these fantastical AI solutions,
• 20:43
  SPEAKER_00: does it mean that we're all out of a job or something like that, right?
• 20:46
  SPEAKER_00: And, like,
• 20:47
  SPEAKER_00: I always remind people, like,
• 20:48
  SPEAKER_00: if you go back to the 90s, right,
• 20:49
  SPEAKER_00: we were having this exact same conversation in the 90s about lawyers because,
• 20:53
  SPEAKER_00: like, everybody was saying, well...
• 20:55
  SPEAKER_00: If the junior lawyers don't have to go into the library to do the review anymore,
• 20:59
  SPEAKER_00: how would they learn how to be good lawyers, right?
• 21:01
  SPEAKER_00: And like LexisNexis and Westlaw sort of changed everything,
• 21:04
  SPEAKER_00: but they didn't change the fact that we still have very good lawyers, right?
• 21:08
  SPEAKER_00: And so I actually view,
• 21:11
  SPEAKER_00: and maybe I'm a bit Pollyannish or whatever,
• 21:13
  SPEAKER_00: but like I actually view like,
• 21:15
  SPEAKER_00: my perspective is that,
• 21:16
  SPEAKER_00: you know,
• 21:17
  SPEAKER_00: younger people are going to engage with the latest and greatest technologies
• 21:21
  SPEAKER_00: on average, generally sooner than the rest of us will, right?
• 21:25
  SPEAKER_00: And they're going to figure out how to use it in whatever task you're setting out
• 21:28
  SPEAKER_00: for them to do.
• 21:29
  SPEAKER_00: And they're going to figure out how to work with it and make it work for them.
• 21:34
  SPEAKER_00: But they're still going to be able to learn the things that will make them
• 21:37
  SPEAKER_00: a good lawyer,
• 21:38
  SPEAKER_00: the judgment, the strategy, the empathy, and the ethical applications, right?
• 21:42
  SPEAKER_00: I mean, those will all still be things that humans will need to do.
• 21:45
  SPEAKER_01: So how do you talk with either people on your team
• 21:48
  SPEAKER_01: that sounds like you're mentoring them in their own AI journey,
• 21:51
  SPEAKER_01: so about that line between...
• 21:55
  SPEAKER_01: The tool and the judgment or on the customer side,
• 21:57
  SPEAKER_01: because I assume you must have customers coming to you all the time who say like,
• 22:00
  SPEAKER_01: we did this perfectly and it produced something that was problematic for our firm.
• 22:04
  SPEAKER_01: And often that I assume will be a result of the human error, not the AIs.
• 22:08
  SPEAKER_00: For the longest time, I've been using the phrase a human in the loop, right?
• 22:11
  SPEAKER_00: I think we all have human in the loop, human in the loop.
• 22:14
  SPEAKER_00: It's not a tool to replace humans.
• 22:15
  SPEAKER_00: You need a human in the loop.
• 22:17
  SPEAKER_00: I just recently heard a better phrase, the phrase that I like better as human on the hook.
• 22:22
  SPEAKER_00: I think it's very appropriate for the legal profession.
• 22:26
  SPEAKER_00: When I'm a lawyer, I am on the hook for my legal output.
• 22:32
  SPEAKER_00: I might use an AI tool to help me do it more efficiently,
• 22:36
  SPEAKER_00: faster, more cost-effective, broader, whatever.
• 22:39
  SPEAKER_00: But I'm still on the hook for that, right?
• 22:42
  SPEAKER_00: And that's the thing I remind people that you read it, don't trust it.
• 22:47
  SPEAKER_00: You have to make sure that it's giving it the correct answer.
• 22:52
  SPEAKER_00: And then the other thing I always remind people is...
• 22:55
  SPEAKER_00: You can over-index on that too.
• 22:57
  SPEAKER_00: And lawyers are the worst at this, right?
• 22:59
  SPEAKER_00: You ever heard the phrase, don't let the perfect be the enemy of the good, right?
• 23:03
  SPEAKER_00: Lawyers are the masterclass at that, right?
• 23:05
  SPEAKER_00: They go to law school to learn how to spot all of the issues.
• 23:08
  SPEAKER_00: Then they go to law firms where they have high billing rates
• 23:13
  SPEAKER_00: and your clients are paying the high billing rates.
• 23:15
  SPEAKER_00: And so perfection is kind of a thing that they aspire to deliver, right?
• 23:20
  SPEAKER_00: But when you become in-house, I don't necessarily want perfect all the time, right?
• 23:26
  SPEAKER_00: If it takes five hours to do something to get it 95% correct,
• 23:30
  SPEAKER_00: whatever that is, or it takes 21 hours to get it 99.7%,
• 23:36
  SPEAKER_00: For a lot of things, I'll take the shorter period of time.
• 23:39
  SPEAKER_00: And so that kind of goes with how you use these tools too.
• 23:43
  SPEAKER_00: Use the tools for the things where you don't necessarily need the perfection.
• 23:49
  SPEAKER_00: And maybe they can make you a lot more efficient because you can trust them.
• 23:52
  SPEAKER_00: And when you need to get to the part where you need to get closer to perfection,
• 23:56
  SPEAKER_00: use the tools,
• 23:57
  SPEAKER_00: but bring a much greater level of review and discipline to managing those outputs
• 24:03
  SPEAKER_00: than you might otherwise do.
• 24:04
  SPEAKER_01: It's very interesting.
• 24:05
  SPEAKER_01: I hadn't thought about it until you just said it,
• 24:06
  SPEAKER_01: but I would think that the technology maps differently in that respect
• 24:10
  SPEAKER_01: onto in-house counsel versus law firms.
• 24:12
  SPEAKER_01: In-house counsel,
• 24:13
  SPEAKER_01: I think,
• 24:13
  SPEAKER_01: are incentivized to be fast,
• 24:15
  SPEAKER_01: as fast as possible, and maybe not as accurate as possible as well.
• 24:19
  SPEAKER_01: Law firms,
• 24:20
  SPEAKER_01: I think,
• 24:20
  SPEAKER_01: both because they're trying to do justice to the billable rate,
• 24:23
  SPEAKER_01: but also because they're trying to increase billable hours,
• 24:25
  SPEAKER_01: often want to do more and more.
• 24:27
  SPEAKER_01: And so efficiency,
• 24:27
  SPEAKER_01: I think,
• 24:28
  SPEAKER_01: in some circumstances would work actually against the law firm business model.
• 24:31
  SPEAKER_01: Do you experience that distinction at all?
• 24:34
  SPEAKER_00: One of my great mentors used to say, like,
• 24:36
  SPEAKER_00: you know,
• 24:37
  SPEAKER_00: the billable hour is the worst possible way for lawyers
• 24:41
  SPEAKER_00: to meet out what they do other than all the other ways.
• 24:44
  SPEAKER_00: Right?
• 24:45
  SPEAKER_00: But the point she was trying to make is like,
• 24:46
  SPEAKER_00: you know,
• 24:46
  SPEAKER_00: you can have fixed fee arrangements, but unless you really trust the other side.
• 24:52
  SPEAKER_00: The problem with fixed fee arrangements is one side always feels like
• 24:54
  SPEAKER_00: the other side is getting one over on the other side.
• 24:57
  SPEAKER_01: And sometimes both sides feel
• 24:58
  SPEAKER_00: that way.
• 24:59
  SPEAKER_00: Oftentimes both sides feel that way, right?
• 25:01
  SPEAKER_00: One day the billable hour will be gone, right?
• 25:05
  SPEAKER_00: But I think every single technological revolution has come by also with
• 25:09
  SPEAKER_00: the prediction that this was the thing that was going to put the nail in the coffin
• 25:12
  SPEAKER_00: on the billable hour.
• 25:13
  SPEAKER_00: And so I think AI and AI efficiency continues to drive us in that direction,
• 25:19
  SPEAKER_00: but I don't necessarily think it's the nail in the coffin, right?
• 25:22
  SPEAKER_00: If I'm an in-house person, which I happen to be, right?
• 25:26
  SPEAKER_00: And for years,
• 25:27
  SPEAKER_00: like law firms have been telling me the billable hour is the best way
• 25:30
  SPEAKER_00: to do something.
• 25:33
  SPEAKER_00: I'm going to hold them to that now that,
• 25:35
  SPEAKER_00: you know, that the economics might be on the other side of the foot here.
• 25:38
  SPEAKER_00: If something used to take 40 hours and now can be done in four,
• 25:42
  SPEAKER_00: you know,
• 25:43
  SPEAKER_00: you have to be having discussions with your law firms about how
• 25:46
  SPEAKER_00: that efficiency is going to get, you know, attributed.
• 25:50
  SPEAKER_01: You've written about your team's vendor privacy review workflow.
• 25:54
  SPEAKER_01: Can you talk a little bit about that?
• 25:56
  SPEAKER_00: We have a bunch of different workflows, right?
• 25:58
  SPEAKER_00: Just about for everything at this point, right?
• 25:59
  SPEAKER_00: But, you know,
• 26:00
  SPEAKER_00: the idea is if you can systematize these things,
• 26:04
  SPEAKER_00: then you can get efficiencies out of it, right?
• 26:05
  SPEAKER_00: And so with our vendor privacy review,
• 26:09
  SPEAKER_00: we look at the different vendors that we deal with and we deal
• 26:12
  SPEAKER_00: with like 200 vendor engagements a month or something like that, right?
• 26:16
  SPEAKER_00: And they range everywhere from like, you know, catering, right?
• 26:19
  SPEAKER_00: To an event,
• 26:20
  SPEAKER_00: to like adding a new subprocessor that's going to have access to customer data
• 26:24
  SPEAKER_00: and content, right?
• 26:24
  SPEAKER_00: And so like, From a privacy perspective, not much can go wrong with catering.
• 26:30
  SPEAKER_00: On the data subprocessor, a lot can go wrong, right?
• 26:33
  SPEAKER_00: And so sort of first things first,
• 26:35
  SPEAKER_00: we want to try to match the level of review on data privacy to the risk associated
• 26:39
  SPEAKER_00: with it, right?
• 26:40
  SPEAKER_00: So not a lot of time spent on one, a lot of time spent on the other.
• 26:43
  SPEAKER_00: So that's the first stage,
• 26:45
  SPEAKER_00: is just sort of like using the tool to help us like scope,
• 26:48
  SPEAKER_00: like, hey, how much does this really matter for us to get right?
• 26:52
  SPEAKER_00: And then once we do that, we also then...
• 26:56
  SPEAKER_00: If it does touch,
• 26:57
  SPEAKER_00: you know,
• 26:57
  SPEAKER_00: if it touches usage data,
• 27:00
  SPEAKER_00: if it's going to be used in the service,
• 27:01
  SPEAKER_00: but not necessarily touching data,
• 27:04
  SPEAKER_00: customer data and content, that's also still going to be very important.
• 27:07
  SPEAKER_00: It's maybe not quite as important as if it's touching customer data,
• 27:10
  SPEAKER_00: but it's adjacent and we want to make sure we get that right.
• 27:13
  SPEAKER_00: So first off, it's just like scoping the risk.
• 27:17
  SPEAKER_00: And then we have our set of playbooks, right?
• 27:20
  SPEAKER_00: That like every company does on our perspective
• 27:23
  SPEAKER_00: on how data privacy should be enabled between a vendor and its client.
• 27:29
  SPEAKER_00: And we use AI to sort of address those things.
• 27:32
  SPEAKER_00: So I can get somebody else's paper and we can quickly sort of map it
• 27:36
  SPEAKER_00: onto our own frameworks, which then speeds up the review process.
• 27:39
  SPEAKER_01: I think...
• 27:40
  SPEAKER_01: In this conversation,
• 27:41
  SPEAKER_01: I've just assumed in the back of my head that the barriers to adoption
• 27:45
  SPEAKER_01: for Harvey are sort of lawyer skittishness about the technology.
• 27:48
  SPEAKER_01: And now I'm realizing that's perhaps a silly assumption.
• 27:51
  SPEAKER_01: When you kind of look forward and you're looking at like the possible market in
• 27:55
  SPEAKER_01: the legal field,
• 27:56
  SPEAKER_01: what do you see as the barriers and to what extent are they driven by kind
• 28:00
  SPEAKER_01: of like uncertainty about AI or feelings about the technology?
• 28:03
  SPEAKER_00: I definitely think in the early days of Harvey, there was skittishness.
• 28:06
  SPEAKER_00: Like one of my favorite stories is like we had one of our Then
• 28:11
  SPEAKER_00: a prospective customer now,
• 28:12
  SPEAKER_00: an actual customer,
• 28:13
  SPEAKER_00: tell us that on the same 36-hour period,
• 28:17
  SPEAKER_00: they got a generic letter from one of their banking clients,
• 28:22
  SPEAKER_00: you know, do not use AI in any of our matters without further notice.
• 28:26
  SPEAKER_00: And then 36 hours later,
• 28:28
  SPEAKER_00: they got an RFP from that same banking customer saying,
• 28:31
  SPEAKER_00: That said, like, we want you to tell us how to do this, this, and this.
• 28:34
  SPEAKER_00: Oh, and by the way,
• 28:34
  SPEAKER_00: make sure you have a discussion about how you're going to use AI
• 28:38
  SPEAKER_00: to make this more efficient and cost-effective for us, right?
• 28:40
  SPEAKER_00: And so, like, there was this little bit of this duality, right?
• 28:44
  SPEAKER_00: And, you know, lawyers actually have ethical canons that govern us.
• 28:48
  SPEAKER_00: One of those ethical canons basically says,
• 28:50
  SPEAKER_00: and I'm going to get this wrong at the detail level,
• 28:53
  SPEAKER_00: but, like, at the high level, it's, you know, you...
• 28:57
  SPEAKER_00: Or doing yourself,
• 28:58
  SPEAKER_00: your clients a disservice if you're not using technology in efficient ways for them.
• 29:03
  SPEAKER_00: So at some point,
• 29:05
  SPEAKER_00: you know,
• 29:05
  SPEAKER_00: and I'm beginning to think we're getting there now,
• 29:09
  SPEAKER_00: you know,
• 29:09
  SPEAKER_00: if you're not using AI and you're a lawyer,
• 29:12
  SPEAKER_00: you're questionable whether or not you're beginning to get
• 29:16
  SPEAKER_00: on like ethical shaky ground.
• 29:18
  SPEAKER_01: Uh-huh.
• 29:18
  SPEAKER_01: And can we actually just unpack that a bit because I want to make sure,
• 29:21
  SPEAKER_01: I mean, I took the ethics exam as well.
• 29:23
  SPEAKER_01: Sure.
• 29:24
  SPEAKER_01: It's not ringing a bell, but I assume it's part of just delivering excellence for your client.
• 29:28
  SPEAKER_01: And the idea is if you swear off of all technology,
• 29:31
  SPEAKER_01: then presumably you'll be taking longer to deliver a worse work product.
• 29:35
  SPEAKER_01: Is that the basic thing?
• 29:36
  SPEAKER_01: That's exactly right.
• 29:37
  SPEAKER_00: I honestly think it was a throwback to the fact that lawyers were the last people
• 29:41
  SPEAKER_00: to start catching on to the technological bandwagon.
• 29:45
  SPEAKER_00: And I think some smarter members of the profession were like, well, we should...
• 29:49
  SPEAKER_00: Put it into the canons that they should be using.
• 29:51
  SPEAKER_00: They have to explore it.
• 29:52
  SPEAKER_00: They have to understand what it can do.
• 29:54
  SPEAKER_00: And they're remiss if they don't ultimately bring it into their practice.
• 29:58
  SPEAKER_00: And that's where we are now.
• 29:59
  SPEAKER_00: I'm not here to say if you're not using AI, you're violating ethics.
• 30:03
  SPEAKER_00: But at some point, that'll be a true statement, right?
• 30:07
  SPEAKER_01: So if you were starting law school today,
• 30:08
  SPEAKER_01: what would you focus on or invest in to become a great lawyer?
• 30:12
  SPEAKER_00: I think the practice of law is changing, right?
• 30:17
  SPEAKER_00: And to go back to my,
• 30:18
  SPEAKER_00: you know,
• 30:19
  SPEAKER_00: the software engineers were the first people to adopt generative AI,
• 30:23
  SPEAKER_00: and it's really weird to me that lawyers are the second, right?
• 30:25
  SPEAKER_00: But I do think there's lots of good reasons for that.
• 30:28
  SPEAKER_00: I think lawyers and software engineers actually do the same thing.
• 30:30
  SPEAKER_00: They use language and logic to achieve repeatable structured outcomes, right?
• 30:35
  SPEAKER_00: And so one of the things I think about is...
• 30:40
  SPEAKER_00: If you want to know where the legal profession is going from
• 30:43
  SPEAKER_00: a technological perspective,
• 30:45
  SPEAKER_00: it's probably okay to look at what's going on with software developers and say,
• 30:49
  SPEAKER_00: oh, that's probably a good bellwether for where things are coming to lawyers.
• 30:54
  SPEAKER_00: And the biggest shift that's happening right now is this shift to agents, right?
• 30:59
  SPEAKER_00: And agentic models, right?
• 31:02
  SPEAKER_00: And what that generally means,
• 31:03
  SPEAKER_00: like we discussed, is,
• 31:04
  SPEAKER_00: you know,
• 31:05
  SPEAKER_00: you can tell an agentic model what the outcome is that you want,
• 31:08
  SPEAKER_00: and then it can go and figure it out.
• 31:10
  SPEAKER_00: It might come back to you with some questions,
• 31:12
  SPEAKER_00: it might ask for some additional data,
• 31:13
  SPEAKER_00: it might ask for additional context,
• 31:15
  SPEAKER_00: but it can develop the plan on how to deliver on what you want.
• 31:19
  SPEAKER_00: And so...
• 31:21
  SPEAKER_00: If you think about what that means from a practice perspective,
• 31:24
  SPEAKER_00: like lawyers used to be the ones who go out and did all the work, right?
• 31:29
  SPEAKER_00: Just like coders used to be the ones that used to go out and write all the code.
• 31:33
  SPEAKER_00: And, you know,
• 31:34
  SPEAKER_00: I could cherry pick the statistics,
• 31:36
  SPEAKER_00: but some large percent of code is no longer being written by software engineers.
• 31:41
  SPEAKER_00: Just like I think a lot of basic legal documents are no longer going
• 31:43
  SPEAKER_00: to be like first drafts are not going to be coming from people sitting at
• 31:47
  SPEAKER_00: a typewriter picking it out, right?
• 31:49
  SPEAKER_00: And...
• 31:50
  SPEAKER_00: And so what this means is your job as a lawyer to practice law is going to change
• 31:57
  SPEAKER_00: because what you're going to do is you're going to instruct agents to do the things
• 32:03
  SPEAKER_00: that you want them to do.
• 32:03
  SPEAKER_00: You're going to let those agents cook and then you're going to review the output
• 32:08
  SPEAKER_00: and do what you need to do with it after that.
• 32:10
  SPEAKER_00: Right.
• 32:11
  SPEAKER_00: And so you're going to become sort of like an orchestrator of agents.
• 32:14
  SPEAKER_00: Right.
• 32:15
  SPEAKER_00: And so But that to me is like a fundamental shift in how you think about how you practice law.
• 32:21
  SPEAKER_00: That's a skill set that you don't necessarily come out of law school prepared to do, right?
• 32:26
  SPEAKER_00: Like how do I tell, how do I keep multiple agents juggling at any given time?
• 32:31
  SPEAKER_00: And in fact,
• 32:33
  SPEAKER_00: I've been scaring my team recently by telling them like, hey,
• 32:37
  SPEAKER_00: we're going to start tracking from nine to five the number of agents
• 32:40
  SPEAKER_00: that you have going at any given time.
• 32:42
  SPEAKER_00: And the answer needs to be greater than one, right?
• 32:45
  SPEAKER_00: Right.
• 32:46
  SPEAKER_00: That's going to be something that we're going to measure.
• 32:48
  SPEAKER_00: And because that's how we know that you're getting maximal throughput out
• 32:52
  SPEAKER_00: of what you're doing, we're seeing that in the software space.
• 32:56
  SPEAKER_00: So to go back to your fundamental question, what would I do differently?
• 32:59
  SPEAKER_00: I would want to make sure I'm paying attention to this change
• 33:03
  SPEAKER_00: and understand what it means for me to no longer be the guy who sits in
• 33:07
  SPEAKER_00: the data room and reviews all the documents and how I sort of direct that instead.
• 33:11
  SPEAKER_00: And I think what happens is You need to develop a very keen ability to assess the output of AI.
• 33:19
  SPEAKER_00: And I think that's where I'd put a lot of my effort.
• 33:21
  SPEAKER_01: It is interesting.
• 33:22
  SPEAKER_01: I mean, one of the...
• 33:23
  SPEAKER_01: One of the things I experienced in my career was like the challenges of
• 33:26
  SPEAKER_01: the transition from individual contributor to management.
• 33:29
  SPEAKER_01: And you don't get a lot of training often in management,
• 33:31
  SPEAKER_01: even in places where like they provide trainings.
• 33:33
  SPEAKER_01: It seems like kind of what you're describing is that almost on day one,
• 33:37
  SPEAKER_01: you're a manager and you're learning how to manage and whether it's agents or
• 33:41
  SPEAKER_01: if you end up having a sizable team as you do,
• 33:43
  SPEAKER_01: you're learning the skill set that you would need in that position.
• 33:45
  SPEAKER_00: That's a very good analogy, I think, right?
• 33:47
  SPEAKER_00: Like you're becoming a manager from day one and at the same time
• 33:52
  SPEAKER_00: where you're also still
• 33:55
  SPEAKER_01: There are obviously like a lot of uncertainties,
• 33:57
  SPEAKER_01: anxieties now about the impact that AI will have on various different job markets.
• 34:02
  SPEAKER_01: It sounds like your sense overall is that tools like Harvey will be positive in
• 34:05
  SPEAKER_01: the legal profession.
• 34:06
  SPEAKER_01: Is that right?
• 34:07
  SPEAKER_00: I'll give you a real world example, right?
• 34:09
  SPEAKER_00: So I've worked on a couple like bet the company style M&As in my lifetime,
• 34:14
  SPEAKER_00: but most of them aren't that, right?
• 34:16
  SPEAKER_00: Most of them are like you're bringing in a small team of 10 people and like, you know, and so...
• 34:22
  SPEAKER_00: At Harvey in the last year, we've done four M&A events, right?
• 34:25
  SPEAKER_00: None of them were about the company.
• 34:26
  SPEAKER_00: They were just all these small, relatively small ones.
• 34:30
  SPEAKER_00: And when you're doing a small M&A,
• 34:32
  SPEAKER_00: historically speaking,
• 34:33
  SPEAKER_00: before AI, you would have to make some cost-benefit analysis, right?
• 34:39
  SPEAKER_00: Do we ask our outside counsel to review every single document in the data room?
• 34:44
  SPEAKER_00: For a relatively small M&A, the answer is probably no, right?
• 34:47
  SPEAKER_01: Yeah.
• 34:48
  SPEAKER_00: But you still want to know, like,
• 34:50
  SPEAKER_00: what are the top,
• 34:51
  SPEAKER_00: you know,
• 34:51
  SPEAKER_00: customer contracts and what are the top vendor contracts and things like that.
• 34:54
  SPEAKER_00: So you would eyeball these, like, rough heuristics.
• 34:56
  SPEAKER_00: Like, I want to see every customer contract that's above a million dollars.
• 34:59
  SPEAKER_00: And I want to see every vendor agreement.
• 35:02
  SPEAKER_00: And you would make those calls and you would hope that in the stuff
• 35:04
  SPEAKER_00: that you weren't reviewing wasn't a real big,
• 35:07
  SPEAKER_00: you know, pile of turd or something like that, right?
• 35:10
  SPEAKER_00: But now with AI, I can instruct my outside counsel or we can do it ourselves.
• 35:15
  SPEAKER_00: We review every document.
• 35:17
  SPEAKER_00: There's no document that we don't review, right?
• 35:19
  SPEAKER_00: So we're doing more legal work, right?
• 35:22
  SPEAKER_00: We're just using AI to help us, right?
• 35:24
  SPEAKER_00: And that's a relatively straightforward example,
• 35:27
  SPEAKER_00: but I think that concept is going to apply here.
• 35:31
  SPEAKER_00: The other thing that's going on is, do you remember when Amazon first started?
• 35:36
  SPEAKER_00: Right.
• 35:37
  SPEAKER_00: Right.
• 35:37
  SPEAKER_00: Right.
• 35:54
  SPEAKER_00: Beyond their expectations on speed, they're like, oh my God, John, thank you.
• 35:57
  SPEAKER_00: That was awesome.
• 35:59
  SPEAKER_01: That then becomes your expectation going forward, right?
• 36:01
  SPEAKER_00: And then that becomes the norm, right?
• 36:02
  SPEAKER_00: So combine those two things together,
• 36:04
  SPEAKER_00: I don't think we're going to be talking about a world with less lawyers in it,
• 36:09
  SPEAKER_00: right?
• 36:10
  SPEAKER_00: And my sort of like overarching theme on this is like AI is going to get used everywhere.
• 36:16
  SPEAKER_00: It's going to make the world move faster.
• 36:18
  SPEAKER_00: As the world moves faster, it gets more complex.
• 36:21
  SPEAKER_00: And I just think that lawyers shine in complexity, right?
• 36:24
  SPEAKER_00: So I think there's going to be strong demand for legal work
• 36:29
  SPEAKER_01: for the foreseeable future.
• 36:31
  SPEAKER_01: You talk about from a product perspective that Harvey uses multiple different models.
• 36:34
  SPEAKER_01: I'm curious if you can talk a little bit about that.
• 36:37
  SPEAKER_01: How do you pick and choose models?
• 36:39
  SPEAKER_01: What's the rationale there?
• 36:40
  SPEAKER_01: And why do you do it?
• 36:42
  SPEAKER_00: So when Harvey first came on the scene,
• 36:46
  SPEAKER_00: Um, we, we use primarily, um, models from open AI, but we weren't beholden to them.
• 36:52
  SPEAKER_00: We just thought that they were the best quality models out there, right?
• 36:56
  SPEAKER_00: There's been competition in the model front and,
• 37:00
  SPEAKER_00: you know, we're an open opportunity, you know, customer.
• 37:03
  SPEAKER_00: So we, we use models from Gemini,
• 37:05
  SPEAKER_00: we use models from Mistral,
• 37:07
  SPEAKER_00: we use models from Anthropic, and we obviously still use models from open AI, right?
• 37:11
  SPEAKER_00: But that's not what we're doing.
• 37:12
  SPEAKER_00: We're not just like wrapping up your question and passing it to it.
• 37:17
  SPEAKER_00: What we're really doing in our harness is we are deconstructing your question
• 37:23
  SPEAKER_00: and we're sending it to all these different model calls.
• 37:26
  SPEAKER_00: And for some parts, we might send it to Gemini.
• 37:28
  SPEAKER_00: And for some parts, we might send it to an Opus class model.
• 37:31
  SPEAKER_00: For some things, we might send it to a Sonic class model.
• 37:33
  SPEAKER_00: And we make all of these decisions about where It should go.
• 37:37
  SPEAKER_00: And we also have this lab, which is our benchmarking thing.
• 37:42
  SPEAKER_00: So we have all this data about which models are best for which kinds of questions.
• 37:48
  SPEAKER_00: And we use that information to inform our harness about where it's going
• 37:52
  SPEAKER_00: to route particular kinds of questions to it.
• 37:54
  SPEAKER_01: So simultaneously,
• 37:55
  SPEAKER_01: as you're working with the model companies to deliver your own product,
• 38:00
  SPEAKER_01: they're also increasingly rolling out legal services tools.
• 38:03
  SPEAKER_01: So I'm curious how you guys are experiencing that.
• 38:07
  SPEAKER_01: Like you're both a customer and a competitor.
• 38:09
  SPEAKER_01: What is that dynamic like?
• 38:10
  SPEAKER_00: So before coming to Harvey, I was at a company called Snowflake.
• 38:14
  SPEAKER_00: And this very same dynamic actually played out in Snowflake's history, right?
• 38:18
  SPEAKER_00: So Snowflake, they hate this characterization.
• 38:21
  SPEAKER_00: It's data warehousing in the cloud, right?
• 38:23
  SPEAKER_01: Mm-hmm.
• 38:24
  SPEAKER_00: And they were platformed on top of Azure,
• 38:26
  SPEAKER_00: they were platformed on top of AWS, and they were platformed on top of GCP.
• 38:30
  SPEAKER_00: Turns out each of the big hyperscalers has their own data warehousing platform.
• 38:34
  SPEAKER_00: Like Google's got BigQuery, Azure's got Synapse, AWS has Redshift, right?
• 38:39
  SPEAKER_00: So Snowflake was selling on top of those customers something that they competed with, right?
• 38:46
  SPEAKER_00: But still, A,
• 38:47
  SPEAKER_00: still found a way to partner with each of those three hyperscalers
• 38:50
  SPEAKER_00: who were delivering the data layer to their customers with absolute focus, right?
• 38:55
  SPEAKER_00: So I always tell people we need to do the exact same thing at Harvey.
• 38:58
  SPEAKER_00: We're going to compete with the model providers.
• 39:02
  SPEAKER_00: We're going to partner with the model providers and we're going
• 39:04
  SPEAKER_00: to stay laser focused on what it is that we need to do.
• 39:08
  SPEAKER_00: And whereas I think we'll be able to bring these 200 legal engineers to bear
• 39:13
  SPEAKER_00: and help with the change management and we'll be able to build out the features
• 39:16
  SPEAKER_00: that we know lawyers want because that's all we're focused on.
• 39:19
  SPEAKER_00: I think that's going to give us a depth of product that I don't expect
• 39:24
  SPEAKER_00: the model providers to be able to match with us in the short term.
• 39:27
  SPEAKER_01: In the history of technology,
• 39:28
  SPEAKER_01: it seems like it's always the case that the providers up the stack start looking
• 39:33
  SPEAKER_01: for ways to close out the competitors down the stack and they're well positioned in
• 39:37
  SPEAKER_01: the ecosystem to do that.
• 39:39
  SPEAKER_01: I'm curious if that's something that you're wary of.
• 39:43
  SPEAKER_00: I still think the best solution is just build a better mousetrap.
• 39:47
  SPEAKER_00: If you're the better mousetrap, the better mousetrap is going to win.
• 39:50
  SPEAKER_00: I think the other thing that I like to remind people of is there's this concept
• 39:53
  SPEAKER_00: of like model FOMO that exists in the world, right?
• 39:57
  SPEAKER_00: So when you're a lawyer,
• 39:58
  SPEAKER_00: like you only want to use the best because that's what we're – we're a special lot,
• 40:04
  SPEAKER_00: right?
• 40:04
  SPEAKER_00: So if when I go and I'm looking at like a Harvey or like one of the model providers
• 40:11
  SPEAKER_00: for legal, right – Maybe I choose one of the model providers for legal, right?
• 40:15
  SPEAKER_00: But then all of a sudden I'm locked into their models, right?
• 40:17
  SPEAKER_00: And at the time that I made that buy decision,
• 40:19
  SPEAKER_00: their models probably were the best in class, right?
• 40:22
  SPEAKER_00: But they won't always be.
• 40:23
  SPEAKER_00: Like there's so much great competition going on, right?
• 40:25
  SPEAKER_00: Like six months from now, it's going to be Gemini.
• 40:27
  SPEAKER_00: And six months after that, it's going to be anthropic.
• 40:30
  SPEAKER_00: It's always going to be shifting.
• 40:31
  SPEAKER_00: And so if you choose a specific model's provider's offering,
• 40:36
  SPEAKER_00: when that model provider is not the top model,
• 40:38
  SPEAKER_00: whatever that means, you're going to feel like you're not getting the best.
• 40:41
  SPEAKER_00: And that's going to weigh in on you.
• 40:42
  SPEAKER_00: Whereas if you use an AI application layer company like Harvey,
• 40:46
  SPEAKER_00: we can route to whomever we want to.
• 40:48
  SPEAKER_00: And that changes dynamically over time as new models come on the scene.
• 40:51
  SPEAKER_00: So we can help a little bit with the model FOMO.
• 40:54
  SPEAKER_01: I'm curious how you're sort of sizing up the DC conversation around these issues now.
• 40:58
  SPEAKER_01: Like there's conversation about like, should we relax antitrust laws for the hyperscalers?
• 41:03
  SPEAKER_01: There are questions about should we look actually at investigations
• 41:07
  SPEAKER_01: or moving forward with antitrust enforcement?
• 41:09
  SPEAKER_01: Is that all sort of like chatter on the other coast for you or is it something
• 41:13
  SPEAKER_01: that is material for you?
• 41:14
  SPEAKER_00: I mean, right now it's just chatter on the other coast for me.
• 41:16
  SPEAKER_00: I'm not paying that much attention to it.
• 41:18
  SPEAKER_00: I'm paying attention to what's going on in Sacramento a little bit, right?
• 41:23
  SPEAKER_00: But less so on that particular point.
• 41:25
  SPEAKER_01: So California actually has taken some steps to think about regulation in this area.
• 41:30
  SPEAKER_01: So- One of the bills, SB 574,
• 41:33
  SPEAKER_01: was one bill that we actually had been tracking ourselves and were thinking
• 41:35
  SPEAKER_01: about implications for the portfolio, including you guys.
• 41:38
  SPEAKER_01: Curious how that process played out from your point of view.
• 41:40
  SPEAKER_00: Yeah.
• 41:41
  SPEAKER_00: So now Harvey has a feature called horizon scanning,
• 41:45
  SPEAKER_00: which I wish we had a couple months ago because it would have told us about 574
• 41:50
  SPEAKER_00: before.
• 41:50
  SPEAKER_00: But good for us.
• 41:52
  SPEAKER_00: You were an investor with us.
• 41:54
  SPEAKER_00: You actually told us about it, right?
• 41:55
  SPEAKER_00: And so you put it on a radar screen.
• 41:57
  SPEAKER_00: And one of the fundamental things that we noticed about the original version of
• 42:00
  SPEAKER_00: the bill, and the bill basically talks about, like, it's a great bill.
• 42:04
  SPEAKER_00: Basically,
• 42:04
  SPEAKER_00: it says, like,
• 42:05
  SPEAKER_00: don't use privileged and confidential information with,
• 42:08
  SPEAKER_00: like, a publicly available tool, right?
• 42:10
  SPEAKER_00: Like, make sure you've got the right, you know, protections in place, right?
• 42:15
  SPEAKER_00: But it didn't have a definition of what a publicly available AI system was.
• 42:21
  SPEAKER_00: So, we've Very politely pointed that out and said,
• 42:25
  SPEAKER_00: by the way,
• 42:26
  SPEAKER_00: the California Bar Association just came up with a definition
• 42:29
  SPEAKER_00: that we think is perfectly fine.
• 42:31
  SPEAKER_00: Maybe use that definition in the bill.
• 42:33
  SPEAKER_00: And that's how the bill passed ultimately, right?
• 42:36
  SPEAKER_00: And so, again, I thought it was a perfectly good bill.
• 42:39
  SPEAKER_00: I just think it needed some more definition.
• 42:41
  SPEAKER_01: Yeah, it's sort of a rare example, I think, of...
• 42:44
  SPEAKER_01: Different components of the policy process working well,
• 42:46
  SPEAKER_01: like the lawmaker who was pushing it was interested in doing this thoughtful thing,
• 42:51
  SPEAKER_01: I think,
• 42:51
  SPEAKER_01: and how the industry would be regulated,
• 42:53
  SPEAKER_01: but the definition was one that would have had unintended consequences.
• 42:57
  SPEAKER_01: It would have let open litigation, right?
• 42:59
  SPEAKER_01: They were really receptive to feedback to make it more consistent with its intent,
• 43:03
  SPEAKER_01: which is great.
• 43:04
  SPEAKER_01: John, thanks so much for coming on the podcast.
• 43:06
  SPEAKER_00: Thanks so much.
• 43:06
  SPEAKER_00: Appreciate it being here.
• 43:10
  [music]