Latest / Herbert Smith Freehills Kramer Podcasts / Legal Tech Deciphered Trust, Truth and Hallucinations EP2: eDiscovery in 2026: GenAI Disruption or TAR Reinforcement?
Transcript
- 0:00SPEAKER_00: So maybe the real question that we should be asking here is it's not tar or or gen AI.
- 0:04SPEAKER_00: It's actually so what's the right approach for this matter, for this data and this client in these circumstances?
- 0:10SPEAKER_00: And that that I think is is the real the real thing that we need to be asking about and talking to our clients about when they come to us, so we make sure that we're advising them with the right tools, right, in the right way.
- 0:40SPEAKER_00: So far in this series, we have taken a case study-led approach and explored the application of Gen AI, looking at one particular matter.
- 0:48SPEAKER_00: Today we're going to take a step back a little to examine the wider technology landscape that is shaping e-discovery, including Gen AI.
- 0:57SPEAKER_00: So, to introduce myself and the other panel members of the podcast today, my name is David Beck.
- 1:03SPEAKER_00: I'm head of eDiscovery here in London, covering the UK and EMIA.
- 1:07SPEAKER_00: I advise on data scoping and management, disclosure strategy, governance, and the effective use of advanced review technologies, including technology assisted review and emerging generative AI across both complex disputes and investigations.
- 1:23SPEAKER_00: Joining me are Danby Kim.
- 1:25SPEAKER_01: Hello everyone.
- 1:31SPEAKER_00: She leads digital legal delivery strategy and capability across the US market with a focus on innovation, e-discovery, and technology-enabled legal services.
- 1:41SPEAKER_00: She works closely across HSF Kramer to drive the adoption of new tools and ways of working that enhance efficiency, insight, and client experience across complex legal matters.
- 1:54SPEAKER_00: Megan Ryan is a senior manager specializing in e-discovery and is based in our UK team in London.
- 2:03SPEAKER_00: As part of the legal digital legal delivery team, she works on strategy and delivery of large-scale matters with hands-on experience of modern technology assisted review workflows and evolving AI tools used to support document review.
- 2:17SPEAKER_00: In the UK, she is our AI review expert and advises our legal teams on their use as part of eDiscovery workflows.
- 2:25SPEAKER_00: So, to bring us to the central theme of today and the underlying question What's the difference between technology assisted review or TAR and generative AI?
- 2:35SPEAKER_00: And this question is something that we're getting from clients, lawyers, and is being discussed in the wider market.
- 2:41SPEAKER_00: Gen AI is dominating industry conversations as we all know, but the reality about its uses is much more nuanced.
- 2:48SPEAKER_00: So this episode aims to clarify not hype.
- 2:52SPEAKER_00: The outline of what we'll be covered is the evolution of TAR, Gen AI strengths in e-discovery, why tar remains widely used, but also we won't be covering any deep work deep dive workflows or implementation details.
- 3:06SPEAKER_00: So with the stage set and our speakers introduced, let's dive into the heart of today's discussion.
- 3:11SPEAKER_00: What really distinguishes technology assisted review from generative AI in the world of e-discovery?
- 3:17SPEAKER_00: Megan, perhaps you could help clarify.
- 3:20SPEAKER_02: Sure.
- 3:21SPEAKER_02: I'll start off with just some simple definitions.
- 3:23SPEAKER_02: I think everyone's pretty comfortable with what Gen AI is.
- 3:27SPEAKER_02: It's based off of prompt criteria, it works off the LLMs.
- 3:31SPEAKER_02: You would feed a tool a prompt, and based on that criteria, it would return the documents that you're looking for.
- 3:38SPEAKER_02: TAR, while it stands for technology assisted review, which should encompass all technology assisted review, it's commonly associated with predictive coding.
- 3:47SPEAKER_02: And predictive coding is based on machine learning technology.
- 3:51SPEAKER_02: That's still AI, it's just not generative AI.
- 3:54SPEAKER_02: So machine learning is based on human input.
- 3:57SPEAKER_02: So instead of learning from a prompt, it is learning off of a reviewer coding.
- 4:02SPEAKER_02: So if a reviewer codes a document is relevant, it will learn from that document and find other documents that are similar and say, oh, these documents are also most likely to be relevant based on that reviewer input.
- 4:13SPEAKER_02: So still AI.
- 4:14SPEAKER_02: It does involve a lot more user input.
- 4:16SPEAKER_02: It involves a lot more training than generative AI, but it's still a form of AI.
- 4:22SPEAKER_02: People often use the use the terms tar or predictive coding or cal.
- 4:28SPEAKER_02: They're all kind of the same thing.
- 4:30SPEAKER_02: Tar and predictive coding are exactly the same, but not all tar is cal.
- 4:36SPEAKER_02: Cal means uh continuous active learning.
- 4:39SPEAKER_02: And it's it's when tar is learning continuously as reviewers code.
- 4:43SPEAKER_02: So the more documents that reviewers code, it continues to learn from those documents.
- 4:47SPEAKER_02: Um it's different than the older version of tar, commonly known as tar 1.0.
- 4:54SPEAKER_02: Um, in the UK, we're commonly using Cal, while in the US is still using Tar 1.0 for lots of reasons.
- 5:00SPEAKER_02: Um and I'll hand over to Danby to explain that further.
- 5:04SPEAKER_01: Yeah, so you know, for um the TAR that we are using, for example, the uh government investigation related to merger, second request, uh the government requires you to present them the evidence how your TAR was conducted or performed.
- 5:23SPEAKER_01: So for example, um you need to um give the metrics in first round how many uh samples were uh went in to train the model, how many uh positive or negative samples in each round, and how the score uh metrics change over time.
- 5:42SPEAKER_01: So those are one of the requirements that you typically see in those cases that government require.
- 5:47SPEAKER_01: So if you do the uh continuous active learning, that will be really hard to do.
- 5:52SPEAKER_01: Also uh because we because of that, and also because we want some more control in conducting those uh workflows.
- 6:01SPEAKER_01: So even even though we're using the uh continuous active learning uh capable model, but then we we control what to put in, when to update the rank so that it can be um run as if it's a TAR 1.0 uh workflow.
- 6:20SPEAKER_01: So, you know, the uh many many clients come and say that what's the difference between Torah and then Gen AI?
- 6:29SPEAKER_01: Uh, but I think the TAR is the one that has been used for more than 10 plus years, and then it has been accepted in the court proceedings by government and uh by the court, and then there is a some structured way of proving that the discovery and the e-discovery rebuke has been conducted in a uh uh more defensible way or um the proven way uh with the validations.
- 6:57SPEAKER_01: Uh but then Genai is a new tool.
- 7:01SPEAKER_01: But Megan, can you shed more light on you know what uh the difference that you see in Gen AI?
- 7:08SPEAKER_02: Yeah, um, so as you said, TAR is the only tool that's officially approved by the courts for now.
- 7:15SPEAKER_02: And you do report statistics based on your Cal or TAR 1.0 models, such as recall, precision, and that's how you defend your model.
- 7:24SPEAKER_02: So based on those statistics, the court says, okay, this is accepted.
- 7:27SPEAKER_02: Um, we agree with how you review the documents.
- 7:30SPEAKER_02: I think what people don't realize is that you can get those same statistics with uh genitive AI.
- 7:35SPEAKER_02: It's a little bit different instead of um doing it at the end like you do with a Cal model, you do it at the very beginning.
- 7:41SPEAKER_02: So in your prompt iteration um based on a statistical sample, you would use that to um find recall and precision and know how accurate your gen AI output is.
- 7:52SPEAKER_02: Um so you still get those statistics.
- 7:54SPEAKER_02: It's just that it hasn't officially been approved in the court yet.
- 7:57SPEAKER_02: And I think that's why some um clients are hesitant to use it since it hasn't officially been accepted.
- 8:03SPEAKER_02: Um, there are some benefits, however, of Gen III versus um tar or cow.
- 8:08SPEAKER_02: Um, with predictive coding, you do you get the relevance decisions, a reviewer will look at it, tag it as relevant, and that's what gets disclosed.
- 8:17SPEAKER_02: Wherever GenI, a lot of tools like Relativity Air will also provide you with the rationale.
- 8:22SPEAKER_02: So it will tell you why I thought that document was relevant and the citations within the document.
- 8:26SPEAKER_02: So a lot of lawyers um are appreciating that because it further backs up why that document is relevant.
- 8:31SPEAKER_02: And that's something that you won't get on the face of tar.
- 8:35SPEAKER_02: You would have to ask every reviewer to put an explanation of why they thought a document was relevant, which is kind of intensive, where generative AI will just do that automatically.
- 8:43SPEAKER_02: So that's kind of a main benefit of generative AI.
- 8:46SPEAKER_01: So it became a less of a black box now.
- 8:49SPEAKER_02: Yes, exactly.
- 8:52SPEAKER_00: Yeah, thanks, Megan.
- 8:53SPEAKER_00: I think that's um that's actually you know the interesting point, isn't it?
- 8:57SPEAKER_00: That obviously clients used to come to us and they used to say, Oh, you know, we need to run these keyword searches.
- 9:01SPEAKER_00: Now they're coming to us and saying, Well, actually, okay, what technology have you got?
- 9:05SPEAKER_00: But actually, what are the risks around using it?
- 9:07SPEAKER_00: Is it been court-approved, like you said, etc.?
- 9:09SPEAKER_00: Right.
- 9:09SPEAKER_00: So a lot of the time, I mean, as we've talked about, right?
- 9:12SPEAKER_00: Danby touched on this, you know, Gen I is the is the buzzword, it's the latest thing, it's the exciting, exciting new, shiny.
- 9:19SPEAKER_00: Um, but actually, that difference between what the Gen AI tool will do against what Tar does um is not necessarily always understood.
- 9:28SPEAKER_00: And that difference between those two systems and actually how they can be used most effectively on different client matters, different, you know, requirements for production, etc., is really important.
- 9:40SPEAKER_00: But we have seen clients come to us and say, well, actually, you know, what's your what are your options when it comes to review?
- 9:48SPEAKER_00: Um, how can you best help us to filter down, get to those relevant documents and all that kind of thing?
- 9:55SPEAKER_00: And it's you know, it's a difficult thing because actually, of course, in whilst they always want the newest and fanciest, sometimes the newest and fanciest isn't quite ready for them, or it's not quite ready for what we need.
- 10:08SPEAKER_00: And again, we've touched on you know, actually doesn't have that validation step really, not formally, that a tar platform does in terms of how it's checking the results that it's got and all of that sort of stuff.
- 10:21SPEAKER_00: So, you know, maybe, maybe Megan, you could go a little bit into some of those sort of points around what what are those things that we can provide in a tar that we can't provide in a gen AI.
- 10:35SPEAKER_02: Um, well, I think one benefit of tar is that it does learn from other documents.
- 10:40SPEAKER_02: And that's another question we get a lot with Gen AI.
- 10:43SPEAKER_02: Well, um, a benefit of Gen AI is that you don't have to train a model.
- 10:47SPEAKER_02: It's already trained, you give it input, it provides the output, it doesn't learn from other documents.
- 10:52SPEAKER_02: So if the Gen AI does get it wrong, and if a reviewer looks at it and says, oh, that's actually not relevant, and tag it as such, it's it doesn't adjust, it doesn't readjust to say, okay, now other documents like this are also not relevant.
- 11:05SPEAKER_02: Um, and that's why a lot of times we suggest using a combination of these tools where you can use Gen AI to kind of get a start, find that initial population of the relevant documents.
- 11:16SPEAKER_02: And then for the documents where the tools say, okay, it's kind of borderline, might not be fully relevant.
- 11:22SPEAKER_02: You can put that through a cow model and then use the technology in cow to quickly split the documents between relevant and not relevant if the Gen AI tool couldn't quite get it right.
- 11:32SPEAKER_02: Um, so the yeah, so the tools aren't really a replacement for one another, and a lot of times they can work together.
- 11:38SPEAKER_02: Um, and that's kind of how we've been approaching our workflows of, you know, can we use Gen II for one part of the workflow, tar for another?
- 11:44SPEAKER_02: Some matters it does fully work with a Gen II workflow.
- 11:47SPEAKER_02: Um, it all depends.
- 11:51SPEAKER_01: Yeah, that's right.
- 11:53SPEAKER_01: I think if I could just add some more color to it, is that, you know, I I attended the Legal Week uh in March and there was a mock trial about proponent of the Gen AIUs and the opponent of the Gen IUs.
- 12:07SPEAKER_01: And then the proponent uh interestingly proposed to use Gen AI tool with the TAR as a QC tool together to give more uh more assurance or defensibility against the other other party uh that who's opposing to it.
- 12:25SPEAKER_01: So I think it's it's not as as Megan mentioned, it's not one or another.
- 12:30SPEAKER_01: It could be combination of it uh that that will work better in in a very um simplistic way.
- 12:39SPEAKER_01: I always think that Genai is another algorithm, it's another model that put into the you know machine learning or you know technology assisted review workflow.
- 12:50SPEAKER_01: So depending on your data, depending on your need and goal, you can plug in what model or which tool that you want to use to get to the most um the efficient or effective way of conducting your review.
- 13:05SPEAKER_01: So I think it's a very complementary uh and then it will go hand in hand um in the near future.
- 13:13SPEAKER_02: Um another question we get often is which one's faster?
- 13:17SPEAKER_02: And there's a fine answer to that.
- 13:20SPEAKER_02: Which one's quicker?
- 13:21SPEAKER_02: Because I think what people think about so with with tar, yes, you do need reviewers to actually code the documents, you need the model to stabilize for it to actually know how many documents to review.
- 13:32SPEAKER_02: And yes, that takes time.
- 13:33SPEAKER_02: But then what people forget is that Gen I also does take time.
- 13:37SPEAKER_02: You do have to do a few iterations of the prompt.
- 13:39SPEAKER_02: What we're finding is a lot of our workflows, you're you don't get it right on the first prompt or the second prompt.
- 13:44SPEAKER_02: You do have to a few iterations, and it's a lot of work up front.
- 13:47SPEAKER_02: Um, but once you get it right, then the savings come at the end because you get it right and then you cut out a bunch of not relevant documents, and then you have your relevant set.
- 13:55SPEAKER_02: So there is a benefit, but I think we are having trouble um convincing teams of that because they think, oh, Gen II is magic.
- 14:01SPEAKER_02: So let me write a prompt.
- 14:02SPEAKER_02: Oh, we'll say all these documents are relevant, I'm done.
- 14:05SPEAKER_02: That's not the case.
- 14:05SPEAKER_02: It still has a lot of work up front.
- 14:07SPEAKER_02: So both tools do require a little bit of work and they work differently.
- 14:10SPEAKER_02: But yeah, there's not, I can't really say one's quicker than the other.
- 14:14SPEAKER_01: Yeah, you know, I get a lot of those.
- 14:17SPEAKER_01: They're saying that, oh, so so Danby, you have a generative AI tool.
- 14:21SPEAKER_01: Can we finish this review tomorrow?
- 14:24SPEAKER_01: Or is it any any quick and cheap uh way of reviewing these documents with Gen AI?
- 14:29SPEAKER_01: Uh, but you know, in order to have defensible review or really uh controlled review, I think we need the time, as Megan said, and then QC, and um so so that it is not uh something that, you know, well what has been proven is the generative AI or AI in in general need uh the quality control and humans double checking uh the work you know results.
- 14:55SPEAKER_01: So I think it's it's a you know it's a laughing matter in our conversation, but it is serious when you talk up with the client, you know, what what they want, what they want to achieve.
- 15:05SPEAKER_01: So managing that expectation is a little bit of uh uh difficult at times, but you know, this we we hear a lot about you know Gen AI magically done it overnight type of uh expectation.
- 15:20SPEAKER_00: Yeah, I think that's right, Danby.
- 15:22SPEAKER_00: And of course, you know, the other challenge that we have, you know, primarily so far, we've been talking about the difference between like Tar and an e-discovery gen AI tool.
- 15:31SPEAKER_00: But of course, the the other big challenge we have is is you know, is is facing up to the no, please don't just take all of your documents and throw them into Copilot or whatever else you've got, right?
- 15:42SPEAKER_00: It it cannot do a document review, and it and it comes back to that point you just made, it cannot do it defensively, it cannot even do it repeatedly.
- 15:50SPEAKER_00: You're not necessarily going to get the same result every single time you run a prompt, right?
- 15:54SPEAKER_00: Um and so you know, these are the things that that that that can present a real challenge and actually a bit of a risk, and it's all about for us at least as eDiscovery professionals, managing that, you know, no, no, sorry, you you still have to use the right tools in the right way, and therefore you still have to talk to us.
- 16:12SPEAKER_00: You can't just go away and put it in co-pilot and see what happens.
- 16:15SPEAKER_00: Um, I mean you could, but it wouldn't be good.
- 16:18SPEAKER_00: Um, but actually that's that's not to say that some of these sort of more I'd say tradition traditional gen AI, if we can even have traditional gen AI at this point, but that's not to say that some of those can't add some of the values of things that we that we use, right?
- 16:33SPEAKER_00: And I think we've probably all had instances where we've you know we've ended up with a pool of documents, and actually what the legal team really need next or the client needs next is a chronology.
- 16:43SPEAKER_00: So actually we can export those documents, put them into a a traditional Gen AR tool, if I can use that phrase, generate and generate that chronology.
- 16:52SPEAKER_00: Or, you know, we've we've we've look been looking at using it for can we generate keywords that we could then use to run and you know, not everybody likes keywords, but actually it it turns it on its head if you take some documents from your review that that the client or the legal team know are relevant to their to what they're looking for, get gen AI to generate those keywords from that and use that as your keyword.
- 17:18SPEAKER_00: Suddenly a keyword search suddenly makes a lot more sense than it does in in the old school, right?
- 17:22SPEAKER_00: Of just making up some words and see what see what falls out almost.
- 17:26SPEAKER_00: Um and so, but but yeah, I and I guess you know the key thing though, regardless, is is as we said at you know at the start of this section, it just it's not a substitute for document review, it's not a substitute really for getting eyes on, at least at some level at some point in the process and the workflow, and just you know, understanding the limitations of some of these tools, whether they are generative AI standard traditional tools, or whether they're e-discovery, um, you know, you've you've still got to be very careful about how you're using them in a in a defensible way, right?
- 18:00SPEAKER_01: Yeah, that's right.
- 18:01SPEAKER_01: It's it's not the question of whether it's a gen AI or tar.
- 18:05SPEAKER_01: I think it's it's uh, you know, within the tools that we have, whether it's any other uh generative AI tool, what's the best fit for your client's purpose?
- 18:16SPEAKER_01: Yeah, what's what's the um whether it's a cost uh concern or whether timing concern, what is the right one for this?
- 18:24SPEAKER_01: And then discuss with your client and select the ones that that that solve the issue uh as best as we can.
- 18:32SPEAKER_01: I think that's where we are at right now.
- 18:36SPEAKER_00: So that's absolutely right, Danby.
- 18:38SPEAKER_00: And and and so maybe the real question that we should be asking here is it's not tar or or gen AI.
- 18:43SPEAKER_00: It's actually so what's the right approach for this matter, for this data and this client in these circumstances?
- 18:49SPEAKER_00: And that that I think is is the real the real thing that we need to be asking about and talking to our clients about when they come to us, so we make sure that we're advising them with the right tools, right, in the right way.
- 19:00SPEAKER_00: Um, and so actually, yeah, that that's something that we'll look at um a little bit more in the next episode, where actually um you know, we'll look at practical strategies how um you know for navigating this this evolving landscape and and how combining some of these tools together can be the right approach.
- 19:20SPEAKER_00: And with that, I'll say thank you, Megan and Danby.
- 19:23SPEAKER_00: And uh thank you everybody for listening.
- 19:26SPEAKER_00: See you next time.