Latest / Herbert Smith Freehills Kramer Podcasts / Legal Tech Deciphered EP5: From Chaos to Clarity: Building the Data Layer That Unlocks AI
Transcript
- 0:00SPEAKER_02: So this is very much the garbage in, garbage out episode, looking at whether AI can really deliver value if the data beneath it is disorganized, outdated, or simply unmanageable.
- 0:12SPEAKER_00: 60% of AI projects will be abandoned due to poor data readiness.
- 0:34SPEAKER_02: In the last episode, we talked about how organizations often jump straight to solutions and generally assume that AI is going to solve their issues before they really have an understanding of the problem they're trying to solve.
- 0:47SPEAKER_02: In this session, From Chaos to Clarity, Building the Data Layer that unlocks AI, we're going to take a step back even further.
- 0:55SPEAKER_02: Because one of the biggest problems we see isn't the technology, it's the data itself.
- 1:00SPEAKER_02: So this is very much the garbage in, garbage out episode, looking at whether AI can really deliver value if the data beneath it is disorganized, outdated, or simply unmanageable.
- 1:13SPEAKER_02: I'm David Beck, head of eDiscovery in London, covering UK and EMIA.
- 1:17SPEAKER_02: I advise on data scoping and management, disclosure strategy, governance, and the effective use of advanced review technologies, including technology-assisted review and emerging generative AI across complex disputes and investigations.
- 1:30SPEAKER_00: Thanks, David.
- 1:31SPEAKER_00: So hi, I'm Casey Ballard.
- 1:33SPEAKER_00: I'm head of the strategic applications team, also covering UK and EMIA, spending most of my time aware the legal problems, process design, and technology meet, both across transactions and disputes work.
- 1:45SPEAKER_00: Uchi, over to you.
- 1:47SPEAKER_01: Thanks, Casey.
- 1:48SPEAKER_01: And I'm Uchi Jawaduna.
- 1:50SPEAKER_01: I lead our data analytics team.
- 1:52SPEAKER_01: We specialize in collecting, analysing, and presenting complex data in clear, actionable ways to help clients make confident decisions in complex situations.
- 2:06SPEAKER_02: And as you can tell from that, we're each coming at this from a slightly different angle, be it contentious, transactional, or data operations.
- 2:12SPEAKER_02: So hopefully you'll get a few different perspectives on the same problem.
- 2:18SPEAKER_02: Well, let's start with the reality check.
- 2:20SPEAKER_02: Across most organizations today, according to LinkedIn, around 80 to 90% of enterprise data is unstructured.
- 2:28SPEAKER_02: That's emails, documents, messages, and recordings.
- 2:32SPEAKER_02: Whereas Dataversity.net say that only about 10% of that data is actually used for insight or decision making, and roughly half of all enterprise data is dark, that is collected and stored, but never used at all.
- 2:47SPEAKER_02: So when we're talking about AI transforming businesses, we're often talking about applying it to data that organizations don't understand, don't trust, and in many cases, don't even use.
- 3:18SPEAKER_02: One of the key themes for this episode is just that AI is frequently a catalyst.
- 3:23SPEAKER_02: Usually within itself, it's not a solution.
- 3:26SPEAKER_02: There are really two ways we see AI being used: analyzing organizational data to extract insight, or sitting on top of document systems, helping people search and retrieve information and collating it into new documentation or communications.
- 3:42SPEAKER_02: And in both of those cases, if your data isn't organized, classified, and maintained, the result is the same.
- 3:49SPEAKER_02: You get something that's more expensive, more risky, and less accurate than it should be.
- 3:55SPEAKER_00: Exactly.
- 3:55SPEAKER_00: And we're seeing this particularly with Gen AI.
- 3:58SPEAKER_00: People are taking huge volumes of content and saying, let's just put this all in and see what we get out.
- 4:04SPEAKER_00: No guidance as to how to interpret the data that's then been fed in or what good actually looks like.
- 4:10SPEAKER_00: But what you get back isn't necessarily wrong.
- 4:13SPEAKER_00: It's just not reliable.
- 4:14SPEAKER_00: The biggest concern is on the face of it, it can look really impressive until you actually delve into the output of content.
- 4:21SPEAKER_01: I agree, Casey, 100% with you.
- 4:23SPEAKER_01: But I think there's a bigger issue here also.
- 4:25SPEAKER_01: Using all that can potentially lead to incorrect conclusions.
- 4:30SPEAKER_01: Simply because you may have a situation where the sheer volume of data simply obscures the relevant elements.
- 4:38SPEAKER_01: And the consequence of using all that data in an AI environment is that the output generated can turn out to be completely incorrect or ineffective.
- 4:49SPEAKER_02: Exactly, Uh.
- 4:51SPEAKER_02: AI doesn't fix bad data, it amplifies it.
- 4:54SPEAKER_00: It does.
- 4:55SPEAKER_00: And we see this a lot where clients actually say, we've got three years worth of data.
- 5:00SPEAKER_00: Can we just use it all?
- 5:01SPEAKER_00: The reality is when you look at that, the rules and regulations have changed, the business processes have changed, templates and outputs of formats have changed, but none of that is reflected or identified within the data you're feeding it.
- 5:15SPEAKER_00: So you're feeding a system multiple versions of the truth with no guidance or guardrails.
- 5:21SPEAKER_01: Indeed.
- 5:21SPEAKER_01: And this becomes very important simply because instead of having relevant data, this addition of irrelevant data, again, as I mentioned earlier, just confuses the outcome, especially given that AI has no inherent way of distinguishing what's current, what's obsolete, or what's relevant.
- 5:41SPEAKER_01: It will just process everything it's given and treat it equally.
- 5:45SPEAKER_02: Exactly, you see.
- 5:46SPEAKER_02: And actually, this is where e-discovery becomes quite a useful lens, because we've been dealing with this problem for years.
- 5:52SPEAKER_02: When data isn't well managed, what you see is lots of duplication, inconsistent metadata, poor naming conventions, lack of classification, and massive volumes of irrelevant material.
- 6:05SPEAKER_02: And all of that has a direct impact on the time and cost of any discovery exercise.
- 6:10SPEAKER_02: If the exercise is regulatory related, not having proper control of your data adds to the risk as well.
- 6:17SPEAKER_00: Exactly.
- 6:18SPEAKER_00: And taking that from a practical point of view, it can mean more data to process, more data to review, whether that's manually or using Gen AI.
- 6:27SPEAKER_00: This then incurs higher costs, longer timelines.
- 6:30SPEAKER_00: So you're essentially paying to sort through the noise.
- 6:33SPEAKER_00: When volume and messy data dumps are already substantial in both transaction and disputes matters, this can be hugely magnified.
- 6:42SPEAKER_01: Thanks, Casey.
- 6:43SPEAKER_01: Aside from paying extra to shift through all this irrelevant data, you may actually end up with an ineffective process.
- 6:51SPEAKER_01: You may end up compounding the issues as these exercises are rarely single-step processes.
- 6:58SPEAKER_01: Layer upon layer, you may end up compounding these issues.
- 7:02SPEAKER_01: You may actually end up building something on a very shaky foundation.
- 7:06SPEAKER_01: And as we all know, what you build on a shaky foundation is going to fall apart sooner or later.
- 7:14SPEAKER_02: Yeah, and also importantly, if your metadata is wrong, your conclusions can be wrong.
- 7:18SPEAKER_02: That's something that's very well understood in eDiscovery, but something the wider business world is only now starting to realize in the context of AI.
- 7:26SPEAKER_02: And this isn't just theoretical.
- 7:28SPEAKER_02: According to Forbes, poor data quality costs organizations on average around$12.9 million a year in wasted resources and lost opportunities.
- 7:38SPEAKER_02: So this isn't just a technical problem, it's a financial one.
- 7:42SPEAKER_01: Completely agree, David.
- 7:44SPEAKER_01: A lot of this comes down to behavior as well.
- 7:47SPEAKER_01: People either don't have the time to decide what's relevant or they just want to be safe.
- 7:52SPEAKER_01: So they keep everything.
- 7:54SPEAKER_01: And that's also seen as the safer option, keeping all the data just in case.
- 8:00SPEAKER_01: Everything gets saved, everything gets archived, and nothing gets cleaned.
- 8:04SPEAKER_01: The end result is having to troll through a lot of irrelevant material.
- 8:10SPEAKER_01: Or worse, having troll having to troll through all this irrelevant material may mean that you may actually miss out what you're looking for, leading to suboptimal decisions.
- 8:21SPEAKER_01: Now, keeping all this data can actually lead to another headache.
- 8:26SPEAKER_01: Historically, storage was never a major concern.
- 8:29SPEAKER_01: You had low volumes of data, which meant smaller storage costs.
- 8:33SPEAKER_01: But with all the recent tech developments, including AI, the volumes of data have skyrocketed.
- 8:40SPEAKER_01: Storage costs now are considerable.
- 8:43SPEAKER_01: According to research from Comprise, 73% of enterprise organizations spent more than 30% of their total IT budget on storage.
- 8:54SPEAKER_01: And I think Casey also has something to say about all this data.
- 8:58SPEAKER_00: Thanks, Uchi.
- 9:00SPEAKER_00: It's definitely worth flagging that the issue isn't just around the storage cost.
- 9:04SPEAKER_00: In fact, the more data you retain, the larger your potential surface risk becomes.
- 9:09SPEAKER_00: It's about the risk around security and privacy.
- 9:12SPEAKER_00: And in turn, that's about your regulatory exposure.
- 9:15SPEAKER_00: Ultimately, the key is understanding what information really matters to your organization and what's actually an acceptable risk.
- 9:23SPEAKER_02: Thanks, Casey.
- 9:25SPEAKER_02: And when it comes to risk, one thing that's been in the headlines a great deal in the last few years has been data breach.
- 9:32SPEAKER_02: And in the context of that, we often talk to or talk about organizations needing to identify their crown jewels.
- 9:38SPEAKER_02: The data that creates the most value for the business, or the data that would create the greatest harm if it were lost, exposed, or misused.
- 9:46SPEAKER_02: And the problem is that when everything is retained and very little is classified, those crown jewels become harder to identify.
- 9:54SPEAKER_02: And if a breach does occur, that's when the problem really comes into focus.
- 9:58SPEAKER_02: Because the first questions asked will be what data has been accessed?
- 10:03SPEAKER_02: How sensitive is it?
- 10:05SPEAKER_02: Which customers, employees, or counterparties are affected, and what do we need to report and when?
- 10:11SPEAKER_02: And if you don't understand your data state before an incident, it's incredibly difficult to answer those questions during a breach occurring.
- 10:20SPEAKER_01: And David, if I can just add to that, speed really matters in these types of situations.
- 10:26SPEAKER_01: The organizations that actually respond best to a breach are usually not the ones with the most amount of data.
- 10:34SPEAKER_01: They are the ones that know where their most important data is, who owns it, and how it's protected.
- 10:41SPEAKER_01: Because they are always able to focus immediately on the areas that matter most.
- 10:48SPEAKER_01: And that's another reason why data governance isn't just an administrative exercise.
- 10:54SPEAKER_01: Good governance helps you extract value from data.
- 10:58SPEAKER_01: But it also helps you understand risk, prioritize protection, and respond more effectively when something actually goes wrong.
- 11:07SPEAKER_02: Yeah, that's right, Uchi.
- 11:09SPEAKER_02: So as we've seen, now while there can be a temptation to think that keeping everything gives you options in the future, in practice, keeping everything often means you've got more data to secure, more data to manage, and more data to investigate when a problem occurs.
- 11:23SPEAKER_02: Which brings us neatly on to one of the most basic but almost important foundations of good data management: structure, classification, and naming conventions.
- 11:34SPEAKER_02: And probably the most common scenario that we see is that organizations technically they have policies, but in practice, those policies are not followed or they're not enforced.
- 11:46SPEAKER_02: And so over time, the data state just grows and becomes more complex and more fragmented.
- 11:52SPEAKER_00: That's so true.
- 11:53SPEAKER_00: And sometimes it really is the most basic things, like the naming conventions you mentioned, David.
- 11:58SPEAKER_00: If you've got 12 documents all called the same thing, and you ask a system to extract, review, or even just find the final version, it doesn't know which one that is unless you clearly defined it as the final on the document or in the metadata or even just having a signed copy.
- 12:15SPEAKER_00: And unless you've put very deliberate guardrails, you're effectively asking the system to make a judgment call without the information needed to actually do that.
- 12:23SPEAKER_02: Yeah, thanks, Casey.
- 12:25SPEAKER_02: And the interesting thing is, of course, this isn't just a litigation problem.
- 12:28SPEAKER_02: It actually affects both sides in the contentious world.
- 12:31SPEAKER_02: More data means more cost, more noise, and more risk.
- 12:34SPEAKER_02: But in the transactional world, poor data makes it harder to reuse, analyze, or integrate information.
- 12:41SPEAKER_02: Different contexts, but fundamentally the same underlying issue.
- 12:45SPEAKER_01: Mergers are probably the clearest example of this.
- 12:49SPEAKER_01: You bring together two organizations with different systems, different formats, and different standards.
- 12:56SPEAKER_01: And what tends to happen is that the data gets combined but never properly integrated.
- 13:01SPEAKER_01: And if timelines get tight, usually they do, data structure is usually the first thing to go.
- 13:09SPEAKER_00: Exactly.
- 13:10SPEAKER_00: Because if you lose that structure, you're not just making the data harder to manage either.
- 13:16SPEAKER_00: You can compromise the solution itself, particularly if you're trying to overlay Gen AI.
- 13:21SPEAKER_00: And then either you get a weaker outcome or you spend time and money building workarounds at a later date.
- 13:28SPEAKER_02: Yeah, exactly.
- 13:29SPEAKER_02: And then afterwards, people will say, well, we'll tidy it up later.
- 13:32SPEAKER_02: But business as usual resumes and it never happens.
- 13:35SPEAKER_02: It's like moving house.
- 13:36SPEAKER_02: You don't have time or inclination to clear out before you move, thinking you can do it when you're in your new home and nicely settled.
- 13:43SPEAKER_02: But you never do it.
- 13:44SPEAKER_02: Life takes over.
- 13:46SPEAKER_02: And the challenge is this problem isn't stabilizing, it's accelerating.
- 13:50SPEAKER_02: According to techdogs.com, global data volume is projected to reach 221 zettabytes in 2026, a 22% increase on 2025.
- 14:02SPEAKER_02: Whilst per statista.com, it's not expected to slow down either.
- 14:06SPEAKER_02: Driven by AI and Internet of Things adoption, global data volumes are forecast to reach 527 zettabytes by 2029.
- 14:15SPEAKER_02: So organizations are capturing, generating, and retaining more information than ever before.
- 14:21SPEAKER_00: Such huge numbers.
- 14:23SPEAKER_00: And that's what makes this so interesting.
- 14:25SPEAKER_00: And exactly the point where companies are investing heavily in AI to extract value from their information.
- 14:31SPEAKER_00: According to Precisely, only 12% of organizations actually say their data is sufficient quality and accessibility for effective AI implementation.
- 14:41SPEAKER_00: And Gartner predicts that 60% of AI projects will be abandoned due to poor data readiness.
- 14:48SPEAKER_00: So the amount of data is exploding, but the proportion that is genuinely AI ready actually remains remarkably small.
- 14:57SPEAKER_02: And of course, all that suggests is that AI isn't exposing a technology problem.
- 15:02SPEAKER_02: It's exposing a data problem.
- 15:04SPEAKER_02: The question isn't whether organizations have enough data anymore.
- 15:07SPEAKER_02: Most have more than they know what to do with.
- 15:10SPEAKER_02: The real question is whether they understand it, trust it, and can actually use it.
- 15:16SPEAKER_01: And David, if I can add to that, the burden of all this is that in addition to the data being irrelevant, you are adding to the storage costs by carrying a lot of unnecessary data.
- 15:27SPEAKER_01: You're also likely to be burning through extra tokens to run AI across it, which adds further to the cost.
- 15:35SPEAKER_01: This is becoming critical, especially as software companies shift away from flat rate subscriptions to consumption-based pricing.
- 15:44SPEAKER_00: Exactly.
- 15:45SPEAKER_00: And it doesn't just add to the cost either.
- 15:47SPEAKER_00: It adds complexity.
- 15:49SPEAKER_00: Because once you carry unnecessary data through the process, every step then has to deal with it.
- 15:54SPEAKER_00: From scoping to processing to review to the output.
- 15:59SPEAKER_02: And so what we're saying is that in addition to data volume, one of the biggest gaps we see, which will only increase with that volume, is classification.
- 16:07SPEAKER_02: Without proper classification, organizations fall back on things like keyword searches to find or exclude the data that they're looking for.
- 16:14SPEAKER_02: And that's just not reliable anymore.
- 16:16SPEAKER_02: You cannot depend on keywords to manage complex data sets in 2026.
- 16:20SPEAKER_02: Whereas even if you've only got partial classification in place, even if it's only 50 or 60% effective, you're already in a much better position than having none at all.
- 16:30SPEAKER_00: Definitely.
- 16:31SPEAKER_00: There's also a really important risk question here.
- 16:34SPEAKER_00: If we are relying on data that hasn't been structured or validated, who actually owns that risk?
- 16:40SPEAKER_02: And from a legal context, of course, that really matters because you have to stand behind your data.
- 16:45SPEAKER_02: Unfortunately, you cannot just say the AI told us.
- 16:48SPEAKER_00: Exactly.
- 16:49SPEAKER_00: I think there's also a bit of a reality check here.
- 16:51SPEAKER_00: So running a theme from our last episode, there's no magic wand.
- 16:55SPEAKER_00: If you send large, messy data sets, it doesn't just become clean because you apply AI to it.
- 17:02SPEAKER_02: Yeah, thanks, Casey.
- 17:04SPEAKER_02: And so, in summary, challenges of successful AI implementation aren't really an AI problem.
- 17:09SPEAKER_02: It's a data management problem.
- 17:11SPEAKER_02: And the organizations that are starting to get value from AI are the ones that have invested in data governance, applied some level of classification, and taken an overall proactive approach rather than waiting until something goes wrong.
- 17:25SPEAKER_01: And if I can add to that, David, and where we do see this working well is where organizations move towards more managed, ongoing approaches, treating data as something that's maintained over time rather than something that's dealt with reactively or on a case-by-case basis.
- 17:46SPEAKER_02: Yeah, exactly, thank you.
- 17:48SPEAKER_02: And even if you solve all of that, there's still another big challenge emerging, which is where the data actually sits across messaging platforms, mobile devices, personal apps.
- 18:00SPEAKER_02: So in the next episode, we'll look at what happens when your data isn't just messy, it's everywhere.
- 18:06SPEAKER_02: Thank you for listening.
- 18:07SPEAKER_02: We'll see you next time.
- 18:08SPEAKER_02: Casey Nucci, thank you very much for joining me.
- 18:11SPEAKER_00: Thanks, David.
- 18:12SPEAKER_00: Bye bye.
- 18:13SPEAKER_01: Thanks, David.