Latest / ThinkData Podcast / S4 | E10 | Why AI initiatives fail to deliver measurable results - CDO @ Precisely

Transcript

S4 | E10 | Why AI initiatives fail to deliver measurable results - CDO @ Precisely · ThinkData Podcast

• 0:03
  Alex Hutchings: Welcome to the Think Data podcast brought to you in partnership with Mydataworks. If you want to stay up to date with the latest breakthroughs and trends in the world of data and artificial intelligence, and if you're curious about some of the strategies that companies and founders use to launch data and AI products, then you're in the right place. Our aim is to bring together a diverse lineup of fantastic guests from the founders, through to accomplished leaders and product owners at some of the most fascinating data and AI companies worldwide. They will each offer you their own unique insight into what it takes to launch and scale a great data business. Thanks for tuning in, and I hope you enjoy the episode. Welcome to the Think Data podcast and today's guest is Dave Schumann. He is the Chief Data Officer at Precisely. Precisely is a global leader in data integrity. They help organizations trust, manage and maximize their data to help organizations come to better decisions whilst also working on analytics and AI at a true scale. Precisely released a data report in 2026 and we're going to take a closer look at that and also taking a look at why AI initiatives fail. and how to really turn organizational ambition with regards to AI into actual results. And actually what's really refreshing is I have a fully-fledged Chief Data Officer on the podcast. I'm looking forward to really looking at kind of that evolving role because there's been a huge amount of press around where does the CDI role sit, you know, how much involvement are they having on the business strategy, and the fact that Precisely is so well positioned in this space, especially around AI and data integrity. Yeah, I'm really looking forward to this one. Thanks for coming on this morning, Dave. I appreciate it's early your time. So thanks again. But for people who've not come across you, I know obviously you're ex-Cloudera, spent five, six years over there. But with regards to Precisely, what is your role over there and what brought you to them?
• 2:07
  Dave Schumann: Alex, good morning. I lead enterprise-wide innovation across data governance, modeling, analytics, and agentic AI. I'm a hands-on technologist at heart, but I love building.
• 2:18
  Alex Hutchings: it's scaling high impact teams so whether it's about data engineering architecture process automation bi or now ai interesting and i one one thing i alluded to in my intro is obviously this data report we're going to take a closer look at that but i think going back a bit you know ai has obviously been this huge you know we've seen a wholesale shift in how organizations are looking at. productivity, workforce, hiring, firing. But, you know, they're looking at, a lot of companies are trying to align AI to their broader goals. So actually what they're saying is this is the key driver for us. But actually you produced a report that said only 31% link that to real KPIs. What's the disconnect here? Why is one thing saying one and one thing showing the other?
• 3:09
  Dave Schumann: Alex, this is pretty clear in the data. There's a misconception that AI will solve everything without a clear data structure in place. I kind of put that like this AI has a sort of fairy dust capabilities and we sprinkle it over our existing systems and processes and magically we expect different things to happen. So the enterprises are putting a heavy focus on experimentation and model creation and a limited emphasis on sustainment and value realization. I also see in this, I call this the squirrel syndrome. We're chasing the next AI capability that's coming down the pike instead of defining what's measurable about the problem that we're trying to solve. And so this lack of focus, this lack of discipline is part of the problem in realizing and linking the outcomes to real business KPIs.
• 4:01
  Alex Hutchings: Why do you think we've got to that spot in the market then? Because I know obviously AI in the truest sense isn't. anything necessarily new you know artificial intelligence machine learning has been around for a number of years but we've obviously got to this inflection point where there's obviously this belief that ai is going to transform organizations why do you think that's happening why is everyone just getting on the bandwagon without necessarily thinking are we ready to go full in on ai alex
• 4:28
  Dave Schumann: should you point that out because as you looked at some of the the stats in the birth of the ai ml era data scientists spending 80 to 90 percent of their time in data preparation rather than in data science. And we should have looked at that as a leading indicator that the data infrastructure is actually the friction point. So rather than focusing on these, not to consider below the waterlines, and data governance is not a sexy area. It's routine, consistent execution that creates its own flywheel. But you have to have that drumbeat of effort against that. And it's not the glamorous dashboards or demos or new functionality that comes out of AI. It's very much about execution at its core level. And that doesn't get the excitement or the investment that the new shiny objects get.
• 5:27
  Alex Hutchings: Yeah, it's a really interesting point. I guess that's where I touched it again in the intro about the kind of the role of the chief data officer or the role of the most senior kind of, whether it's head of data governance or head of data transformation. How important is that role now in organizations to really ensure that, yes, the new shiny tools are out there? Yes, the business objectives are saying one thing. But ultimately, if there's no governance data, proper data warehouse environment set up. you know, poor data quality, you know, then it all falls apart. So how important is that role now?
• 6:01
  Dave Schumann: It's incredibly important. And when you start to look at what does the modern data landscape look like, and for us, it's a lot of SaaS applications. So each of the applications kind of builds its own silo of operational data, whether that's in your CRM system, like a Salesforce or Microsoft Dynamics, it's in your ERP system, like your NetSuite or your SAP, Each of those builds... data that for them is functional to the operation of the tool, but not necessarily to the function of the enterprise. And so the role of the CDO is to break down those silos and build context across these different application systems. And it's kind of funny, we're in the 2020s now, and we're talking about the same problem we've talked about before, which are data silos that are preventing business contacts from being established. The problem's just gotten more rapid because of the ease of being able to implement these SaaS tools.
• 6:59
  Alex Hutchings: Yeah, it's such a really interesting point. I think the world of B2B SaaS and specifically around kind of data analytics, workflow automation, it seems relatively straightforward or easy for companies to go and sign up on the latest tool, deploy that into the environment. But what are some of the mistakes you're seeing companies make? when they're going with these SaaS companies before they've actually taken a look at their own data environment?
• 7:25
  Dave Schumann: They are thinking about the connective tissue between all of these different applications. So I think fundamentally you have to go back to what is the process flow for the things you're trying to accomplish. So you have words often with the word to in the middle of it. So it's lead to opportunity, opportunity to cash, issue to resolution. And all of those should be a signal to us that there's a linkage between these. these actions that we want to take that potentially cross systems. So building out that data catalog, pulling out the core definitions so that you're understanding in each system what the context is of the customer or the employee in each of those realms, and watching how that flows between systems and ensuring that the systems are exchanging their identity just at its core architecture, making sure that it's a... person or customer transitions from system to system, their own identity is moving with them. And so that you can stitch that back into a common data fabric.
• 8:28
  Alex Hutchings: Yeah, it's really interesting. That's obviously where Precisely play a big part. So you said for people who haven't come across Precisely, if you're going into these organizations or these organizations are approaching you, you know, what A, do you do? And B, how would you approach that kind of almost data audit for these organizations? What would you focus on first?
• 8:46
  Dave Schumann: Yeah. Often we lead with a strategic services offering from Precisely that can help evaluate the state of your data environment. That helps to provide a third party that can come in and say, well, this baby may in fact be ugly. It's often a hard truth to hear if you're the owner of those systems that there's the system itself or even your processes around that will need to be adjusted in order for you to leverage agentic systems or even. in some cases, common BI interfaces. I often have these conversations with my colleagues, and they talk about the scenario where they walk into a meeting, and each of the persons in that meeting has generated their own set of data and insights to bring to the meeting. They're not working from a common catalog. They're building out their own definitions of the events they're trying to look at, whether that's sales or bookings or revenue. The early part of that meeting is debating who pulled the numbers the best. Where did that report come from? And they're doing things like lineage and data quality in the context of the meeting rather than having those fundamentally be part of the organization. That friction. If you see it relevant, if you see it in those meeting rooms, it's going to exist also when you try to put agentic capabilities on top of that same data set. So removing that friction, getting to a common set of definitions, making sure we're all singing from the same hymnal as we walk into whatever our business process is, allows us to build agentic on top of this. But I use that BI example. as a sort of a canary in the coal mine and an early indicator that you have noxious gas in your systems. And that you have to address those in order to be able to build the next set of capabilities.
• 10:35
  Alex Hutchings: Yeah, it's a really valid point, isn't it? Because companies want to get, they don't want to be left behind, right? They've obviously, there's these new technologies, they're competitors, they've got a co-CEO who they know, they're doing some really interesting stuff. So how do you know if you're an organization, you're ready to deploy true AI? How do you know that your current data environment is ready? Is that a, is that too broad a question or is that actually, there's so many different silos you need to take a look at first?
• 11:02
  Dave Schumann: There's so many different pieces. And I think organizations are conflicted because when I look at the data that's coming back from the report, 87% say that they've invested in the systems to be able to be agentic. And so that to me is... almost a procurement exercise. We've worked through the contracts. We have the ability to do this. But the same organizations that have said that they have the infrastructure and capabilities turn around and say that's one of their biggest barriers to deploying agentic AI. And that, to me, drives down to a skills gap. We don't have the people in place to be able to execute that in this. And the data shows this. 51% of the respondents... cited that skills gaps were one of the primary barriers to implementing agentic AI. And it was spread across all the disciplines. It wasn't just the data engineers or the context analysts. It was really spread equally across all roles. So I think we have to think about our people and invest in upskilling them to understand how to operate in an agentic realm. And that's more than just deploying Copilot out to your organization.
• 12:18
  Alex Hutchings: I'm really pleased you said that, even though as a person, actually my day job is hiring and obviously it benefits me when organizations are going to market to find specialist talent. But I also know from experience, there's far more roles out there that are commanding specialist skill set than the availability of talent. So actually for you and Precisely, it's refreshing to hear you are investing internally on developing those next cohorts of specialists. How does that work? process begin? Because ultimately, they're almost leaping into the unknown, right? They've kind of got this new technology, new tool, new way of working. So how does it work internally?
• 12:54
  Dave Schumann: Our efforts within Precisely are co-led by IT and HR. And that's by design. We have to ensure that as we're dealing with new technology being implemented into the organization, new ways of thinking, that is a people problem more than a technology problem. And so the adoption of AI and In fact, really just thinking through how we need to re-engineer our processes to leverage that, not just to take exactly what we've been doing all along, slap AI on top of that and hope that that's going to create better outcomes or better results. It's really rethinking the entire workflow and where we want to have our people engaged in the loop.
• 13:36
  Alex Hutchings: Yeah, really interesting. Actually, if people are listening to this, it's refreshing to hear. It's not just about hiring the next person, hiring, hiring, hiring. It's actually investing into your current folk, making sure they're ready. And as you say, tying that into it's a people problem is really valid. And you mentioned about the report. I know you touched on some of the percentages you've seen, some of the trends you've seen, but is there anything from that report which surprised you? Is there anything from that report which actually is some key headline figures you can share?
• 14:05
  Dave Schumann: I thought one of the interesting ones was what would we ask? What's the? primary influence driving your data strategy. And over half of the respondents now say that AI is that primary influence. And that kind of gets away from the, for us, the core data strategy is building these data assets of high integrity for both transactional and analytic use. This is getting into saying the shiny object in AI is driving where our data strategy is. The other thing that I saw in that is where I'm tired. Data strategy for AI was differentiated from data strategy for the organization. And folks were creating, I think what's 31%, were creating separate organizations to govern AI versus integrating that into their data strategy. And what we saw in the results is that companies that did an integrated data strategy where AI was part of their overall data strategy, rather than having separated governance for AI and data, were far more successful in implementing AI.
• 15:09
  Alex Hutchings: Yeah, that's fascinating. So AI is leading the strategy as opposed to being included into the wider business strategy. They're actually being led by the trend, the technology. That's fascinating. Where do you think this will, do you think that will flip? Do you think that will change eventually?
• 15:25
  Dave Schumann: I think that AI is going to become part of the ecosystem. Rather than being the driver of investment, it will be seen as a tool within the overall arsenal that we have to deploy. I was around back in the dot-com era. We sort of went through this sort of releasing out these in sort of the early or mid-90s, these sort of catalog websites. It was a single destination place that you went to and you did something at that site. And we didn't know that at the time. It was called Web 1.0, but that was only in retrospect when there was a Web 2.0. And what Web 2.0 introduced was a bunch of capabilities for sites to connect with sites. So APIs really unlocked the power of the web. And we started to have this really interconnected experience. I say that now, the interconnectivity, because I feel we're at .0 here. .0 is when we have these autonomous agents talking and communicating with each other. And there's a couple of things we're starting to see on the early edge of what are the APIs that unlock the AI experience. The Nanda project at MIT, MCP, which came out of Anthropic, A2A, which came out of Google and Linux Foundation, are all fundamentally focused on how do agents talk to agents and communicate in that way. We're a year into that journey right now. I don't think we've come anywhere near unlocking that potential. Because now we're going to get out of general purpose AI and we're going to start to develop models and capabilities that have very specific skills. And so it's tying that action to the ability of the model to, within the bounds of its governance, do autonomous actions. And so when agents can now talk to agents within their realm of autonomy, we start to see a completely different process. That's where I think we're going. We're not anywhere near that yet.
• 17:22
  Alex Hutchings: And it's such a valid point and thing for people to be mindful of. As soon as we're allowing agentic AI to speak to agentic AI, you know, in an unsupervised manner, then ultimately those guardrails and those kind of governance frameworks become, you touched right at the beginning, it's not the most fun or sexy part of data, but it will become probably the most important.
• 17:44
  Dave Schumann: What's interesting about this, so we are now deploying autonomous agents within Precisely. And what we're discovering in this is it's. very straightforward, I wouldn't say easy, but straightforward to code for all the known conditions. In our minds, we say, oh, my user is going to do this. They're going to come through this input method. They're going to type in these words to us. We're going to interpret those and drive them down this action. And for me, that's coding for all the ones and forgetting about the zeros in a binary tree. All of those other options in there are where this takes the time to get past the pilot or POC phase, and get into the reality of what are the real inputs we're going to be taking into these systems from real end users. And that point of launch for a capability is not where you've reached the finish line. At that point, when you've launched it, you are into the next phase of that. And the models that are, the agents that are going to be successful in this are going to be able to learn from feedback and also retain the context. So that way, The next action that that individual or that group has with the model, it learned from what we did before. And that context is already there. So I'm not coming back to ground zero every single time I initiate contact with an agent.
• 19:03
  Alex Hutchings: Yeah, as I said, they're becoming more efficient, more effective. The ramp time, the deployment time is less. And actually, they're becoming more effective. And, you know, I understand. My concern at the moment is... All these people, organizations are spending millions of dollars, you know, on becoming truly AI native. But there's a big debate in the moment about how organizations, I'm talking about the C-suite, the CFO, and obviously at your level, how they measure the ROI of this. What is your answer to that? Because I know a lot of people on my podcast, people come on, they talk about the latest tool and the vendor which they represent. But one question which they may get asked is, how do I measure the ROI of this solution? What's your answer to that?
• 19:46
  Dave Schumann: I am. And that's a key one. And it goes back to the results that were in the study. Only 31 percent of respondents could tie the initiative with AI. back to real business metrics.
• 19:58
  Alex Hutchings: Wow.
• 19:58
  Dave Schumann: That's a pretty significant gap.
• 20:00
  Alex Hutchings: See, 9% couldn't. Yeah.
• 20:04
  Dave Schumann: Which means they weren't measuring it, but they were measuring, likely, volume over value. So if you see someone's come back and they say, well, we just deployed this new capability, and you're like, well, what are your metrics on it? Well, we had 10 users today that interacted with the system, and they did 17 queries that were answered back. And we said, okay, but... What did you get from that? Where is the real ROI? How do you tie that to business value? And that's where you get the sort of blank stare. Well, we had 10 users that were on the system today. So we're ready. And it's almost easy to measure utility. It's much more difficult to tie that back to business value and make that something more attributable than a softer fuzzy. Like it was faster to do this. Talk about this in procurement cycles and you're the head of procurement. You're like, well, now it was faster for us to be able to execute this agreement. But where's the value in that back to the organization? And you can say, well, the value is that we were able to provide access to that tooling faster than we would have before. And we go back to that. Well, what was the value that you drove out of that? So it's really fundamentally asking that question. How do you tie this to the core business drivers? And those often look like revenue, cost, risk and service. If I think of the four key branches of where we're going to assign value and time savings is great. I don't want to take that away. And the fact that we can give our people a better use of their time to focus on critical needs. Cool. That's great. But what did we do? Where do we make more money? Where do we take cost out of the system? Where do we give our customers a better experience? And how do we take risk out of the organization?
• 21:53
  Alex Hutchings: yeah it's a great answer and i will certainly quote on those four points as well when we go live because i think it's really interesting and it goes back to the point of ai leading uh you know the cart leading the horse isn't it almost like the ai is leading this strategy whereas actually should be the business metrics and how we're going to become more efficient how we're going to save time how we're going to increase revenue how we're going to increase retention these type of things as opposed to there's the tool let's deploy it then worry about the metrics softwoods.
• 22:19
  Dave Schumann: Yeah. And often those are early stage metrics. So as we're developing a new capability to be able to interpret unstructured data, aka documents. So the first thing we're measuring is, are we able to do that accurately? And that's a significant amount of work because you have to do an A-B comparison. Our model came back with this result. A human who would have been doing that job otherwise came back with these results, compare and contrast. That piece is tedious. But once you get past the tedium to be able to get your quality metrics established, build trust in the process, now you come to what actions can I take? And so if I know that by interpreting this document, say that we're doing expenses and I have a receipt that was just uploaded, and we're able to evaluate whether it's in or out of policy in the moment, what action will I take with that information? How do I then improve the outcome for the business? And that moved away from the core capability is, can we read the document to what are we going to do with this new source of data into our organization that allows us to take different types of actions? And if you just keep the existing processes in place, you're not going to take advantage of this new signal to your process that allows you to get that value. So I think that's where companies really have to think about, yes, you've evolved the capability. It worked in POC, which is. Almost always like shooting fish in a barrel. Like the POC is such a controlled environment. Governance is built in by high observation of the POC participants. Often the data that you're bringing into a POC is highly curated. I'm going to take this data from this place. We're going to watch the effects all the way through. And we're typically evaluating the positive outcomes. Great. Now we start to work in the rollout and we start to observe what does this do with wild. What does it do with access to everything it could see in the graph and the information it could pull back? What does it do when our users aren't necessarily trained in what we expected it? And so they put their own inputs into the system unfettered by our like, no, that's not in the POC doc. And being able to take that feedback and incorporate that is how you roll out successful agents. But as an organization, you have to have the investment and discipline. to realize that the launch event is not completion. It's really just the first step of many more iterations on this.
• 24:53
  Alex Hutchings: Yeah, no, it's a fantastic point. That ties really nicely into the role of the CDO or the most senior kind of data person within the organization. Would you say, and again, I mentioned at the kind of top of the intro that there's a lot of debate around where the CDO should sit. But based on what you just said there, I'm assuming it's that there's almost that hybrid, you know, the data custodian, owning the business strategy, that's what we should see is a fully fledged. CDO these days. Is that accurate?
• 25:24
  Dave Schumann: I would agree. The role of CDO has changed dramatically over the past 30 years. If you take us back into the late 90s, the CDO was, in essence, the custodian of key punch entry. So we were ensuring that we had set the policies and processes up for how we wanted data to be inputted in our system. The modern CDO, I think, is sort of the anchor of this transition to agentic, ensuring that every algorithm is
• 25:50
  Alex Hutchings: tethered to a semantic model every project is measured by its bottom line impact and that's a pretty significant shift over just a few decades yeah and i think it's a it's a really good point i think uh but you've said i've certainly seen it over recent times in terms of organizations not quite getting that alignment right so you know are they on the technology side under the finance team are they operations but for you it's really they need to have their fingers and eyes and everything by the sounds of it to be truly effective at this transformation.
• 26:22
  Dave Schumann: No, I do truly believe that. And I think from an enterprise perspective, the CDO should be looking at how what she or he does is directly impacting their customer's experience and it's directly impacting their employee's experience. But the third piece is how are you generating enterprise value? And that, when I look at that sort of, you know, the three bosses I report to is our customers. our employees, and our investors. And we are a company that is built on data. So the quality of our data directly drives to our enterprise value. It's how we tell our story. And if you cannot confidently tell that story, then you're eroding enterprise value.
• 27:05
  Alex Hutchings: Yeah, that makes sense. And what's, last one at least, what's next for Precisely? I know you guys have... grown you know as i mentioned earlier really kind of cutting edge solution you're adding huge value to those enterprises and subsequently to those customer base but what what's next what can people take a look at on your site and and get to grips with i mean they can download the report is that accurate so they can actually get on and see the report absolutely
• 27:28
  Dave Schumann: that's available on our website at precisely.com what i think you're going to see for precisely is where we sit in this this ecosystem that is going to be driven around ai's capabilities And so in this, how do we use AI to sort of unlock the value that companies have in their data at scale? And people often ask me, like, how much data do you have? And we often express this in, is it in petabytes? Is it in rows and columns? I tend to think of this at a cellular level. So each cell within our data sets, the rows times the columns, is the data that's under management. And the scope and scale of that. for an enterprise the size of Precisely, and for companies in our peer size, is pretty overwhelming. Especially if you think you're sort of hand coding your data quality rules about this. So this for us is where AI really can create an advantage using things like summarization on our own data, summarization on the outputs of our data quality rules to understand what the trends and patterns are, and really help our customers build scale around governance and quality. to lead to execution.
• 28:42
  Alex Hutchings: Yeah, and really interesting. I've really enjoyed the conversation. I feel it's all well and good interviewing kind of people from kind of AI startups and vendors, but it's really nice and really refreshing to speak to someone so entrenched in a kind of the data and which is obviously supporting all of these really cool vendors that I get the pleasure of speaking to and recruiting for, but actually data analytics, BI governance, that ultimately without that, Then there's no AI. So I've really enjoyed it, Dave. And for anyone listening here, check precisely out online. We'll make sure that we tag in the report as well, which is a really interesting report. So yeah, thanks again, Dave. It's been really interesting.
• 29:24
  Dave Schumann: Alex, thanks for having me on.
• 29:25
  Alex Hutchings: Thank you.