Latest / The Adults In The AI Room / Earley AI Podcast - Episode 99 Data Governance, Business Context, and Why AI Makes the Old Problems Worse with Zoher Karu

Transcript

Earley AI Podcast - Episode 99 Data Governance, Business Context, and Why AI Makes the Old Problems Worse with Zoher Karu · The Adults In The AI Room

• 0:14
  SPEAKER_00: Early AI podcast is sponsored by Early Information Science for digital transformation journey with design and deployment of innovative technology solutions.
• 0:24
  SPEAKER_00: As well as Vector.
• 0:26
  SPEAKER_00: Now enjoy the shop.
• 0:28
  SPEAKER_01: Welcome to the Early AI Podcast brought to you by Vector.
• 0:32
  SPEAKER_01: I'm your host, Seth Early.
• 0:34
  SPEAKER_01: In each episode, we explore how artificial intelligence and data are reshaping a business strategy and operations.
• 0:39
  SPEAKER_01: Today we're talking about data governance, AI readiness, and one of the most persistent problems in enterprise AI.
• 0:46
  SPEAKER_01: The gap between promise of intelligent decision making and the mess and the messy, siloed, poorly governed data reality that most organizations are actually working with.
• 0:55
  SPEAKER_01: The tools have never been more powerful.
• 0:57
  SPEAKER_01: The data behind them has never been more consequential.
• 1:00
  SPEAKER_01: And every organization I talk to has the same reaction to generative AI.
• 1:04
  SPEAKER_01: It feels like magic.
• 1:05
  SPEAKER_01: And if you understand what's behind the scenes, if you ever try to program a neural network to play tic-tac-toe, watching a model write code and generate a slide deck, really does feel like magic.
• 1:16
  SPEAKER_01: But the organizations that treat it like magic are the ones that are getting burned because the same old problems, the same data silos, the multiple sources of the truth, the missing business context do not disappear.
• 1:27
  SPEAKER_01: They just get expressed faster with more confidence.
• 1:30
  SPEAKER_01: My guest today has spent his career on the data and analytics side of exactly the problem, running enterprise-wide data and analytics teams at Sears Holdings, eBay, Citibank, and Blue Shield of California and building personalization programs before personalization was something that a large language model could attempt.
• 1:48
  SPEAKER_01: Zoher Caru has a phrase for what he is seeing right now.
• 1:52
  SPEAKER_01: And I think it's one of the best summaries of the moment that I've heard.
• 1:56
  SPEAKER_01: Data governance has become sexy again.
• 1:58
  SPEAKER_01: Zoher, welcome to the show.
• 2:00
  SPEAKER_03: Thanks, Seth.
• 2:01
  SPEAKER_03: Thanks for having me.
• 2:02
  SPEAKER_01: So give us uh the audience, uh, the audience a version of your background you gave me.
• 2:06
  SPEAKER_01: Technical training, McKinsey, then a run of enterprise data and analytics leadership roles.
• 2:10
  SPEAKER_01: Most people will recognize.
• 2:12
  SPEAKER_01: How did you start?
• 2:12
  SPEAKER_01: How did you end up thinking about AI the way you do?
• 2:15
  SPEAKER_03: Well, you know, it's you're you're right.
• 2:17
  SPEAKER_03: My background is varied across multiple different industries and you know, technical bioeducation, bachelor's, master's, PhD, electrical engineering.
• 2:24
  SPEAKER_03: But then I went straight into McKinsey, did management consulting, where you learn a lot about a lot.
• 2:30
  SPEAKER_03: And one of the things you certainly learn is you start making decisions on partial information, which is a lot of what you know even AI is doing these days, right?
• 2:37
  SPEAKER_03: Is trying to make uh decisions based off of probabilistic guesses and models and everything else.
• 2:42
  SPEAKER_03: And so from the beginning after McKinsey, I did several worked at several smaller startups doing interesting things like tracking customers in retail stores using video cameras or analyzing phone conversations at call centers, but then as you mentioned, leading large enterprise teams at bigger companies, ears, uh eBay, Citibank, Blue Sheet with California, trying to use business uh data analysis to help accelerate business objectives, right?
• 3:06
  SPEAKER_03: So partnering with different stakeholders around the organization to accelerate business outcomes.
• 3:10
  SPEAKER_03: So I've seen you know data being used for a while in different forms, uh, whether that's people walking in a store or conversations or numbers in a database, and trying to think through how that can be best used to drive business.
• 3:24
  SPEAKER_03: Uh typically people are using data just to make some reports, but not really using it to make decisions that drive their business.
• 3:30
  SPEAKER_01: So you told me uh data governance has become sexy again.
• 3:33
  SPEAKER_01: That is a great line.
• 3:34
  SPEAKER_01: But go ahead and unpack it because most people think generative AI made governance less relevant rather than more relevant.
• 3:41
  SPEAKER_03: I think uh, well, certainly we're in the middle of a transformation.
• 3:45
  SPEAKER_03: I mean, it certainly is a hype cycle around the the latest version of AI, which is these generative models and transformer models.
• 3:52
  SPEAKER_03: But uh, you know, at the end of the day, it's still and it will be transformational despite the hype.
• 3:57
  SPEAKER_03: I mean, for sure it is step-changing a lot of the way we are.
• 4:02
  SPEAKER_03: But at the end of the day, it is based on data and where is that data?
• 4:06
  SPEAKER_03: And do you, you know, where is it located?
• 4:09
  SPEAKER_03: Do does everybody understand what it means?
• 4:12
  SPEAKER_03: Because uh machines are also trying to understand what it means.
• 4:15
  SPEAKER_03: And so in the past, there was a lot of effort around data governance, around like, okay, you know, even in the banking world, these are critical metrics, we need to make sure they're calculated correctly, you know, what is the traceability on all of them?
• 4:27
  SPEAKER_03: And so even and today you can point, you know, some of these powerful tools at your data, but if the data isn't uh clean, then you you'll generate answers, but you're not sure the answers are correct, right?
• 4:39
  SPEAKER_03: And so uh these these uh AI models like Chat GPT and others have a certain a certain knack for always sounding like they're right, but you never know if they're actually right.
• 4:50
  SPEAKER_01: A good consultant, like a good consultant, right?
• 4:52
  SPEAKER_03: Yeah, so it's it's definitely you know the classic problems like data silos or multiple sources of truth or ambiguity about how things connect together, they always existed, they still exist.
• 5:04
  SPEAKER_03: Uh and just because you're po pointing powerful, more powerful engines at it, you otherwise you'll make bad decisions faster, basically.
• 5:11
  SPEAKER_03: And so the need for that basic work is still present.
• 5:16
  SPEAKER_03: And you know, it it's not that it has to be done serially, meaning let's just go clean up all our data and then we'll start using AI, like that one ever, right?
• 5:24
  SPEAKER_03: Uh, you need to take a very use case-driven approach about what are we trying to solve for, what data is needed to solve that, and then kind of clean clean as you go, if you will, right?
• 5:34
  SPEAKER_01: So yeah, no, that makes that makes tremendous sense.
• 5:37
  SPEAKER_01: And talk a little bit about what you run into in terms of classic misconceptions.
• 5:42
  SPEAKER_01: Uh, you know, for for a long time, you know, uh and even now there are some executives that think that the AI will solve the problems and deal with poor data quality.
• 5:52
  SPEAKER_01: And how how have you seen that evolve or change?
• 5:55
  SPEAKER_01: Are you still seeing executives or hearing about executives make that claim?
• 6:00
  SPEAKER_01: Or do you think the industry has matured a little bit beyond that and start to realize, oh yeah, we do really need this, it's not going to solve the problems.
• 6:07
  SPEAKER_01: But what are the misconceptions you've you've come across in your answer?
• 6:10
  SPEAKER_03: Wow, the you know, the misconceptions are, you know, usually start somewhere where it sounds it sounds something like we need to be using some AI.
• 6:20
  SPEAKER_03: Let's just use some AI without really understanding what problem we're actually solving.
• 6:24
  SPEAKER_03: And then just because you point a large language model at a part of your database or the database, doesn't mean it can figure out exactly what's what, right?
• 6:33
  SPEAKER_03: Like there might be four columns called sales.
• 6:36
  SPEAKER_03: How does it know which one you actually meant?
• 6:38
  SPEAKER_03: Because they don't have different numbers in it.
• 6:39
  SPEAKER_03: And, you know, it might be able to guess, you know, based on a lots of reading, lots of code and lots of this, but it's it's hard.
• 6:47
  SPEAKER_03: And, you know, like for example, I when I was at Citibank and then running analytics across multiple countries, everybody had a different definition for active customers.
• 6:56
  SPEAKER_03: And so you couldn't compare a number of active customers against each other because everyone had a different definition of what it meant to be active.
• 7:01
  SPEAKER_03: And until you started actually trying to jam it all together, uh you realized that we need to at least standardize some of these things, right?
• 7:09
  SPEAKER_03: Like, and so you know, executives can say, okay, yeah, just go compute this.
• 7:14
  SPEAKER_03: You can compute something, but it won't, you're not sure it's right, okay, unless you actually, you know, look through how it got to that answer.
• 7:22
  SPEAKER_03: Uh and if there's multiple versions of the truth, then then you're not sure if it picked the right one.
• 7:27
  SPEAKER_03: I mean, the way I the analogy I use is uh data, you know, before it was like data is a new oil, but I call data like blood, right?
• 7:35
  SPEAKER_03: Blood in your body.
• 7:36
  SPEAKER_03: And there are many, you know, the body's like an enterprise, and there are many organs in your body, and each one is a different department, and each one needs the blood for a different purpose, and they do different things with it.
• 7:45
  SPEAKER_03: That's great, but there's only one version of the blood.
• 7:47
  SPEAKER_03: And, you know, you're all sharing it.
• 7:49
  SPEAKER_03: And you're not, uh it's not contaminated, it's not leaking out of your body, it's you know, it's shared across.
• 7:56
  SPEAKER_03: And so it's that that is the mentality that you should take and need to take when it comes to managing what is, you know, you hear the cliche all the time, but what is the most uh valuable enterprise asset?
• 8:08
  SPEAKER_03: That is your data, right?
• 8:09
  SPEAKER_03: So that is the way to start approaching it.
• 8:13
  SPEAKER_01: Yeah, and you know, you you still come across organizations where they'll nod their heads, right?
• 8:18
  SPEAKER_01: The executives will not say, Oh, yeah, data is important, data's really good, data, good data, you know, we want data.
• 8:24
  SPEAKER_01: And then you start to talk about what it's going to cost, and they say, well, it's not that important, but AI is really forcing that function and saying, no, it is, right?
• 8:32
  SPEAKER_01: And you can't.
• 8:32
  SPEAKER_01: And I think you made a great point of saying you really have to be focused on what data do you need for the use case, right?
• 8:39
  SPEAKER_01: Because there will be different sources.
• 8:41
  SPEAKER_01: And and the the other thing to think about is I, you know, I just wrote an article about rather thinking about a source of truth.
• 8:47
  SPEAKER_01: Now you do have to think about a source of truth in certain areas, right?
• 8:50
  SPEAKER_01: You know, you do have to have certain definitions, you do have to have certain reference data, but there's also a source of authority because data will change, right?
• 8:58
  SPEAKER_01: Because especially in unstructured content, you know, the the anal, you know, the story is, you know, you have an engineer that'll create some specifications, you have a marketing person that'll create some specifications, you have a support person that'll create specifications, whose specifications are right.
• 9:13
  SPEAKER_01: Well, it depends on the context, it depends on the the audience, but you do need to you and you do need to adjudicate that, and you do need to bring these different sources together.
• 9:22
  SPEAKER_01: And AI is very good at that, but it's not necessarily going to decide, well, what's the right answer for this context?
• 9:28
  SPEAKER_01: And that needs to be adjudicated by a source of authority, right?
• 9:32
  SPEAKER_01: Somebody needs to be responsible for that.
• 9:34
  SPEAKER_01: So it's it's kind of like they we know that data is going to change as it's transformed.
• 9:38
  SPEAKER_01: As one department does something with it, it's gonna create an output.
• 9:42
  SPEAKER_01: And that output is going to be based on what they needed to do with that.
• 9:46
  SPEAKER_01: So, what what are your thoughts about that?
• 9:48
  SPEAKER_01: That really gets down to kind of process and ownership and decision making.
• 9:52
  SPEAKER_01: So, do you have any thoughts about that?
• 9:55
  SPEAKER_03: Well, you know, I guess my perspective is that the root cause of a lot of these data problems is that people, human behavior is they will use data for their own need and they won't care about data for somebody else's need.
• 10:09
  SPEAKER_03: Right.
• 10:11
  SPEAKER_03: That's right.
• 10:12
  SPEAKER_03: You know, like I gotta, I gotta do this task, I'm gonna do it.
• 10:16
  SPEAKER_03: If I have to make my own little access database on the side to do it, I'll just do that.
• 10:19
  SPEAKER_03: Like, I mean what is figure out a way to do it.
• 10:21
  SPEAKER_03: And, you know, like one analogy I can use again for my banking example.
• 10:25
  SPEAKER_03: Like, I asked somebody, how many loans did we sell yesterday in Malaysia?
• 10:30
  SPEAKER_03: And they're like, Well, apparently the hard word in that sentence was the word yesterday because they only got monthly reports.
• 10:36
  SPEAKER_03: I'm like, all right, fine.
• 10:37
  SPEAKER_03: So go find out.
• 10:37
  SPEAKER_03: And they got the number and like, great, now give it to me by time of day.
• 10:40
  SPEAKER_03: Like, what do you mean time of day?
• 10:42
  SPEAKER_03: Like, it doesn't matter what time of day it was for the salesperson's commission.
• 10:44
  SPEAKER_03: Like, it doesn't like it's irrelevant.
• 10:46
  SPEAKER_03: I'm like, yeah, but marketing cares and call center cares and all these other people care.
• 10:50
  SPEAKER_03: Like, don't don't you think beyond your own silo, right?
• 10:53
  SPEAKER_03: Um, and so it that that is the core part of the problem because then people start inventing their own version of the number to do their task, and then the definitions are different.
• 11:04
  SPEAKER_03: Not that each person is wrong, they're just different, right?
• 11:06
  SPEAKER_03: Um, because they did it for different purposes.
• 11:09
  SPEAKER_03: So the context of why you're gonna need it matters.
• 11:12
  SPEAKER_03: And the power of AI is connecting across domains.
• 11:15
  SPEAKER_03: Okay, if you're trying to solve within one domain, okay, it's fine.
• 11:18
  SPEAKER_03: Though you'll get some value.
• 11:19
  SPEAKER_03: But the power that AI has, and these the recent, when I say AI, I mean like the recent transformation, uh transformer models, you can start looking across areas of your organization.
• 11:28
  SPEAKER_03: Like, how does this call center behavior compare to the person's login behavior on my website?
• 11:33
  SPEAKER_03: Like, okay, like people, you can start asking questions like this if you can start connecting these dots.
• 11:37
  SPEAKER_03: And if you suddenly have different versions of what a customer is or different versions of the truth, it will be hard to do these kinds of things.
• 11:44
  SPEAKER_03: And so that's that's that I think that is really the problem.
• 11:48
  SPEAKER_03: People are just so focused on their own area of the business, and that's how you run into these data silos and different definitions and things like that.
• 11:56
  SPEAKER_01: So you really need to think about the data lineage and the data provenance, and you need to think about the value chain as that data goes through the organization and is transformed.
• 12:06
  SPEAKER_01: And then, hey, maybe it's not going to go back into the main flow of of data.
• 12:11
  SPEAKER_01: It is going to kind of dead end in this process, right?
• 12:13
  SPEAKER_01: In other words, yeah, you're solving your problem.
• 12:16
  SPEAKER_01: But but one of the things that you know you talked about was that you know, many times you you your your scenarios when when when you have data that's not correct or when the AI is making a mistake, the more dangerous scenarios when it uh when it is not when it fails visibly, right?
• 12:33
  SPEAKER_01: It's when it fails silently, when it starts generating answers that seem plausible, but people don't know what that they are wrong.
• 12:41
  SPEAKER_01: So talk a little bit about that.
• 12:42
  SPEAKER_01: And what does that look like in practice?
• 12:44
  SPEAKER_03: Yeah, I mean, this, you know, this gets back to what maybe why people are talking about how a lot of AI pilots they fail, oh, you know, when they go into production.
• 12:53
  SPEAKER_03: Because look, you can you can try something with a very curated set, like, oh no, no, I'll just clean up these Excel columns and then I'll upload it and then we'll show that that works.
• 13:03
  SPEAKER_03: And okay, fine, it works, right?
• 13:05
  SPEAKER_03: But then once you start putting it in production with a whole bunch of dirty data flowing around and things like that, it will still operate the algorithm on the data you provide.
• 13:14
  SPEAKER_03: It will still generate an answer unless something catastrophic happens, like, oh, the entire column is missing, in which case it's actually a good thing because then something broke, right?
• 13:22
  SPEAKER_03: But if everything looks like it's present, it will generate answers.
• 13:26
  SPEAKER_03: And you're not, you won't be sure that it's correct, right?
• 13:29
  SPEAKER_03: And that is the worst scenario, actually, when handling data, is that it if it breaks entirely, somebody will go investigate it to understand why.
• 13:37
  SPEAKER_03: If it's off by 30%, who's gonna notice, right?
• 13:40
  SPEAKER_03: Like I mean, and so what how is that gonna get figured out?
• 13:44
  SPEAKER_03: And that's an important discipline most organizations don't have is they'll launch something, pat themselves on the back, put it on autopilot, and then wonder why is nothing correct is happening, right?
• 13:55
  SPEAKER_01: So yeah, and I and I I've seen a couple of examples of that of both model drift and semantic drift, right?
• 14:04
  SPEAKER_01: So semantic drift is when a definition will change when you have a new term that was not part of your original lexicon, right?
• 14:11
  SPEAKER_01: And so the meaning when you change the definition of customer, when you change your billing cycle, you change something that that materially impacts that that that the meaning of that data, right?
• 14:22
  SPEAKER_03: Yeah, but in the in the in the world of healthcare, there's always new regulations coming out from Medicare or you know, CMS and things like that.
• 14:29
  SPEAKER_03: And they're changing what it means to be a high-risk, whatever, or changing, you know, what this means.
• 14:36
  SPEAKER_03: If you don't update the meaning of the data, you know, then I mean there's the data itself, but then there's the meaning of the data, right?
• 14:42
  SPEAKER_03: And so both are needed.
• 14:44
  SPEAKER_01: Yeah, absolutely.
• 14:45
  SPEAKER_01: And and and that has to be part of governance, and that has to be part of uh what what travels with the data.
• 14:50
  SPEAKER_01: I'm also thinking about model drift.
• 14:53
  SPEAKER_01: When when uh I think I told you the story, and I might have mentioned on the podcast previously, but when I was producing reports, we do these uh analyses of you know different types of situations, and one was for looking at the retrieval of of content on a website.
• 15:08
  SPEAKER_01: And I started noticing very subtly that the that the formatting was a little bit off.
• 15:14
  SPEAKER_01: And then I noticed that it was a lot off.
• 15:17
  SPEAKER_01: And then I realized, wait a minute, you're not following the process here, you're not following the rule about formatting.
• 15:22
  SPEAKER_01: And when I questioned the AI, I said, Why didn't you format it correctly?
• 15:26
  SPEAKER_01: He said, I skipped it.
• 15:28
  SPEAKER_01: You skipped it.
• 15:28
  SPEAKER_01: What do you mean you skipped it?
• 15:29
  SPEAKER_01: That's part of your rule, that's part of the the role.
• 15:31
  SPEAKER_01: He goes, Well, I got absorbed in the problem.
• 15:34
  SPEAKER_01: That's what the AI said.
• 15:35
  SPEAKER_01: I got absorbed in the problem.
• 15:36
  SPEAKER_01: And the analogy is if you created a one, some great report, some great analytic report, and I said, So you didn't format this.
• 15:44
  SPEAKER_01: You'd be like, Seth, what are you crazy?
• 15:45
  SPEAKER_01: You know, it formatting's not that important.
• 15:47
  SPEAKER_01: It's it's the answer, the quality of the answer is there.
• 15:50
  SPEAKER_01: That was so interesting because you know, I said, Well, you're telling me I can give you a rule and you may not follow it.
• 15:55
  SPEAKER_01: And it said, Yeah, pretty much, because it was statistically less relevant to the problem at hand.
• 16:01
  SPEAKER_01: And so you start to see these things drift, and that can be very subtle where people don't notice it until it is wrong, right?
• 16:09
  SPEAKER_01: So, so that's where so how do you address that?
• 16:12
  SPEAKER_01: I mean, that's kind of a needing an LLM to check on an LLM.
• 16:16
  SPEAKER_03: This is the you know, the it falls in the same umbrella of governance, right?
• 16:20
  SPEAKER_03: Like, what are you doing to keep track of what's going on, right?
• 16:24
  SPEAKER_03: And people, you know, it let's say before even the Z LLM days, people would like launch marketing campaigns and and you know, congratulate themselves, and then they would wonder if it's working or not, and they're like, oh, they forgot to think about how to measure it, right?
• 16:39
  SPEAKER_03: And just like when you're out launching LLM solutions, do you if you did you even think in advance about how you're going to continually check if it's right?
• 16:46
  SPEAKER_03: Like it doesn't have to be, you know, daily, obviously, but like what do you even have a process by which you're going to try to revalidate what's going on?
• 16:55
  SPEAKER_03: And my guess is most people will not, right?
• 16:57
  SPEAKER_03: And and then you'll be caught with your pants down sometime in the future where it's not doing what you thought, and you will wonder to yourself when it went off the rails, right?
• 17:06
  SPEAKER_03: So uh because data changes, yeah, semantics change, things change, right?
• 17:10
  SPEAKER_03: The environment we live in is not a bubble and constant, you know, the economy changes, like the decisions that you made in high interest rate environments are not the same decisions you would make in a low interest rate environment, but unless you kept updating things, right?
• 17:23
  SPEAKER_03: Like you how are you, you know, what are you going to do to keep it up to date, right?
• 17:27
  SPEAKER_01: So yeah, and and it it it is interesting because when you start looking at these the experiments that organizations are running, right, they're trying different mechanisms, they're trying to improve retrieval.
• 17:39
  SPEAKER_01: They may use a different vendor, a different model, a different uh ingestion approach, a different uh taxonomy, a different content model.
• 17:47
  SPEAKER_01: And and what's difficult is getting an apples to apples comparison across those environments, something that we're helping organizations with.
• 17:55
  SPEAKER_01: Give them the you know, you have that gold set of truth, right?
• 17:58
  SPEAKER_01: The golden set of questions and answers, and then have multiple as multiple variables to any retrieval environment.
• 18:05
  SPEAKER_01: And then you have to kind of you know stack them up in an equivalent way in order to in order to show the consistency.
• 18:12
  SPEAKER_01: But I agree with you that you know there's there's so many different variables here that that it's difficult to get that apples to apples comparison.
• 18:19
  SPEAKER_01: It's also difficult to monitor over time.
• 18:22
  SPEAKER_03: Yeah.
• 18:22
  SPEAKER_03: I think you know, the models themselves, my perspective is they're moving towards commoditization, right?
• 18:28
  SPEAKER_03: They're they're ingesting all of the same data and trying to get it all.
• 18:31
  SPEAKER_03: And like, okay, there's times when some are a little bit more better at this taste than this test, but it's moving towards similarity.
• 18:39
  SPEAKER_03: The differentiating factor will become how well have you, you know, yes, you got to clean up your data, but I will argue even cleaning up your data is not enough.
• 18:48
  SPEAKER_03: You need to be able to the the models need to understand your business.
• 18:52
  SPEAKER_03: I mean, imagine hiring, you know, some whiz bang analyst off the street, and you know, she knows how to query data like nobody's business and and other, but then you're like, okay, go manage my supply chain.
• 19:04
  SPEAKER_03: You're like, what?
• 19:05
  SPEAKER_03: Like I I I I know how to write queries, but I I don't know how to run your supply chain, right?
• 19:10
  SPEAKER_03: Like, and where's this business knowledge?
• 19:11
  SPEAKER_03: And you have to train this employee, like, okay, this is how you handle VIP customers, and this is how you handle international shipments, and this is how you handle this, and all of that knowledge is somewhere, and it's usually in Bobby's head or Susie's head or somebody else's head, and you have to go get it out of their head and get trained, right?
• 19:27
  SPEAKER_03: And so uh, and when you're working with an LLM, you can give it all the data you want, and it can be pristine, but if you don't tell it the context around the way to use that data, that's going to be the next wave of problems, right?
• 19:38
  SPEAKER_03: Um to try to solve.
• 19:40
  SPEAKER_03: And that's what organizations' real secret sauces.
• 19:43
  SPEAKER_03: They say your, you know, your data is your enterprise asset, which it is, but it's also the way you run your business that is also your asset, right?
• 19:51
  SPEAKER_03: And that is typically captured loosely in documents, maybe some Slack messages, emails here or there, or maybe not even captured anywhere, right?
• 20:00
  SPEAKER_03: Uh, it's just in people's heads and tribal knowledge.
• 20:03
  SPEAKER_03: That is what is going to start differentiating things because yes, you can you can use these LLMs for simple efficiency gains, and it does get you in efficiency gains, like write my code for me and do this.
• 20:14
  SPEAKER_03: But the real value will be on helping you actually make decisions that help you run your business, right?
• 20:20
  SPEAKER_03: And that is not always captured in just data.
• 20:24
  SPEAKER_01: So right.
• 20:24
  SPEAKER_01: That's a really good point.
• 20:25
  SPEAKER_01: And I think that it's that tacit knowledge that people come into the office knowing about the business.
• 20:31
  SPEAKER_01: And it's and it's that collective wisdom and institutional knowledge of how you do things.
• 20:38
  SPEAKER_01: And then a large language model isn't going to have that publicly, right?
• 20:41
  SPEAKER_01: Because that's your IP, that's how you differentiate, that's how you deliver your services differently than another company in your exact same industry serving your exact same customers.
• 20:52
  SPEAKER_01: There are differences in how you do the job and how you service that market.
• 20:56
  SPEAKER_01: And that is based on that that that context of how this organization does it, that, that, that secret sauce, right?
• 21:04
  SPEAKER_01: Which is knowledge in people's heads, but also knowledge that's written down, knowledge that's embedded in systems and processes, in Slack groups, as you say, you know, and email communication.
• 21:15
  SPEAKER_01: So that's the challenge is how do you codify that, right?
• 21:18
  SPEAKER_01: How do you capture that and codify it, harvest it, capture it, codify it, and then apply that as context as business problems arise?
• 21:25
  SPEAKER_03: Yeah.
• 21:25
  SPEAKER_03: I was at the Snowflake conference uh earlier this summer, and you know, before maybe a year ago, it was all about agent, agent, agent.
• 21:32
  SPEAKER_03: This year is about context, context, context, right?
• 21:35
  SPEAKER_03: Like because agents can go act, but if they don't have the context in which to act, you know, what what are they going to do?
• 21:40
  SPEAKER_03: I mean, like I still revert to the same analogy.
• 21:43
  SPEAKER_03: Like you can get brand new super smart people off the streets, they don't have enough information.
• 21:47
  SPEAKER_03: They you still need to train them, right?
• 21:49
  SPEAKER_03: And and in the same way, you need to give a model more information than just data.
• 21:55
  SPEAKER_03: And you that those rules, like, okay, VIP customers, VIP customers get Five days to pay and everyone else 30 days.
• 22:02
  SPEAKER_03: It's probably coded in some system somewhere, right?
• 22:05
  SPEAKER_03: And which is fine.
• 22:06
  SPEAKER_03: And that code runs and it operates.
• 22:08
  SPEAKER_03: But that information is locked into that one piece of code.
• 22:11
  SPEAKER_03: It's not disseminated across the organization where everybody can take advantage of that knowledge.
• 22:16
  SPEAKER_03: And then, you know, customer service and marketing and everybody else can start using the same rule instead of recoding the same rule over and over and over.
• 22:24
  SPEAKER_03: And so that platforms that are going to allow you to start capturing knowledge about your business context, ontology, whatever, you know, the words that you want to use, that's going to be the next wave of what we're going to see.
• 22:36
  SPEAKER_03: That's my opinion.
• 22:37
  SPEAKER_01: So yeah, I like to say that the ontology kind of codifies the knowledge relationships in an organization.
• 22:44
  SPEAKER_01: Right.
• 22:45
  SPEAKER_01: What are the things that are important?
• 22:47
  SPEAKER_01: If you had a you know, a reference librarian will know, you know, if you want to uh if you want to learn about, you know, uh a particular historical figure, you know, here's the book you want to read about the War of 1812, if you want to know about, you know, that that uh Jackson, for example, right?
• 23:02
  SPEAKER_01: Or if you want to know about some, it's not necessarily in the title, it's not necessarily obvious, but it's conceptually related.
• 23:09
  SPEAKER_01: So for consultancy, it's what are the methodologies that get you this outcome for this particular industry?
• 23:15
  SPEAKER_01: Well, that sometimes is in the head of a senior consultant and not necessarily in the head of or in the or written down someplace.
• 23:22
  SPEAKER_01: And so that's what you have to do is capture those relationships, and that's where you start building that knowledge network or that knowledge scaffolding in the organization.
• 23:30
  SPEAKER_01: We talked about that a little bit.
• 23:32
  SPEAKER_01: Do you think that that buzzword of ontology or knowledge architect or information, do you think that's getting through, or people tossing that term around, or or what what are your thoughts about that?
• 23:42
  SPEAKER_01: How how important is that how much awareness is there?
• 23:46
  SPEAKER_03: I think okay, the awareness is definitely building, just like people are now using agent in sentences, right?
• 23:52
  SPEAKER_03: Uh, and so I think people will start using ontology or context or whatever uh more and more.
• 23:57
  SPEAKER_03: It's it's it is a critical piece of information.
• 24:00
  SPEAKER_03: I mean, you know, okay, there's two large hurdles to overcome.
• 24:04
  SPEAKER_03: One, all these external models are trained on public data.
• 24:07
  SPEAKER_03: Yep.
• 24:08
  SPEAKER_03: And they're scanning the Library of Congress and they're doing whatever they can to get their hands on whatever data they can get their hands on, right?
• 24:13
  SPEAKER_03: I mean, you saw in the headlines the other day, I think Google bought Spirit Airlines data.
• 24:19
  SPEAKER_03: Like, I mean, they're not running an airline, but they just bought your data, right?
• 24:21
  SPEAKER_03: That's right.
• 24:22
  SPEAKER_03: And so that that's just an indication of company data internal to your organization is, first of all, a lot bigger than public data collectively, right?
• 24:32
  SPEAKER_03: And more valuable, right?
• 24:34
  SPEAKER_03: And so how are you take cleaning and then taking advantage of your own internal data?
• 24:38
  SPEAKER_03: And how are you taking advantage of your institutional knowledge, right?
• 24:43
  SPEAKER_03: And not hard coding that knowledge one task at a time, but making it available to everybody so that everybody can scale, right?
• 24:50
  SPEAKER_03: Oh, and so both of those things I think it's people will start to realize the this the need for the second.
• 24:57
  SPEAKER_03: I think a lot of people are worried about okay, do I have the data right?
• 25:01
  SPEAKER_03: Or instead of using the right data, and they'll use rag models and okay, fine, I'll give it my proprietary data and it'll give me the right.
• 25:07
  SPEAKER_03: Okay, fine, you're making progress, but you're still having to tell the model how to use the data every time almost, right?
• 25:14
  SPEAKER_03: Like, you know, like you see sometimes people posting their prompts and there's super long prompts of like, okay, okay, I'm this type of person, and okay, I've had this much experience, and well, don't forget about this, and don't forget about that.
• 25:26
  SPEAKER_03: And all you just have to give it all this information.
• 25:28
  SPEAKER_03: Imagine having to do that every single time you want something done, right?
• 25:31
  SPEAKER_03: Like, can't you just put that somewhere and have it captured?
• 25:34
  SPEAKER_03: And so that is what we are starting to move towards, and I think people are starting to recognize that.
• 25:40
  SPEAKER_01: So yeah, yeah.
• 25:41
  SPEAKER_01: We we uh spend a lot of time uh using the LLMs to harvest our decades of knowledge in our repositories.
• 25:49
  SPEAKER_01: And you know, the other day I was coming up with some best practices for taxonomy design, and it referenced uh a paper or a framework that was written in 2012, and it still was valid, and it said this is the best practice, this is what you have.
• 26:02
  SPEAKER_01: But we have you know annotated work that uh that I've forgotten about, right?
• 26:08
  SPEAKER_01: And then the engine is able to find it and then collate it and then put it into a repository.
• 26:14
  SPEAKER_01: And then that's what we use to provision our projects, right?
• 26:17
  SPEAKER_01: So that they're always getting the right context for that particular now.
• 26:20
  SPEAKER_01: There's all sorts of problems with that still, because you still can fragment your knowledge and there's no perfect solution there, but but it but it is a way of kind of grabbing that institutional knowledge and the knowledge that has walked out that door.
• 26:32
  SPEAKER_01: You know, somebody caught somebody captured it, you know, 14 years ago.
• 26:37
  SPEAKER_01: I long forgot about it.
• 26:38
  SPEAKER_01: I didn't know it existed, but but the the system was able to pull it up and then and apply it.
• 26:44
  SPEAKER_01: So let's talk a little bit about the idea of an information product or data as a product.
• 26:49
  SPEAKER_01: Uh you you approach data with a product management mindset rather than technology mindset.
• 26:53
  SPEAKER_01: What does that mean in practice?
• 26:54
  SPEAKER_01: And how does that change the way data teams work with the rest of the business?
• 26:59
  SPEAKER_03: I think you know, when it comes to the world of data and analytics, you can do most anything once, right?
• 27:06
  SPEAKER_03: Like, I need you to go figure out how to compute the something, something for this customer and or their lifetime value or their whatever, and go off and do your little research project and come back with the answer and do your PowerPoint presentation.
• 27:19
  SPEAKER_03: But then you're typically asked, okay, that was great.
• 27:22
  SPEAKER_03: Now I want you to recompute it every single day.
• 27:24
  SPEAKER_03: You're like, well, whoa, okay, wait a minute.
• 27:25
  SPEAKER_03: I didn't I didn't think about that, right?
• 27:27
  SPEAKER_03: And so when I say manage it like a product, it's like, who are your customers?
• 27:32
  SPEAKER_03: What are the features and value propositions that they want?
• 27:35
  SPEAKER_03: You know, why do they want this?
• 27:37
  SPEAKER_03: And uh your product has uh is repeatable, it's scalable, it's iterative.
• 27:42
  SPEAKER_03: Like, okay, I'm gonna compute lifetime value using these three inputs, but next time I'm gonna do the version two of this and it's gonna be faster, or version two of it, it's gonna have more inputs, or version two of it's gonna have whatever, right?
• 27:54
  SPEAKER_03: And so you think about iterating and constantly developing what you've created as opposed to treating everything as a one-off you know project, if you will.
• 28:02
  SPEAKER_03: And so this is the classic difference between what is a project and what is a product, right?
• 28:06
  SPEAKER_03: And so if it's a product, you want to do it again and again, and you want to listen to your customers, and you want to get feedback, and you want to make sure that you're delivering a value, value to them, right?
• 28:17
  SPEAKER_03: And so that's that's the mindset I think that needs to get adopted.
• 28:22
  SPEAKER_01: So yeah, and you have a and you have an owner, there's a product owner, and there's a budget to maintain that product, and there's specifications and there are needs of the users and their scenarios.
• 28:30
  SPEAKER_01: And just like you said, building the use cases, that those use cases say, here's the information product that's going to come in as an input to this process.
• 28:39
  SPEAKER_03: Like, for example, go to some organization that has you know a thousand dashboards, which is typical EB Store, pick one and say, Who's the owner of this dashboard and see if you get an answer, right?
• 28:48
  SPEAKER_03: You're like, oh yeah, uh, I don't know, that person left a while ago.
• 28:52
  SPEAKER_03: Yeah, yeah.
• 28:53
  SPEAKER_01: That's right.
• 28:53
  SPEAKER_01: And that's why there's a new dashboard that's created that replicates the other dashboards, right?
• 28:58
  SPEAKER_01: Yeah.
• 28:58
  SPEAKER_03: Um so you need an owner that says, I am going to make this better and better.
• 29:04
  SPEAKER_03: And I track, like, for example, I did also a bunch of dashboards.
• 29:07
  SPEAKER_03: I tracked value by usage.
• 29:09
  SPEAKER_03: If nobody used it, we just deleted it, right?
• 29:11
  SPEAKER_03: Like, I mean, it was as simple as that, right?
• 29:13
  SPEAKER_03: I and people who like your product use your product.
• 29:17
  SPEAKER_03: Simple as that, right?
• 29:18
  SPEAKER_01: So and then you have continuity, you have a handoff, right?
• 29:21
  SPEAKER_01: When when people have a portfolio of products, uh, they don't just get abandoned and orphaned, they get passed on to the next person, and it's becomes part of the whole uh continuity process and the whole data governance process.
• 29:34
  SPEAKER_01: Yeah, no, it makes a lot of sense.
• 29:36
  SPEAKER_02: Yeah, yeah.
• 29:37
  SPEAKER_01: So one of the stories uh that we talked about was that there's a pro there's a team member who built a program years ago that saved the company tens of millions of dollars.
• 29:47
  SPEAKER_01: And it was and it was had to do with structuring content and and be able to standardize that information and made it into a Harvard Business Review article.
• 29:58
  SPEAKER_01: And now with this new wave of AI work, sometimes he doesn't necessarily have a seat at the table.
• 30:04
  SPEAKER_01: And uh and it's very interesting how the voices are the loudest voices are being heard that may not have the portfolio of success.
• 30:12
  SPEAKER_01: What are your thoughts about that?
• 30:14
  SPEAKER_01: Because again, if you say here's somebody that solved the problem, well, you I would give that person a seat at the table and and listen to what they say, but sometimes that's not happening.
• 30:23
  SPEAKER_01: The having the answer isn't always uh helpful.
• 30:25
  SPEAKER_01: What are your thoughts about that in politics?
• 30:27
  SPEAKER_03: You know, my my what I told my team when it comes to the seat of the table analogy is like if you don't have a seat at the table, just bring a folding chair.
• 30:36
  SPEAKER_03: Like you need to show up, right?
• 30:38
  SPEAKER_03: Uh and just you need to be stand in the room.
• 30:41
  SPEAKER_01: Yeah.
• 30:41
  SPEAKER_03: That's right.
• 30:42
  SPEAKER_03: Because the the bad version of operating is there's some people who are, you know, solving a problem, they think they got to the solution, they throw it over the wall and say, okay, just build this for me, or do this for me, or query this data for me, and just give me the answer, right?
• 30:57
  SPEAKER_03: And without the person on the receiving end really understanding the why or the context, or like, you know, or the you know, this might be really hard, but if we just did it this other way, it'd be a lot easier.
• 31:09
  SPEAKER_03: But I don't know what you're trying to do.
• 31:11
  SPEAKER_03: So I okay, fine.
• 31:12
  SPEAKER_03: It's gonna take me three weeks instead of three days.
• 31:15
  SPEAKER_03: Okay, fine, right?
• 31:16
  SPEAKER_03: Uh and so getting in getting the different functional experts together at the same time is actually useful.
• 31:23
  SPEAKER_03: Like, you know, having HR at the table is useful, right?
• 31:26
  SPEAKER_03: Just like having your data analytics expert at the table is useful.
• 31:29
  SPEAKER_03: Like having everybody there trying to understand what you're trying to solve for actually leads to better outcomes, honestly, right?
• 31:36
  SPEAKER_03: Because different perspectives and can provide different approaches to solving the problem as to, you know, uh like it's like working with lawyers.
• 31:44
  SPEAKER_03: Okay, I need you to write me a contract to do the following, blah, blah, blah, just to, you know, just do it.
• 31:48
  SPEAKER_03: Like, okay, what are you doing again?
• 31:50
  SPEAKER_03: Like, you know, so like you you just need you you need to really get everyone together.
• 31:56
  SPEAKER_03: And so that's and so, and like your expert that you talked about, they have expertise in a certain area and they can help you solve the problem in maybe a way you didn't think about solving, right?
• 32:06
  SPEAKER_03: And so that's where the best ideas come from is from cross-functional diversity, right?
• 32:10
  SPEAKER_03: Yeah, diversity of thought, you know, is critical in an organization.
• 32:14
  SPEAKER_03: That's my opinion.
• 32:15
  SPEAKER_01: And sometimes the quiet, methodical, you know, players that solve the problems in the background don't get the voice that they need.
• 32:23
  SPEAKER_01: And the people who are have have a louder voice, but unencumbered by expertise and knowledge.
• 32:31
  SPEAKER_03: No, you're just slowing the process by not laying all the cards at the table at the same time.
• 32:35
  SPEAKER_03: Like, okay, do this.
• 32:36
  SPEAKER_03: Okay, now that wasn't quite right.
• 32:38
  SPEAKER_03: Could you do this?
• 32:39
  SPEAKER_03: Do this, like, okay, what are you trying to do again?
• 32:42
  SPEAKER_03: So, like, why don't you start with that, right?
• 32:44
  SPEAKER_03: So yeah, yeah.
• 32:45
  SPEAKER_01: So you had built personalization and engines, some of the largest consumer companies in the world.
• 32:51
  SPEAKER_01: What did generative AI unlock that was not possible with predictive models?
• 32:55
  SPEAKER_01: And what does getting it right actually require today?
• 32:58
  SPEAKER_01: So what carries on?
• 33:00
  SPEAKER_01: What what what what still applies, right?
• 33:03
  SPEAKER_01: And because we talked about the mad libs approach, which I think is still valid.
• 33:07
  SPEAKER_01: Now it's filling in the gaps and the blanks in those mad libs, right?
• 33:12
  SPEAKER_03: That's right.
• 33:12
  SPEAKER_03: Like you have a template, like an email template with seven boxes, and you can put anything you want in those seven boxes.
• 33:17
  SPEAKER_03: What do you put for you?
• 33:18
  SPEAKER_03: Right.
• 33:18
  SPEAKER_03: Uh, and so it it is about trying to under at the end of the day, understand each customer, not at like the segment level where you're still grouping people together, like all Medicare patients are the same.
• 33:30
  SPEAKER_03: No, they're not, right?
• 33:31
  SPEAKER_03: Uh, and so how do you get down to individual behavior?
• 33:35
  SPEAKER_03: And that's been going on for a long time.
• 33:37
  SPEAKER_03: Like, okay, I'm gonna build a predictive model to try to predict your likelihood to respond to this coupon.
• 33:42
  SPEAKER_03: So I'm gonna target who I send this email to.
• 33:45
  SPEAKER_03: And okay, fine, that's been going on a long time.
• 33:47
  SPEAKER_03: Or direct mail, which households do you mail this thing to?
• 33:50
  SPEAKER_03: You don't mail it to every house in America, right?
• 33:52
  SPEAKER_03: You direct mail to certain people.
• 33:54
  SPEAKER_03: That's all based on predictive models and AI.
• 33:56
  SPEAKER_03: And so now with what has happened in recent versions with personalization, there's an opportunity, opportunity to do personalization on steroids because instead of doing my 12 variable predictive model with my random forest methodology, I can do 12,000 variables, right?
• 34:14
  SPEAKER_03: Like, okay, whoa.
• 34:16
  SPEAKER_03: Like I can talk, I can look at you from so many different perspectives from the you know, your Facebook behavior to your, you know, magazine subscriptions to your, you know, the current last car you purchased to what I got, I don't know what like you can just start bundling all kinds of data in and the model can start predicting off of that.
• 34:33
  SPEAKER_03: So one is volume of data, right?
• 34:35
  SPEAKER_03: Uh and two is speed of update, right?
• 34:39
  SPEAKER_03: And so the ability to compute the answer as opposed to I'm going to refresh my propensity score once a month.
• 34:45
  SPEAKER_03: You know, you can refresh it every few seconds if you want to, right?
• 34:48
  SPEAKER_03: Um, because things are changing in in real time and you can getting that real time context.
• 34:52
  SPEAKER_03: And I think the third thing that's different also is the ability to generate multiple variations very quickly of what to show somebody, right?
• 35:00
  SPEAKER_03: Um like, okay, I've got the seven box email template, and in box number one, I have four choices of what to show.
• 35:07
  SPEAKER_03: I can show a picture of a single person or a person with a family or a person of this race or a person of like whatever.
• 35:12
  SPEAKER_03: Like you can, but now there's almost almost an infinite number of variations of what to show, right?
• 35:18
  SPEAKER_03: Um and so you can generate a vast variety of things that can be even more tailored uh as opposed to sort of picking out the out of the box, right, of which one to do, which is better than just doing the same for everybody, no doubt.
• 35:31
  SPEAKER_03: But uh there's an opportunity to do much better, right?
• 35:34
  SPEAKER_03: And so I think it's a combination of that more input, more velocity, more you know, variations, I'll call it.
• 35:40
  SPEAKER_03: That is can be uh leveraged even more for personalization in this day and age, right?
• 35:46
  SPEAKER_01: Yeah, and I think about how well Claude knows me, right, based on my patterns and that data.
• 35:51
  SPEAKER_01: Oh my goodness, that's way beyond uh what organizations are capturing today.
• 35:55
  SPEAKER_01: I mean, it's it's capturing your your thinking, your mindset, your patterns of of of thought, your your behaviors.
• 36:02
  SPEAKER_03: It's really incredibly and it's and it's getting better and every time you use Claude, in theory, it's getting better and better because it's learning more and more about you, right?
• 36:12
  SPEAKER_03: And so it that's an example of more data about, right?
• 36:15
  SPEAKER_03: And so, yeah.
• 36:16
  SPEAKER_01: I uh I actually asked Claude to describe my writing style and my thinking style, and it came up with all of these dimensions that I had to ask what the definitions were, right?
• 36:26
  SPEAKER_01: And it gave me the definitions and it gave me and it gave me all sorts of like temporal density, right?
• 36:31
  SPEAKER_01: Whatever temple, you're right, what it were all these strange terms, but it had a definition.
• 36:36
  SPEAKER_01: And then I started realizing I said, Well, you're putting this into human language, but it's really mathematical, it's a construct, and and so it's lossy.
• 36:44
  SPEAKER_01: So you lose the fidelity when you're taking multi-dimensions, thousands, now dimensions, and you're projecting it onto something that's more flat, which are English facets, right?
• 36:54
  SPEAKER_01: English language descriptors and attributes.
• 36:57
  SPEAKER_01: But really, it's it's much more complex than that.
• 37:00
  SPEAKER_01: And it defies translation into language because it is a mathematical construct that you have to interpret.
• 37:06
  SPEAKER_01: So if you can act on those mathematical constructs, right, which which organizations will be building, they'll be building those models of their consumers and their customers based on all of those interactions and based on the subtleties that the large language models can derive and and and and be able to apply uh again without necessarily having that that language translation, which is which loses that fidelity.
• 37:31
  SPEAKER_01: It's very interesting when you think about it for a moment.
• 37:33
  SPEAKER_03: Oh, it's an interesting problem for sure.
• 37:35
  SPEAKER_03: Like, I mean, if you're trying to personalize an email message and you're like, well, I could write this message in a very formal tone to you, I could write it in a very casual tone to you, which one are you going to better respond to?
• 37:44
  SPEAKER_03: And by the way, what is the definition of what is formal and what is casual?
• 37:48
  SPEAKER_03: And that's a mathematical construct, as you said, right?
• 37:50
  SPEAKER_03: Like, how does the machine know this is a casual paragraph and this is a formal paragraph?
• 37:54
  SPEAKER_03: It's somehow encoded, right?
• 37:55
  SPEAKER_03: Uh in a series of vectors of the choice of words used and whatever else, right?
• 37:59
  SPEAKER_01: And so it's formal to me, may not be formal to you, yeah.
• 38:04
  SPEAKER_01: That's right.
• 38:04
  SPEAKER_01: That's right.
• 38:05
  SPEAKER_01: Interesting.
• 38:06
  SPEAKER_01: So let's kind of summarize some of your thoughts when when you what should executives do differently?
• 38:11
  SPEAKER_01: So senior leader navigating vendor claims, technical terms, business takeholders, all pulling in different directions.
• 38:17
  SPEAKER_01: What are some of the most important things to get right before deploying AI at scale, or as they're deploying AI, as they're looking at vendor claims?
• 38:25
  SPEAKER_01: What are your what are your thoughts?
• 38:26
  SPEAKER_01: What what what do they need to do differently?
• 38:28
  SPEAKER_03: I think, you know, there are many vendors knocking at everybody's doorstep saying, I can solve this problem for you.
• 38:34
  SPEAKER_03: And then the next sentence will be give me all your data.
• 38:36
  SPEAKER_03: Like, okay, hold on, right?
• 38:37
  SPEAKER_03: And so so it's you know, the pro I would think about okay, let's see if I can frame this.
• 38:44
  SPEAKER_03: The easiest place to start uh with using some of these AI tools is productivity improvements, right?
• 38:50
  SPEAKER_03: Like, hey, I got call center agents writing notes after a phone call.
• 38:54
  SPEAKER_03: Wouldn't it be nice if they could automatically generate some notes and save some time for them?
• 38:58
  SPEAKER_03: Right, right.
• 38:59
  SPEAKER_03: Or I've got engineers taking five days to write this piece of code.
• 39:02
  SPEAKER_03: You could do it in five minutes if I leveraged you know AI.
• 39:05
  SPEAKER_03: Okay, great.
• 39:06
  SPEAKER_03: And you will get benefits from doing that for sure.
• 39:08
  SPEAKER_03: Uh and you should start there because it will start demonstrating value relatively quickly.
• 39:14
  SPEAKER_03: But the real value of AI is not getting process improvements, it is thinking about business change, business process change.
• 39:22
  SPEAKER_03: Like, for example, on that call example, yes, you can automate the notes after a phone call.
• 39:27
  SPEAKER_03: Have you thought about why you're taking phone calls at all?
• 39:30
  SPEAKER_03: Right?
• 39:30
  SPEAKER_03: Like, have you thought about that?
• 39:31
  SPEAKER_03: Right?
• 39:32
  SPEAKER_02: I haven't thought about that.
• 39:33
  SPEAKER_03: So and so that that's okay, you'll get there eventually, but start with that and be very careful of trying to solve very point problems at the same like I'm gonna use AI and I'm gonna solve this problem, and then when I get to the next problem, I'll use a different vendor, and when I get to the next problem, I use a different vendor, right?
• 39:49
  SPEAKER_03: Like that's gonna create a large mess on your hands.
• 39:52
  SPEAKER_03: And you think about the sort of a platform perspective like what is the suite of capabilities that I'm looking for so that you at least have a roadmap as opposed to throw use case darts at the wall, right?
• 40:03
  SPEAKER_03: Uh and try to do one at a time.
• 40:05
  SPEAKER_03: So that's a great point.
• 40:06
  SPEAKER_03: That that's the way I think about it, right?
• 40:08
  SPEAKER_01: So yeah, no, it's a great point.
• 40:09
  SPEAKER_01: It's you're solving classes of problem that may be comprised of multiple symptoms and multiple pain points, but you have to abstract a level and and say, well, what is what's going to have the greatest coverage across these classes of problems, these types of problems, and it is based on those use cases.
• 40:25
  SPEAKER_02: Yeah.
• 40:26
  SPEAKER_01: Well, this has been uh terrific.
• 40:29
  SPEAKER_01: You know, if if people take one thing for this conversation, it's it's this the tools are getting dramatically more capable, but the underlying discipline does not go away.
• 40:37
  SPEAKER_01: It's gotten more important.
• 40:38
  SPEAKER_01: Uh, data governance is sexy again because not because the technology demands new governance, but it the old kind of governance was always the actual bottleneck.
• 40:46
  SPEAKER_01: And now the costing, uh, the cost of skipping that shows up faster with more confidence behind the wrong answer.
• 40:51
  SPEAKER_03: So, you know, and there's even more things to govern now, right?
• 40:54
  SPEAKER_03: Like it's you, you know, so you're you're I mean, look, governance.
• 40:58
  SPEAKER_03: One way to think about governance is these controls and government, they're not meant to be internal bureaucracy, right?
• 41:04
  SPEAKER_03: They're not meant to be like mud you have to slog through.
• 41:07
  SPEAKER_03: They're meant to be like, I'll call it the brakes in your car.
• 41:11
  SPEAKER_03: The reason you're able to go around a curve at high speed is you know you have brakes in the car.
• 41:16
  SPEAKER_03: You know how fast you can possibly go, right?
• 41:19
  SPEAKER_03: If you didn't know you had brakes in the car, you'd probably inch your way around the curve because you're like, I don't want to fall off the edge of the cliff, right?
• 41:24
  SPEAKER_03: Uh and so you'd go slowly and be cautious and whatever.
• 41:28
  SPEAKER_03: You need controls and governance to know what the absolute limits are and operate at the limit, right?
• 41:34
  SPEAKER_03: And so that's the way I would think about it.
• 41:36
  SPEAKER_01: So that's fantastic.
• 41:38
  SPEAKER_01: Thank you.
• 41:38
  SPEAKER_01: So Herr, thank you for joining me in and what decades of working at the intersection of data analytics, enterprise decision making teaches you about what it takes to make AI work.
• 41:48
  SPEAKER_01: And thank you to our listeners.
• 41:49
  SPEAKER_01: Thank you for tuning in to the Early AI podcast.
• 41:52
  SPEAKER_01: Be sure to subscribe for more conversation on how AI is shaping the future of business.
• 41:56
  SPEAKER_01: And again, thank you to our sponsor, Vector.
• 41:58
  SPEAKER_01: Um, so Harold, thank you again, and we will stay in touch.
• 42:02
  SPEAKER_03: Yeah, for sure.
• 42:03
  SPEAKER_03: It was great.
• 42:03
  SPEAKER_03: I enjoyed it.
• 42:04
  SPEAKER_03: Thank you.
• 42:04
  SPEAKER_03: Excellent.
• 42:05
  SPEAKER_01: Bye now.
• 42:07
  SPEAKER_00: Thank you for joining us on another deep dive into AI innovation.
• 42:11
  SPEAKER_00: Tune in next time when we introduce another industry expert and discuss how to maximize AI in the world.
• 42:17
  SPEAKER_00: The Early AI podcast is sponsored by Early Information Science and Vector.
• 42:22
  SPEAKER_00: To learn more, visit early.com at e L E dot com.
• 42:28
  SPEAKER_00: Thank you for tuning in.