Latest / The Data Splash / From Data Police to AI Enabler – The Evolution of Data Governance - with Tiankai Feng
Transcript
- Ido Bronstein: Welcome everyone to the Data Splash Podcast, brought to you by Upriver. If you are a data engineer, a data leader, or just someone trying to make sense of what the hell is going on with data and AI right now, you are in the right place. Here with me today is Feng, a data and AI leader with deep expertise specifically in data governance and also author of Humanizing Data Strategy and Humanizing AI Strategy. We are going to talk about the evolution of data governance. Tiankai has a lot of experience from the side and from the consulting side, and he's famous for making it something people actually want to do. So ⁓ we ⁓ deep into much, we have a small tradition We call it the 30 Second Splash! No time to think. Just instinct. Tiankai are you ready for that? Tiankai Feng: Yes, I'm ready. Let's go. Ido Bronstein: Let's go. Governance teams, sherpas or police? Tiankai Feng: Sherpas Ido Bronstein: What breaks first, the model or the metadata? Tiankai Feng: Metadata. Ido Bronstein: Dream vacation destination? Tiankai Feng: Maldives Ido Bronstein: Maldives which instrument explains data governance the best? Tiankai Feng: the conductor staff. Ido Bronstein: The conductor, yeah, nice. Is AI going to replace data stewards? Yes or no? No. Okay. Tiankai Feng: ⁓ won't replace it. No. Ido Bronstein: So let's ⁓ we are diving in, ⁓ I think we will have a bit for the audience to get a little perspective of you, what you do ⁓ what you did in the past a bit and what you are doing today? Tiankai Feng: Yeah, sounds good. So I'm Tiankai I'm currently working at Thoughtworks, a consultancy focusing on and data solutions. and yeah, I have been working on data all my career and AI, right? And I had various roles product owners, as analyst, as data scientists, as a little bit of an engineer, and now being more focused on strategy and governance. And ⁓ I'm really passionate about the human side of data and AI, which I always say, which is why I wrote two books about it called Humanizing Data Strategy and Humanizing AI Strategy. And I really think that the humans are the make or break of any of these ⁓ aspects, right? Of making data and AI work. ⁓ it's important that we focus on people being part of the journey and not being against the journey, I would say. And that's why I really love to talk about it and how to make strategy and governance work with humans. Ido Bronstein: Amazing. First of all, I cannot ⁓ agree with you more. I think that like as long as technology advanced, all the technical part is a bit being solved. But what is never being solved is the human part. So love to drill down into that. Tiankai Feng: Exactly. Exactly. Ido Bronstein: So in 2020 Tiankai Feng: Absolutely, sounds good. Mm-hmm. Ido Bronstein: you left analytics for governance. ⁓ And a transition that ⁓ most of the people are afraid of. Tiankai Feng: Mm-hmm. Ido Bronstein: Because governance also more again, because it's people, not technology, ⁓ in manner. So if you can share us a bit how was this transition was and what attracted you in the governance and the things that you want to correct from like the governance side. Tiankai Feng: Mm-hmm. Yeah, absolutely. so I mean the reason why I actually switched from analytics to governance was driven by my passion for data quality. what basically was happening, right, in my analytics role, I was dealing a lot with internal product data at the job at the time. And I realized that a lot of the quality really wasn't suboptimal, right? So there was a lot of issues in timeliness, completeness, and accuracy of the data. And ⁓ I basically a frequent ⁓ complainer to the data governance team saying, look, there's something wrong with the data quality here, right? Like can we please fix this? And at some point, basically the data quality side, ⁓ data governance side, they said, do you want to help solve the problem, right? Instead of just complaining about it, why don't you join us ⁓ we fixed the problem together. That's why what sound like a really interesting ⁓ proposition, right? Like let me see what's happening there and why is it so hard to fix. Ido Bronstein: Yeah. Tiankai Feng: So I switched to the governance side and I realized why it's so hard to fix, right? but it was ⁓ definitely a very exciting eye opening experience. Ido Bronstein: Ha ha ha. ⁓ if you cannot win them, join them, right? ⁓ Tiankai Feng: Yeah. That's true. That's true. Ido Bronstein: but what I love about that is that it's really the passion of driving value, right? Because you can do analytics as much as you want, you can complain about all the organization what the data is this, why that and that. But in order to really fix that, you need to come back to the foundations, to the processes, to the people behind the data. And I really like that you take a proactive step there. Tiankai Feng: Yeah, that's true. Yeah. Absolutely. I do also think that it really comes down to mindset, right? So the thing is, especially if you're in an enterprise environment, it's so easy to blame others for things not being right and just sitting back and waiting for someone else to fix it, right? But what is much more valuable and much more effective if you know how it should be correct, to then use your own expertise to help fix the problem together, right? Instead of saying it's their fault and that it's not my problem and I'm just gonna basically hide from it. Ido Bronstein: Yeah. Tiankai Feng: Right, why don't we come together more often and try to bring our expertise together to fix problems together instead of throwing it over the fence and waiting for it to be fixed, right? So overall I think that could be a much better mindset that we all can have. Ido Bronstein: This is for sure. I think that ⁓ especially in data collaboration is like everything cause data handoff between so many stakeholders. So if you are not coordinating, it will end up like a swamp. But you said that like when you move to the governance side, you understand why it is hard and why is it challenging. So ⁓ will be nice, I think, to the audience that are mostly like data engineers, they usually used to complain. Tiankai Feng: Exactly. Exactly. Exactly. Yeah. Ido Bronstein: A bit to understand like why this is so difficult. Tiankai Feng: Yes, absolutely. I think I mean there are two things to it, right? The first one is the overall mismatch of what its intended usage of the data and what's the actual usage of the data. So what I mean by that, right, that ⁓ of the data that we create in organizations for the first time are for operational reasons, right? So for example, we set up product data for the first time in product management systems, we set up custom made data for the first time in CRM systems, etcetera, et cetera. All of these things are for reasons where we want to do the data directly to do something with it, right? Like logistics or marketing, whatever, and so on and so on. But over the recent years now, the use cases for that data became so much more, right? So suddenly we use analytics to create reports, we create data science models to basically predict stuff, we use Gen AI now to create new stuff with it, right? And do some stuff with it. And those are all use cases that were not intentionally like initially thought of when the prod when the data was created there. Right. And so what basically is happening is that the requirements towards the data and the actual user of the data is not really coming together nicely. So the first step is basically to first understand what are all the requirements towards the data that actually exists and not just the one that we assumed in the beginning, right? And collect all the requirements down. That's I think the number one big step. And then the thing is, okay, let's have clear requirements and now we know exactly how we define good quality. Then we realize the data is not matching the quality that we want to have, right? So we do root cause analysis and try to understand where it comes from. And then it usually boils down into three aspects, right? You have the human error, you have the process error, and you have the technical error. So technical error, I think is the most straightforward. A pipeline broke down, right? A system is down, something is off. Okay, we just fix it. Ido Bronstein: Yeah. Yeah. Tiankai Feng: We ⁓ prevent it next time from happening and it's going well. A process problem is a little bit trickier already because it's basically it's about time and sequence, right? So for example, the data is only ready two weeks before the deadline, but they need it actually six weeks before the deadline. And so we have a structural problem there already. So how do we get that working? the last and most difficult one is human error, right? When the data is originally created as a manual entry by one human being, Ido Bronstein: Yeah. Tiankai Feng: And ⁓ when that is happening and it doesn't match the requirements, then the only person who can fix it is the human who was putting the data in the first place, because there's no other reference to what is actually correct, right? We can only say that it's wrong, but only one person knows what the correct record looks like. And that means we have to train people to actually do it better and we create better interfaces for them to input the data right? So in the end, really that is I think the most difficult part. Ido Bronstein: Yeah. Tiankai Feng: that the more human-centric the ⁓ source of the problem is of data quality, the more difficult it gets. ⁓ as data engineers who are really just downstream, right? How do we train people who are working in completely different departments how to input data, right? So it becomes a much bigger change management that we need to actually do. Ido Bronstein: Yeah. Amazing. So think that you framed it perfectly really resonate with like how I feel the problems in my previous work as a data engineer. ⁓ And things that under control, like the technical part, it's always easy. Like it's first of all your fault. So okay, I build a non-schedulable pipeline, I have some edge case that I didn't thought about that. But when it's coming to ⁓ Tiankai Feng: Yeah. Ido Bronstein: human uh things that are out of your control, this is when you need an organizational view in order an organizational alignment, first of all, on the fact that this is important and we need to invest in that because a lot of the time people do not understand that. ⁓ And this is the really difficult part in governance and basically job in that manner. Tiankai Feng: Absolutely. Absolutely. Mm-hmm. Yeah, that is true. That's true. Ido Bronstein: Yeah. I'm sure that as you work as a like in the enterprise and in the consultancy, you take the journey from like walking ⁓ the or even in the data governance area to running and jumping. So if you will describe what the transitions that the company in a wider scope is having when you have a good data governance. Tiankai Feng: Yeah. Mm. Ido Bronstein: To create like good data quality. Like ⁓ Tiankai Feng: Yeah. I think Ido Bronstein: how does success look like in that manner? Tiankai Feng: I think that ⁓ the main like maturity we can measure and that we should be very obvious is how we move from being reactive to being more proactive overall, I would say. Right. So reactive means right a lot of bad things in data happening and data governance comes in and tries to fix it with like reactive solutions, right? Like Data quality is bad. Let's come up with a data cleaning algorithm, right? Or some compliance issue. Okay, let's pay the fine first, and then we're gonna figure out a policy how to prevent this from happening next time, right? So ⁓ go from this fixing pinpoint issues first, and trying to fix whatever is burning, then slowly into more routine, right? And at some point you will then ⁓ later down the maturity line. see that we can prevent things too, right? Like not about only reactively fixing stuff. Let's build in guardrails. Let's build in policy as code. Let's build in like automatic processes that can help us to already predict what might go wrong and we can prevent it from happening in the first place. Then we actually move away from this reactive fixing and trying to like ⁓ be firefighter into let's plan ahead. Let's see what the journey will look like and what is needed. And we can already build the right modules and the right building blocks, basically, to really prevent decision from happening in the future. And is where I think really the maturity is coming from. When things are more embedded and governance is then more seen as basically default that every data product and data set and asset is now in the future having governance in mind and not just reactively repurposed, right? Then I think the maturity has really increased. Ido Bronstein: Yeah. Something that you talked earlier that ⁓ really resonated with me, that you need to start with the use case in this manner. Like having ⁓ a table with ⁓ zero nulls, it's but it's not a goal. ⁓ the idea is how you drive a business value. So when I'm thinking and on a proactive and love to hear your thought on that, like when I'm thinking ⁓ on a proactive Tiankai Feng: Yes. Yeah. Correct. Ido Bronstein: It's an organization that first of all know what what he wants to do with the data now and then through actively creating a process and not manual process but automatic process that can sure that the data is correct ⁓ correct for this use case from the human in the start to the ⁓ technical staff down the line. Tiankai Feng: Yes. Mm-hmm. Ido Bronstein: ⁓ and I think that like organizations that I saw in a very good state from their a governance perspective, exactly. They did not pursue around fixing data quality here, finding PII there ⁓ they asked themselves, okay, what does the business want to achieve right now with the data, and make sure that the stakeholders doing that involve Tiankai Feng: Yeah. Absolutely. Yeah. Ido Bronstein: in this business use case have aligned and put data as a first citizen because it's critical, it's a business asset. Tiankai Feng: Yeah, absolutely. and that that is right, because I think on a psychological note, use cases purpose. But if you just create something so it exists, it doesn't really have a purpose. And without purpose, it's really to convince people. We know that. The good thing about use cases in organizational context is they usually come with clear requirements and a clear ⁓ quantification of the impact as well. So not only can we say Okay, this is what we exactly need from this. We can also say we need this so we can make so and so much money, or we can save so and so much money, right? For example. And having that kind of surrounding is like an undeniable justification of it. So not only does it say it's unclear what we want to do, so we all are misunderstanding each other, but also we can say we have to do this because otherwise we are not able to achieve that impact. So we better all get motivated to really do this. So in a way it becomes that objective truth. To say, We need to do this and we all have the facts to back this up. Let's get this done, right? And that's how governance can really bloom and not just become like a fluffy thing on the side. Yeah. Ido Bronstein: Yes, exactly. Yeah. It helps to connect it not just to the policies and the data ⁓ quality but also for the business at what our goal is like where it leads the the organization for. Tiankai Feng: Exactly, yeah. Ido Bronstein: Yeah. And today we cannot finish an episode on data without talking about AI. ⁓ is no ⁓ way out of that. And I think that AI and I want to talk with you on AI in two different perspectives. The first one and me and how AI changed the perspective on data governance in the company ⁓ and Tiankai Feng: Yeah. That's true. Yes. Mm-hmm. Mm-hmm. Ido Bronstein: do you see like data governance in the role where we want to enable AI in the organization? And the second one, how ⁓ can use AI in order to enable better ⁓ governance. So let's focus first on the first one because it's connected to what we talked earlier. But yeah, where do you see data governance in the of like making an ⁓ organization to adapt AI? Tiankai Feng: Yeah, Ido Bronstein: and to be AI ready? Tiankai Feng: Yeah. think that ⁓ AI and all the hype around it has forced data governance to ⁓ not only become in the center of attention, but also to evolve. So because Basically with AI, we try to the things work, right? And the moment we put it into production we realize the data is not really supporting the use case enough. So what can we do? Suddenly now it's basically all a problem. ⁓ we should have governed the data better is the conclusion And then we go to governance. And ⁓ then we realize though that even with the traditional governance concepts, it's not covering all the new things that we need to do. And most importantly, for example, is that while what's, for example, reporting use cases only needed maybe maximum the last two to three years of data. Now suddenly with AI we want a lot more historical granular data to train models. So it's we have to go back in time a lot more, but all of that was never in scope for governance. So suddenly we have to go back all that time and realize what data is usable and what isn't. And that is a really tricky kind of environment to be in. The other one is that with Gen AI the focus shifted from structured data a lot more to unstructured data. And data governance has now, in the recent years, much more focused just on structured data. So governing unstructured data in documents and pictures and videos, right? ⁓ is really a new thing. And suddenly like we have to understand what the difference is between governing normal tables to actual PDFs SharePoints. How do we even do this, right? And how do we create like a governed way around it? And that I think is a really tricky thing as well. and yeah, I think are the main issues, of why data governance is not more in focus, but they also need to are forced now to evolve to meet the requirements of the AI work. Ido Bronstein: Yeah. Interesting. So also put data governance more in the center because now it's more urgent to adapt AI, but create new challenges. First of all, like looking on historical data that nobody cared beforehand. And like unstructured data. And this is a huge challenge. How do you think like about governing unstructured data? It's like for me it's a whole new approach that we need to understand. Tiankai Feng: Yeah. Exactly. Exactly. Yeah. I think My first reaction to this ⁓ ⁓ a of the quality comes from curating unstructured data properly. And what I mean by that is that we all know how SharePoints work and shared drives, and we have like ⁓ like fifty versions of the same document in one folder, for example, Like this is like PDF version one, PDF version fifty, PDF final, right? All of these versions are in the same folder, Ido Bronstein: Yeah. Tiankai Feng: And if we just let an AI go on it, right? An agent or a model, it would be completely confused by all of the different versions and conflicting information on there. So I think in that analogy, we as the experts and the business ⁓ experts need to curate better, right? So what is actually the source of truth? What are the correct and up to date documents? And basically label them or curate them first, And then I think once that happens, right, we can create like more better ways to like save them in knowledge products and combine them with semantics and so on, all these hype words around context, right? That that is helpful. But it starts first with even knowing and understanding which are the right documents and which ones are the right unstructured data that we want to have governed to have a clear scope. Ido Bronstein: Yeah, And sounds exactly the problem that we solved like ten years in structure data, right? curating the data, finding ⁓ that we have one source of truth, just need to do that with different tools today, maybe semantic tools, maybe ⁓ different storage behind the scene. But the data governance still have the same ideas of like Tiankai Feng: Exactly. Absolutely. Absolutely. Yeah. Ido Bronstein: how it's enabled organizations. And how do you see AI helps you as a data governance steward to do all those stuff? Tiankai Feng: Yeah, I think this is where it's really I'm really optimistic about it and I'm seeing really good things in the market already, I think AI capabilities have really started to support better in like the day-to-day tasks in data governance and data stewardship as well. simply because it can take away a lot of the tedious repetitive tasks. So let's think, for example, just basic policy writing, And now that you have AI, you can let it ⁓ draft something up for you and just basically Finalize it with some human touch, but you don't have to write it all from scratch anymore. That is just from yesterday. You don't have to take care of all of that anymore. And also with like data profiling. There's really smart ways for it automatically to run and just see anomalies and any kind of patterns in there without you having to run all the queries on your own and then check the results yourself anymore. There's just really great tools out there that can do it. And also the same with monitoring and alerting, right? Where like it can identify Ido Bronstein: Yeah. Tiankai Feng: any kind of incidents, it can already identify drift for you, And all these things. And you don't have to just build all the solutions on your own. Where most of the things came out of the box already. And that's really good. And when we think on top of agents as well, Like ⁓ agents who can now you don't have to click as a steward manually into a dashboard anymore every day. But as an agent can capture all of that for you and informs you about what you need to know every day and proposes actions for you. That you just have to approve and it does things for you, then even that is also great, Like it can do a lot of the things, I think, for you. I it's in that in that way, right? ⁓ Governance become so much more leaner and we as humans can focus much more on making right decisions and deciding the right things. And we're not so occupied anymore with the tedious manual tasks anymore with it, which I think is a really good thing. Ido Bronstein: Yeah. Yeah, I agree with you. And I think it's like in a lot of different worlds that we see out there. AI help us to reduce the amount of technical and tedious work that we need to do and help us to focus on what's important. The people, the processes, making sure that everybody aligns on the use case. yeah, agent and profiling. You can do your own the with ⁓ with the data itself once you decide what you want to do. Tiankai Feng: Exactly. Exactly. Yeah. Yeah, exactly. Yeah. Ido Bronstein: Yes. one question before we close in. You to reveal us what is going to be in the next hit on the data governance songs that you are making? Tiankai Feng: to be honest, I don't know yet. I mean, ⁓ even my first one that was an unexpected hit, right? Governors of Data. I didn't expect it to be still so relevant. Like even like now it's like four years later and people are still sharing it to each other and ⁓ using it to communicate data governance better. I'm very happy that I can use my music to make it more approachable, right? But I don't really have a plan for my music. It just happens when I have an idea and then I just do it. Ha ha. Ido Bronstein: already looking for the next hit. ⁓ really looking forward. It will be something with AI, ⁓ I have guess, but ⁓ let's ⁓ Very So one last If you need to to a new steward what to do today in order to be the best steward in five years from now, what should you do? Tiankai Feng: Thank you. Yeah, very likely. Very likely. Yes. Hmm. I think two things. is never forget to focus on what's important, right? And that will evolve over time because what is important today might not be important anymore tomorrow. So always keep an eye on for business value and ⁓ basically skills, On what are the right problems to solve and do you have the right skills to solve the problems. That is, I think, number one. And the second one is ⁓ don't just stagnate in your day to day. If you feel any ways of optimizing your day to day with AI or without AI, go for it, right? That never hurts to reflect and optimize your workload. And it takes some conscious decisions to do that because it's so easy to get stuck in your day-to-day routine. But we shouldn't afford it because we can do things so much more efficient nowadays. Ido Bronstein: Exactly. You can never stop and stand in the place. Tiankai Feng: Yeah, exactly. Exactly. Ido Bronstein: Amazing Tiankai. Thank you so much. It was a great conversation. I learned a lot ⁓ what is really important in data governance that is focusing on the use case ⁓ ⁓ allowing the organization to aim to the same business goal and then of course fix all the problems from the technical to the human and the in order to help us to build a better data quality data set ⁓ and ⁓ The most important thing to drive value in the organization. So thank you so much. alright, that a wrap. Again, huge thanks to you, Tiankai, to bringing ⁓ the insights and keeping it real. If you enjoy this, follow the Data Splash Podcast. We've got a lot more coming your way. Until next time, keep your data flowing and we will see you in the next stream. Thank you very much, Tiankai. Tiankai Feng: Thanks for having me.