Latest / State of the AI Union / Habib Basiri: AI Data Governance: Trust & Transparency with Habib Basiri
Transcript
- 0:05Hi everybody, and welcome to this week's episode of the State
- 0:08of the AI Union. Today we're joined by Habib
- 0:11Basiri, Director of Technical Product Management for AI
- 0:14Products and Extreme Networks. Habib is a visionary product
- 0:17leader who spent his career building the foundations that
- 0:20make enterprise AI possible. From leading development of AI
- 0:24core data, governance and knowledge graph systems and
- 0:27Extreme, to also architecting AI platforms at eBay, his work sits
- 0:31at the intersection of trust, data, and scale, and to try to
- 0:35build AI systems that people can actually rely on.
- 0:38So Habib, do you want to introduce yourself?
- 0:40Tell us a little bit more about you and the work that you're
- 0:42doing at Extreme. Yeah, sure.
- 0:44I'm Habib Basiri. As Laura mentioned, I've been
- 0:47leading the product in the last few years.
- 0:50I don't know with more than 20 years experience but my focus
- 0:54was build the platform and trusted data to enable AI and
- 1:02also a genetic platform to use that has been started from eBay
- 1:08which we start to build all them AI and data governance platform
- 1:12with knowledge graph based catalog to to enable data and
- 1:17govern data for different use keys.
- 1:20And recently AI is coming everywhere, but this been years,
- 1:26it's been use case for years and we can go why it has been
- 1:30important in a different company.
- 1:31But in extreme, the focus is extreme is building 0 touch
- 1:36network management with AI and is going to use a lot of
- 1:40continuous and reactive agent and conversational action with
- 1:44talking to the data and knowledge.
- 1:46But how we provide real and near real time data for agents in the
- 1:52semantic and contextual understanding.
- 1:55It was the mission I have here as a product owner and my focus
- 1:59was enabled near real time and real time data for models and
- 2:04also basically democratize AI for everybody.
- 2:07That was briefly what I'm doing. Also, well, that sounds amazing
- 2:11and I, I can just imagine at eBay there was just like such
- 2:15large scale data availability and data platforms and now
- 2:20you're trying to, do you know, something similar at extreme,
- 2:22you're also working off of like tons and tons of data.
- 2:24Tell me about the role of the knowledge graph and how that has
- 2:28evolved over time. What has changed in how
- 2:31enterprises think of data for AI?
- 2:32Well, of course there is a big change with coming AI in the
- 2:36picture. But regardless of the AI
- 2:38knowledge graph always I mean last 1520 years, a strong role
- 2:45how we semantically understand the data and how we integrate
- 2:50and interoperate data and bring the silos of a data in the in
- 2:57the manner we understand the meaning that that was in the I
- 3:01mean picture for years. And an eBay which is the most
- 3:05your biggest e-commerce catalog they have and with more than 50
- 3:10maybe metadata source. It's a critical to understand
- 3:14who owns what type of a data, for how long and which purpose.
- 3:19If you want to answer this question on enterprise level,
- 3:22you need to start to create your canonical version of ontology
- 3:27for all data and data source you have and then discover it and
- 3:32then connect it with that integration at the business
- 3:36meaning on that and unified or harmonized data for all the
- 3:41users. These users can be application,
- 3:43can be batch, can be person, can be auditor from the governance
- 3:46perspective and privacy team, but also can be the AI those the
- 3:50all these AI was not that much main user of the of that.
- 3:57But knowledge graph itself improve a lot of by models
- 4:02because if you want to have more contextual and related and deep
- 4:06graph, of course NLP and in ER plays a strong role there to
- 4:11build that. But the stage is change.
- 4:13The stage is how I can practically, I mean we can
- 4:17discuss it how it change I think is the big change is this from
- 4:22model centric we change to the trusted data and intelligent
- 4:26workflow distance with that change means we need trustable,
- 4:31contextualized and continuous AI embedded to the any, every,
- 4:36every business process you have. So this is the big change we
- 4:39have. But traditionally same as
- 4:41before, still building a graph is same.
- 4:43Of course there are some improvement has been done, but
- 4:45graph is a graph. But The thing is how your you
- 4:49feed your AI with the right and meaningful business data and
- 4:55keep this consistency between all the business units you have.
- 4:58I think that's a big change from the consumer perspective of the
- 5:01data AI. Without right data, it's not.
- 5:06Again, the big changes is in that area.
- 5:09But of course there will be some help for building the better and
- 5:14accurate ontology with agents later on.
- 5:17But the tooling and skill of the building ontology and graph is
- 5:23the same as the issue, same as before and the same issue.
- 5:26Sounds like it sounds like now as we are using these graph
- 5:30databases for AI, the near and real time aspect of the data
- 5:35availability availability becomes more important.
- 5:38Is that what I'm hearing? Yes.
- 5:40And getting the right data with right meaning for consistency
- 5:45for all the business units you have.
- 5:47That's how your semantic version and knowledge graph has a
- 5:50backbone of that. Place there and so you, you
- 5:53mentioned that like the knowledge graph improves data
- 5:56availability both for humans and for the AI to use.
- 6:01How does that translate into greater trust in AI models and
- 6:05decisions? Like you said, we're moving from
- 6:07models to trust and intelligence, right?
- 6:10So it it it has key rolled it because if you build your
- 6:15knowledge graph in a way you have full discovery of the all
- 6:20the data you have and understand the relation between them.
- 6:24Then the biggest problem today with AI is the trust.
- 6:28And the trust means how you explain the steps of the data
- 6:34are coming to the your visibility or end to end
- 6:37visibility is the biggest part of the trust of AI because you
- 6:41need to explain each steps and also you need to say what type
- 6:45of a data use for which models. And you keep all this versioning
- 6:51and crack the flow and the, the knowledge graph is the best
- 6:55place you can get the lineage, you can understand the relation
- 6:59and also you can keep all the versioning and everything's for
- 7:03that purpose with building that back phone in your data
- 7:07infrastructure. So it will be easy especially in
- 7:10your catalog because any company wants to be successful in AI in,
- 7:15in my opinion future is them who has a right semantic and good
- 7:19knowledge graph and also cataloging of the data and keep
- 7:23all those relation and the governance of those data under
- 7:27monitored into invisibility. Into invisibility is a big part.
- 7:30So the trust comes if you can explain what type of a data is
- 7:34coming and it is aligned with all the privacy and regulatory
- 7:37asks for jurisdiction, which model is using and then you can
- 7:42test it your fair bias hallucination and everything's
- 7:47for every single of data sets you have in your catalog.
- 7:51And then you have all in your based on the graph that that's
- 7:54how it shows. So because everything's rely on
- 7:57the data and then if you have full visibility on your data
- 8:01between all the business lines and organization you have then
- 8:06it will be easily to explain and easily to track and easily also
- 8:10to debug any issue. So for that cost is gradually
- 8:14will be because you are very transparent to your user what is
- 8:19happening in your data and who is using your data, who is your
- 8:22who or you are sharing your data because that's the most of the
- 8:26concern customer has today. If we give you your sensitive
- 8:29data, how you want to guarantee is only using for us, not for
- 8:35anybody else and how also we can see the model is using and what
- 8:40is happening there based on the model.
- 8:42I think those are the key functionality knowledge graph
- 8:45you can build and the catalog and start.
- 8:49Then you will start the trust from there and then you will
- 8:51learn in during the journey how you bring and keep or add more
- 8:57component and the layers or dimensions to the your data to
- 9:00bring more trust. So, you know, actually that's
- 9:03really interesting, right? Because what you're saying is
- 9:05that trust comes with being more transparent and being able to
- 9:10explain what the AI is doing, what you're doing with the data,
- 9:14not when you hide it behind something and it's a black box.
- 9:18You don't know what's going on with it.
- 9:19So so that that I guess is is how trust is developed.
- 9:22Because the observability today is not observability on the
- 9:25models or the application Now observability is end to end.
- 9:30From the time you accurate the data until consumed by models
- 9:34and then you store it the result, you should have full
- 9:37visibility and observability on that.
- 9:40Very transparent to the users. So let's.
- 9:42Talk about data governance, right?
- 9:44You and I met at the leadership circle at Devref leadership
- 9:48Circle in San Jose, and the Secretary Rice was a speaker
- 9:52there and she talked about how the US innovates, China
- 9:56imitates, and Europe regulates. And I love, I love what she said
- 10:01there. So let's talk about this data
- 10:02governance and the regulation of data.
- 10:04Sometimes that tends to like slow things down.
- 10:07What has changed in how companies design governance
- 10:11models that maybe actually help the acceleration, help
- 10:14accelerate AI adoption? Yeah.
- 10:16Definitely, yeah, that was the interesting topic.
- 10:19I think she brought that to that conference, which is really
- 10:22valid. I think is what is changing now
- 10:25and compared to a few years ago before 2020 maybe is any
- 10:31governance if you consider or the regulation or compliance was
- 10:36part of the legal layer and compliance layer and it's never
- 10:42been considered governance and privacy as a feature of the
- 10:47product. So any company is struggling
- 10:52with that this days because they and also culturally the
- 10:56governance and be compliant was a very reactive culture.
- 11:00So these days we need proactive culture, which means you should
- 11:04be proactively manage all their compliance or the regulation
- 11:09from the date time you're ingesting data or accurate data.
- 11:13Yes, it is a it is slowing the companies which hasn't been
- 11:18founded rightly under the governance side.
- 11:21As the platform governance has a two side 1 is a process.
- 11:25Yes, it is time consuming. But if you automate the
- 11:28governance and all the policy automation then is not
- 11:32bottleneck anymore. So the big shift is happening as
- 11:35I mentioned from reactive to active, which means you need a
- 11:39very comprehensive platform to support that active governance.
- 11:43Plus you need to also consider as a feature of the product.
- 11:47It is not anymore legal or compliance layer.
- 11:51It is the feature of the product.
- 11:53You need to be sure from any data acquisition, any model
- 11:57development and model deployment you have full governed data and
- 12:02the models. So this is the how is going to,
- 12:06I mean it is slowing because there are things you I think
- 12:10there was a Gartner or any other research shows 90% of the
- 12:14enterprise hasn't governed and their data ready for AI.
- 12:18Wow. OK, So if you if you go with
- 12:20that, even if you go with 60% and it is huge, it is huge.
- 12:26Now everybody's jumping to the rag.
- 12:28Yes, the rag is working, but rag is is rag.
- 12:30Rag is not the AI enablement here or the yes it is slowing.
- 12:36The company hasn't invested on the building the right
- 12:40governance platform and automation in place now.
- 12:44It is if any company invested in last few years or even recently
- 12:50to build the right governance platform and the process, I
- 12:55think it is not slowing anymore. It is a big investment because
- 12:59just one simpler from EUAIA. If you any data consumer or
- 13:06processor cannot comply its percent of GMV or a minimum $30
- 13:12million if I'm not wrong or $20 million they need to pay as a
- 13:15fund. But that is a direct cost.
- 13:17But indirect cost is trust which is huge.
- 13:21In that scenario. You see it is early investment
- 13:24and any early investment it's slow, but the payoff is very
- 13:28high. But with AI is now changing a
- 13:31lot, although we don't have that much except EUAIA, but which is
- 13:35not 100% activated. Not a lot of states already put
- 13:41the compliance and the govern governance around the AI.
- 13:44But I'm surely next one to three years.
- 13:47A lot of countries will come with that.
- 13:49So still not that much late, still late, but not not that
- 13:53much. But I think all the jurisdiction
- 13:56will add responsibility to The Who is providing AI enterprise
- 14:00solution to govern in the way they're asking.
- 14:04I think everybody is learning from EU how to manage that
- 14:07because EU still is a top regulation for Data Act and also
- 14:11EU AI Act. But I think a lot of country are
- 14:15countries are learning gradually to add more for that.
- 14:18Yes, the governance is slowing for WHO hasn't been thinking for
- 14:22that to enable data and has an active catalog now or active
- 14:27compliance or proactive compliance now is a big issue.
- 14:30But who invested I think is now is a return of investment time.
- 14:35You know I. Think, I think what is what's
- 14:37interesting is if you are building an AI native company,
- 14:41let's say you stood up your company in the last two years,
- 14:45sort of like post GPT era, You know, in a sense, data
- 14:49governance is something that you already are thinking about.
- 14:51So it's it's easier for you to like, you know, stand it up
- 14:55right and, and incorporate those governance things.
- 14:57What would you advise companies that you know have been around
- 15:01for a long time that like you said, maybe you're not really
- 15:03like weren't really thinking about it yet, but now they are
- 15:06ready to like do AI and their data is like not yet in
- 15:11governance state. What's like the first two steps
- 15:13that you you would recommend them do I think?
- 15:15Is the first two steps is if they want to go to AI road map
- 15:19or not that's area they need to take a decision yes.
- 15:22So how much they want to adapt to the AI, if that's the key
- 15:27that will define how, how much they need to govern the data as
- 15:33active governance and compliance.
- 15:36I mean that, that is the definition of that.
- 15:38If, if they want to go fully adapt to the AI, I think I think
- 15:42is there are two steps. Generally we say is 1 is they
- 15:46need to think about how they where they want to be in the
- 15:50data governance maturity limit, what they are today, what they
- 15:55are targeting because as I'm saying it, maturity start
- 15:58normally from zero to 5 or we are saying nothing in place or
- 16:01food fully optimized. They need to understand where
- 16:04they are today and what is their today data management maturity
- 16:08and governance maturity. If they understand that and then
- 16:12understand the scale of a data they have based on that, then
- 16:15they will put the priority. But the biggest part as I
- 16:18mentioned for that is the discovery of the data and
- 16:21discovery after and building the ontology of that then change
- 16:25management of the data. I think those are the big steps.
- 16:28My advice for everybody also they need to consider they're
- 16:32going to build in house or they want to use the third party tool
- 16:35because that's a two different approach.
- 16:37If you if they go to that route, then they they need to evaluate
- 16:42which platform works or what they want to build for for
- 16:47themselves. Then, then they can start from
- 16:49their, the old companies struggling with a lot of
- 16:53metadata and data source and also legacy systems.
- 16:57And also they need to start from the data are going to be more
- 17:03used by AI first, not all the data first.
- 17:05I mean, who is using more and then or what data are going to
- 17:09be used more by AI? They need to start from that
- 17:11area, yeah. And and so maybe the trade off
- 17:13here, like you said, is first determining like how much do you
- 17:16want to expose to the AI, right? Because that can really
- 17:19determine your scope and, and maybe organizations don't have
- 17:22to expose like everything. Maybe it's just a small use base
- 17:25and it still be significantly more efficient, you know, by by
- 17:29implementing AI and the trade stuff is that like you said, the
- 17:33AI maybe doesn't have the full context of the picture if they
- 17:35don't expose all of it, but then there's another big time value.
- 17:40You can start with smallest step and then expand it.
- 17:45At the end of the day regards of AI you need the graph catalog in
- 17:49my opinion in your organization at least for governance and
- 17:52compliance. But then also it is the cost of
- 17:54the your AI use case is real time data access for near real
- 18:00time access because it is completely different approach
- 18:03and is the cost also is different.
- 18:05This exactly is a trade off here.
- 18:06It's also trade off between the coverage and also accuracy.
- 18:09You want to go small coverage, high accuracy, you want to go
- 18:12big coverage, low accuracy. And those are the all the
- 18:16trade-offs that I mean they need to consider what do you.
- 18:19Think is everything is is next after LLM start reasoning over
- 18:25these structured graphs. How are they going to how are we
- 18:27going to be shaping the next generation of enterprise AI
- 18:30platforms? I think.
- 18:31Is LLM is going to be commodity after a while, what enterprise
- 18:36is going to get benefit who has harmonized contextualized data
- 18:41to feed and trade LLM? That's the key.
- 18:45So who builds that I think is the key.
- 18:48The other area in my opinion is going to play a strong role in a
- 18:52knowledge graph, which is part of the also what has been
- 18:54changed. Every company are specializing
- 18:57every vertical entity by graph. They will win more and more
- 19:02because they can train agents, they can feed and train the
- 19:06agents and models with more accurate data.
- 19:09What does that mean? Is is it's let's go in some
- 19:13specific, let's say you have a model for only for supply chain
- 19:18or for HR data. If you have your specific
- 19:22knowledge graph, there are two companies I know they built they
- 19:25already have like C3 AI and Palantir.
- 19:27If you have vertical graph here industry or entity let's say
- 19:33like supply chain or HR networking or if you are
- 19:37specialized on that graph then your Google train agents and
- 19:42models more specifically and accurately for your purpose for
- 19:46that data. That's another most important
- 19:50part is not only building now graphing scale but also building
- 19:56subgraph in vertical entities. You have more accurate response
- 20:01from the your models and also your agent can work it.
- 20:06Correctly on the data read and understand and act it correctly
- 20:10and also your reasoning becomes more accurate per vertical which
- 20:16is the key for your reasoning and also semantic as a
- 20:20consistent layer for all the reasoning in your organization.
- 20:23I think those will play key role.
- 20:25Wow. That's amazing.
- 20:26Yeah. So I think so Verticalized AI
- 20:29guys, graphs be like awesome. Habib, thanks so much for the
- 20:37conversation. We learned a lot today about the
- 20:40knowledge graphs, where they're going, why you need it.
- 20:42Is there anything that you wanted to leave the audience
- 20:44with before we wrap up here? Yeah, I can have a couple of
- 20:47things. I mean as I mentioned is we are
- 20:50moving from model centric to workflow centric.
- 20:54There are a couple of things happening in next three to five
- 20:57years in my opinion. One is data foundation as a
- 21:00competitive mode which is going to play key role there.
- 21:03Second, agent workflows will adapt to autonomous or semi
- 21:07autonomous as a semi autonomous agent on live business data and
- 21:12the live business data, it's a key part of that.
- 21:15And also the other important part is how we implement the
- 21:19zero copy data for agent. It's a key part of the graph.
- 21:22I think we deal with that. Then AI governance we discuss,
- 21:26yes, the AI governance responsible AI explainability is
- 21:30not anymore part of the compliance layer.
- 21:32It is the product feature. You need to look at how you
- 21:36build your product from beginning with that approach as
- 21:39with included feature. Then we discuss about the AI and
- 21:44ML model commings as a service or commodity and how your data
- 21:47will play big role. But the other things I think we
- 21:51need to consider is there, what are the challenge we have.
- 21:54I mean, I didn't talk too much about the challenge, but there
- 21:57are a couple of challenge I think we have in the data area.
- 22:02It's 1 is still semantic drift is a big things or change manage
- 22:08from all your ontology. It's a big one how you want to
- 22:11manage that that one in your organization that's the key.
- 22:14Second is scale and performance. When you have a big graph and
- 22:19your storage costs, your query cost and all are the your
- 22:25factors to decide how you trade off between all those promises.
- 22:28There is governance and ownership of that because there
- 22:33are a lot of ambiguity about the governance and ownership in your
- 22:36organization. How you define the ownership?
- 22:39Ownership is is a big role on catalog governance and knowledge
- 22:44graph how you want to define ownership and because ownership
- 22:47will dictate retention because of the which domain you own,
- 22:51what is the purpose of activity of the data and who can use this
- 22:55data. And of course, there's also
- 22:58classification and other part of that those that's another area
- 23:01we have the biggest issue is them contextual dependency and
- 23:06ambiguity. Different meaning, different
- 23:08words has a different meaning, but some sometimes translating
- 23:11organization as A1 meaning. But for example user user can
- 23:15have hundred types of the definition in different company
- 23:18and for each domain has different user.
- 23:21How you fix this ambiguity of the entity and and basically any
- 23:26object in your organization, that's a key part of the graph
- 23:30which will add a bit synonym and understanding meaning of that
- 23:34and adding and dictionary and clarifying in your organization.
- 23:38It's a big part. Then the other challenge we have
- 23:43is the tooling and scheme building a knowledge ontology.
- 23:49It's a manual is not going to be fully automated very soon.
- 23:53But how it can be semi automated or pragmatic works is as much as
- 23:57you create your craft, you can't train your agents in some
- 24:02specific entity when you integrate to the new source how
- 24:06to map to the new ontology. I think that's still is a big
- 24:11problem we have there. The other part of that AI is
- 24:14going to be the last part in my based on my idea is a
- 24:19localization and also multilingual semantic layer.
- 24:23If you want to act AI globally, how you want to build that
- 24:27localized and multilingual graph and semantic layer, I think
- 24:31that's the key part of the challenge we are going to face.
- 24:35Wow. Awesome.
- 24:36Hey, one thing that you mentioned earlier that I just
- 24:38want to double click on was you said, you said one of the key
- 24:40things here was like implementing 0 copy data.
- 24:43Tell me more about that. What is that?
- 24:44So traditionally when you want to data science, they do, they
- 24:48copy the data, they bring it to their environment and work on
- 24:51that and do different processes and cleansing to data to be
- 24:56ready. But now with AI, we don't need
- 25:00if you want to copy data different places and all the
- 25:03users of AI use, it's going to be very on.
- 25:07I mean managing is a big issue and the governance and the
- 25:09compliance and security is on other issues and of course the
- 25:12cost. But yes, the approach is when
- 25:17any agent is coming and asking or sending query can be SQL or
- 25:21can be a natural language query when it comes how you understand
- 25:26the meaning of that and then how you create a query and go to the
- 25:30source and get the data and send it back to the agent or any
- 25:34query comings there. That's a key part of all the
- 25:37technology you need to build this.
- 25:40And the biggest part is if it goes more difficult, if your
- 25:44query is not only calling one API, if it goes to the multiple
- 25:48APIs, then how you are compiling all this query together and
- 25:52putting in one package and send it to the downstream.
- 25:55It's the key part. And in that scenario you are not
- 25:57copying your data everywhere. You are just getting from the
- 26:00source sending to the user and it's very clean and still data
- 26:05owner owns their data. We are not copying data
- 26:07anywhere. It's the same approach, but also
- 26:10is a key part for the real, especially for real time data
- 26:14access. That's that's the key part.
- 26:16But also with that integration between source, it's a key.
- 26:19And also any, any users or let's say BI users, they can get their
- 26:24data or mothers, they can get data from the source easily
- 26:27without accessing to the full data or copying the full data.
- 26:30Wow. Awesome, Habib.
- 26:32Thanks so much. Really appreciate you joining
- 26:34us. And to our listeners, if you
- 26:37enjoyed this content, please let us know, send us like a comment
- 26:40and until then we will see you next time.
- 26:42Thank you. Thank you.