Latest / The Adults In The AI Room / Revolutionizing Data Pipelines, Unifying Metadata, Knowledge Graphs, and Generative AI - Alexander Schober - The Earley AI Podcast with Seth Earley and Chris Featherstone - Episode #038

Transcript

Revolutionizing Data Pipelines, Unifying Metadata, Knowledge Graphs, and Generative AI - Alexander Schober - The Earley AI Podcast with Seth Earley and Chris Featherstone - Episode #038 · The Adults In The AI Room

• 0:01
  SPEAKER_00: This is the early AI podcast.
• 0:06
  SPEAKER_00: As we developed the patient, and experiences of leadership expectations to talk about what's possible with artificial intelligence.
• 0:14
  SPEAKER_00: The early AI podcast is sponsored by early information science.
• 0:18
  SPEAKER_00: The digital transformation journey is designed to deploy innovative technology solution as well as DMSI.
• 0:25
  SPEAKER_00: Enjoy the show.
• 0:31
  SPEAKER_01: Okay, well, welcome to uh the Early AI podcast.
• 0:36
  SPEAKER_01: My name is Seth Early.
• 0:38
  SPEAKER_01: And I'm Chris Featherstone.
• 0:40
  SPEAKER_01: And uh we're very excited to introduce our guest today.
• 0:43
  SPEAKER_01: Uh, we're going to be discussing using knowledge graphs to improve large language model uh accuracy.
• 0:49
  SPEAKER_01: We're going to talk about some of the misconceptions uh as well as some better practices or best practices.
• 0:56
  SPEAKER_01: I always have an issue with best practices.
• 0:58
  SPEAKER_01: They're practices, who's to say they're best for using large language models, some accepted practice practices.
• 1:04
  SPEAKER_01: We'll talk about metadata, you know, the topic that's near and dear to my heart uh around uh and data engineering for AI, and then talk a little bit about the data infrastructure.
• 1:14
  SPEAKER_01: So, our guest today is the data and AI project owner at MoSIUS.
• 1:20
  SPEAKER_01: Uh, he manages a team of tech experts focusing on machine learning, knowledge graphs, and data analysis.
• 1:26
  SPEAKER_01: He has prior experience at Siemens Technology, which involved uh pioneering research in federated learning and self-supervised methods for anomaly detection.
• 1:37
  SPEAKER_01: Uh, he has extensive experience in AI and data analysis.
• 1:41
  SPEAKER_01: Uh, Alexander Schober, welcome to our show.
• 1:45
  SPEAKER_03: Thank you so much for the kind introduction.
• 1:47
  SPEAKER_03: Happy to be here.
• 1:48
  SPEAKER_01: Absolutely.
• 1:49
  SPEAKER_01: So, um, what I like to start, and we talked about this before when we were doing our intro call, is what are the big misconceptions that you're seeing in the space in terms of large language models, in terms of knowledge graphs, in terms of data preparation?
• 2:04
  SPEAKER_01: I mean, there's a lot of noise and confusion and you know, fear, uncertainty, and doubt, and all of the nonsense that uh, you know, that vendor hype and so on.
• 2:14
  SPEAKER_01: But what are you seeing?
• 2:16
  SPEAKER_01: What do you think are some of the bigger misconceptions?
• 2:20
  SPEAKER_03: So, yeah, like um maybe I can start a little bit off of the story, um, like how I got to all these topics, which I also mentioned in the past.
• 2:29
  SPEAKER_03: So um, like when GPT came out, I think everybody started thinking about hey, we want to have this one chat interface that will answer all our questions.
• 2:39
  SPEAKER_03: So I think that was in everybody's mind.
• 2:41
  SPEAKER_03: And then um you quickly figure out like, okay, what is missing to actually get there?
• 2:46
  SPEAKER_03: Um, I think one use case you see quite often is that you're able to do some document search.
• 2:53
  SPEAKER_03: Um, there's plenty of approaches there uh where you retrieve some text.
• 2:57
  SPEAKER_03: Um and I think there's a great value in those use cases as well.
• 3:02
  SPEAKER_03: Um we are working a lot with um companies in the mechanical engineering sector, and they have these huge manuals um which you need to search through.
• 3:11
  SPEAKER_03: So these are quite quite interesting use cases.
• 3:14
  SPEAKER_03: But then if you think about hey, we want to actually answer all the questions we have, then you also get um topics where you um need to look at data.
• 3:24
  SPEAKER_03: So, like more structured forms of data.
• 3:26
  SPEAKER_03: Um, and then you what what you then find is like, hey, there's approaches where you do text to SQL um and text to cipher and so on and so forth, right?
• 3:35
  SPEAKER_03: That's where then also already knowledge graphs come in.
• 3:38
  SPEAKER_03: Um but like if you ask again the question, how well does this work?
• 3:43
  SPEAKER_03: Yeah, it well it works well when you already have the data prepared.
• 3:46
  SPEAKER_03: Like, as long as I have data in a table where I can just write the SQL query for, um, that actually works quite well.
• 3:54
  SPEAKER_03: I think the challenge comes then when you don't have all your data in uh in a state ready for for these kind of queries.
• 4:02
  SPEAKER_03: Um and yeah, that's the the the question which has been uh yeah uh bugging me for the last I think year now.
• 4:11
  SPEAKER_03: Um, how do we actually get data in a state where we can use large language models to interface with it?
• 4:16
  SPEAKER_03: And then like let's imagine uh I am in a factory and I want to answer questions like okay, uh, which parts should I maybe order now?
• 4:25
  SPEAKER_03: Uh or um which should I produce so that I don't have any delays in my manufacturing?
• 4:30
  SPEAKER_03: Um which parts might be delayed in the next three months?
• 4:34
  SPEAKER_03: So even do some predictions on it.
• 4:36
  SPEAKER_03: And I think um in order to get there, and these are some questions people have, um, there are some things we need to put in place.
• 4:43
  SPEAKER_03: And um knowledge graphs are one thing which I think is important, especially when we think about the semantics of of the data which we need to cover.
• 4:51
  SPEAKER_03: Um but yeah, so I think there is actually like I I did a lot of uh interviews also with uh other experts in the field, and I've like I broke it down into six points, which I say uh think are quite interesting.
• 5:06
  SPEAKER_03: So, first one we need is kind of data discoverability.
• 5:09
  SPEAKER_03: So we need to be able to find the relevant data and we need to be able to access it.
• 5:13
  SPEAKER_03: And um if like if if your data is on some local Linux server under some table, it's really hard to access it uh with LLMs.
• 5:24
  SPEAKER_03: Uh even like if you do some data engineering projects or data analytics projects, it will take weeks until you you have access.
• 5:31
  SPEAKER_03: Um and so yeah, that's that's one thing which which holding back, like using LLMs on data, I think.
• 5:37
  SPEAKER_03: Then the second one, if the data is not trustworthy, right?
• 5:40
  SPEAKER_03: Um you can't um trust the output the LLM produces.
• 5:45
  SPEAKER_03: Um, and I think data trustworthiness is also a really, really important one.
• 5:49
  SPEAKER_03: You see this also that like data lineage is uh really important.
• 5:53
  SPEAKER_03: Also, like uh you have these concepts like that, uh active metadata.
• 5:57
  SPEAKER_03: Uh Atlan is, for example, pushing for where you have like you want to be able in your dashboards to understand all the lineage and where the data is coming from, which transformations have been applied.
• 6:07
  SPEAKER_03: So that I, as a consumer of the information, can then really trust uh where where it's coming from.
• 6:14
  SPEAKER_03: So I think that's another really important aspect.
• 6:17
  SPEAKER_03: Yeah.
• 6:18
  SPEAKER_03: Um uh yeah, but like going further, I think then another one uh which is uh is important is data uh availability and timeliness.
• 6:27
  SPEAKER_03: So like uh data in general, I think is most or like insights are most valuable um if they're served hot.
• 6:35
  SPEAKER_03: I think uh I hope you can say that in English.
• 6:37
  SPEAKER_03: Um again, if there's if they're what?
• 6:40
  SPEAKER_03: If it's uh served hot.
• 6:42
  SPEAKER_03: So like uh if if if you want to have the insights right now, because like let's imagine I'm a um retail uh company and I want to predict my like how much sales will I do in my online store for the next week, and I get this information in two weeks, then it's completely useless.
• 6:58
  SPEAKER_03: Sure.
• 6:59
  SPEAKER_03: And so like we want to uh to have it as real time as possible, or like as we can afford, I think.
• 7:04
  SPEAKER_03: Um then another thing which I think also goes into the topic of knowledge graphs, is like the the whole topic of data governance.
• 7:12
  SPEAKER_03: Like, which data am I actually allowed to see um and and use?
• 7:17
  SPEAKER_03: And for what purpose am I allowed to use this data?
• 7:20
  SPEAKER_03: And obviously, data governance has like way more aspects, but in the context of LLMs, I think it's especially important.
• 7:27
  SPEAKER_03: Uh, if I um yeah, want just have an LLM on all my data, uh, I need to be able to kind of tell it, hey, you're able, uh, like this user is able to uh look at this data for these use cases.
• 7:40
  SPEAKER_03: Right, right.
• 7:41
  SPEAKER_03: Um yeah.
• 7:43
  SPEAKER_01: Well, there was an interesting point you were making about uh product uh manuals before, and saying that if you have the product manual, you know, you can access that.
• 7:53
  SPEAKER_01: Um what what uh what I was talking with one of the uh large technology vendors who we built uh a component authoring environment for and built the governance and so on, they use their components, they do create ones publish everywhere, and they use their components to for lots of different purposes.
• 8:12
  SPEAKER_01: But one of the things we're talking about is the fact that those components still need the context of the overall document.
• 8:19
  SPEAKER_01: So if you say I have an error code, well, for what product, right?
• 8:24
  SPEAKER_01: What model, you know, what module, et cetera.
• 8:27
  SPEAKER_01: So are you seeing the need to kind of uh ingest components with overarching metadata that would provide context for those individual components?
• 8:40
  SPEAKER_03: Okay, now uh this will be a consultant answer.
• 8:43
  SPEAKER_03: I think it depends on the use case.
• 8:45
  SPEAKER_03: Um, but uh I see how, for example, like if you have a large set of documents, you want to maybe it depending on how the user wants to interact with the system, you might do some filtering of like which documents am I actually looking into?
• 9:00
  SPEAKER_03: Um, so for example, uh let's imagine uh I'm like a service engineer at a factory, uh and I like have some heater in front of me, for example.
• 9:12
  SPEAKER_03: Uh I want to ask LLM uh, hey, I have this error code, uh line three, two, four, uh, for a heater.
• 9:19
  SPEAKER_03: Can you please tell me how to resolve it?
• 9:22
  SPEAKER_03: Right.
• 9:22
  SPEAKER_03: And so, like, if I had now all the manuals of this company in the system, it would be really hard to infer from this context, like what manual or like which heater is he actually talking about.
• 9:34
  SPEAKER_03: Um, and so I think that's where then uh it becomes relevant to combine like more structured forms of information with uh the retrieval combination um capabilities of LMs, where you say, like, hey, I know the context in which the person is operating, and I maybe know the customer.
• 9:52
  SPEAKER_03: I know they have these, they bought these kind of heaters, I can already filter it down, um, and and so on and so forth.
• 9:58
  SPEAKER_03: So I think that's how you um yeah, can make the task easier uh by already providing some structured context.
• 10:06
  SPEAKER_03: Um yeah.
• 10:08
  SPEAKER_01: Yeah, that makes a lot of sense, and I think that's a piece people are missing is that they don't understand how you need to retain context uh for information when you have large documents or manuals or large numbers of documents, because again, just pointing the LLM to it, the LLM will not know how to uh disambiguate or you know differentiate between one error code for one product and then another error code for another product, unless you tell it what product it's for, right?
• 10:38
  SPEAKER_01: So um, so yeah, so that that's certainly interesting in our world.
• 10:42
  SPEAKER_01: Um so when you talk about, talk a little bit more about uh data preparation when um when you're looking at using large language models, um, you know, what are the key considerations when people need to think about uh their data preparation?
• 11:02
  SPEAKER_01: And I know it's all dependent upon use case and application, but what are the practices that you recommend, you know, when an organization needs to kind of evaluate either their knowledge or their data for a large language model application?
• 11:15
  SPEAKER_01: So what kind of things do you think about and do you walk in with?
• 11:19
  SPEAKER_03: Yeah, absolutely.
• 11:20
  SPEAKER_03: Um, so okay, I think um then I would again split it into two parts.
• 11:25
  SPEAKER_03: Like one is okay, we have some text or unstructured information which we want to search through.
• 11:30
  SPEAKER_03: And on the other hand side, the the structured information.
• 11:33
  SPEAKER_03: Um I think uh for for the the unstructured information, I would look into okay, which are actually the use cases where we can uh the most easily get benefits out of.
• 11:45
  SPEAKER_03: Um and that there are some things which are harder and some things which are easier to retrieve, and then I would focus on the low-hanging fruits.
• 11:52
  SPEAKER_03: Like, for example, if you look at the manuals, you also often have like um yeah tables or visualizations, which are definitely harder to retrieve and accurately uh accurately um yeah, reproduce, then it's maybe for like um some some just uh plain text.
• 12:13
  SPEAKER_03: So if you if you do decide based on that, like what are the use cases, which are the lower hangar fruits, I would I would go for the easier ones.
• 12:21
  SPEAKER_03: Um and then like when when I'm talking about the structured text, I already talked about these four aspects.
• 12:26
  SPEAKER_03: I think we need to have describability, we need to have trustworthiness, we need to have a uh availability and governance.
• 12:33
  SPEAKER_03: Uh, but the other thing which, and then we're like it's uh connected to the idea of knowledge graphs uh or ontologies is uh the semantics, um, which is also really, really important.
• 12:45
  SPEAKER_03: So um there's also this example where um I talked to an airline and they had uh they wanted to calculate how long is our average flight.
• 12:55
  SPEAKER_03: And now imagine you ask your LLM that how long is is our average flight?
• 13:00
  SPEAKER_03: Um they they thought, hey, this is actually an easy question.
• 13:03
  SPEAKER_03: We're gonna just go to the airports um and uh get uh get the data when our flights departed and when they arrived, and then we just take the average.
• 13:11
  SPEAKER_03: Uh, that should be easy.
• 13:12
  SPEAKER_03: But what you then find is that, hey, uh for example, Frankfurt uh calculates uh or considers a plane taking off as soon as the wheel uh lifts off the ground, and then Munich, on the other hand, side might uh say, hey, actually a plane departs when it moves from the gate.
• 13:31
  SPEAKER_03: Right.
• 13:32
  SPEAKER_03: And now you have a discrepancy.
• 13:33
  SPEAKER_03: And if you like if you start at the source uh where data is coming from and you lose this kind of semantics, um how uh how should an LLM ever figure out um how to calculate this and accurately?
• 13:47
  SPEAKER_03: Right.
• 13:47
  SPEAKER_03: So um I think like having the semantics in your data is really, really important if we want to um interface it with an LM.
• 13:56
  SPEAKER_03: And I think um I I I like from the what I uh learned so far this year is like there's two main uh things how companies uh approach it, right?
• 14:07
  SPEAKER_03: One thing was uh would be hey, we could uh do it on a metadata level, like um uh define a business uh ontology, which then maps into the specific implementations for different units, for example, and interface the LLM on top of that.
• 14:23
  SPEAKER_03: Or you find companies who kind of do like an end-to-end value stream for one particular user and um yeah, ingrain the whole semantics in there.
• 14:32
  SPEAKER_03: So you what you see is then, for example, there are companies who do this for the manufacturing space, where you can have one interface where you can answer ask all the questions a specific persona would have.
• 14:43
  SPEAKER_03: And underneath that, they then have like an ontology, which um kind of provides the semantics for the data.
• 14:49
  SPEAKER_03: And underneath that, they have some integration for the relevant data sources in these kind of use cases, and then you what you get is kind of an end-to-end insights platform for one specific use case, and then I think building this more general for the for a whole enterprise becomes really, I think, difficult because then you need to think about okay, how do I create these semantics for my whole enterprise?
• 15:14
  SPEAKER_03: How do I maintain it?
• 15:16
  SPEAKER_03: Um, and and so on and so forth.
• 15:18
  SPEAKER_03: So, yeah.
• 15:19
  SPEAKER_02: So, how do you, Alexander, then um help companies guide like, you know, or what what would be you know your approach to like governance and guardrails, system access, human access, um, you know, clean rooms, things like that.
• 15:36
  SPEAKER_02: Well, what would how do you actually you know like approach that?
• 15:39
  SPEAKER_02: Because we we get hit with that all the time right around I've got this data, I need it anonymized, but I need it shared.
• 15:45
  SPEAKER_02: So I need this clean room concept, but I need the guardrails, you know, kind of concept put on.
• 15:50
  SPEAKER_02: How do you how do you you know what do you what do you talk about when you know with with maybe suggestions and ideas there?
• 15:59
  SPEAKER_03: Um so yeah, in the in uh in these kind of projects, um when when we're talking to customers, usually or many of our customers are not yet that far that they have a really good concept.
• 16:11
  SPEAKER_03: So what we usually start with is like an assessment of where are they actually.
• 16:15
  SPEAKER_03: Like if we if you look like at these six aspects, for example, there is other frameworks which you can use to assess like the readiness.
• 16:22
  SPEAKER_03: Yeah, um, and we we start with them and try to figure out like, hey, where are you and where are you lacking?
• 16:28
  SPEAKER_03: Um, and if if they're really in the beginning, I think uh it's always like a challenge to to set up huge projects for data governance for like this.
• 16:38
  SPEAKER_03: Also, like uh I'm a big fan of knowledge graphs, but setting up knowledge graphs is not a cheap endeavor.
• 16:44
  SPEAKER_03: Um so you really need to find out like where is like the ROI in this journey and what are like some specific use cases where I can say, like, hey, if I'm um gonna approach this, I should start here because I can prove it will it will have a positive impact.
• 17:00
  SPEAKER_03: And so what we usually then do is try to come up with like a target architecture of how it where we want to get to, and then we want to find like at least two, we call it lighthouses, where we say, hey, like if we if we if we uh deploy these two use cases, both of them will have like an ROI which is measurable, which we can kind of uh argue for, and but both of them should already target the same um target architecture in the end.
• 17:31
  SPEAKER_03: And then that's how we uh usually approach uh approach these kind of setups within companies, um, trying to help them move there and getting all the stakeholders on board.
• 17:41
  SPEAKER_01: What have you seen for um high value clear ROI use cases and uh and what what is the technology stack that you're typically working with?
• 17:53
  SPEAKER_01: What kinds of uh knowledge graph tools and and so on?
• 17:56
  SPEAKER_01: So just curious, what what you're seeing is kind of a clear ROI without you know a huge long let heavy lift to kind of help people understand the value, uh because that's challenging.
• 18:08
  SPEAKER_03: Yeah, that the that's a uh good question.
• 18:11
  SPEAKER_03: So I think the the best use cases is when there is some regulation in place.
• 18:18
  SPEAKER_03: So um, and I think if, for example, if you need to do some kind of reporting for environment, health and safety, that's something where it could be, hey, we actually need to kind of provide the provide this data to the to some uh regulators.
• 18:32
  SPEAKER_03: Um, so we need to also kind of get it together.
• 18:35
  SPEAKER_03: Um, we could do that in in the ontology, which we can then map to whatever the regulator needs.
• 18:40
  SPEAKER_03: So that's for example a use case where you where you can start.
• 18:44
  SPEAKER_03: Um and then I think like the generator for how to find these kind of use cases where there's higher RRI or where there's a larger benefit of using a knowledge graph compared to uh something else is if you have a lot of data linked.
• 18:59
  SPEAKER_03: So if you if if you would if you have questions which require a lot of joins within tables, then and you have a lot of hops, that's when you start seeing that like a knowledge graph actually provides uh additional uh additional benefits.
• 19:13
  SPEAKER_03: Um the next kind of cluster I uh I I heard a lot uh during during the last year was hey, we actually have uh a core entity uh where we have a bunch of data sources that uh contributed to it, and we all want to bring those together.
• 19:32
  SPEAKER_03: So for example, I talked to um yeah, some um uh some people in working in in uh in the public sector, and for example, they they were interested in the concept of a citizen and all the data related to the citizen, um that that they created as like an ontology, and then they mapped all the data sources which they could find about the citizen within in this um in this ontology.
• 19:59
  SPEAKER_03: Um and so I think in these cases where you have like a bunch of data sources contributing to the same concepts, uh that's when also you you might be interesting to um concept of knowledge graphs and ontologies and so on.
• 20:13
  SPEAKER_01: Yeah, yeah.
• 20:14
  SPEAKER_01: So customer 360 types of things or exactly, yeah, exactly.
• 20:18
  SPEAKER_01: So this is also something uh we we look into uh into uh quite one of the things that we talked about um is a unified metadata model and why it's critical in the context of data engineering and AI.
• 20:32
  SPEAKER_01: Do you want to just for the audience who may not be as familiar with what that might entail, or there may be a lot of different interpretations when we talk about unified metadata models?
• 20:43
  SPEAKER_01: I look at that as an enterprise architecture and you know, kind of the standards and um the standard organizing principles across multiple domains and multiple areas.
• 20:54
  SPEAKER_01: Do you want to talk about your uh interpretation of um a unified metadata model and how that fits in?
• 21:01
  SPEAKER_03: Sure.
• 21:01
  SPEAKER_03: Um so again, like when I when I drilled down over this year, um I did a bunch of conversations, and what I from from my perspective uh perspective, what I found was that there's a lot of metadata around data pipelines and the data engineering, but this is often kind of disjointed.
• 21:20
  SPEAKER_03: So um, so for example, you have data modeling tools in one world, uh, then you have uh data catalogs, which uh supposedly are like the metadata platforms, but they are also kind of disjointed from the pipelines.
• 21:34
  SPEAKER_03: Then you have um data observability, uh, which is again producing metadata about the quality of the data, right?
• 21:42
  SPEAKER_03: Uh and the state of the data, which is again somehow disjointed from the data catalogs.
• 21:48
  SPEAKER_03: So you now see them, I guess, moving closer together.
• 21:52
  SPEAKER_03: Um, but what I want to say is basically, from my point of view, it looks like that these kind of tools, which all Deal with metadata are kind of disjointed and not unified in one uh single platform.
• 22:06
  SPEAKER_03: Um and I think that provides challenges.
• 22:10
  SPEAKER_03: So, like if you, for example, what I also find interesting, there's all these data modeling tools.
• 22:16
  SPEAKER_03: Um, yet um especially for the logical and physical layer, like for the conceptual layer, I think there's not that many good ones.
• 22:26
  SPEAKER_03: Um this information is not really reused when I build my pipeline.
• 22:31
  SPEAKER_03: So um, hey, I'm actually defining like uh my constraints on my data somewhere, but then when I build a pipeline, um it seems like I need to rewrite a lot of this stuff, right?
• 22:44
  SPEAKER_03: So, like let's imagine in dbt you want to create the dbt um quality and integrity checks.
• 22:50
  SPEAKER_03: Like this information should already be there if I did some data modeling uh to some extent.
• 22:56
  SPEAKER_03: Um, and so I I think there is a lot of um disjointment between the tools.
• 23:01
  SPEAKER_03: I think Atlan and DBT are now moving closer together um and trying to bridge this gap, but I think there's a yeah, it's not yet fully um yeah, corrected.
• 23:13
  SPEAKER_02: And so I was just saying, would you agree with the statement that although AI is not uh you know, you you know you're not required, you know, to have a graph environment for an AI model, regardless if it's discriminative or or um generative, but would you say that it absolutely enhances all the abilities of AI if you do have a graph?
• 23:39
  SPEAKER_02: Not required, but enhances the ability.
• 23:41
  SPEAKER_03: Is that fair?
• 23:43
  SPEAKER_03: Yeah, so like my background is more in AI than data engineering, right?
• 23:46
  SPEAKER_03: So I would say there I need to make a distinction, like which kind of AI?
• 23:51
  SPEAKER_03: AI like is a broad term.
• 23:54
  SPEAKER_03: Um if I talk about LLMs and the capability of LLMs to kind of answer questions based on my structured data, then I would say probably yes.
• 24:07
  SPEAKER_03: If I think about other use cases for AI, then I I I would maybe come to a different thing.
• 24:11
  SPEAKER_02: So I'm glad you brought that up, and I'm glad that you um, you know, in terms of the delineation between the two, because we see that there's those that, you know, especially when they're getting their data prepped, that you you are arguably have to have a good knowledge infrastructure and and you know information um architecture, right?
• 24:29
  SPEAKER_02: And that's one of the really, really um, you know, key areas that Seth you know pushes hard is you just can't have good AI without a good information architecture.
• 24:39
  SPEAKER_02: I get it, right?
• 24:40
  SPEAKER_02: Um, not required, but I think you know, we get into scenarios, yeah, for you know, different use cases and different AI types and approaches where I think it starts to become essential for some of those.
• 24:54
  SPEAKER_02: In fact, we're getting into scenarios now where utilizing a discriminative model to actually create that pattern recognition and be a data generator is absolutely essential before actually interacting with, let's say, a large model to actually do something neat, interesting, and innovative.
• 25:11
  SPEAKER_02: So, because that becomes our input into you know, utilizing to retrieve that data back to actually make something interesting and maybe more human, reasonably reasonable reasoning with that type of information, right?
• 25:24
  SPEAKER_02: But I think you know, people by and large miss.
• 25:27
  SPEAKER_02: In fact, we're getting also into scenarios, and I don't know, Alexander, if you're you know, if you've been had experience with, I'd love to get your take if utilizing generative you know models and LLMs to actually generate graphs and RDF files, because we're getting to those scenarios now, too.
• 25:42
  SPEAKER_02: Right.
• 25:43
  SPEAKER_03: So, yeah, that's something we experiment with um uh right now.
• 25:47
  SPEAKER_03: Like uh again, if customers come to us, uh usually they want to have like the quick wins.
• 25:53
  SPEAKER_03: And there usually are quick wins.
• 25:55
  SPEAKER_03: Uh so we didn't get yet to the uh, or mostly didn't get to the like the really big use cases where you then might to uh need such a system.
• 26:05
  SPEAKER_03: Um but personally, I think like if you would now give me 100 million dollars, I would try to build that.
• 26:13
  SPEAKER_03: So basically, I would uh be interested in um building a self-constructing knowledge graph, which also does some kind of uh entity disambiguation or information resolve, like conflict resolvement within the knowledge graph, uh and then build an LLM on top of that.
• 26:31
  SPEAKER_03: So, like the why I think that that's kind of relevant is like let's imagine I'm I'm just I just have an LLM interface on top of my confluence.
• 26:41
  SPEAKER_03: And in my confluence, I have uh two pages which have conflicting information.
• 26:47
  SPEAKER_03: So one says, like, hey, Napoleon was born around uh at the same time as Christ, and the other one said, like uh um yeah, when he was actually born.
• 26:57
  SPEAKER_03: So then the the question is, I I asked the LLM when was Napoleon born?
• 27:02
  SPEAKER_03: Um and yeah, okay, maybe based on the information it got from the internet, uh like the external context, it might be able to answer it.
• 27:09
  SPEAKER_03: But if it's now like something really specific, which is my company internal information, so like this product uh has these features versus this product has these features, um, how should it know?
• 27:21
  SPEAKER_03: And I think then it would be interesting to bring both of these information points of information into a knowledge graph and then have some kind of reasoning or resolvement of these conflicts, which is um transparent or explainable, so that when I then ask it, like, hey, when was Napoleon born?
• 27:40
  SPEAKER_03: It cannot only tell me, like, hey, I'm he was born here, but it can also tell me, okay, based on these other facts with I which I know to be true in my knowledge graph, I'm 70% confident he was born then.
• 27:54
  SPEAKER_03: But I also have information which results from these sources which say something else.
• 27:59
  SPEAKER_03: And so I think that's where like I think that that would be a cool, uh, cool place to go.
• 28:07
  SPEAKER_02: Interesting.
• 28:08
  SPEAKER_02: Yeah, most of most of the stuff that we think about too is centered around um, you know, I do a lot of a lot of work in the telecommunications and in you know, common communication service for service providers, but where we look at it to help utilize um, you know, these these the ability to look at all the communication nodes within a given graph environment and then dynamically allocate you know scenarios around some of that information to drive, let's say, optimal usage and or optimal communication types and generate you know those RDF files and or and also utilize generative models to look at it from a perspective for um you know architectural approval processes and things.
• 28:53
  SPEAKER_02: So get it's starting to get interesting centered around utilizing some of these techniques, like you're talking about around how to represent the right information structure, you know, for some of these systems that normally would be super, super difficult to query.
• 29:10
  SPEAKER_02: And now it's giving you know everybody the ability, or let's say the more general availability of being able to interact and query and understand those very, very complex systems because of things like you know the complexity around the context and things, but also you really get insightful you know information around it, you know.
• 29:31
  SPEAKER_03: So I totally agree.
• 29:33
  SPEAKER_03: So, like we also like we have a bunch of knowledge craft projects, and uh um so far, what we usually do is we go to the user uh and ask him, like, what actually the questions you want to answer, and then we provide template, like we provided template questions, so that like you can just insert uh given points, and like um that will help you answer this question.
• 29:56
  SPEAKER_03: But now actually it became way easier, so now you can just use LLMs and create like maybe you in the background you can still have some uh templates for the the common questions, but then for the other ones, you can just try to generate the the cipher, and so like the utility of a knowledge graph just gets way, way bigger, um just by like being able to answer more and more questions for people who actually don't know how to write those cipher queries for the data which is already there.
• 30:22
  SPEAKER_03: Um, so yeah.
• 30:24
  SPEAKER_03: I think enabler, yeah.
• 30:27
  SPEAKER_01: Yeah, and and you know, just going back to uh building a um uh building out a um uh a data pipeline, you talked a little bit about you know creating an enterprise scale pipeline using metadata.
• 30:44
  SPEAKER_01: Do you want to just talk a little or expand a little bit more on how that works when you think about uh that pipeline?
• 30:50
  SPEAKER_01: You know, what is the role of metadata there?
• 30:52
  SPEAKER_01: And there's lots of different roles, I realize, but as you kind of see constructing that enterprise scale pipeline.
• 30:59
  SPEAKER_03: Sure.
• 31:00
  SPEAKER_03: Um, so actually that's something I recently uh tried out with some colleagues because uh we started from this point where I said, like, hey, uh we see that the data catalogs and the pipelines are actually disjointed.
• 31:12
  SPEAKER_03: Can we yeah, maybe generate uh dbt integrity checks based on the metadata I have uh in my data catalog?
• 31:20
  SPEAKER_03: And um, so yeah, that worked.
• 31:22
  SPEAKER_03: It's actually not a very difficult task to do.
• 31:25
  SPEAKER_03: So then we thought, okay, can we go even further?
• 31:28
  SPEAKER_03: Like, how far can we go of using metadata to generate pipeline code and then also again, like all the quality checks, uh, all the integrations and so on and so forth.
• 31:39
  SPEAKER_03: Um, and it turns out uh it works surprisingly well.
• 31:42
  SPEAKER_03: Uh so what you can do is uh like these uh large language models have actually the capacity to draft whole data pipelines.
• 31:50
  SPEAKER_03: So you just tell them, hey, these are my sources, these are like the semantics between uh behind them.
• 31:56
  SPEAKER_03: Like what do they mean?
• 31:57
  SPEAKER_03: So let's as a concrete example, um, use customer lifetime value.
• 32:03
  SPEAKER_03: So I have a raw customer table with these kind of uh columns, that's what they mean um in plain text.
• 32:09
  SPEAKER_03: So I I provide some metadata.
• 32:11
  SPEAKER_03: Um and then I I have a second table which is a raw sales table.
• 32:17
  SPEAKER_03: I can tell it, hey, these are the two tables I'm interested in, and I I want to create a customer lifetime value out of this, and then I tell it it should have um or like a customer report and it should have these columns, and this is how they should be calculated.
• 32:32
  SPEAKER_03: Um, so what the LLM now can do is actually it can first propose you like a DAC of how to come from your source to your target.
• 32:40
  SPEAKER_03: So it will do that, and then once you if you break it down far enough, you you can already uh propose some transformations, and then you if you interface well with your metadata catalog, you can also propose, hey, this is this should be a data steward.
• 32:56
  SPEAKER_03: Um, this should be um the governance attributes, this should this should be the uh privacy classification for the data you're like the steps in between you're generating.
• 33:05
  SPEAKER_03: So and I think that would be that's actually quite um interesting because uh also in in my talks over last year, I heard over and over again that uh data engineers or are often in in this kind of a dilemma where they uh feel like we don't have enough time to build data pipelines properly.
• 33:27
  SPEAKER_03: So we need to cut corners here and there.
• 33:29
  SPEAKER_03: And this would and what is usually cut um seems to be the data quality checks, uh, the metadata definition, the documentation.
• 33:39
  SPEAKER_03: So I I quickly build something, but then in the in the end it becomes really hard to make.
• 33:43
  SPEAKER_03: And if we are able to generate the things which are usually cut, um my hope would be that it actually will like make sure people actually do it.
• 33:53
  SPEAKER_03: Uh, and then over time this will increase like the metadata quality, but also the data quality, the pipeline robustness uh within a company.
• 34:02
  SPEAKER_03: So uh some things I'm trying out uh with some colleagues like lately.
• 34:08
  SPEAKER_01: And what tools do you typically use?
• 34:10
  SPEAKER_01: Like what um are you using uh commercial or open source, or what kinds of technologies do you typically use?
• 34:18
  SPEAKER_03: Uh depends on where uh where we are.
• 34:22
  SPEAKER_03: Um, so I think usually we go with whatever uh our customers uh already use, usually they have a preference.
• 34:31
  SPEAKER_03: So then um that's what we go for.
• 34:33
  SPEAKER_03: Uh we have like a a structure in uh which we call fluid structure, so we are able to kind of select freelancers based on the given projects, and so that's why we actually can can do that quite well.
• 34:46
  SPEAKER_03: Um, but then yeah, oftentimes we go with open source.
• 34:49
  SPEAKER_03: I think like for for example, now if you go for uh transformations, I think TBT now became like uh the gold standard.
• 34:57
  SPEAKER_03: Um but then yeah, uh I think we would need to talk a little bit more which which aspects.
• 35:02
  SPEAKER_01: But yeah, sure, sure.
• 35:04
  SPEAKER_01: Um, one of the things that we had uh talked about uh was the importance of data contracts and of course uh governance.
• 35:12
  SPEAKER_01: Do you want to talk a little bit about what those data contracts entail, how they're used, and then what's unique about governance and generative AI?
• 35:22
  SPEAKER_03: Sure.
• 35:23
  SPEAKER_03: Um, so that's again something I came across on my journey to answer like how what what is um holding us back from putting LLMs on top of our data stack.
• 35:33
  SPEAKER_03: And then um I already mentioned the data availability and timeliness, and I think that that's maybe where you can put data contracts.
• 35:41
  SPEAKER_03: So if uh one problem you find if you talk to people is if I make a change upstream, um uh and and don't consider like where my data is used downstream, um, I might just break something.
• 36:00
  SPEAKER_03: And then like combine that with uh not optimal documentation, it becomes a real pain because you uh you break something and it takes maybe two, three weeks to figure out like what actually like where did it now break.
• 36:14
  SPEAKER_03: And so um I the idea of the data contracts is hey, I I will have an agreement between producers and consumers of data.
• 36:22
  SPEAKER_03: What actually like, yeah, how does my data look like?
• 36:26
  SPEAKER_03: What is like the quality attributes, what is the L SLA of this data, and so on so forth.
• 36:32
  SPEAKER_03: And then it will be enforced so that I like I can't make upstream changes which violate my data contract, so I can prevent data pipelines from breaking.
• 36:39
  SPEAKER_03: I think Chad Sanderson is like uh the the one pushing uh pushing for these these kind of topics.
• 36:46
  SPEAKER_02: I I I know you have you know definitely have an opinion too, around like just in general terms, like model drift, right?
• 36:54
  SPEAKER_02: And when when things get stale, what are some things that our listeners should know and understand about that in terms of what to look for, right?
• 37:03
  SPEAKER_02: What are the going to be the key aspects to making sure that your models stay relevant all the time and evolve with you over time?
• 37:12
  SPEAKER_02: Will you talk about that a little bit?
• 37:13
  SPEAKER_02: Because I know you have an opinion on it for sure.
• 37:16
  SPEAKER_03: Yeah, uh definitely.
• 37:17
  SPEAKER_03: Um, so if we look at machine learning models, for example, um they are trained on a given distribution, right?
• 37:26
  SPEAKER_03: Uh on a set of data.
• 37:28
  SPEAKER_03: And what now happens if data changes over time, it might uh it become a different distribution, and the model won't perform that well.
• 37:38
  SPEAKER_03: So let's imagine um, again, industrial uh context, uh I'm heavily biased, like you have some some um yeah anomaly detection model or some predictive maintenance model, um, which uses as one feature the temperature.
• 37:54
  SPEAKER_03: You train it in the summer and it works brilliantly.
• 37:58
  SPEAKER_03: But then over the year uh it becomes colder in your factory.
• 38:01
  SPEAKER_03: Uh and in the winter, uh the temperature drops significantly.
• 38:05
  SPEAKER_03: So now the data which is going into your model is something different.
• 38:10
  SPEAKER_03: And what likely will happen is that your model doesn't perform that well anymore because kind of the input, like the situation in which it's like living in uh changed.
• 38:22
  SPEAKER_03: And so um I think that's really important that you kind of uh monitor like how loud how does your data um drift, how does your data change over time so that you're at the right time uh uh understand that that's happening, and then you are able to retrain your models.
• 38:41
  SPEAKER_03: If you don't, like it's kind of you need to do some kind of uh predictive maintenance for your models in that sense as well.
• 38:47
  SPEAKER_03: Um if if you don't, like they will just degrade over time and and the predictions will suffer.
• 38:52
  SPEAKER_02: Um yeah, I always I often talk about it's you know, it's the the not necessarily constant, but it's the the the health checks and the care and feeding of the intelligence of these things to actually stay relevant and and make sure that it's either accuracy that you're looking for and and go ahead and test that, right?
• 39:10
  SPEAKER_02: Accuracy, or if you if you've had maybe even any model poisoning that had happened, right?
• 39:16
  SPEAKER_02: With maybe adverse data that you put in there that you didn't plan for, and now all of a sudden, boom, you're there.
• 39:22
  SPEAKER_02: I mean, right?
• 39:23
  SPEAKER_02: We got Samsung putting their code out into a public model.
• 39:26
  SPEAKER_02: No bueno, because of what that can provide, right?
• 39:30
  SPEAKER_02: But I think it's the care and feeding.
• 39:32
  SPEAKER_02: Anyway, you're gonna say something.
• 39:33
  SPEAKER_03: No, uh yeah, I absolutely agree.
• 39:35
  SPEAKER_03: I think the the challenge in in some applications might be that you don't actually get the real accuracy.
• 39:42
  SPEAKER_03: Like um, yeah, if, for example, in the production line you want to sort out is uh like are my products good or bad, you would need to do manual labeling in the end to be able to figure out how well your uh model works, and that kind of defeats the purpose.
• 39:59
  SPEAKER_03: Um, in that case, is I think like um monitoring the data stream coming in and figuring out how does like the distribution change is is a way to kind of approach it.
• 40:12
  SPEAKER_03: Um yeah, yeah, so yeah.
• 40:17
  SPEAKER_01: Can can you talk about a recent challenge that you've solved for a customer and kind of walk through what what they were facing and then what those steps were and kind of the quick wins and so on?
• 40:28
  SPEAKER_01: Can you walk through uh recent um problem?
• 40:32
  SPEAKER_03: Um yeah, which one to pick?
• 40:36
  SPEAKER_02: Um, I think um I can definitely definitely pick the one that that folks are gonna be like, I have that problem.
• 40:44
  SPEAKER_02: Yes, I need help.
• 40:46
  SPEAKER_03: Okay, like um, okay, then it's ongoing and I uh we didn't figure out all the the the kinks yet.
• 40:52
  SPEAKER_03: But um one thing I see is like um, especially like in German SMEs, um, which are in the manufacturing industry, is the state of their master data for parts and materials.
• 41:05
  SPEAKER_03: So um, like one challenge uh I saw and like it resonated a lot was like, hey, uh there was this new regulation and this coding um is no longer allowed.
• 41:16
  SPEAKER_03: So now we need to figure out on which parts is actually this coding.
• 41:21
  SPEAKER_03: Um and for like one of our customers, this sparked a whole data science project because um this information was only in their technical drawings, it wasn't even in like some some kind of database.
• 41:34
  SPEAKER_03: Um even if it was, then you usually have like this uh massadata problems, like people put in the wildest stuff uh for from codings, uh materials, um, weight of parts.
• 41:48
  SPEAKER_03: Um, and so this seems this seems to be a huge challenge, um, which kind of uh within multiple companies.
• 41:56
  SPEAKER_03: And that's something where we we looked into okay, can we actually use machine learning to kind of resolve some of these challenges, at least um provide assistance with um cleaning it up?
• 42:07
  SPEAKER_03: And then um what we want to build ultimately is like a single point of truth for all your uh part metadata, uh part metadata, uh material metadata, and then enabling a lot of use cases.
• 42:19
  SPEAKER_03: And I think that that again could be a really nice knowledge graph use case.
• 42:23
  SPEAKER_01: I wanted to try off on that solution where you're talking about the codings uh issue or the regulatory issue, and you said that a lot of that content uh is in you know spec sheets and PDFs.
• 42:35
  SPEAKER_01: That's a common problem for organizations trying to harvest any kind of product data, and it's an area that we've been experimenting.
• 42:43
  SPEAKER_01: So, how far along did you get with that type of solution?
• 42:47
  SPEAKER_01: Is it a matter of um you know processing those PDFs to understand where that particular coding substance was using and then tracing that back to you know an ERP system that could kind of tell you uh where that was um uh where that was where else that was used, or what other assemblies, what other products, how did that kind of play out?
• 43:11
  SPEAKER_03: Yeah, sure.
• 43:12
  SPEAKER_03: Um so um the codings were not the only information which was only stored in in the technical drawings, right?
• 43:21
  SPEAKER_03: There's a bunch of other information which like over the course of the last 80, 100 years, uh was just captured in these drawings.
• 43:28
  SPEAKER_03: And you can't go and extract all of it.
• 43:31
  SPEAKER_03: Uh again, you need to find like the the most crucial pieces which are actually technically feasible.
• 43:36
  SPEAKER_03: Um also like extracting information from technical drawings is not uh an easy feat.
• 43:41
  SPEAKER_03: There are so many edge cases.
• 43:43
  SPEAKER_03: Um, but yeah, so what we did in this project was like initially that was the the the first uh problem.
• 43:49
  SPEAKER_03: Hey, we want to extract uh information from technical drawings, and we need to because we need it right now.
• 43:55
  SPEAKER_03: But then the next thing will be hey, can how can we make sure that in the future we don't Need these kinds of projects anymore where we go in and have like computer vision specialists try to uh extract this information with uh OCR and some pattern matching and um maybe some AI, right?
• 44:13
  SPEAKER_03: Um and so that's where where we want to go.
• 44:15
  SPEAKER_03: And I think then it starts making sense to actually connect uh the systems where the data originates from.
• 44:22
  SPEAKER_03: So, like if you have like a uh CAT system uh where you actually produce these kind of data, um, you should not put it first in the PDF, but then directly into a knowledge graph.
• 44:34
  SPEAKER_03: So that's I think you wanna um you you wanna on one side clean up what you need to clean up uh from from the past and then come up with a system which will in the future make sure that yeah you will have everything accessible and a good quality and and so on.
• 44:52
  SPEAKER_01: So much of our our efforts around AI and machine learning are making up for our past sins in towards the culture, right?
• 45:01
  SPEAKER_01: Not having systems and processes.
• 45:03
  SPEAKER_01: So that's been great.
• 45:04
  SPEAKER_01: By the way, um uh you're based in in uh Europe.
• 45:07
  SPEAKER_01: Where what do you uh what do you do outside of work?
• 45:10
  SPEAKER_01: What's uh your your world like uh outside of knowledge graphs and uh yeah outside of knowledge graphs, um yeah, I I really love to play basketball.
• 45:19
  SPEAKER_03: Um so that that's uh it would be also American thing.
• 45:22
  SPEAKER_03: Uh but um no, no, uh I really really enjoy it.
• 45:26
  SPEAKER_03: Um that that that's my hobby.
• 45:28
  SPEAKER_03: Uh where are you where are you based, Alexander?
• 45:31
  SPEAKER_03: I'm in Munich.
• 45:32
  SPEAKER_03: Um city of the October first, I guess.
• 45:35
  SPEAKER_02: So if anybody rewrote basketball, it's Nowitzki, right?
• 45:38
  SPEAKER_02: Come on.
• 45:39
  SPEAKER_02: Yes, you've got you know from the best of all time that's you know a nice one of your fellow countrymen and it's phenomenal.
• 45:45
  SPEAKER_03: So yeah, I always try to do the um fadeaway with yeah, my trainer hated it uh because it never went in that as well.
• 45:55
  SPEAKER_03: But uh yeah.
• 45:56
  SPEAKER_01: Uh you do some skiing?
• 45:58
  SPEAKER_01: Do you do some skiing?
• 45:59
  SPEAKER_03: I yeah, sure.
• 46:00
  SPEAKER_03: Uh from now and then.
• 46:01
  SPEAKER_03: I like Munich is so close to the Alps.
• 46:04
  SPEAKER_03: Um it's just beautiful.
• 46:06
  SPEAKER_01: That's great.
• 46:07
  SPEAKER_01: That's great.
• 46:08
  SPEAKER_01: And what's on tap for you next year?
• 46:11
  SPEAKER_01: Anything uh interesting and exciting?
• 46:13
  SPEAKER_01: Um, your uh any new initiatives or just uh continuing to refine the stuff that you're you're working on?
• 46:21
  SPEAKER_03: Yeah, so I'm I'm really interested in these topics, like especially how the different tools in the data uh space are disjointed.
• 46:29
  SPEAKER_03: I think there's still some opportunities.
• 46:32
  SPEAKER_03: Um and then yeah, also like cleaning up these this metadata for SMEs is also something which I I feel is really valuable.
• 46:41
  SPEAKER_03: Um so I'm gonna just look these are the topics which are really interesting for me.
• 46:46
  SPEAKER_03: And I want to understand better like what are actually the the problems, what are the the ways we can tackle this on a scalable in a scalable fashion.
• 46:54
  SPEAKER_03: Yep.
• 46:54
  SPEAKER_03: Um that's what what's driving me, and that's why I'm also talking to a lot of people in the industry to to learn about these things.
• 47:00
  SPEAKER_01: Yeah, that's what gets you excited.
• 47:02
  SPEAKER_01: We can uh your company is motius, m-o-t-i-us, so um motius.com, so we can find that.
• 47:10
  SPEAKER_01: And then you're on LinkedIn Alexander dash uh show s-c-h o-b-e-r, and those will be in the show notes as well.
• 47:19
  SPEAKER_01: And um, and I guess you're also on Twitter, yes?
• 47:22
  SPEAKER_01: Or actually I'm just on LinkedIn.
• 47:25
  SPEAKER_01: Uh the LinkedIn, okay.
• 47:27
  SPEAKER_01: All right, I thought you had some Twitter, but the the the company formerly known as Twitter.
• 47:32
  SPEAKER_01: Anyway, thank you so much for your time today.
• 47:37
  SPEAKER_01: It's been great to have you.
• 47:38
  SPEAKER_03: Yeah, thank you so much for having me.
• 47:40
  SPEAKER_03: It was a pleasure talking to you.
• 47:41
  SPEAKER_03: It was really interesting.
• 47:42
  SPEAKER_03: Um, and thank you, Alexander.
• 47:44
  SPEAKER_03: It's great, great to meet you.
• 47:46
  SPEAKER_01: Thank you.
• 47:46
  SPEAKER_01: Uh thanks to the audience, the folks that are listening in.
• 47:50
  SPEAKER_01: I hope you learned something today and uh uh found something of interest.
• 47:54
  SPEAKER_01: So uh again, uh Alexander, uh, thank you so much for your time today.
• 48:00
  unknown: Thank you.
• 48:01
  SPEAKER_00: Thank you for joining us on another deep dive into AI innovation.
• 48:05
  SPEAKER_00: Tune in next time when we introduce another industry expert and discuss how to maximize AI in your world.
• 48:12
  SPEAKER_00: The Early AI podcast is sponsored by Early Information Science and EMS Wire.
• 48:17
  SPEAKER_00: To learn more, visit early.com.
• 48:19
  SPEAKER_00: That is E-A-R L E.com.
• 48:24
  SPEAKER_00: Thanks for tuning in.