Latest / The Data Splash / What Exactly is Knowledge Engineering? - with Jessica Talisman
Transcript
- Ido Bronstein: Welcome everyone to the Data Splash Podcast brought to you by Upriver. If you are a data engineer a data leader or just someone trying to make sense of what the hell is going on with data and AI right now, you are in the right place. And today with me is Jessica Talisman Hey Jessica, very happy that you join us. Jessica Talisman: Thank you so much for having me. Ido Bronstein: So Jessica is a librarian who became one of the top voices in Knowledge Engineering. She built knowledge graphs at Adobe and Amazon, created the Ontology Pipeline framework, and her book comes out soon. September Jessica Talisman: September. Ido Bronstein: very soon. Yes. Jessica Talisman: Yes. Ido Bronstein: So I'm really excited that you are with us today to talk about knowledge engineering, context engineering. And what everybody is talking about as the "company brain". There is a lot of names to those things. But before we are diving deep too much, we have a small tradition here. We call it the 30 Second Splash. So I'm going to ask a quick question. You're going to answer in one word. We have 30 seconds for that. It's no time to think just instinct. So you're ready to jump in? Jessica Talisman: Yeah, let's go. Let's go. Ido Bronstein: Let's go. So knowledge or context? Jessica Talisman: Knowledge. Ido Bronstein: Most misused data word of 2026? Jessica Talisman: Context Ido Bronstein: Is AI going to take over the world? Yes, for sure. Jessica Talisman: Yes, yes, yes. Ido Bronstein: Yes. Can AI build a taxonomy on its own? Jessica Talisman: No. Ido Bronstein: No. Best thing about Santa Cruz that has nothing to do with data. Jessica Talisman: Mm, the ocean. Ido Bronstein: The ocean. for sure the ocean. Thank you very much. So before Jessica Talisman: Thank you. Ido Bronstein: I'm talking more and diving into the detailed topics, if you can give a bit of overview of your career and how you arrived to your position today, I think the audience will love to hear that. Jessica Talisman: Sure. I actually started my journey in the late 1990s building one of the largest visual history catalogs in the world using brand new technology. I worked for Steven Spielberg for a project called Shoah Foundation for Visual History. So it's an incredible project. It's still very much alive. It's a database of more than 11 and a half years of footage of interviews and testimonies organized into three to six minute clips that are searchable. So that was like a way ahead of its time in the late 90s. It was very Ido Bronstein: Yeah. Jessica Talisman: early and we were still using VHS tapes. It was before large scale. It was very much at the beginning and forefront of that digitization of moving images. So I've worked in cultural heritage, I've worked as you know I have a master's in information and library science. I've worked as an academic librarian. So yes, I'm a librarian, but then I've been building semantic systems ever since the late 90s and only went over onto the corporate side around 2018. So you know I've worked for Amazon and I've worked for Adobe. I've done projects for Microsoft. I'm currently independent. So I consult and I also help lead the Knowledge Graph Academy. And we've graduated over a hundred students teaching them how to build controlled vocabularies, ontologies, and knowledge graphs. Ido Bronstein: First of all, congratulations. I will love to Jessica Talisman: Thank you. Ido Bronstein: participate in this academy sounds super interesting. Tell me a bit, what is the real difference between like working in, you know, the academic world compared to working in the industry? Jessica Talisman: I would say the biggest market difference is in industry, you know, there's the ethos of build fast and break things. In academia, you don't really have that pressure. It's not about build fast and break things. It really is turned outwards towards the user. So you're building ultimately and always for the benefit of the user. And in enterprise, I think we get really caught, you know, there tends to be a little bit more competition internally, where I find that there are teams that work against each other rather than rowing the boat in the same direction. So the coordination, even though it's faster, I think that tech would do well to slow down a little bit. in order to coordinate better so that we're not having to deal with so much tech debt or uncoordinated efforts. And that's one of the instrumental things about knowledge is knowledge is not a necessarily fast thing. Ido Bronstein: Yeah. So I think that a lot of the time companies use this term of like build fast and racing as an excuse Jessica Talisman: Yes. Ido Bronstein: to do like mediocre work in general. And I guess Jessica Talisman: Yes, yes. Ido Bronstein: that things need to move fast, but in the end, what really matters is the outcome. and Jessica Talisman: Yes, yep. Ido Bronstein: the work isn't good and the experience is not that good. it doesn't matter and I agree with you that knowledge it's like from its raw form, it's like it's not fast. It takes time to like gather and build and curate. and we will talk about it later, but I think it's really interesting to think about it in the area of the AI that everybody expects that everything will be automatic and and immediate. Jessica Talisman: Yeah. yeah. And that's the thing is like AI is not new. It's new to many people because there's now this commercial there are commercially available products in these frontier models. and in fact there's a very long lineage with AI and building things like taxonomies and ontologies, that is a form of AI and that's lost on many people. Ido Bronstein: Yeah. So before we are talking about that, I think that a lot of our audience will need to go back a bit to the basic about like what is knowledge, what is taxonomy, and why do you in the end need those. Jessica Talisman: Well, knowledge is and has always been a bedrock of society. So you know, throughout history, I think you can't uncouple knowledge from history because, you know, as you said, it's the brain. Well, it's the brain of society, it's the collective brain of society. knowledge is textured and it's layered, it's not one thing. It exists in many shapes and forms. It's how do we communicate knowledge within our digital systems? So many people are familiar with libraries. that is a repository for knowledge. A library is a repository for knowledge. and what's interesting is that you know, we see knowledge as being coupled. I've seen some wild explanations in the past week where people think knowledge is context plus data. context has become a very nebulous word. It's sort of the catch all for anything that's sort of related to or surrounding knowledge. It can be as little as a sentence and as much as an entire document. It can be data tables, it can be metadata, it can be taxonomies, it can be any of these things. And and so when someone talks about context as it relates to knowledge, What they're really talking about is the relationships between things. That's what context actually is, is how you can have objects in space, but how those things relate to each other is really where you start to codify context, where it actually becomes knowledge. So taxonomies enter the picture. A taxonomy is a hierarchy. It's a hierarchy of defined concepts. And the way that I build taxonomies and teach how to build taxonomies is using the SKOS or Simple Knowledge Organization System. It's a W3C standard. It's a data model in RDF that is very intentional in its construction of structuring broader, narrow concepts. And each concept is rich with alternative labels, definitions, scope notes, hidden labels, and each concept within a taxonomy carries a unique identifier with it to provide stability of that thing in space. So taxonomies are instrumental to actually describing and defining things, which is one single step in the knowledge, you know, space. It's defining things is where we begin with knowledge. Ido Bronstein: Yeah. So I agree with you that like knowledge is much more broader than data context and it's kind of our ability to like capture everything that we know on how the and like the digital environment that we store in our organization. I think that what confuses a lot of the people that I'm talking with in in the industry and our audience, they were data people all their life, right? And Jessica Talisman: Right, right. Ido Bronstein: like a data engineer and data analysts, we used to think on those roles as a very technical roles, writing SQLs, analyzing tables, distribution, statistics. This is what those roles used to be. But now when like the system that we want to feed AI does not satisfy from those statistics and those raw data. They need knowledge in order to really act. The roles that we have need to mature from data to knowledge. Jessica Talisman: Yes. Ido Bronstein: So if you can help like give a tip to a data engineer that they have like a huge data lake, a lot of tables, raw files. What do they need to do in order to start thinking about this as knowledge. Jessica Talisman: Think beyond your database. So that's the biggest advice. And I'm seeing this in real time working with clients, big enterprise clients right now, and helping them to parse like what is knowledge? So if most organizations store things like financial data, right? customer data, those sorts of things in databases and data tables and data lakes, that's a fraction. Of your organization's actual knowledge. You have Confluence, those things may not, you know, assets may not live in those tables, in those rows, in those columns. You have SharePoint. You have you know, S3 buckets where you have just things stored generally, right? I mean, it's usually just sort of a yard sale. Knowledge right now exists as just sort of a 52 -card pickup, you know, across your organization. But what I see Is you have the analytics view where you have what is, you know, being marketed as the semantic layer, you know, which is to help make sense of dashboards. That's one sliver or slice of knowledge. Combine that with the metadata, from your data lakes and your tables and your rows and your warehouses. That's another slice. That's another sliver. But I'll bet you that. the documents and the confluences and the and the Jira tickets and all of those things are not stored in your database. So the thing about knowledge is it's a holistic view of a topic, a subject, an idea, an item. So if you're only giving that little sliver of knowledge about your, you know, your financial information, in most cases, what I'm seeing is like that's not necessarily the soundest deployment for AI to begin with? Like, do you want to be feeding that to your LLM number one? If you think about it from a security standpoint, how do you start describing a thing? If it be you know, the if I'm structuring for a big financial institution and I'm trying to make sense of investment data, am I gonna go directly just to the database or the data lake or the data warehouse and think that's where knowledge is? I'm gonna be looking for you know, prospecti or perspectives'. I'm gonna be looking for Confluence pages. I'm gonna be looking for how the outside world defines these things, not just how I define things in my database. So I think that we're still approaching this from too narrow. I think it's a hangover from the big data era, is that we have this bias towards everything that matters in our organization lives in databases, and that's therefore that's where knowledge lives. But it's not. Like if you on your own personal quest for knowledge and you're interested in a topic, I doubt the first thing you're going to do is go look in a database. You're going to go and Google and find papers to read and documents to read. And you're going to try to understand the topic and consume it from that level before getting to that state of maturity where you're looking to, you know, define it according to terms that a machine can understand. Ido Bronstein: Yeah. And something that I like that you said that it's not like going back to the data. Think about the topics that is most interesting to you. So in Jessica Talisman: Yes. Ido Bronstein: a sense, the one thing that like I'm hearing from you that you first need to decide what's the topics that you want to construct the knowledge around them and then build it like top down in a sense. Understand the topic. understand what I want to construct and then go back to the data and understand Jessica Talisman: Yes. Ido Bronstein: it's not just tables it's the Confluence it's the web and then capture all the relevant things they have but going the other way around first look on all the data ask what I can do with all those tables and start with that just will give you a bunch of random relationship between points. Jessica Talisman: Yeah, yeah. And we tend to be, you know, the other thing that I think is interesting, and this is very much sort of a library science perspective, is when you approach topics, we have our own internal biases. And it's really important to remember that if you're working with some of these commercial AI products, they come already trained on data. And that training data primarily is from the open web. So the internet. So if you know how it arrives and what knowledge it brings to the table already, it's in everyone's best interest to also define things relative or at least disambiguate how things relate to the outside world. Ido Bronstein: Yeah. And I think it's very connected to the next topic that I want to talk about. And it is like the company brain. Today this is a very Silicon Valley buzz word like it's my phrasing for that. a corpus of yes, documents and tables that someone created graph out of that. Usually Jessica Talisman: Yes. Ido Bronstein: it's a code repo with Confluence, as you said. So maybe I already said my opinion about that, but what do you think about that? How do you see like enterprises and companies generally approaching that today? Jessica Talisman: Yeah, you know, and I personally have not witnessed a lot of organization around from companies around grabbing a hold of their SharePoint and their Confluence and really trying to come up with a framework that helps to preserve knowledge while also preserving meaning. You know, anyone can wildly tag things in Confluence, for example. But unless you have someone that's managing and governing Confluence in order for everything to roll up to sort of defined tagging terms, it's going to be very difficult to scale. So it really is in a thoughtful approach an architecture. So, you know, something that I think we aren't bringing enough strategy to our approaches. As of now, as far yes, the company brain does exist, but we've seen things wildly be overhyped and marketed in terms of context, which has been very, very dangerous. the idea of tacit knowledge. And even by definition, people are saying, well, there's a lot of tacit knowledge that exists, but what they really mean is knowledge that hasn't been tagged. labeled classified because tacit knowledge means it's not printed anywhere. It has not been codified. It has not been captured even. It's in someone's head. That's the whole idea. But as soon as it enters a Slack thread, it's no longer tacit knowledge because it's recorded. So Ido Bronstein: Yeah. Jessica Talisman: it's like, you know, Using the right terminology and understanding knowledge, I know this sounds harsh, but I call it Epistemic Squatting. Which is there's this assumption that people can come into the world of knowledge and just, you know, that we can just lift and shift practices from data over into knowledge practices. It doesn't really work that way. So it's an entire domain and discipline that's been shaped over 3,000 years. It's not a new thing. It goes back to scrolls, it goes back to the printing press. It goes, It has a lineage and it has a history, especially in digital systems. So Trying to invent something new is not in anyone's best interest. The idea is in order to form a company's brain. again, that holistic approach, but there has to be strategy. And that's the thing, you know, in terms of doing an audit. What do we have in our system? In what shape, in what format do those things exist? Like, are we going to come up with a spec? I call it, you know, the first step I work through with customers is functional requirements. for a given system. And those functional requirements usually define how not only the approach and the strategy, but what needs to be captured and what needs to be addressed in order for knowledge to exist. Ido Bronstein: Yeah. And it's all going back to your point of like the industry used to like move fast & break things and there is things that like it won't work from the beginning. So it won't help you to Jessica Talisman: No. Ido Bronstein: move fast in that manner. and we need to come back to the foundations and create a strategy about our knowledge before we are declaring Jessica Talisman: Yeah. Ido Bronstein: we have one. Jessica Talisman: And it's fascinating because like the other side of it is many organizations for the past three years made AI tooling available to their employees, but Ido Bronstein: Yeah. Jessica Talisman: without any guardrails and without any plan or strategy. So then you have the byproduct. The hundreds of thousands of millions of billions of documents and assets that people have created using LLMs. Ido Bronstein: Yeah. Jessica Talisman: And that creates a whole nother knowledge problem, especially with the upcoming EU Ido Bronstein: Didn't think about that. Jessica Talisman: AI Act. It's really bad. It's actually very bad because the thing is that it's become all too easy to generate a document, whether someone's reading any of those documents, or you know, you can copy and paste and drop it onto a Confluence page. There may be duplicates, triplicates, things may not have been versioned. There's no saving of prompts everyone has it's been just the wild west. So I think that we're going to start seeing a shift in the next two years because there's going to be EU AI Act requirements, whether companies feel like they need to meet those standards. The financial penalties are big, but again, that goes back into knowledge. You can't organize, think of the GDPR days, you can't organize and serve up and label something if you can't find it. Ido Bronstein: Yeah. Jessica Talisman: Like a fundamental problem. Ido Bronstein: Yeah, we just digging ourselves a bigger hole, but it will be fine. In the end we will solve that. Yes. Jessica Talisman: It'll be fine. Yeah. Ido Bronstein: Yeah. So something that you wrote about and I really like the idea of like the symbolic AI. how do you think about like the next phase of AI and what it is a symbolic AI in that matter? Jessica Talisman: Yeah, so I mean, I don't know if you see the same, is that we're still in this kind of big data era of collecting and querying data. warehouses, SQL, SQL is meant for querying, warehouses were meant for storing, databases were meant for storing, for collecting. Like that was the whole movement for the past two decades. Scaling laws have proven that more data doesn't make AI more reliable. and In that it has brought attention, renewed attention, especially because there's been tons of papers about this in the last three years. it's renewed attention to things like taxonomies, ontologies, the sorae, knowledge graphs, as a way to drive truthfulness and reliability for AI deployments. The reason being that it is a form of symbolic AI. So we call it GOFAI, good old-fashioned AI. It's like the OG of Ido Bronstein: Okay. Jessica Talisman: AI. And so it's starting, and that's related to things like world models that we've, you know, most people have gotten kind of excited about that idea. So the combination of symbolic AI with generative AI is called neurosymbolic AI. And it's not a new discipline, domain, or area of research. It's more than 30 years old. So it's you know a lot of people discovering it think that they've like Yreka discovered gold, you know, that they've discovered something new. But There's like a huge domain of research and practice that goes back actually to the 1940s around symbolic AI. So the idea is describing being able to describe problems well enough, to be able to describe things well enough that you're able to you know create that knowledge, that brain, in order to drive reliability. But traditionally in AI, and I think this is where this more recent wave of AI has hurt people more than helped, is there's always been a practice in AI of augmentation before automation, which means that you necessarily put a human in the loop at strategic points where you're not quite certain if this is the best implementation, or you need to implement something like reinforcement learning with a human in the loop or whatever. normally you don't want to go zero to sixty and automate something without having a care in the world, right? And so there are certain disciplines and practices around these types of deployments and usages that again, I feel like we're in such a hurry to automate that we're not really thinking of the long term. Ido Bronstein: Yeah. So If I understand correctly, say that like we started with a very brute force approach to AI, like just do gradient descent on all the internet, and it's not reliable. So we don't fully trust that. And we need to do a lot of the things that AI do for us, in a sense, in the end, is there still humans in the loop, and now we're trying to patch it with some kind of reinforcement learning that tries to learn the human judgment. Jessica Talisman: important. Yeah. Ido Bronstein: To understand like if AI is correct or not correct. Jessica Talisman: Yes. And that sort of verification aside from that, if something is in someone's head and you get to a certain point in the workflow that you know is questionable, can this be audited? Or, you know, can this be automated? So, Ido Bronstein: Yeah. Jessica Talisman: so those are natural segues or points or breakpoints in the automation cycle where it is in your best interest to have a human involved. So you can start recording that tacit knowledge that how Expert or subject matter expert engages and presses the buttons and does all the things. Like you need to have that information. When we talk about recording tacit knowledge, that's one form of recording tacit knowledge, is not automating right away. It's knowing, is having the knowledge yourself and understanding of the technology that if there's tacit knowledge involved and you're not quite certain in order to build the reliability, you're gonna have to take the time to record and to include a human human feedback. Ido Bronstein: Yeah. And from that you can teach the AI on a good curated knowledge rather Jessica Talisman: Yes. Ido Bronstein: than like the all words that a human write that humankind have ever write. Jessica Talisman: Well, and this is like what it goes back to human-computer interaction. If you look at the history of AI, you're going back to the 40s and 50s and going into the 60s with the very devices we're engaging with right now, right? Your computer, your screen, your mouse, your keyboard, those are all forms of augmentation. Not necessarily there's augment, you know, there's automation in there as well. But if something is hard to articulate. If a person cannot articulate, if it's experiential, if it's brains to fingers on a keyboard, that's your option is to include humans along the way, whether it be a pipeline, a workflow, I don't care what, and devise the innovation happens in how you're going to capture what we are calling tacit knowledge. Ido Bronstein: Super interesting! I have one last question. So your book is out this Fall. What Jessica Talisman: Yes. Ido Bronstein: is the one thing that readers has to know and has to read your book for it? Jessica Talisman: My book, it's after about 10 years of implementing these types of systems, you know, where I've built a knowledge program within different organizations. The framework is meant to guide, and it's really a pragmatic framework. You could be an individual builder, but especially for organizations, I think one of the biggest struggles is where do I start and how do I start? and I see too many organizations going straight to taxonomies or straight knowledge graphs, like trying to implement knowledge graphs. And they haven't cleaned up, reconciled, or defined their knowledge yet, their data yet. And so that preparatory stage is very important. It's the same thing like with databases, how we have our conceptual, our logical, and our physical models. It's not too dissimilar from knowledge graphs, from building knowledge. Like there's preparation, there's necessary work that needs to be done in order to successfully implement ontologies and knowledge graphs. The second thing is that knowledge graphs aren't architecture. We talk about them as if they're just this universal thing that you can adopt. There are wildly different architectures for knowledge graphs, depending on your technologies in play, your organization, how big is your organization, how many data sources. I mean, it could just goes on and on and on. So that's the second part. but the most important thing is that this is meant to be a book, a real reference book that can help guide people through the most difficult part of building knowledge in an organization. Ido Bronstein: Amazing. I'm already waiting for it and I'm sure that it's going to help a lot of practitioners in order to understand how to really Jessica Talisman: I hope so. Ido Bronstein: build knowledge inside their organization. In that optimistic ending, thank you so much Jessica. I really enjoyed our conversation. Jessica Talisman: Thank you. Thank you for having me. This was great. Ido Bronstein: Yes, so thank you very much. And this is a wrap for the episode. If you enjoyed this, follow the Data Splash Podcast. We have got a lot more coming your way. Until next time, keep your data flowing and we will see you in the next stream. Thank you very much, Jessica. Bye bye. Jessica Talisman: Thank you.