Latest / The Adults In The AI Room / It’s All About the Data - Kirk Marple - Earley AI Podcast with Seth Earley & Chris Featherstone - Episode # 031

Transcript

It’s All About the Data - Kirk Marple - Earley AI Podcast with Seth Earley & Chris Featherstone - Episode # 031 · The Adults In The AI Room

• 0:01
  SPEAKER_00: This is the Early AI Podcast, hosted by Seth Early and Chris Featherstone.
• 0:06
  SPEAKER_00: Join us as we delve deep into the passions, expertise, and experiences of thought leaders and practitioners to talk about what's possible in artificial intelligence.
• 0:15
  SPEAKER_00: Early AI Podcast is sponsored by Early Information Science, your digital transformation journey with the design and deployment innovative technology solutions.
• 0:26
  SPEAKER_01: Welcome to today's podcast.
• 0:28
  SPEAKER_01: My name is Seth Arui.
• 0:29
  SPEAKER_01: And I'm Chris Featherstone.
• 0:31
  SPEAKER_01: And I am really excited about today's guest.
• 0:34
  SPEAKER_01: He is a customer-focused technology leader with extensive expertise in cloud-based microservice, multimedia data ingestion, and computer vision integrations.
• 0:44
  SPEAKER_01: He has a great deal of expertise in knowledge graphs and AI in general.
• 0:48
  SPEAKER_01: His expertise with AWS and Azure Services, Chief Executive Officer and Technical Founder at Unstruck Data, Kirk Marple.
• 0:55
  SPEAKER_01: Welcome to the show.
• 0:57
  SPEAKER_01: Thank you so much.
• 0:58
  SPEAKER_01: Glad to be here.
• 0:58
  SPEAKER_01: Looking forward to the conversation.
• 1:00
  SPEAKER_01: So, so, and I am as well.
• 1:02
  SPEAKER_01: And so, what I wanted to start off with is what are the misconceptions that you are coming across in the industry around knowledge graphs and around AI?
• 1:12
  SPEAKER_01: You can take that as two separate questions or you can combine them.
• 1:15
  SPEAKER_01: So, what are your thoughts about it?
• 1:16
  SPEAKER_02: Yeah, I think a lot of people, if they hear the word knowledge graph, they think of, I mean, the semantic web, I mean, kind of what Google has proposed.
• 1:23
  SPEAKER_02: I mean, they're their classic things, not strings article back in the day.
• 1:27
  SPEAKER_02: And people think of it as maybe just a way to organize web data.
• 1:32
  SPEAKER_02: But they're really your applications for um business and how to organize um, I mean, business specific data for different verticals in a knowledge management um kind of approach.
• 1:44
  SPEAKER_02: And I think, I mean, that's one of the misconceptions of thinking it's just about the web.
• 1:48
  SPEAKER_02: I mean, really.
• 1:49
  SPEAKER_02: And and I think about misconceptions about AI of you have to build your own models.
• 1:53
  SPEAKER_02: You have to have data scientists.
• 1:54
  SPEAKER_02: A lot of people underestimate just how fast things are moving that to abstract from leverage AI for your business, but don't lean in so far that you overcommit to one specific ML model or one specific API because commoditation happens very, very quickly in this market.
• 2:15
  SPEAKER_01: Interesting.
• 2:16
  SPEAKER_01: And what would you say the message to executives would be around AI, especially generative AI, which is the rage these days?
• 2:25
  SPEAKER_02: No, it's it's everything's changed in the last six, nine months so, so fast.
• 2:29
  SPEAKER_02: And I think there's just capabilities that I've used different, I mean, text analytics models and video analytics and audio analytics over the last several years.
• 2:38
  SPEAKER_02: And the the step change that's happened with with open AI and large language models.
• 2:44
  SPEAKER_02: It's easy to get overwhelmed.
• 2:45
  SPEAKER_02: And can I even commit to anything?
• 2:47
  SPEAKER_02: Because it's going to change in three months.
• 2:49
  SPEAKER_02: But I think the the real takeaway is it's about the data.
• 2:53
  SPEAKER_02: I mean, organize your data in a way that you can leverage the new models that come out and kind of abstract yourself from that.
• 3:02
  SPEAKER_02: Um, so I think it's be pragmatic is really the big takeaway.
• 3:05
  SPEAKER_03: So double click on that a bit too, because um, you know, just in terms of of how people miss with their data models.
• 3:13
  SPEAKER_03: Because I'm sure you get into so many scenarios around consulting.
• 3:16
  SPEAKER_03: I know someone does all the time, but where do they miss?
• 3:19
  SPEAKER_02: We had worked in um some areas of like kind of uh the built world, I mean construction engineering and um uh railways and things like that.
• 3:28
  SPEAKER_02: And I think there's this classic tendency to treat data, unstructured data specifically in the kind of classic IT sense.
• 3:35
  SPEAKER_02: I mean, it's like, hey, I throw it in a SharePoint folder.
• 3:38
  SPEAKER_02: I mean, and I mean, thinking about file naming and folder name folder conventions.
• 3:42
  SPEAKER_02: And I think it's really a different way that the miss is really you have to think about it in a different way.
• 3:48
  SPEAKER_02: That I mean, you it's a data set, not just a hard drive.
• 3:52
  SPEAKER_02: I mean, a drive of data.
• 3:54
  SPEAKER_02: And the ability to organize that in a way that you could pull out the the entities that are interesting, the people, places, and things.
• 4:03
  SPEAKER_02: Um, I think it's it's it's a bit of a phase shift for people to really think that way.
• 4:07
  SPEAKER_02: Um, I mean, you're starting to see the the thing that I've really seen the last six or nine months is with large language models and with, I mean, be able to talk to your data, what we all think of as like retrieval of entity extraction, people are not talking about it.
• 4:23
  SPEAKER_02: In in like it's people are it's it's becoming more commonplace.
• 4:26
  SPEAKER_02: And so I think there's now the spaceshift that people have kind of caught up maybe to things that were kind of maybe behind the scenes the last couple of years, last 10 years probably.
• 4:35
  SPEAKER_01: So so when you come back to uh organizing unstructured information and knowledge, a lot of folks think, well, the AI is going to do it, but obviously, you know, uh generative AI is a mechanism to generate content based on patterns that it has learned.
• 4:54
  SPEAKER_01: So how does it fit in?
• 4:56
  SPEAKER_01: How does generative AI fit in with knowledge processes, knowledge management, and organizing that knowledge?
• 5:03
  SPEAKER_02: What you're starting to see is there's a pattern that's developed, I mean, really over the last probably three, six months of long-term memory for large language models.
• 5:11
  SPEAKER_02: So large, I mean, the large language models and also say GPT style model are very useful, but because you can only give it so much context to answer from, um, it's very limited.
• 5:23
  SPEAKER_02: And so there, you have to have some structure to pull from, say, hey, point it at a SharePoint directory, I mean, a uh directory, or scrape this website and then be able to ask questions and have it generate from that.
• 5:36
  SPEAKER_02: OpenAI released their um open source a retriever plugin.
• 5:40
  SPEAKER_02: And it's starting to be a pattern.
• 5:42
  SPEAKER_02: I mean, everybody's kind of having to follow this because it's a way to say, look, I can answer this.
• 5:47
  SPEAKER_02: The models are very powerful, but where's my my sort of history?
• 5:52
  SPEAKER_02: Um and so what I really see is the power is in organizing that historical data, organizing that long-term memory.
• 6:00
  SPEAKER_02: Um, and you're starting to see a lot of kind of knockoffs of like, hey, upload your PDF and chat with it.
• 6:06
  SPEAKER_02: But it's in a business sense, like, what if I have a lot of historical data sitting in SharePoint or SAP or something like that?
• 6:15
  SPEAKER_02: How do I format that and organize that into a structure that the model can can answer from?
• 6:21
  SPEAKER_01: So if it's just crappy, messy, unstructured data with poor naming conventions and black tagging or inconsistency, uh, you're saying that our AI magic is not going to work on it?
• 6:35
  SPEAKER_01: Let's borrow it.
• 6:36
  SPEAKER_02: It's a great point.
• 6:37
  SPEAKER_02: I mean, I think in and that's where I think the the things that um I mean, folks like yourself have have done for years in terms of nauticalization of the data and providing taxonomy is I think it's only gonna help because right now, I mean, the model is only as good as it it's been trained on.
• 6:56
  SPEAKER_02: Um, but the idea of, I mean, can it differentiate a company name versus, I mean, I don't know, like a street name.
• 7:03
  SPEAKER_02: And if they happen to be the same thing, that semantic value of of organizing that data is is super important.
• 7:11
  SPEAKER_01: Yeah.
• 7:12
  SPEAKER_02: Yeah.
• 7:12
  SPEAKER_01: I would agree.
• 7:13
  SPEAKER_01: And go ahead, Chris, Vina.
• 7:15
  SPEAKER_03: I was just gonna ask, you know, centered around, um, you know, what's what's your perspective then on, you know, when you got data evolutions that happen, right, of that information around it so that there's you know governance, not only from the people side, but because they're adding to it, but from the system and process side.
• 7:33
  SPEAKER_02: Yeah, I think I mean data lineage uh is super important because you're gonna have to track, I mean, hey, I'm pulling in data from S3 bucket, I'm pulling it in from from Dropbox or SharePoint and be able to track that history of, especially now when you see some of these semantic search engines um providing um uh what do you want to say, attribution of, hey, I found it in this chunk of data in this article.
• 7:58
  SPEAKER_02: If you could go to the lineage and be like, okay, it's this article, but where did I grab it from?
• 8:02
  SPEAKER_02: What S3 bucket did I grab that from?
• 8:04
  SPEAKER_02: Um, that I don't, I mean, we're thinking about that, but I haven't seen that commonplace that a lot of people are thinking that direction.
• 8:11
  SPEAKER_03: I I think you're spot on.
• 8:13
  SPEAKER_03: I think there's they almost um, well, one, who's gonna document the lineage, right?
• 8:19
  SPEAKER_03: And also put together all the attribution for it so that for instance, um, you come to it and you so know exactly what the data is, where it's being used, who it's being used by, right?
• 8:32
  SPEAKER_03: Almost that you know, like you're saying, that that that lineage, but it contains all the audit trail, but also the data sources that make it up and everything else.
• 8:40
  SPEAKER_03: I think you're spot on.
• 8:41
  SPEAKER_03: They don't we don't see that as like say a first class citizen yet in in designing up front as opposed to, oh, let's do that after the fact.
• 8:50
  SPEAKER_03: In fact, this week we um I was working with a customer and they actually utilized some generative AI techniques to actually do that, you know, drive the lineage of their stuff and generate all the metadata so that it it was in natural language almost query, you know, to a perspective, but for the metadata.
• 9:11
  SPEAKER_02: It's really interesting.
• 9:12
  SPEAKER_02: And I mean it like Microsoft Purdue could kind of do that, but for the structured world, and that's I think another thing.
• 9:19
  SPEAKER_02: There's a lot of tooling and data ops platforms that are so focused on the SQL world.
• 9:24
  SPEAKER_02: And I mean, my thesis is I mean, there's just a lack of that tooling.
• 9:29
  SPEAKER_02: There are a lot of the same patterns, but I mean, where's the purview for unstructured data?
• 9:34
  SPEAKER_02: I mean, and and those kind of things.
• 9:36
  SPEAKER_02: And so I think that um that's really where I believe the next couple of years are going to become pretty interesting of kind of taking some of those patterns and and applying it more to unstructured data.
• 9:47
  SPEAKER_01: So talk a little bit about the role of a knowledge graph in some of these applications.
• 9:52
  SPEAKER_01: And again, you mentioned talked about metadata on content.
• 9:57
  SPEAKER_01: Um the way I looked at it is that those tags are important for the embeddings, right?
• 10:01
  SPEAKER_01: Because signals about the content.
• 10:04
  SPEAKER_01: For example, if you're trouble, if you need troubleshooting from a uh an installation manual for some router or equipment, you kind of need to know, well, what piece of equipment is there, what is the model and what is the make, and what is the you know, what is the version, and then what kind of error codes you use.
• 10:21
  SPEAKER_01: And all of that is a metadata that the content needs to be tagged with so that you can pick up those hints from utterance from what people are asking for.
• 10:30
  SPEAKER_01: Hey, I need help with a router.
• 10:32
  SPEAKER_01: What router?
• 10:33
  SPEAKER_01: Right?
• 10:34
  SPEAKER_01: Um, and and so so talk a little bit more about that and how knowledge graphs fit in.
• 10:39
  SPEAKER_02: Well, that's where I I find it really interesting.
• 10:41
  SPEAKER_02: So there's, I mean, there's the data extraction side of pulling out the entities, finding the people places, things, as we always say, and the products and things like that.
• 10:50
  SPEAKER_02: But the other part that I don't think is talked about enough is the data enrichment.
• 10:55
  SPEAKER_02: It's the correlation of that data to the com the data you have other places in SAP, in a snowflake.
• 11:02
  SPEAKER_02: Where's my SKUs for all my widgets and screws and wrenches in my organization?
• 11:08
  SPEAKER_02: And how can you then make the language model know that it's like a real thing?
• 11:14
  SPEAKER_02: It's not just a word.
• 11:16
  SPEAKER_02: And I think that's where I think there's going to be a lot of growth in that area of like, can I do a join between a SQL table and a bunch of PDFs and make it sort of join on like a SKU?
• 11:28
  SPEAKER_02: And that's really what we're trying to attack because I think there's a lot of flow-hanging fruit of just like making that that entity resolution work properly, um, disambiguation, um, be able to correlate the same sort of logical thing across multiple data sources.
• 11:44
  SPEAKER_02: Um, that I that's the part I find really interesting that that has been touched as much.
• 11:50
  SPEAKER_01: So talk a little bit about those applications.
• 11:53
  SPEAKER_01: So when you talk about you know needing a SKU and then saying, okay, what's all related to this?
• 11:57
  SPEAKER_01: Is New York e-commerce and product data is very important.
• 12:01
  SPEAKER_01: So how to talk about an application or problem that can be solved.
• 12:06
  SPEAKER_02: I mean, think about um competitive research is is an easy one.
• 12:11
  SPEAKER_02: I mean, you have a company.
• 12:12
  SPEAKER_02: Um, I mean, you could basically scrape the web, you could scrape Reddit, you could scrape like customer websites.
• 12:19
  SPEAKER_02: Um, and then you might want to correlate that to an Airtable um spreadsheet, your CRM.
• 12:26
  SPEAKER_02: Um, you could correlate it to your Slack channel where you're discussing that uh that information.
• 12:31
  SPEAKER_02: And so the problem then is, I mean, you can dump all of that as raw text into something and then have the GBT drain query over it, but you're not really giving it enough context to know.
• 12:45
  SPEAKER_02: I mean, hey, this is actually like a company and here's its website, because I mean, you you may want to quote join on different properties.
• 12:53
  SPEAKER_02: Um, and I think that's where it's not as simple as just, hey, I found a word in five different places.
• 13:00
  SPEAKER_02: It's, I mean, you may have it spelled differently, you may have somebody reference the URL versus the name.
• 13:06
  SPEAKER_02: Um, and so that's where putting all that metadata together, where it's a company, but that company is a logical entity that has, I mean, a name, an address, a URL, um, XYZ.
• 13:19
  SPEAKER_02: And so it's just that's where I think you have to correlate all that data from different places.
• 13:24
  SPEAKER_02: And I often say it's in the customer data platform world where you're trying to create like a 360 view of a customer, it's a similar kind of concept, but for any entity.
• 13:34
  SPEAKER_02: And so that's that's really what um what we've been looking at.
• 13:38
  SPEAKER_01: I that's great.
• 13:39
  SPEAKER_01: And can you talk a little bit about a specific solution or problem you solved for a customer recently and maybe uh put that into kind of a hint of plus value?
• 13:48
  SPEAKER_01: Yeah.
• 13:49
  SPEAKER_02: Yeah.
• 13:49
  SPEAKER_02: I mean, one one of the extreme use cases is um, I mean, actually in in the podcast domain, I mean, we've been, I was talking to uh one of the other podcast um hosts that I was on, and there's a concept of basically being able to correlate all the deep information.
• 14:05
  SPEAKER_02: Like, I mean, even in this podcast, we're talking about companies, we're talking about topics, and they wanted the ability to correlate from different sources, not just the transcripts, but other web content, um, I mean, maybe social media, different things like that, and kind of create that 360 view of their podcast.
• 14:25
  SPEAKER_02: And that's actually super interesting to me because I had had this theory of, I mean, in a podcast, there's so much information that's dropped on the floor that you you can search transcript, but it's not really, I mean, how do you follow links?
• 14:38
  SPEAKER_02: How do you kind of get more like sort of do it as a research tool?
• 14:42
  SPEAKER_02: Um, and so for me, that's a a really a passion of mine of, I mean, how do you learn better from content and from from that information?
• 14:51
  SPEAKER_02: Um, and that's that's an area that we're looking at that I think is is super interesting as a research tool.
• 14:57
  SPEAKER_01: Well, when you get that uh uh figured out and deployed, let me know.
• 15:02
  SPEAKER_02: Yeah, no, it's definitely over the next that's uh uh over the next couple of months we'll have that released.
• 15:06
  SPEAKER_02: And it's uh I think it's super exciting.
• 15:08
  SPEAKER_02: So it's awesome.
• 15:09
  SPEAKER_03: I was I was um I was looking at at some um some different models and uh you know as I was thinking about it from the perspective of where these things break down and the limitations of such and um one thing that kind of stood out to me was you know, if you're gonna, you know, utilize maybe a large language model or something to do something generative, just know that like a screenshot or whatever, just know that you're not gonna get too much historical information past, let's say, 2021, you know, or something like that, right?
• 15:40
  SPEAKER_03: So, you know, and I thought about that and I I I I was like, you know, that that becomes a really rich opportunity for these organizations that may have all this deep historical information and given points.
• 15:52
  SPEAKER_03: But like there's not a it's it's more of an art, not a science, because there's not a one size fits all model that will fit for every single thing.
• 16:00
  SPEAKER_03: So you know, maybe you do a large language model from let's say one of the core vendors out there that's that do all the investment for you.
• 16:07
  SPEAKER_03: Maybe you also take then some of your own historical records and generate your own model off of that, right?
• 16:13
  SPEAKER_03: So I'd like to get your take and perspective on what balance should be.
• 16:18
  SPEAKER_02: Yeah, it's really absurd.
• 16:19
  SPEAKER_02: I mean, and and the pattern right now is I mean, they talk about fine-tuning versus it retrieval, um, sort of memory retrieval.
• 16:26
  SPEAKER_02: Fine-tuning is expensive today.
• 16:29
  SPEAKER_02: So the idea would be is I mean, you take a model and kind of wrap another layer of the onion skin around it of knowledge and fine-tune an existing model.
• 16:37
  SPEAKER_02: And I think, I mean, that's gonna get cheaper.
• 16:40
  SPEAKER_02: I think it's it's only inevitable that that'll become more accessible.
• 16:44
  SPEAKER_02: But the really the only sort of cost-effective model is to take an existing model and give it memory.
• 16:49
  SPEAKER_02: Um, but then there's the idea of, I mean, could you then um have a a like we're seeing this now with like tax GPT?
• 16:59
  SPEAKER_02: I mean, it's a model specifically fine-tuned around, I mean, a bunch of data for a specific vertical use case.
• 17:06
  SPEAKER_02: And I really do think um those vertical model verticalized models will proliferate.
• 17:11
  SPEAKER_02: I mean, and I think, but the good thing in the in the way I'm kind of looking at it is that is part of monetization.
• 17:18
  SPEAKER_02: I mean, it's just an API call at the end of the day.
• 17:20
  SPEAKER_02: And so as long as whatever solutions you build, you can say, well, look, I can start with GPT 3.5, Turbo, I mean GPD 4, um, when I get access to it.
• 17:30
  SPEAKER_02: Maybe there's a new model like the Bloomberg model that just came out.
• 17:33
  SPEAKER_02: I mean, maybe a specific.
• 17:35
  SPEAKER_02: Um, I think you just want to abstract your data pipeline from that and just say, look, it's just an API endpoint that I know how to get data in, I know how to get data out.
• 17:44
  SPEAKER_02: I think the patterns are now evolving enough that I mean, you can start to look at that's just swap, and we'll just swap it out.
• 17:51
  SPEAKER_02: I mean, like AWS just announced bedrock this week.
• 17:54
  SPEAKER_02: I mean, hey, maybe GPT's, I mean, not right for my application and I want to go use the their new model.
• 18:00
  SPEAKER_02: Um, I mean, I saw this.
• 18:02
  SPEAKER_02: I mean, uh, we've been working more in the computer vision side of this a couple of years ago, where I mean, hey, you want to do image analysis.
• 18:08
  SPEAKER_02: I mean, now there's a million of them on Hugging Face.
• 18:11
  SPEAKER_02: Now, I mean, you can build your own with Roboflow.
• 18:13
  SPEAKER_02: It's so easy to do this.
• 18:15
  SPEAKER_02: So if you can abstract the pattern of enrichment and the data storage and the search side of it, um, it just gives you much more future-proofing, um, is the way I look at it.
• 18:26
  SPEAKER_01: Um, do you want to talk a little bit more about uh what Unstruct Data does?
• 18:30
  SPEAKER_02: Yeah.
• 18:30
  SPEAKER_02: So I mean, we call ourselves an unstructured data platform.
• 18:34
  SPEAKER_02: I mean, I think we've we've been focused on these things we've been talking about of, I mean, taking anything from documents, images, video, 3D, creating useful knowledge from that.
• 18:44
  SPEAKER_02: And we had been focused kind of in, as I said, in the built world um community where they were um having CAD drawings and 3D files as well as documents and things just sitting in in SharePoint and kind of creating um essentially a semantic search and a and a visualization um tool for that.
• 19:02
  SPEAKER_02: But what we've actually seen is the value of there's so much capability to build verticalized applications on a platform like this.
• 19:10
  SPEAKER_02: So we have a new product that we're actually um releasing later this quarter called Graphslit, which is a uh a platform for anybody to build tools like this on this essentially this graph data pipeline.
• 19:23
  SPEAKER_02: And so it's super exciting using um large language models for data extraction, kind of pulling the entities out of the data, as well as um semantic search and building um prompted queries on that.
• 19:36
  SPEAKER_02: So kind of like a like a Hasora or a superbase where it's like a back end of the service.
• 19:41
  SPEAKER_02: Um so we're super excited to open this up to anybody to now build these capable apps on top of.
• 19:48
  SPEAKER_01: Wonderful.
• 19:48
  SPEAKER_01: And uh talk a little bit about the role of say company specific uh reference models or ontology.
• 19:55
  SPEAKER_01: So looking at large large language models, you know, they have term relationships and concept relationships that are generalized, but then when you look at an organization, maybe their specialized terminology that is part of their IP or the competitive advantage.
• 20:11
  SPEAKER_01: And um, so talk a little bit about how that needs to be kind of considered and integrated.
• 20:17
  SPEAKER_02: So that's a good question.
• 20:18
  SPEAKER_02: Yeah.
• 20:18
  SPEAKER_02: I mean, I've we've started off with kind of the layer following schema.org, um, their the JSON LD kind of model where I mean you you kind of get this wide range of common um object model.
• 20:30
  SPEAKER_02: I mean, essentially the that taxonomy that we can basically apply.
• 20:33
  SPEAKER_02: And that covers a pretty large swath of um of the data sources that that we see.
• 20:38
  SPEAKER_02: I mean, in terms of people and products and organizations and things like that.
• 20:43
  SPEAKER_02: But I think, I mean, we actually haven't done a lot in the very bespoke area of like healthcare or um things like that, where there might be other kind of object models on top of that.
• 20:53
  SPEAKER_02: But um, you could essentially think of it.
• 20:55
  SPEAKER_02: I mean, I come from an auditorian background.
• 20:57
  SPEAKER_02: I mean, you could subclass different different objects and different entity types and kind of add your own properties, um, is the way we look at it.
• 21:04
  SPEAKER_02: But for us, it's really about extracting to the few dozen common entities that um that are most common and then kind of extending, extending from there.
• 21:16
  SPEAKER_02: But I know I know that there's a ton of value in the the more I mean verticalized bespoke areas, but we we just haven't actually um gone down that road too much.
• 21:24
  SPEAKER_02: Sure, sure, sure.
• 21:26
  SPEAKER_03: Hey, I'd love to get your take too, because there's also been um, well, I mean, some of this still feels very, very wild west, but you know, what is what what is there any part of these these um these models and some of these approaches that make you nervous?
• 21:40
  SPEAKER_03: Or that you think about it from uh you know your company like uh okay, we've got to figure out ways to, you know, to not allow for data poisoning to happen, or we gotta figure out ways to make sure that the data set is so good that you know hallucination can't happen, or whatever.
• 21:56
  SPEAKER_03: What what are some of those maybe that you know for our listeners like what Some of the things that make you nervous if they do?
• 22:02
  SPEAKER_02: It's a great question.
• 22:03
  SPEAKER_02: I think, I mean, there's a couple of things even that's come up recently on this.
• 22:06
  SPEAKER_02: I mean, I think one is just it's a bit of a closed loop.
• 22:10
  SPEAKER_02: I mean, if you think about what it's trained on, it's trained on, I mean, a data set, as you said, up to 2021.
• 22:16
  SPEAKER_02: Um, it's not being fed a lot of new information.
• 22:20
  SPEAKER_02: Um, I mean, it's being said um reinforcement from sort of a human feedback on how good it is or not.
• 22:26
  SPEAKER_02: But one concern I have is I thought of this last week is if a lot of new content is generated via these language models and they get fed back into the top of the funnel, you're not injecting a lot of new knowledge into the corpus of the data set.
• 22:42
  SPEAKER_02: And I think that becomes kind of interesting is like, do we lose value in the models over time of not having fresh data?
• 22:51
  SPEAKER_02: Um, and so I mean, I know that there's I mean, it's it's from scanning books and scanning, I mean, Reddit and Wikipedia and all those things, but I I wonder, I mean, how do you keep filling the top of the funnel with fresh, fresh data?
• 23:04
  SPEAKER_02: Um, I mean, I think that's one of one of the areas.
• 23:07
  SPEAKER_02: And then the other is really rent security is, I mean, I know, I mean, an insurance company isn't really going to be happy putting their data up in open AI servers.
• 23:16
  SPEAKER_02: And there's gonna be a big push, I think almost a backlash back to on-premise or at least private cloud.
• 23:23
  SPEAKER_02: And that's something we've been anticipating with this idea of, I mean, a back into the service, making sure that we can partition the data and say, hey, I'm building an app that's sort of a bespoke app.
• 23:32
  SPEAKER_02: I want to make sure my data stays away from everybody else's data.
• 23:35
  SPEAKER_02: And so the multi-tenancy side of it, um, we actually had already done.
• 23:40
  SPEAKER_02: And so we're actually that's a day one thing for us, but I don't see that in a lot of the tools out there today where they're like, okay, it's just a pool of data, but through like SaaS multi-tenant, um, I think is we're gonna start to see a lot of people kind of catch up with that, I think.
• 23:57
  SPEAKER_03: Yeah, I do love the perspective though, of uh at some point we're gonna reach the, you know, if we don't feed it, we're gonna reach the limits of what that data can and should look like.
• 24:08
  SPEAKER_03: And I wonder if it's gonna end up being like the uh, you know, when you create a duplicate of a duplicate of a duplicate, you know, like multiplicity you get, right?
• 24:15
  SPEAKER_03: You get something that's far from what it started at, but super limited in what its capabilities are too, right?
• 24:22
  SPEAKER_02: So it's it's kind of, yeah, if you think about it, I mean it generating marketing copy, generating web copy, and then it just gets fed back in.
• 24:28
  SPEAKER_02: It's kind of a like what's that gonna end up being?
• 24:31
  SPEAKER_02: So I hope that there's, I mean, new people continue to uh write the new written words.
• 24:36
  SPEAKER_01: Right.
• 24:37
  SPEAKER_01: Right.
• 24:37
  SPEAKER_01: That's where human creativity comes in.
• 24:39
  SPEAKER_01: And right now we're mining the uh treasure troves troves of human creativity blame, right?
• 24:46
  SPEAKER_01: So that's an interesting kind of philosophical question.
• 24:49
  SPEAKER_01: Um what should what should organizations be concerned with besides that uh secure that data security?
• 24:56
  SPEAKER_01: Is there are there other things that that uh folks should be concerned with?
• 25:01
  SPEAKER_02: I think cost, cost is one.
• 25:03
  SPEAKER_02: I mean, just the one thing we're doing is because we're a usage-based platform, is giving good exposure because it's really easy to let these things kind of go off the rails and spend a ton of money.
• 25:15
  SPEAKER_02: I mean, uh like any cloud service, but to really give visibility to um, I mean, how many tokens are being used?
• 25:22
  SPEAKER_02: I mean, how much data do you have?
• 25:24
  SPEAKER_02: Um, caching is a is a big thing.
• 25:26
  SPEAKER_02: I just saw this week.
• 25:27
  SPEAKER_02: Somebody's like, oh yeah, now you don't have the all open AI for everything.
• 25:31
  SPEAKER_02: And it's some of them are just software engineering 101 that I think people are catching up with, but I think it's it's gonna be, I mean, giving visibility as people build more complicated applications, um, it's really easy to just spend a bunch of money um on this stuff.
• 25:46
  SPEAKER_02: And so I think it's it's always that's always been a kind of side passion of mine of like cloud costs at a granular level at an application for application developers.
• 25:56
  SPEAKER_02: Like you make a code change, how does this affect your costs?
• 25:59
  SPEAKER_02: And I I think it's something that people are in this as we get more API centric, people are just gonna have to be more more aware and have more visibility.
• 26:10
  SPEAKER_03: What about the the thoughts too?
• 26:11
  SPEAKER_03: Because I I I I love and hate the fact that, you know, and this is how it generally goes, right?
• 26:16
  SPEAKER_03: You something new comes out and everybody the mob is fickle, right?
• 26:21
  SPEAKER_03: Everybody rushes to it.
• 26:22
  SPEAKER_03: They think that they've got to build every single thing in their organization on this, you know, this one topic idea or or something that's released, and then it just doesn't work out for them.
• 26:32
  SPEAKER_03: Yeah.
• 26:33
  SPEAKER_03: So where do you where do you often see pitfalls that folks can avoid with some of this kind of work?
• 26:39
  SPEAKER_02: I mean, I think it's almost be be patient.
• 26:42
  SPEAKER_02: I mean, I've the last couple of months have been moving so fast.
• 26:45
  SPEAKER_02: I've actually been doing less building to move as fast.
• 26:50
  SPEAKER_02: Like I've been kind of more doing architecting and just thinking about okay, like the classic, where's the puck gonna go?
• 26:56
  SPEAKER_02: Like, I mean, and it's an interesting area.
• 26:58
  SPEAKER_02: Like, I feel like we've seen this step change in speed to execution, but I think people are kind of losing the architectural vision a bit.
• 27:07
  SPEAKER_02: And so I've been trying to, I mean, hopefully the right direction is just kind of take a step back, kind of where is this really going?
• 27:14
  SPEAKER_02: Like, where's the arc happening?
• 27:16
  SPEAKER_02: And that made me want to abstract more and be like, look, I mean, a new model pops up every week, a new, a new capability pops up every week.
• 27:26
  SPEAKER_02: Don't overcommit to any one thing like that, but anticipate the concepts that are coming up of data retrieval, data organization, data lineage, like we talked about.
• 27:38
  SPEAKER_02: Um, that that's kind of my takeaway right now, is just be pragmatic, but also kind of focus a bit on where this is heading, not where we are today.
• 27:46
  SPEAKER_03: Yeah, I think there's I think there's some definitely some, well, obviously there's some wisdom in that because it's it's how we've had to deal with the the new new, right?
• 27:54
  SPEAKER_03: Of technology that's always come out and and then apply old patterns to make sense of it, you know, centered around, like you said, governance, hard rails, and you know, where you know it's gonna make the most sense to build and generate the right amount of pipelines to give access to those people, systems, and processes that need it, but in a very systematic and auditable way, right?
• 28:20
  SPEAKER_03: Yeah, and then part of that too is you know, which I I try to recommend a ton, is like I said, it's it's it's an art, not a you know, it's an art form, not a science, because you know, it's there's there's not one or two models that may work, but test all test a bunch of different test variations, test some are fine-tunable, some are not.
• 28:45
  SPEAKER_03: Some will give you the ability to, you know, hey, listen, I just want to use maybe this check summarizer over this.
• 28:53
  SPEAKER_03: And then, you know, go back and maybe if you don't find what you need, build your own if you have the additive aptitude to do it.
• 28:59
  SPEAKER_03: You know what I mean?
• 29:00
  SPEAKER_03: I think it's true.
• 29:01
  SPEAKER_02: I mean, that's where we we want to lean in more to the tooling layer around it of saying, hey, I mean, the app leave this in the hands of the app developer, not the end user, and say, look, I mean, like, does this model work better?
• 29:15
  SPEAKER_02: I mean, for some reason, is GPT 3.5 better than four in this situation or um prompt engineering of okay, I want to extract data.
• 29:23
  SPEAKER_02: Like I was doing some stuff with addresses and realized how much better GPT is than like other NER solutions that were out there, like miles better.
• 29:33
  SPEAKER_02: But that needs testing.
• 29:34
  SPEAKER_02: So you kind of get need even, I mean, even though like they want to put in these prompts to leverage in their applications, they need testing tools and validation tools on a test data set.
• 29:45
  SPEAKER_02: And it's kind of a different case.
• 29:47
  SPEAKER_02: Like it's not that's not building a model, that's just having a test set and being able to run prompts like a playground and then publish that to be used in a workflow.
• 29:58
  SPEAKER_02: And and I think that's where I mean, it's what we're planning, but I'm sure other people are too.
• 30:03
  SPEAKER_02: I think it's a really interesting area that I mean, it's prompt engineering from a software development side of things, where you're using it for part of your app, I think is is a really interesting area.
• 30:14
  SPEAKER_02: And then packaging those up in a nice little way.
• 30:16
  SPEAKER_03: Yeah, we still we still will always get into this scenario too, where unfortunately um people try to solution the problem before they actually try to figure out what the end results and outcomes need to be, and then work backwards.
• 30:34
  SPEAKER_03: And then figure out from the perspective of, you know, this is just going to drive like accuracy, but what semantic measure of accuracy do you need for that thing?
• 30:40
  SPEAKER_03: And then go back and look at do you have the foundations to support it?
• 30:43
  SPEAKER_03: If not, then you know where to start, right?
• 30:46
  SPEAKER_03: Building in those pieces.
• 30:47
  SPEAKER_03: But I think there's that uh definitely that data pipeline.
• 30:49
  SPEAKER_03: There's the AIML pipeline that needs to feed the overall, you know, uh DevOps pipeline to help you get the apps out there.
• 30:57
  SPEAKER_02: So that's yeah, I mean it's really, I mean, in the address one, we we had a uh oil rights uh company doing legal documents for oil rights, but you then need to do data validation to be like, well, hey, show it on a map.
• 31:10
  SPEAKER_02: Like, did it extract things like in Europe that I shouldn't be expected and like visual testing and just uh visualization.
• 31:17
  SPEAKER_02: I think there's all those kind of things of validating the interconnections of what you extracted are really super interesting.
• 31:24
  SPEAKER_03: I agree.
• 31:25
  SPEAKER_01: So this morning I read an article uh about how uh a large language model uh worked on was given a uh problem around chemical synthesis and was connected to a cloud lab, right?
• 31:45
  SPEAKER_01: So physical equipment did the research, learned what the chemical synthesis should be, and then performed the steps.
• 31:57
  SPEAKER_01: And one of the things that the researchers had looked at is well, what if we asked it to do illicit substances like methane and it refused, right?
• 32:07
  SPEAKER_01: But there were ways of getting around it, like it wouldn't it wouldn't synthesize heroin, it wouldn't synthesize, it wouldn't phosdine, right?
• 32:15
  SPEAKER_01: These all of these things that are you know chemical chemical weapons, but there were ways to get around it by referring to chemical derivatives or automatic A.
• 32:26
  SPEAKER_01: And so there were ways to kind of fuel the system, but it kind of really uh and and and then novel compounds um recognized, right?
• 32:36
  SPEAKER_01: So it really does open a very, very scary chapter when you start thinking about how these things can interact with the physical world.
• 32:45
  SPEAKER_01: And my goodness, you know, considering uh an AI that you know, and I never really believed in this that that was uh prompted in a strange way, or obviously uh programmed in a nefarious way, uh, would could do serious damage, you know, when you when you when you extrapolate that.
• 33:05
  SPEAKER_02: Yeah, it's in I've seen that that come up uh recently a lot.
• 33:09
  SPEAKER_02: I'm I'm a bit of a skeptic on it because there still has to be a human in that loop.
• 33:13
  SPEAKER_02: I mean, you're not gonna get AI access to the power grid to turn off the power grid or like nuclear weapons or I mean, or or chemical.
• 33:23
  SPEAKER_02: I mean, there's still has they still have to have some physical element of the real world, but I get the I get that they can run off the rails without proper guardrails.
• 33:33
  SPEAKER_02: And there was an interesting project, this baby AGI project, if you saw that, that um you know is doing it.
• 33:40
  SPEAKER_02: I mean, it's essentially GPT in a loop.
• 33:42
  SPEAKER_02: I mean, of um it's like it's a planning execution.
• 33:45
  SPEAKER_02: It's kind of simple in its in its real core form, but it's I mean, infinite loops are bad in software.
• 33:52
  SPEAKER_02: I mean, so but you could essentially just give it and let it kind of keep figuring it out, and you never really know where it's gonna go, but it reminds me of like Conway's game of life or STIMS or things like that.
• 34:03
  SPEAKER_02: And I think we're we're kind of getting back into that world of I mean simulation-based things, but just have a lot smarter planning.
• 34:11
  SPEAKER_02: Um, and just there was a it was actually really cool paper.
• 34:14
  SPEAKER_02: I think Stanford, somebody at Stanford, but it was almost like a Sims world driven by large models.
• 34:20
  SPEAKER_02: But I mean, to me, I'm like, it's not that surprising.
• 34:23
  SPEAKER_02: I mean, they're kind of injecting this concept into simulation technology that we've had for years.
• 34:28
  SPEAKER_02: So maybe I'm maybe I'm being too skeptical, but I think it's it's interesting to think where this goes.
• 34:35
  SPEAKER_03: So maybe the maybe the warning sign should be if you hear somebody call their project whopper and you hear it asking if you want to play a game, run like I know.
• 34:45
  SPEAKER_02: I mean, it's like we've we've all been like seeing seeing the bad movies around these uh these things, but I mean it's all somebody has to like connect the dots, I mean, for for these things to to go off the rails and hopefully we're smart enough not to do some of those things.
• 35:00
  SPEAKER_01: So yeah.
• 35:02
  SPEAKER_01: Well, it is interesting, especially when people talk about you know, a hint of AGI, of artificial of artificial general intelligence.
• 35:14
  SPEAKER_01: And uh and and kind of amazing, you know.
• 35:16
  SPEAKER_01: When I use Chat GPT to do some of my research, it's just you know astounding the level in detail.
• 35:25
  SPEAKER_01: You know, even when it explains itself, it's still mind-boggling.
• 35:30
  SPEAKER_03: The I you know part of this though too is um I think we have a very short memory centered around you know, when we when we came out, like for an example, we came out with e-commerce sites, and then those e-commerce sites got hacked.
• 35:44
  SPEAKER_03: Right, and this data was everywhere, and that that was a good lesson in oh, we need secure sockets then, right?
• 35:52
  SPEAKER_03: We need to figure out a way to do that.
• 35:54
  SPEAKER_03: And you know, now we have you know some some nefarious stuff going on, which will always happen, you know, with prompt injections and things to you know, to fish or to do whatever.
• 36:07
  SPEAKER_03: And and you know, and those I'm assuming that there's there's gotta be, you know, a way for well, I know there's gonna be a way, but but we're gonna have to see some failures first before we start those guardrails, you know, get put into place.
• 36:19
  SPEAKER_03: Is there any any you know, thoughts or or have you thought about those pieces too, just in terms of of customers asked, or is it something more along the lines of hey, keep this in mind?
• 36:31
  SPEAKER_02: Yeah, I mean it it's it hasn't come up specifically yet.
• 36:33
  SPEAKER_02: I mean, we haven't gotten into any worlds where they're could do damage, I mean, or have something.
• 36:37
  SPEAKER_02: I mean, the the data, it's more of a data quality question.
• 36:40
  SPEAKER_02: Um, but I like, I mean, there's some open source projects now, like one, I think it's called literally guardrails, um, that is kind of a it does a prompt and then checks the prompt result the response against sort of a performance pattern.
• 36:54
  SPEAKER_02: Um, in a way, it's just because the the LLMs are non-deterministic.
• 37:00
  SPEAKER_02: And so it's it's I was just reading an article last night of how you could ask that the same thing twice, want to like return JSON from the data, and one time it'll do it, and one time it'll give you a little paragraph after it.
• 37:12
  SPEAKER_02: It's like it's I think some of these are kind of hacks to get around the the weirdness of the the language models today.
• 37:19
  SPEAKER_02: And I think that'll just get cleaned up when I mean OpenAI releases one that's a little more predictable, maybe for developers to use.
• 37:27
  SPEAKER_02: But I think from a conceptual kind of rule-based engine of like, hey, go double check what the answer is and kind of have that um sort of uh eyes on what the like response is.
• 37:40
  SPEAKER_02: Um, I think that'll that'll happen.
• 37:42
  SPEAKER_02: And I think it's gonna be needed, and you're gonna have to have this kind of reinforcement internally just to make sure.
• 37:48
  SPEAKER_02: Because I think the biggest thing for me is people aren't used to programming things that like you put in A and get B and not, and then the next time you get always get B.
• 37:58
  SPEAKER_02: And that is just a different programming paradigm.
• 38:00
  SPEAKER_02: It's like a fuzzy logic kind of thing that I think people are just starting to get their hands around and their head, their heads around.
• 38:07
  unknown: Yeah.
• 38:07
  SPEAKER_01: Where did binary go?
• 38:09
  SPEAKER_02: Yeah.
• 38:11
  SPEAKER_01: I just want to ask you uh more uh personal question.
• 38:14
  SPEAKER_01: Tell us more about you.
• 38:16
  SPEAKER_01: What do you do for fun and where are you located?
• 38:19
  SPEAKER_01: Yeah, that's a little reporting type.
• 38:21
  SPEAKER_02: Um we'll see.
• 38:22
  SPEAKER_02: Um have uh kids now, they're all in their 20s, so a lot more time in my hands.
• 38:26
  SPEAKER_02: So they're um one of them's uh in grad school in the UK and a couple of them in Seattle.
• 38:31
  SPEAKER_02: And um, yeah, I mean, I always wanted to be a chef actually when I was like, I was thinking I was gonna go to culinary school.
• 38:37
  SPEAKER_02: So cooking is my only um sort of big hobby other than just watching Seattle sports, like the crack and gut in the playoffs for the first time.
• 38:46
  SPEAKER_02: So um, but yeah, hockey's become kind of a and uh hockey and baseball are my two big things.
• 38:51
  SPEAKER_02: So um yeah, that's kind of I mean, other than just kind of work on on this uh and always just trying to learn and research.
• 38:58
  SPEAKER_02: I mean, those are those are a couple of passions.
• 39:00
  SPEAKER_01: Um so and where can people find you?
• 39:02
  SPEAKER_01: Um your website is unstruckunst.com.
• 39:08
  SPEAKER_01: Yep.
• 39:09
  SPEAKER_02: And uh LinkedIn's great.
• 39:12
  SPEAKER_02: Um and uh on Twitter a little bit, but um LinkedIn, anybody wants to hit me up, that's usually uh the easiest place to get me.
• 39:19
  SPEAKER_01: Great.
• 39:20
  SPEAKER_01: Well, listen, thank you uh so much for your time today.
• 39:23
  SPEAKER_02: Oh, thank you.
• 39:24
  SPEAKER_02: Now this is a fun conversation.
• 39:27
  SPEAKER_01: Well, and thank you folks for uh listening in.
• 39:30
  SPEAKER_01: Uh if you learned some something today and found this interesting, yeah, let someone know about the podcast.
• 39:35
  SPEAKER_01: And again, um thank you, Kirk.
• 39:38
  SPEAKER_01: I really appreciate it.
• 39:39
  SPEAKER_01: Uh and thank you for your time.
• 39:41
  SPEAKER_01: Yeah, I appreciate the opportunity.
• 39:42
  SPEAKER_02: Exciting stuff.
• 39:44
  SPEAKER_03: Good luck to you, my friend, in the future.
• 39:46
  SPEAKER_03: Yeah, I'll put Amy back in.
• 39:49
  SPEAKER_03: Yeah, definitely.
• 39:51
  SPEAKER_01: This has been another episode of the Early AI podcast, and we will see you all.
• 39:59
  SPEAKER_03: Thanks, Seth.
• 40:00
  SPEAKER_03: Thanks, guys.
• 40:01
  SPEAKER_03: Take care.
• 40:03
  SPEAKER_00: Thank you for joining us on another deep dive into AI innovation.
• 40:07
  SPEAKER_00: Join us next time when we introduce another expert and discuss how to maximize AI in your world.
• 40:12
  SPEAKER_00: Early AI Podcast is sponsored by Early Information Science, CMS Wire, and Marketing AI Institute.
• 40:19
  SPEAKER_00: To learn more about us, visit Early.com.
• 40:22
  SPEAKER_00: At a Z A R L A Y.com.
• 40:26
  SPEAKER_00: See you next time.