Latest / The Adults In The AI Room / Accelerating AI Adoption: How Manish Sharma Sees Information Architecture Evolving - The Earley AI Podcast with Seth Earley - Episode #044

Transcript

Accelerating AI Adoption: How Manish Sharma Sees Information Architecture Evolving - The Earley AI Podcast with Seth Earley - Episode #044 · The Adults In The AI Room

0:01
SPEAKER_00: This is the early AI podcast.
0:04
SPEAKER_00: Early criticality.
0:06
SPEAKER_00: Join that as we can keep into the path and activities and experiences of public leaders and practitioners to talk about what's possible in artificial intelligence.
0:14
SPEAKER_00: The early AI podcast is promoted by early information scientists.
0:18
SPEAKER_00: The digital transformation journey is design and deployment of innovative technologies and well as the MLI.
0:25
SPEAKER_00: Now enjoy the show.
0:31
SPEAKER_03: Okay, welcome to today's early AI podcast.
0:35
SPEAKER_03: My name is Seth Early, and we're really excited to introduce our guests for today to discuss things like the importance of information architecture, of data governance, of risk management, and really looking at this for chief information and security officers.
0:52
SPEAKER_03: We'll look at AI ethics and data privacy.
0:54
SPEAKER_03: We'll talk about managing risks with large language models and generative AI.
0:58
SPEAKER_03: And we have a joint webinar coming up.
1:00
SPEAKER_03: So we'll put that in the show notes and let you know.
1:03
SPEAKER_03: What do we have for the data on that?
1:04
SPEAKER_03: Oh well, let me introduce you.
1:07
SPEAKER_03: Our guest today has over two decades of experience in the technology industry.
1:11
SPEAKER_03: He's an advocate of using generative AI, machine learning, and conversational interfaces to simplify complex operational challenges.
1:19
SPEAKER_03: He has deep understanding of ITSM and AI integration to bring automation to the enterprise.
1:25
SPEAKER_03: He's the co-founder of Resolve.ai.
1:28
SPEAKER_03: Manish Sharma, welcome to the show.
1:30
SPEAKER_01: Thank you.
1:30
SPEAKER_01: Thank you, Seth.
1:31
SPEAKER_01: What a pleasure and privilege to be here on this show.
1:34
SPEAKER_01: Thank you for having me.
1:36
SPEAKER_03: You're quite welcome.
1:37
SPEAKER_03: And let's see, our joint webinar is Wednesday, March 28th at 1 p.m.
1:43
SPEAKER_03: Eastern.
1:44
SPEAKER_03: So keep your eyes open for that, uh, and we'll be diving in more deeply.
1:48
SPEAKER_03: So we like to start with kind of thinking about what are the popular misconceptions?
1:54
SPEAKER_03: What if people not quite understand about uh generative AI, about AI?
2:00
SPEAKER_03: What are you hearing and seeing in the industry?
2:03
SPEAKER_01: So um when AI Genia came, um it has been quite a roller coaster ride, hasn't it been?
2:10
SPEAKER_01: Uh in the first wave, everyone was amazed how great it is, how amazing it is.
2:16
SPEAKER_01: Uh and then what happened, a lot of people made a leap, especially some technology people made a leap that this is fairly simple, right?
2:24
SPEAKER_01: All we need is an LLM model and ability to write a prompt, and we should be able to hack it.
2:30
SPEAKER_01: Um, right.
2:31
SPEAKER_01: Uh so I think one of the big things that people got wrong in the beginning was that taking a chat GPD kind of solution and productionizing and putting into production with a product kind of finished for solving a specific use case is not very simple.
2:51
SPEAKER_01: It's not uh, you know, it's not like, okay, I got it, right?
2:55
SPEAKER_01: So uh sure you can write good prompts for yourself so that Chat GPD can give you great answers.
3:01
SPEAKER_01: Doesn't mean that you can jump to an enterprise great solution with all the necessary safety uh built in, all the biases taken care of, all the edge cases uh you have worried about and figured those out.
3:16
SPEAKER_01: Uh user experience has been optimized, you know, all that kind of things take time.
3:21
SPEAKER_01: And uh people just thought it should be easy, and it was not.
3:25
SPEAKER_01: So that was one thing that definitely people got wrong.
3:27
SPEAKER_01: And second thing, obviously, and I everyone knows this.
3:30
SPEAKER_01: Last year, if you were talking to me exactly a year ago, people were wondering what is going to happen to my job, right?
3:37
SPEAKER_01: Uh, we are all going to be jobless by the end of 2023.
3:41
SPEAKER_01: Uh, thank goodness we all have still seem to have our jobs.
3:44
SPEAKER_01: Uh, US is doing so well in terms of job.
3:48
SPEAKER_01: So that didn't happen, and that was that has been a relief to people.
3:52
SPEAKER_01: Uh, that it still doesn't mean that AI has, Gen AI has completely failed to deliver on the promise of optimization.
3:59
SPEAKER_01: The reality is that optimization is going to come gradually, and it's going, people will have time to adjust to it and elevate their job profiles and job roles and figure out how to use this as an augmentation to their roles rather than replacement that everyone thought in March of 2023.
4:16
SPEAKER_01: So, those are two things that kind of stand out for me in terms of what happened and what people thought they can do and what is the reality.
4:24
SPEAKER_03: And and of course, the whole data curation piece of this, you know, people are thinking that you can just throw all your content into a repository and then use retrieval augmented generation and it's going to magically uh get the information you need.
4:38
SPEAKER_03: You still need quality data sources and a source of truth.
4:42
SPEAKER_03: And we talk quite a bit about that.
4:44
SPEAKER_03: But when you go into an organization, uh, what do you typically see in that in that situation with regard to content and knowledge?
4:54
SPEAKER_01: Yeah, so for content and knowledge, what happens is when you go to there are different strategies and different information, just like anywhere else, which are floating around.
5:04
SPEAKER_01: And um one approach that a lot of people are suggesting is um, for example, copilot, right?
5:09
SPEAKER_01: It's a copilot approach as I like to call it.
5:11
SPEAKER_01: I'm sure the others too.
5:13
SPEAKER_01: Hey, you have your SharePoint, and we can map to the entire SharePoint and give it access uh to everyone based on the permissions, of course, that you have on SharePoint.
5:22
SPEAKER_01: And uh that's very risky because your SharePoint permissions might not be set right, your right documents might not be in the right place.
5:30
SPEAKER_01: It'll expose everyone to a huge amount of risk because you know you are just um going to you know ask a question and and you're going to get answers that you should not be getting answers to.
5:41
SPEAKER_01: For example, you can say, show me the salary of all the VPs in the company, and boom, you'll actually get an answer because someone put a you know document in some folder and that folder didn't have the right permissions or something like that.
5:54
SPEAKER_01: So that's very risky.
5:55
SPEAKER_01: But you're right.
5:56
SPEAKER_01: Um, a lot of people assume that uh, you know, um once we go into um uh advanced gen AI, uh the knowledge management, the hard work of at least organizing the knowledge management, they don't have to do it anymore.
6:12
SPEAKER_01: And that's still not true, right?
6:14
SPEAKER_01: Because, you know, you'll have uh, you know, it's very simple intuitively, right?
6:19
SPEAKER_01: You'll have different versions of information.
6:21
SPEAKER_01: You're going to have conflicting information, you're going to have, you know, uh, you need to organize your information.
6:28
SPEAKER_01: It's the same thing that I tell my kids, you know, get organized, you know, get clean your room.
6:34
SPEAKER_01: Uh, you know, you you just it's the same thing.
6:36
SPEAKER_01: You, you know, they're they're waiting for a bot to come that'll clean their room.
6:41
SPEAKER_01: And I'm afraid it's not gonna happen uh till the time they at least go to college.
6:46
SPEAKER_01: No bot is going to come and clean their room.
6:48
SPEAKER_01: And the same thing is going to happen with that information architecture.
6:52
SPEAKER_01: You got to do the basics, you got to know where your information is, how you're tagging it, right?
6:57
SPEAKER_01: What information do you have?
6:59
SPEAKER_01: Right?
7:00
SPEAKER_01: Don't throw a bundle at AI and say, you know, I hope you'll figure it out.
7:03
SPEAKER_01: And, you know, good luck to us all.
7:06
SPEAKER_03: And you don't clean your room once and be positive, right?
7:11
SPEAKER_03: That's exactly a law of entropy, and especially around knowledge and data and content, right?
7:16
SPEAKER_03: You create more and create more mess, stuff uh supersedes other uh content and data.
7:22
SPEAKER_03: And we really have to have an a handle on that and understand those knowledge flows in the organization, how it's created, how it's consumed, how it's updated, how it's archived, disposed of.
7:32
SPEAKER_03: Because again, the AI is not going to really know the difference between that unless there's some level of metadata uh that that indicates that.
7:41
SPEAKER_03: And we have a good architecture.
7:44
SPEAKER_03: So when you when you think about the perspective of a CISO, what questions should they ask about any AI implementation?
7:53
SPEAKER_01: Yeah, and um, you know, uh we actually are going to talk about it in our next webinar, you and me.
7:58
SPEAKER_01: We are going to break it down into 10 questions, right?
8:01
SPEAKER_01: That uh um, you know, uh what is what is that Netflix series, 13 reasons why it's similar to 10 10 questions for uh you know CISOs to ask.
8:11
SPEAKER_01: But yeah, it's it's um um very important, you know, because CISOs are suddenly being pushed to decide about a technology implementation that did not even exist a year ago, right?
8:24
SPEAKER_01: These technology implementations did not exist a year ago.
8:28
SPEAKER_01: This is a new greenfield area as far as they are concerned, right?
8:31
SPEAKER_01: If you if you were a GI engineer or Gen AI researcher, you knew about this technology four years ago.
8:38
SPEAKER_01: But if you are a CISO, you know, chances are you had no clue.
8:43
SPEAKER_01: And starting middle of last year, uh, POVs have started to take shape, proof of values, right?
8:50
SPEAKER_01: POCs, POVs.
8:52
SPEAKER_01: And this year, implementations are going left, right, and center.
8:56
SPEAKER_01: And um, I had an opportunity to talk to the CISO, and and I realized that um there's a huge gap in understanding and and I don't blame them.
9:06
SPEAKER_01: You know, if you think about how fast this technology is moving, it is difficult even for people like you and me to keep up.
9:13
SPEAKER_01: How can you ask CISOs to uh you know keep up with uh with all the technology?
9:19
SPEAKER_01: So we are trying to guide them with some 10 questions on what they should be asking.
9:26
SPEAKER_01: Um, and and some of these questions are common, right?
9:29
SPEAKER_01: Like, what kind of you know, LLM are you using?
9:32
SPEAKER_01: What guarantees of data security is that LLM offering you, right?
9:37
SPEAKER_01: So, Microsoft, if you go and see the Microsoft LLM, you know, um uh paragraph on security, it guarantees certain I'm not going to reuse your data for training model.
9:50
SPEAKER_01: Important to see that, right?
9:51
SPEAKER_01: Because there are so many LLMs out there.
9:53
SPEAKER_01: So, what kind of API are you using?
9:55
SPEAKER_01: Are you using an enterprise uh uh grade API or not?
9:59
SPEAKER_01: Where is the knowledge of data that you're keeping?
10:02
SPEAKER_01: What kind of permissions exist?
10:04
SPEAKER_01: Who's managing those permissions?
10:05
SPEAKER_01: Are you sharing everything?
10:07
SPEAKER_01: Are you sharing something?
10:08
SPEAKER_01: Um, right?
10:09
SPEAKER_01: So those are some some of the by the way, some of the things where the um the challenges might come is prompt, right?
10:16
SPEAKER_01: Uh, you know that prompt can be hacked in Genii.
10:20
SPEAKER_01: Uh and the question is, how do you make sure what are you doing around your prompt so that it is kept updated to all the threats that are being made by the by the hackers, right?
10:33
SPEAKER_01: So there are a bunch of questions like this that need to be answered, and CISOs should be well equipped with a playbook that they can go to the market with and say, listen, I I know if you're coming to me with a with a genai project, I will tell you these are the 10, 15 things that I'm going to look at.
10:52
SPEAKER_01: And these are different from the traditional AI or non-AI projects that we are dealing with.
10:57
SPEAKER_01: There's something more here.
10:58
SPEAKER_01: Because nobody was talking about prompt in the in the last, right?
11:01
SPEAKER_01: But but now you got to.
11:03
SPEAKER_01: This is one of your questions now.
11:05
SPEAKER_03: And and what trends are you seeing uh in organizations with regard to risk management?
11:10
SPEAKER_03: What kinds of things are people trying to do, or what else is emerging for them?
11:15
SPEAKER_03: Obviously, the IP uh piece is very important.
11:18
SPEAKER_03: We uh had one client where they turned on Microsoft Pilot and it started to spider a um a portfolio uh review application, which had all the sensitive information about clinical trials and companies under you know, consideration for acquisition and stock moving stuff, and they had to like shut it down, right?
11:40
SPEAKER_03: So respecting the security at the at the uh content level, at the repository level, at the item level is very important.
11:47
SPEAKER_03: What else are you seeing in terms of trends around risk and risk management?
11:52
SPEAKER_01: So what we are finding is that different organizations are absolutely in a different places with this, which surprises me.
11:59
SPEAKER_01: Perhaps it should not surprise me.
12:01
SPEAKER_01: Yeah, what is surprising about it?
12:03
SPEAKER_01: Exactly.
12:04
SPEAKER_01: Because some organizations, if you look at the risk management methodology, it has not changed at all.
12:11
SPEAKER_01: Like if you were in 2017 or 2015, uh, they would have the same risk management approach, excels that they give to us, um, that they are doing now.
12:24
SPEAKER_01: Uh, which is as if you know, it's pretending that the Gen AI and the AI revolution is not here, and we are still dealing with almost on-premise or discrete applications that don't talk to each other, and so on and so forth, right?
12:37
SPEAKER_01: That was a different risk uh assessment period, and now is a different risk assessment period.
12:44
SPEAKER_01: So um I'm finding at least 30 to 40 percent of the organizations are very agile and they have moved um and they have started to acknowledge what is happening in the industry technological landscape and and kind of try to incorporate that in risk management.
13:00
SPEAKER_01: And and some have not.
13:02
SPEAKER_01: My recommendation though, two cents will be don't create more Excels, right?
13:07
SPEAKER_01: Like that's not gonna help you.
13:09
SPEAKER_01: You know, you can make your Excel more bulky, doesn't mean that your organization has less risk, honestly, right?
13:15
SPEAKER_01: Like I think, you know, I don't know if you know that people can now respond to Excel's using you know their Gen AI.
13:21
SPEAKER_01: So you can send a big and and I also heard from uh risk management that they are also making because they created such big Excels, now they have to use Gen AI to read those Excels.
13:32
SPEAKER_01: So so just imagine a scenario where Gen AI, they send a big Excel.
13:36
SPEAKER_01: I fill it using Gen AI to give my responses, and they use Gen AI to uh assess the risk.
13:43
SPEAKER_01: Um, you know, that is one way to go.
13:45
SPEAKER_01: Another way is to go to have meaningful conversations with the vendor, uh, identify the key risk that exists.
13:52
SPEAKER_01: The rest of it you can depend on SOC2, you know, um and GDPR compliance and all those kind of things.
13:58
SPEAKER_01: Those are some ones you can take for granted that at least the organization is doing the basics.
14:02
SPEAKER_01: You don't have to ask me whether I'm doing um encryption at risk or uh at rest and at transit.
14:09
SPEAKER_01: I think most serious enterprises uh you know do that, right?
14:12
SPEAKER_01: Any product company will do that.
14:14
SPEAKER_01: So that is not where your risk comes from.
14:16
SPEAKER_01: And I think that will be anyway covered in one of these certifications, right?
14:19
SPEAKER_01: ISO or whatever.
14:20
SPEAKER_01: But your risk comes from uh really from the newer things that are emerging and your ability to understand that and ask the right questions.
14:29
SPEAKER_01: So I'm my two senses, and some of them are already doing this, they're getting us on the call and they're talking about it and they're learning, they're very open.
14:36
SPEAKER_01: They they want to learn in terms of how to manage the informational architecture risk, how to make sure that, like you were saying, the wrong information does not go to the wrong people.
14:45
SPEAKER_01: Because I can have a perfect architecture, but the implementation could be such like Copilot.
14:50
SPEAKER_01: Can anyone blame Copilot for not having a perfect architecture?
14:54
SPEAKER_01: It's one of the largest companies in the world.
14:55
SPEAKER_01: I'm sure they got that right.
14:57
SPEAKER_01: But how you implemented that, you know, you didn't think you need to engage, uh, to be honest, people like you who understand information architecture to make sure that they can assess that risk and manage that risk.
15:10
SPEAKER_01: It's not the product risk per se, it's how you implement it.
15:14
SPEAKER_01: That's where it matters.
15:15
SPEAKER_01: Right, right.
15:17
SPEAKER_03: So when it comes to use cases, how can organizations manage risks by understanding their use cases?
15:25
SPEAKER_03: And what talk about and monitoring those?
15:28
SPEAKER_03: Go ahead.
15:29
SPEAKER_01: Yeah, the the um, you know, there is it starts with the data, like you were saying in the beginning, right?
15:37
SPEAKER_01: It all starts from the data.
15:39
SPEAKER_01: So you have to ask yourself a question what is the data that is being shared?
15:45
SPEAKER_01: Who is being shared with?
15:47
SPEAKER_01: Do you have enough content personalization?
15:49
SPEAKER_01: Right?
15:50
SPEAKER_01: Um, right?
15:51
SPEAKER_01: Is there and and by the way, just because you know there is a PIA doesn't mean the project is a no-go.
15:56
SPEAKER_01: Then the next question you got to say, okay, you're you some PIA is going to be shared with some people.
16:01
SPEAKER_01: Right.
16:02
SPEAKER_01: What are the DLP risk uh that you can you have any DLP solution here that can mitigate that risk?
16:07
SPEAKER_01: Uh, do we know for sure which data is being shared with what kind of people?
16:11
SPEAKER_01: Is there a mechanism to do that?
16:13
SPEAKER_01: Show me that mechanism.
16:14
SPEAKER_01: How do you manage that risk, right?
16:16
SPEAKER_01: Uh show that to me in my implementation.
16:18
SPEAKER_01: Don't show me at a product level, but show to it to me how you're doing in this particular implementation and level.
16:24
SPEAKER_01: And once you understand the risk, you should be okay to sign off, right?
16:27
SPEAKER_01: Like, um, I I am I'm talking about risk, but at the same time, I I would tell you if I were in their shoes, we are not in an era where you're going to say no to everything, right?
16:37
SPEAKER_01: Like, you know, you should know what risk you have and how you're mitigating that.
16:42
SPEAKER_01: But you know, the simplest ways you can say no to everything, but that's not gonna work for your automation.
16:47
SPEAKER_03: Yeah, you're gonna get a lot of shadow IT, you're gonna get a lot of people doing stuff, you know, on uh in unsafe ways.
16:53
SPEAKER_03: You know, we had a prior webinar where we talked about digital adoption tools that can block or direct people to the right application that has been tested and and secure and and so on.
17:05
SPEAKER_03: So saying no is not the answer, as you say.
17:09
SPEAKER_03: Um and so when you think about how how are we monitoring in a way that helps us address the emerging risks, such as hacking techniques?
17:20
SPEAKER_01: Yeah, so that's uh that's a very key area.
17:23
SPEAKER_01: Uh, you know, the the traditional um you know hacking kind of scenarios exist, and I'm not going to talk about them because you know everyone has talked about them enough.
17:35
SPEAKER_01: Uh I'm talking about your prompt getting hacked, right?
17:40
SPEAKER_01: The new threat that exists.
17:42
SPEAKER_01: And um there the thing is how what so if you're a public-facing scenario scenario, right?
17:50
SPEAKER_01: There has to be a different level of security that has to be built in, like what data you are able to share, etc.
17:55
SPEAKER_01: etc.
17:55
SPEAKER_01: Because if they hack the prompt and they can go inside your system, that poses a risk.
18:01
SPEAKER_01: Luckily, not too much has happened in that space.
18:04
SPEAKER_01: The risk, this was a high perceived risk, but the solutions which have come out, like for example, we do a bi-weekly uh risk assessment on hacking of you know, prompt hacking risk.
18:17
SPEAKER_01: Uh, and anything that new that is emerging, we all already incorporate that in our prompt.
18:22
SPEAKER_01: We also are uh much safer because most of our teaming is also uh an important piece of that.
18:29
SPEAKER_01: Yeah, I I would definitely um, you know, uh said that that is that is a great option, and people should think about that.
18:37
SPEAKER_03: So peep so just for baseline understanding, red team is someone who intentionally goes in, or you hire you can hire an organization to break things, try things, intentionally try to hack uh and see what they come up with, and and that's a valuable service.
18:52
SPEAKER_01: Absolutely.
18:53
SPEAKER_01: And and a lot of people use that.
18:55
SPEAKER_01: And and nowadays, like anything that broken is available, that information is generally available in the in the forums and stuff like that.
19:03
SPEAKER_01: So we get to know what is changing.
19:06
SPEAKER_01: Luckily, so far, this has not been a major concern so far.
19:10
SPEAKER_01: Um, we have not heard of a single instance that the uh this got hacked and and valuable information was taken.
19:17
SPEAKER_01: But it's important to watch for it, especially in public-facing scenario, right?
19:21
SPEAKER_01: So those are the kind of scenarios you need to watch for it, and we are watching for it.
19:26
SPEAKER_01: But that's among the 10 questions that CISO should ask.
19:29
SPEAKER_01: Um allowing.
19:31
SPEAKER_01: For example, so I just I just want to add most of our customers use their product from Teams.
19:37
SPEAKER_01: So, you know, um, when you use the product from team or Slack, it's already inside the system, you're not external.
19:42
SPEAKER_01: So that risk goes down, right?
19:44
SPEAKER_01: When you're external, you're facing an uh external facing situation, then the risk are a little bit higher.
19:50
SPEAKER_01: Right.
19:51
SPEAKER_03: So when you think about so information architecture can help with things like data security, access control, making sure that you're you're ensuring privacy and securing uh data within these models.
20:05
SPEAKER_03: And then what about hallucinations within uh the organization?
20:08
SPEAKER_03: And then this is where, again, the retrieval part comes about, right?
20:12
SPEAKER_03: Retrieval augmented generation.
20:14
SPEAKER_03: So, what are your thoughts about uh hallucinations?
20:17
SPEAKER_01: Yeah, so hallucination um was a major problem earlier in the game.
20:22
SPEAKER_01: And to be honest, when we started working on this um one and a half year or so ago, we immediately realized that the fine-tuning of the model, um, you know, that kind of solution, model fine-tuning solution version of the solution doesn't work well.
20:37
SPEAKER_01: We work in areas where accuracy is highly respected.
20:41
SPEAKER_01: So you cannot be approximately right or right most of the time.
20:46
SPEAKER_01: And you know, just imagine HR, you're not right, and you tell a wrong policy, there are repercussions for that.
20:52
SPEAKER_01: So um, so therefore, we started working on rag model even before it was called a rag model, actually, in the industry.
21:00
SPEAKER_01: We didn't know what we were working on, but we were started working on rag model.
21:04
SPEAKER_01: Now, a lot of people work on the rag model, use lang chain and you know, some of those standard uh solutions out there.
21:11
SPEAKER_01: And these solutions also increasing in um uh you know um maturity.
21:16
SPEAKER_01: But we were using our own proprietary, we built our own proprietary model there um on how to chunk it and how to index it and save it.
21:28
SPEAKER_01: Uh, and that is key to our success.
21:30
SPEAKER_01: We we invested the time back there in you know, um, last year, March, April is when we are working on it, and that has really worked very well for us.
21:40
SPEAKER_01: So, yes, that is definitely the right way to go.
21:42
SPEAKER_01: I think industry now commonly acknowledges that RAG model is the right way to go.
21:46
SPEAKER_01: I don't think there's a there's a confusion around it anymore.
21:49
SPEAKER_03: And retreat uh rag is retrieval augmented generation for people who don't know, and there's different ways of doing this, but you're essentially using the LLM to interpret the uh query, you're using.
22:00
SPEAKER_03: Retrieval mechanism, which could be vector-based similarity search or could be semantic search or parametric search, and then you're returning that result and then processing that with a large language model to make it more conversational.
22:12
SPEAKER_01: And then you have to actually one analyst asked me a very interesting question uh the other day.
22:17
SPEAKER_01: He said, Why do you even need LLM going forward?
22:20
SPEAKER_01: Right.
22:21
SPEAKER_01: And the way that some of the solutions are small language models are improving.
22:27
SPEAKER_01: Um it's a you know, one year from now, what kind of discussion we might be having might be entirely different.
22:33
SPEAKER_01: So there are other solutions coming into play where you don't might not even have to go to LLM model, right?
22:40
SPEAKER_01: So some very interesting developments are happening in the industry, which will play out in the next one year.
22:44
SPEAKER_01: But RAG is definitely uh retrieval augment um uh generation model is definitely a way to go, and that's what we do.
22:53
SPEAKER_03: And of course, uh retrieval means you need to curate your knowledge source and your data.
22:59
SPEAKER_03: You need to have the right information architecture, you have to have the right knowledge um uh uh processes and uh and lifecycle and all of that.
23:09
SPEAKER_03: Again, as you mentioned right at the start, it doesn't obviate the need for information architecture, right?
23:16
SPEAKER_03: We need to have that knowledge architecture, information architecture.
23:19
SPEAKER_01: Absolutely.
23:20
SPEAKER_01: You need to know your data and you need to structure it, otherwise, you know you're going to land up in the same place where you were.
23:27
SPEAKER_03: Right, right.
23:28
SPEAKER_03: So I I think we're we're answering the question of why information architecture is so important for CISOs, and that is because it's going to safeguard the data, right?
23:38
SPEAKER_03: Because you're putting access control at the data level, content level, repository level.
23:42
SPEAKER_03: That's part of information architecture, right?
23:45
SPEAKER_03: And being able to monitor and and manage the right access control.
23:49
SPEAKER_03: That's part of the the IA.
23:51
SPEAKER_03: And then, of course, you know, managing the uh the content and the data, which is the source of truth.
23:57
SPEAKER_03: And you know, we you know, you instruct the language model to to only use that data source, and if it doesn't have the answer in that data source to say I don't know, right, rather than making up an answer.
24:09
SPEAKER_03: Absolutely.
24:10
SPEAKER_01: And then I just want to um add one more aspect here.
24:14
SPEAKER_01: Sure.
24:14
SPEAKER_01: You know, different language models are good for different things, but um we have found out that there is a there is a version where you create scripted answers, right?
24:26
SPEAKER_01: So, for example, for a particular question, you want an answer to be given in a particular way.
24:31
SPEAKER_01: You don't want a generated answer, you want the answer to be exactly verbatim the way you have written it due to some reason.
24:38
SPEAKER_01: And this scenario is also playing out in medical, you know, cases, sometimes in HR, sometimes in uh you know uh legal, those kind of scenarios where you want certain scenarios a verbatim answer, precision, yeah, exactly.
24:52
SPEAKER_01: So we are giving uh what we call a scripted response in certain cases, right?
24:57
SPEAKER_01: Where we'll not we know the bits and pieces of the answer and we are going to present them instead of going to the um you know LLNM generating a response, right?
25:07
SPEAKER_01: Um in certain cases we are giving a scripted answer, and that feature is being liked by a lot of people.
25:12
SPEAKER_01: So that's just a little bit of uh information that uh our listeners might be interested in knowing that that is an option too, and yeah, you know, that'll work.
25:21
SPEAKER_01: And and a good information architecture may obviate the need for the LLM.
25:25
SPEAKER_01: Absolutely, absolutely, exactly, exactly.
25:27
SPEAKER_01: If you have your taxonomy right, if you know how you're structuring the entire uh data source, um, how do you structure, you know, what belongs in hierarchy, how, right?
25:37
SPEAKER_01: Uh then then it solves a lot of problems.
25:40
SPEAKER_01: Yep, absolutely.
25:41
SPEAKER_03: So when knowledge storage uh is is and retrieval is so critical for building out, say, proof of value projects, what are the challenges that you have when you start integrating with different platforms like SharePoint or ServiceNow or Confluence?
25:57
SPEAKER_03: What have you seen in those areas?
26:00
SPEAKER_01: Yeah, so I mean, um again, the basics questions remain the same.
26:05
SPEAKER_01: What do you want to share?
26:05
SPEAKER_01: Who do you want to share it for?
26:07
SPEAKER_01: And you know, that's the basic question, really, that we ask.
26:11
SPEAKER_01: And you know, my our recommendation to people is not to throw everything, like don't say, you know what, I'm gonna share my entire SharePoint.
26:20
SPEAKER_01: Uh, we don't recommend that.
26:22
SPEAKER_01: We just recommend a few um sources, like have a few folders within SharePoint, and we recommend knowing what is in this those folders as you're sharing, and you should know use your proper information architecture within those folders you should understand right how you're structuring your data, etc.
26:40
SPEAKER_01: And you should know what data is meant for whom, right?
26:42
SPEAKER_01: Uh ideally tagged uh in some way.
26:45
SPEAKER_01: For example, if there's some information that should not be shared with the contractors of your company, right?
26:51
SPEAKER_01: Right, exactly.
26:52
SPEAKER_01: Yeah, right.
26:53
SPEAKER_01: So you you need you need to wait to very confirm that that it can be done that way.
26:57
SPEAKER_01: That is the kind of things that we want to tell people that you need to think like certain um uh knowledge sources are so complex that only technicians should be able to view them, they're of no value to an average employee, uh, or they are going to get rather confused if they see that.
27:13
SPEAKER_01: Right.
27:14
SPEAKER_03: It's gonna overwhelm them.
27:15
SPEAKER_03: And it's it's it has to be contextual.
27:18
SPEAKER_03: And that also is about uh understanding the user and a metadata model for your customer or your employee to understand what those interests are, level of technical proficiency, and so on.
27:31
SPEAKER_03: So that applies, so information architecture applies to your personas and your your uh users as well.
27:37
SPEAKER_01: That's exactly right.
27:38
SPEAKER_01: That's exactly right.
27:39
SPEAKER_01: You will you will define personas and based on the metadata tags that you will develop in the information architecture to the tablet.
27:46
SPEAKER_01: So it's a so content restriction is a marriage of two things, right?
27:51
SPEAKER_01: It's the attributes of the person on the one side and the metadata and information architecture on the other side.
27:57
SPEAKER_01: You marry these two and you create business tools for two, and that is what you're you're solving for, essentially, right?
28:04
SPEAKER_01: And once you solve for that, you're safe, then you're fine.
28:07
SPEAKER_01: You know the attribute of the person who's talking to the bot or the AI, and you know the uh your your knowledge is properly tagged, and you know right for that audience, then you're done.
28:19
SPEAKER_03: Yeah, that's about contextualization.
28:22
SPEAKER_03: Contextualization is relative to the perspective of that user, whoever they are, and we need to understand that context of who they are and what they're trying to accomplish.
28:32
SPEAKER_03: So, first party data, if it's about a customer and employee data, uh, if it's about an internal user, is all very important for identifying those personas and then making sure that we are providing the right signals to the LLM to be able to retrieve that information, and of course, making sure that the content that is tagged is tagged appropriately for the audience.
28:52
SPEAKER_03: Because you're right, you you know, you could put in a uh a request and it could uh be completely inappropriate at the wrong level of granularity, depending upon who that individual is.
29:02
SPEAKER_03: So when it comes to doing the actual integrations, are the is that just a configuration plumbing?
29:08
SPEAKER_03: Is or is there are there uh any tricks to uh or or obstacles or known challenges to doing the integration with those various sources like Service Now, SharePoint?
29:20
SPEAKER_01: For for now, this is all not an issue anymore.
29:23
SPEAKER_01: It comes out of the box.
29:24
SPEAKER_01: If you go to a product, you can just say I want to configure this, and it's like out of the box.
29:30
SPEAKER_01: Um but uh you know, obviously you need to do the right information architecture, right?
29:34
SPEAKER_01: In the backside, you if once you do that, that plumbing is not difficult anymore for us.
29:39
SPEAKER_01: Um and creating business rules, uh persona-based business rules is also not very difficult.
29:45
SPEAKER_03: So there, you know, when when we start thinking about customization versus um, you know, out of the box, right?
29:53
SPEAKER_03: It's less efficient to do more and more customization.
29:56
SPEAKER_03: It becomes more difficult to manage.
29:59
SPEAKER_03: Do you have a thought on that?
30:00
SPEAKER_03: How much customization is required?
30:02
SPEAKER_03: You know, particularly for your solution, you know, we're not trying to turn this into a sales pitch for resolve, but it's just something that people will need to understand uh about levels of customization and what they can do.
30:13
SPEAKER_03: So I think it's on the data side myself, but tell me what else uh there might be.
30:18
SPEAKER_01: No, this is a very interesting question because honestly, um you know, you can go huge in customization.
30:25
SPEAKER_01: So we were thinking about um some people brought it to us that can the personalization be at a person level?
30:34
SPEAKER_01: Suppose I like to chat with Gen AI, which is you know, chats in a particular way, right?
30:40
SPEAKER_01: Can I do that?
30:40
SPEAKER_01: Can I change the tone of the Gen AI that I'm talking to?
30:43
SPEAKER_01: Right.
30:43
SPEAKER_01: The voice.
30:44
SPEAKER_01: Can I, yeah, exactly.
30:45
SPEAKER_01: Can I see the notifications differently than other people?
30:49
SPEAKER_01: Can I?
30:49
SPEAKER_01: So, how do we um how much control do we give to the central um, you know, uh people, you know, employee support people, versus how much control should be with the employee and in terms of how much they can customize their um, you know, this thing.
31:06
SPEAKER_01: And this is a journey, right?
31:07
SPEAKER_01: Like ideally, I think we are going to reach to the place where employees will be able to customize whatever they want.
31:12
SPEAKER_01: Um, right now, I don't think we are there or anyone else is there.
31:15
SPEAKER_01: We have started from a point where we are handing most of this customization to the configuration to the um, you know, central authorities or you know, whatever you call them, service desk manager or employee support engineers or employee experience managers, whatever.
31:31
SPEAKER_01: They decide for everyone right now, but we're already seeing a trend towards uh giving it at an individual level, right?
31:39
SPEAKER_01: At least there's ask for it, and hopefully, you know, we'll all go towards it.
31:43
SPEAKER_01: Um, some people um don't um um you know uh like that because they they think the counter argument is if we allow each individual to decide, then we will not be able to give consistent experience.
31:59
SPEAKER_01: So there's a point of debate that that exists right now here.
32:03
SPEAKER_01: But the level at which we have reached it right now, that we can do different things for different people.
32:08
SPEAKER_01: So if you're a contractor, if you are a manager in UK, if you are a if you are um you know CEO um right of the company, uh depending on who you are and what you're trying to do, your access level will be different.
32:21
SPEAKER_01: That much we have achieved.
32:23
SPEAKER_01: But giving experience which is unique to you because you are set early is something we have not achieved yet.
32:29
SPEAKER_01: And that's a question of debate right now.
32:32
SPEAKER_01: Yeah, I think it'll just be a matter of time.
32:34
SPEAKER_03: Uh, you know, one of the things people were talking about, of course, with um Gemini was that it was giving, you know, you know, trying to avoid, you know, bias and it inadvertently provide went to the other end of the spectrum.
32:48
SPEAKER_03: But and uh and some groups really hated that, and other groups like that.
32:52
SPEAKER_03: But in the future, I would imagine that you're going to be able to select the tone.
32:56
SPEAKER_03: You're gonna be able to tell it what you want and what your preference is.
33:00
SPEAKER_03: I'm not sure that's a good thing or a bad thing because you know it's gonna increase our echo chambers, but that's that's another story.
33:06
SPEAKER_03: But but it should at least accommodate your preferences in one way, shape, or form.
33:12
SPEAKER_03: And no one size is gonna fit all, you know, especially with these public-facing uh applications.
33:18
SPEAKER_03: Um, and when you start, how do you when you when you're uh in that enterprise market and you're talking to people, how do you communicate the knowledge component and the value of your product when it comes to that, like uh uh knowledge-centric uh case management, KCS, right?
33:37
SPEAKER_03: Knowledge-centric support, right?
33:39
SPEAKER_03: So I I I think that's the right is that the right uh uh name.
33:43
SPEAKER_01: We call it agile knowledge management and uh you know um you know agile knowledge management is a term that I really like using because it kind of um where where the industry is.
33:53
SPEAKER_01: But I I mean to your point, I think the use cases are in front of us, all of us, right?
34:00
SPEAKER_01: Like literally, if you don't have Gen AI, how do you search for information?
34:05
SPEAKER_01: How do you consume information?
34:07
SPEAKER_01: How do you create new content where required, right?
34:10
SPEAKER_01: That's that use case is pretty obvious, like it's it's rights for itself.
34:15
SPEAKER_01: We love to work with those people who are trying to solve for that problem, right?
34:20
SPEAKER_01: Like different people at different stages.
34:22
SPEAKER_01: Some people just want to play with Gen AI, they just want to experience what it means, and that is fine.
34:27
SPEAKER_01: But our customers are, you know, I was having this conversation yesterday with someone within the organization and saying, let's just find people who who are really have a very clear employee experience problem.
34:41
SPEAKER_01: They understand what is enterprise friction, they understand what is process blindness.
34:46
SPEAKER_01: I have coined a term called process blindness, and what that means is an employee is trying to get something done and they don't know how to get it done.
34:54
SPEAKER_01: And I think it's okay for an employee not to know how to get things done.
34:58
SPEAKER_01: Uh, it should be our job of the of the Gen AI or the solution that hasn't been deployed to know how to help them navigate through the done.
35:09
SPEAKER_01: If they are plus blind, that's perfectly okay.
35:12
SPEAKER_03: Right.
35:12
SPEAKER_03: And that's where customer journeys and employee journeys are so important, mapping out that lifecycle.
35:18
SPEAKER_03: What do they need to do?
35:19
SPEAKER_03: And there's going to be lots of them.
35:20
SPEAKER_03: And the other thing that uh we advocate is building, you know, dozens, if not hundreds, of use cases, right?
35:28
SPEAKER_03: They can be classes of use cases that can represent, you know, more use cases and you just simply uh change variables.
35:35
SPEAKER_03: But you need to, we need to understand those use cases for those job roles, right?
35:41
SPEAKER_03: And understand what that journey looks like.
35:43
SPEAKER_03: And that can actually inform the job, it can inform metrics, it can inform the employee, and and by understanding where those bottlenecks are.
35:50
SPEAKER_03: So you're right.
35:51
SPEAKER_03: You know, the employee might, uh especially a new hire, is going to come in, not necessarily know you know everything they need to know.
35:58
SPEAKER_03: They won't know everything they need to know.
36:00
SPEAKER_03: Exactly.
36:00
SPEAKER_03: So we have to accelerate that time to value and reduce the the training and uh make sure that we put in place the the ability to support them throughout their life cycle in their job tasks for that role.
36:15
SPEAKER_01: And I think that's critical.
36:18
SPEAKER_01: And what you said is so timeless because isn't it true that what you said was applicable five years ago and 10 years?
36:24
SPEAKER_01: Oh, absolutely.
36:24
SPEAKER_01: No, so so it doesn't matter.
36:26
SPEAKER_01: Yeah, technology you're using, right?
36:29
SPEAKER_01: Um, it doesn't matter that you're using Jennifer to solve that problem.
36:32
SPEAKER_01: It's great that you're using Jennifer to solve that problem.
36:34
SPEAKER_01: Yeah, because you can now, you know, you have a better tool to solve that problem.
36:39
SPEAKER_01: Yeah, but how you go about solving the problem, the basics of it, they don't change, right?
36:43
SPEAKER_01: Like you still got to do the basics of finding out the employee experience journey and the customer experience.
36:48
SPEAKER_03: Talking and tackling, yeah.
36:49
SPEAKER_03: That's why I tell people that even though my book was written, uh the AI powered uh uh enterprise was written before LLMs, everything in that book is still valid.
36:59
SPEAKER_03: Everything in that book is still valid.
37:00
SPEAKER_03: And then I have the connect the dots uh journal articles that uh that talk about how information architecture will improve uh performance of an LM, especially retrieval augmented generation.
37:12
SPEAKER_03: So, yes, I totally agree that those those principles are still there, they're still valid, you still have to understand the user, you still have to understand your process, your content, your data, your knowledge, uh, and so on.
37:24
SPEAKER_03: And tell me when you when you how quickly do you get to value?
37:30
SPEAKER_03: So in your in your tool, uh, and I imagine you need to have some precursors, you need to know what those use cases are, you need to know what those knowledge sources are.
37:38
SPEAKER_03: Those knowledge sources need to be in decent shape, right?
37:42
SPEAKER_03: Uh, you can't expect garbage.
37:43
SPEAKER_03: But then when it comes to sort of implementing that, uh when you have the right inputs, how long can you show value?
37:50
SPEAKER_01: How long does it take to show value?
37:52
SPEAKER_01: So typical implementation is two to three months to start showing value.
37:57
SPEAKER_01: Um, because uh, you know, um, like you said, if some basics are in place, like you said, you can you can you know very quickly map, it's very automated at this point of time, right?
38:09
SPEAKER_01: And it's simpler than what it used to be um earlier when we had classic AI.
38:15
SPEAKER_01: Classic AI literally meant creating question-answer pairs.
38:18
SPEAKER_01: That's right.
38:19
SPEAKER_01: Yeah, no matter what salesman.
38:21
SPEAKER_03: We weren't even using knowledge engineering.
38:23
SPEAKER_03: I I totally with you.
38:25
SPEAKER_01: Yep, that was all it was.
38:27
SPEAKER_01: And and you wouldn't believe that if you go look at the marketing material of a lot of um companies back then.
38:32
SPEAKER_01: Um, they they almost made it seem like you can just do automatically, but that was not happening.
38:39
SPEAKER_01: And you know, people realize that, right?
38:41
SPEAKER_03: Uh my problem was I was I told people how much it would cost and how long it would take and what you actually had to do.
38:47
SPEAKER_03: And and then other people saying, oh no, it's only twenty thousand dollars and we can do it in three weeks.
38:52
SPEAKER_03: And it's like, try it, you know.
38:54
SPEAKER_03: But then they go down the path and then they start, you know, investing more.
38:57
SPEAKER_03: But yeah, it was uh it was very interesting in the early days of the uh bot.
39:02
SPEAKER_01: Yeah, but now it it is actually simpler.
39:04
SPEAKER_01: You can actually point it to a um knowledge source, um, and uh and the bot will start giving answers.
39:11
SPEAKER_01: Um we do it for demos, you know, people give us their knowledge source, and we and while in the demo we show them, we scroll the website or something and start.
39:21
SPEAKER_01: So it's fairly simple, and we do it, we bring ROA really quickly in the game.
39:25
SPEAKER_03: Yeah, so you're you're you're talking about a full deployment versus a demo or a POV that's gonna show value.
39:32
SPEAKER_03: POV can be done in a very compressed amount of time.
39:35
SPEAKER_01: That's right, that's right.
39:36
SPEAKER_01: That is literally like a week or sometime.
39:39
SPEAKER_01: We are going for a demo.
39:40
SPEAKER_01: What we do is we go to the website of the company and scroll it, uh, crawl that already.
39:45
SPEAKER_01: And in the demo, we start asking questions about the company and the bot answers question.
39:49
SPEAKER_01: So, and and that is literally done five minutes before the meeting.
39:53
SPEAKER_01: So the time to market is is that cycle is shortening for sure.
39:58
SPEAKER_03: It could be done five minutes before, but of course, you put more time and thought into your meetings with customers.
40:06
SPEAKER_03: I call that just-in-time knowledge management.
40:09
SPEAKER_03: There you go.
40:09
SPEAKER_03: There you go.
40:10
SPEAKER_03: That's exactly what it is.
40:11
SPEAKER_03: So uh let me just uh ask a little bit about your background.
40:15
SPEAKER_03: You know, uh, who are you?
40:17
SPEAKER_03: Where did you grow up?
40:18
SPEAKER_03: Where did you come from?
40:19
SPEAKER_03: How did you get to what you're doing?
40:20
SPEAKER_03: And and what do you do for fun?
40:22
SPEAKER_03: I know there's like 10 questions in there.
40:25
SPEAKER_01: Yeah, I I grew up in India in the foothill of Himalayas, very nice uh township.
40:32
SPEAKER_01: Um, uh, you know, a lot of people tell about their childhood was difficult.
40:36
SPEAKER_01: I don't have any bad childhood story, honestly.
40:38
SPEAKER_01: Um, you know, I lived an idyllic child childhood as parents for it.
40:43
SPEAKER_01: It was uh, you know, very free-range um childhood.
40:48
SPEAKER_01: And then I moved to US around 25 years ago.
40:54
SPEAKER_01: And uh, you know, um have been um here worked in sales.
41:00
SPEAKER_01: I used to work for services companies, um and and I realized that every time I was in a services company, I was actually trying to sell a solution or a product, almost a product.
41:10
SPEAKER_01: And then I, you know, I'm not very smart.
41:13
SPEAKER_01: So after like 10 or 15 years of doing that, I realized I'm in the wrong field.
41:16
SPEAKER_01: I should be in a product field.
41:17
SPEAKER_03: Um, where did you move to when you came into the US?
41:21
SPEAKER_01: Uh I was I started from New York.
41:23
SPEAKER_01: I used to work for a company called Infosys, and then I then I moved to um uh you know um um East West Coast.
41:30
SPEAKER_01: I I enjoyed sunny California for around 10 years uh in different services companies.
41:36
SPEAKER_01: Um and then I moved back to New Jersey 10 years ago, and by that time it was enough gestation that I kind of started to realize I got to be doing something else.
41:46
SPEAKER_01: I need to do product.
41:47
SPEAKER_01: Some people, some of my friends tease me because you know, I didn't do product when I was living in Bay Area, which is a very common place to start doing product business, but uh never late than you know, uh late is better than never.
41:59
SPEAKER_01: So start in the product area, it's it's not easy to switch from a services mindset to a product mindset, yes.
42:08
SPEAKER_01: Um and uh and you know, there were uh you know a lot of battle scars to prove it, but uh uh happy with where we are resolved at AI.
42:17
SPEAKER_01: I think we have been able to build a decent brand in the market.
42:21
SPEAKER_01: And uh what do you do for fun?
42:23
SPEAKER_01: Well, I I love tennis, so I love to play tennis.
42:27
SPEAKER_01: Um, I love to watch tennis.
42:29
SPEAKER_01: Um, my me and my kids are um either uh, you know, my kids are into F uh, you know, uh racing NASCAR or F F fun or um or uh tennis, these are two things that really how old how old are your kids?
42:45
SPEAKER_01: My kids, one is going to college, uh, and I have three kids, and younger one is in high school and one about to be high schooler.
42:53
SPEAKER_01: So uh so when you say racing NASCAR, they want like to watch it or do they actually get in cars and drive?
42:58
SPEAKER_01: Well, right now they want to watch it.
43:00
SPEAKER_01: Uh my son, especially, is a NASCAR uh fan.
43:04
SPEAKER_01: Yeah, um, and we have to go to all the other places to watch NASCAR and and and but he wants to drive it eventually, so we'll see.
43:12
SPEAKER_03: I I did a track day, my wife bought a track day formula in a in a uh Formula One car, and that's where you have to take the steering wheel out to get into the car.
43:24
SPEAKER_03: And um, if you if you uh stall it three times, you're done for the day, right?
43:29
SPEAKER_03: Because there's no flywheel and you have to be very careful on the clutch.
43:33
SPEAKER_03: It was a very cold day, and uh somebody people before me were spinning out a lot, and then there's a really famous uh uh Formula One driver named Diddy.
43:44
SPEAKER_03: I forgot which one there's two Didier's, but I forgot which one, but he was in the car and he stalled it twice.
43:51
SPEAKER_03: And he's a super professional, but somebody blew up the uh gearbox before I could go.
43:57
SPEAKER_03: So, but I did it, I did it in a in a believe it or not, a Cadillac CTS, which is an amazing car.
44:03
SPEAKER_03: And it was and it was really understanding those those lines of the track, and you are totally focused on that.
44:10
SPEAKER_03: It's very interesting.
44:12
SPEAKER_03: I I loved it.
44:13
SPEAKER_03: And uh my wife went too, and she had a crazy guy who was yelling at her the whole time.
44:19
SPEAKER_03: I had a very zen guy who was just like really calm.
44:22
SPEAKER_01: And can I ask you one question?
44:25
SPEAKER_01: Do you have you're from Boston?
44:26
SPEAKER_01: Do you um uh remember there was the talk show called Car Talk?
44:32
SPEAKER_01: Yes, of course, of course, yes.
44:34
SPEAKER_01: And those guys from uh Boston.
44:37
SPEAKER_01: Yes, they're in Boston, yes.
44:38
SPEAKER_01: Yeah, yeah, that was I used to love listening to them, and you know they they were amazing.
44:44
SPEAKER_01: They were a bunch of it was all about cards and also life, right?
44:47
SPEAKER_03: It was that's right, that's right, absolutely.
44:51
SPEAKER_03: Well, Manish, this has been wonderful.
44:54
SPEAKER_03: I really appreciate your time.
44:56
SPEAKER_03: Um, this has been a lot of fun.
44:57
SPEAKER_03: Uh, any final uh words before we uh we finish up here?
45:01
SPEAKER_01: Um thank you so much for having me here.
45:04
SPEAKER_01: It was it was really a pleasure.
45:05
SPEAKER_01: And I think, you know, um, you know, I think we there's a lot of things that people are going to hear about information architecture.
45:15
SPEAKER_01: Yep.
45:16
SPEAKER_01: Um, you know, in next around 12 months, I predict this is going to be a center point for a lot of people to think about.
45:22
SPEAKER_01: And um architects within the companies are going to think about information architecture more seriously because of Gen AI.
45:29
SPEAKER_01: So I think I I applaud the work that you guys are doing in this space.
45:33
SPEAKER_01: It has been great uh partnering with you.
45:35
SPEAKER_01: And uh thank you for having us and having this great conversation.
45:38
SPEAKER_03: That's wonderful.
45:39
SPEAKER_03: Thank you again, and thank you to our audience for listening.
45:43
SPEAKER_03: And of course, uh Carolyn and Liam behind the scenes helping out.
45:47
SPEAKER_03: Uh, so we will see you next time on the early AI podcast.
45:52
SPEAKER_03: Thanks.
45:53
SPEAKER_03: Thank you.
45:54
SPEAKER_03: Bye.
45:56
SPEAKER_00: Thank you for joining us in another deep dive into AI innovation.
46:00
SPEAKER_00: Tune in next, I'm going to introduce another industry expert and discuss how to maximize AI in the world.
46:06
SPEAKER_00: The early AI podcast is sponsored by early information designed and DMS cloud.
46:11
SPEAKER_00: To learn more is early.com.
46:14
SPEAKER_00: But it's easy to buy data content.