Latest / Code Story | Startup Podcast for CTOs, CEOs and Technical Founders / The AI Control Loop: AI Discovery isn't just AI - with Tim Ebbers of Wallarm

Transcript

The AI Control Loop: AI Discovery isn't just AI - with Tim Ebbers of Wallarm · Code Story | Startup Podcast for CTOs, CEOs and Technical Founders

• 0:02
  [SPEAKER_01]: Allo listeners, today we are dropping another episode in our series the AI Control Loop.
• 0:07
  [SPEAKER_01]: How Enterprise's govern the AI they've already deployed?
• 0:11
  [SPEAKER_01]: Sponsored by our friends at Wal-Arm.
• 0:13
  [SPEAKER_01]: Wal-Arm is the AI Control Platform for Enterprise AI, protecting every AI workload, API and application in production, giving CSOs the governance they need and CIOs the speed they demand.
• 0:32
  [SPEAKER_01]: using patent and AIML based threat detection and blocking that operates at production traffic speeds.
• 0:39
  [SPEAKER_01]: We all know that you can't secure what you can't see, which is why AI discovery is a first principle for AI security, but what's really required for AI discovery?
• 0:50
  [SPEAKER_01]: It's more than just LLMs and Agents.
• 0:53
  [SPEAKER_01]: Today's episode is entitled AI Discovery, isn't just AI.
• 0:57
  [SPEAKER_01]: And joining us is Tim Ebers, field CTO at Wall Art.
• 1:01
  [SPEAKER_01]: Tim and I discuss the real requirements for AI discovery and why the connections between assets and infrastructure are part of the puzzle.
• 1:11
  [SPEAKER_01]: Tim, thanks for being on the show today.
• 1:13
  [SPEAKER_01]: Thank you for being on code story.
• 1:14
  [SPEAKER_01]: Thanks for having me.
• 1:15
  [SPEAKER_01]: It's pleasure to be here.
• 1:16
  [SPEAKER_01]: I decided to dive into our topic today all around AI discovery.
• 1:21
  [SPEAKER_01]: Before we do, tell me in my audience a little bit about you.
• 1:24
  [SPEAKER_00]: Sure, I've been in and around the industry for many years.
• 1:27
  [SPEAKER_00]: I've spent half my time as a customer of vendors and then the other half as a vendors.
• 1:32
  [SPEAKER_00]: I not only understand what customers go through, but I've been a customer and I've gone through it.
• 1:36
  [SPEAKER_00]: I love being an operator, but I love data as much as being an operator.
• 1:40
  [SPEAKER_00]: And AI seems to put both of them together very quickly, which is why I jumped into the middle of the AI things that we're doing at Walmart.
• 1:47
  [SPEAKER_01]: Yeah, for sure, and it's an exciting time, no doubt.
• 1:50
  [SPEAKER_01]: What do you do for fun?
• 1:51
  [SPEAKER_00]: I actually live on a ranch.
• 1:52
  [SPEAKER_00]: My son and I are ropeers.
• 1:54
  [SPEAKER_00]: So we go help ranchers with their cows and my daughter transports as well as my son does.
• 1:59
  [SPEAKER_00]: So we spent a lot of time on the ranch doing the usual stuff that you do on ranch.
• 2:03
  [SPEAKER_00]: The analog to digital switch is very interesting and exciting though.
• 2:08
  [SPEAKER_00]: I did put the wireless down into the barn this weekend.
• 2:11
  [SPEAKER_00]: This is my kid's bedroom too.
• 2:12
  [SPEAKER_00]: And of course of course.
• 2:14
  [SPEAKER_01]: Okay, let's dive into our topic for today and we're discussing real requirements around AI discovery and let's dive into what security teams often say right they often say you can't secure what you can't see which is totally true and feels just right in the context of AI what exactly do they need to see.
• 2:34
  [SPEAKER_00]: And that's really interesting because when you go from a deterministic world to a non-deterministic world, you have to understand more about what's going on in the data flows rather than just knowing that a request happened.
• 2:44
  [SPEAKER_00]: And so, gone are the days that you can just look at a log and go, hey, that person did this, but that person did this and then did something else.
• 2:52
  [SPEAKER_00]: And so, it's not
• 2:53
  [SPEAKER_00]: just the logs and the flows that you need to care about anymore.
• 2:56
  [SPEAKER_00]: And so security needs to understand the data flows and what that LLM and those models are actually triggering through the request of the response of the of the non-terministic LLM.
• 3:06
  [SPEAKER_00]: And so not only can you, do you need to understand the models?
• 3:09
  [SPEAKER_00]: Do you need to understand the endpoints?
• 3:10
  [SPEAKER_00]: What those models are calling and the data that's flowing between them?
• 3:14
  [SPEAKER_00]: And then really everything around it so that you know that this person did this thing,
• 3:21
  [SPEAKER_00]: with this AI technology, and these are the downstream effects of what that happened in the data that they came back from.
• 3:27
  [SPEAKER_01]: That just makes so much sense, and I'm glad you brought up the deterministic versus non-deterministic or probabilistic nature of AI.
• 3:35
  [SPEAKER_01]: It's one of the biggest concerns I have with AI and with AI ruling the world, quote-unquote, moving into infrastructure, what supporting infrastructure matters most when mapping AI risks, such as APIs, but also cloud services, Kubernetes workloads, data stores, identities, all the things, and external integrations.
• 3:57
  [SPEAKER_00]: I think what you'll find is actually the identity is the most important thing, because the infrastructure is there to support a transaction.
• 4:04
  [SPEAKER_00]: That transaction is there to support a person or an agent doing something.
• 4:09
  [SPEAKER_00]: So in the old days, it was relatively easy.
• 4:10
  [SPEAKER_00]: You have a person doing a thing, or you have a service with a known API key doing a thing.
• 4:16
  [SPEAKER_00]: Now we need to understand what these agents are doing and where they're taking those things.
• 4:21
  [SPEAKER_00]: I don't know that any one thing matters more than the other.
• 4:23
  [SPEAKER_00]: I actually think they all matter the same, but where I would start is probably APIs and gateways, because that's what's going to drive where the data is, because the data is still the most critical thing here, which is what we like to talk about a lot is, where's the data, who's accessing the data and win?
• 4:39
  [SPEAKER_00]: And the problem with MCP servers,
• 4:42
  [SPEAKER_00]: And AI gateways is your identity may change as you move from the initial flow to and through the gateway to an MCP server.
• 4:51
  [SPEAKER_00]: And so that can cause issues as well.
• 4:53
  [SPEAKER_00]: When one identity is making a request on behalf of another, how do you know that first person should actually have the data that you're getting from the the initial database.
• 5:02
  [SPEAKER_01]: the authentication and authorization problems expand now to be on people to agents and knowing who's doing what I definitely am tracking with what you're saying there.
• 5:14
  [SPEAKER_01]: I have a question around shadow AI, and as far as I understand it shadow AI's, the use of AI tools and things, right, without formal approval by IT, so a company has a formal process to approve tools and people are using AI kind of outside of the normal process of their company.
• 5:32
  [SPEAKER_01]: Where does shadow AI typically appear first inside of an enterprise environment?
• 5:37
  [SPEAKER_01]: And I think even more importantly, how can it be prevented?
• 5:41
  [SPEAKER_00]: I think we we see it a lot at the edges of productivity before it shows up in production right so in order to get AI into production you have to develop the thing that's going to do AI and development and most of the people are developing with AI which is where you see it first so the developers the business users is where you typically see AI first and everybody trying to go faster and everybody using AI to their advantage.
• 6:05
  [SPEAKER_00]: is really where you're seeing shadow AI.
• 6:07
  [SPEAKER_00]: And that shadow AI can be really anything from an NCP server that you hook up to cloud on your desktop to do an API that lives somewhere in the data center that you're hitting to get data that you may or may not be able to have access.
• 6:19
  [SPEAKER_00]: But it's really everywhere, as you start your AI transformation journey, there's really no place that it can't be, but it usually starts at the end user's trying to drive speed in their business in their organization.
• 6:31
  [SPEAKER_01]: Yeah, that makes total sense.
• 6:32
  [SPEAKER_01]: How can it be prevented?
• 6:33
  [SPEAKER_01]: How can businesses get in front of that, so to speak?
• 6:37
  [SPEAKER_00]: I would actually tell you that it shouldn't be prevented because then you're going back to the shadow IT ways of the businesses just trying to get stuff done and now you're putting a block in their place so that they can't get stuff done again.
• 6:50
  [SPEAKER_00]: What I would tell you is,
• 6:52
  [SPEAKER_00]: we should watch what's going on so discover and then add policies to that we know the behavior of those agents so that we can then be notified if they're going left or right of that particular policy.
• 7:05
  [SPEAKER_00]: So the policy doesn't necessarily have to prevent somebody from doing something, but it can also just notify you that something happened and it's either good or bad that it happened.
• 7:13
  [SPEAKER_01]: Ah, that makes sense.
• 7:14
  [SPEAKER_01]: So it's shifting the focus from limiting the amount of tools or solutions that people are using and more of understanding who's doing what.
• 7:23
  [SPEAKER_00]: Yes, and it's not that you shouldn't have standards and these are the tools that you should use, but when you get into the into that prevention right away of I want to prevent this from happening, what you're going to do is make enemies out of those people that are trying to make everything go faster, trying to make the organization be better and if you come take a step back and you're just watching and then going and talking with the group of hey, what do you need to do and maybe we can find a better, more secure way, rather than no, you can't do this.
• 7:51
  [SPEAKER_01]: Fantastic.
• 7:52
  [SPEAKER_01]: Now that makes total sense to me.
• 7:54
  [SPEAKER_01]: How do relationships between assets change the risk picture?
• 7:58
  [SPEAKER_01]: And I think to, for example, or maybe to put that in a different terms here, why doesn't matter which API an agent can call or which data source a workflow can reach?
• 8:08
  [SPEAKER_01]: Why does the authorization part of it matter?
• 8:10
  [SPEAKER_00]: Yeah, I think that's it's an important shift that we're watching and the same model becomes high risk as soon as you connect a tool to it that can act on a data source.
• 8:21
  [SPEAKER_00]: So a read only search API versus one that can call payments or an email send API, those are wildly different, but still the same age.
• 8:30
  [SPEAKER_00]: And so
• 8:31
  [SPEAKER_00]: The relationships between an agent and the tools that they can use are really critical and understanding and identifying those tools is really key in creating a policy around those tools as well so that this agent can have access to these tools but not those tools.
• 8:48
  [SPEAKER_00]: And when they try to access some other tool, then you get notified when that happens.
• 8:52
  [SPEAKER_00]: And that risk relationship isn't really a property of the asset, it's the property of the
• 8:59
  [SPEAKER_00]: right?
• 8:59
  [SPEAKER_00]: So an agent isn't necessarily good or bad.
• 9:02
  [SPEAKER_00]: It's the edge between that agent and the tool that you need to put the risk on.
• 9:06
  [SPEAKER_00]: So it changes the way you think about it where an API service always did the same thing.
• 9:13
  [SPEAKER_00]: Your agent may not do the same thing over and over again.
• 9:15
  [SPEAKER_01]: Oh, that makes it clear then.
• 9:17
  [SPEAKER_01]: Okay, let's dig into AI discovery a bit.
• 9:19
  [SPEAKER_01]: What makes AI discovery harder than traditional application or even cloud asset discovery?
• 9:26
  [SPEAKER_01]: What are the similarities and what are the differences?
• 9:28
  [SPEAKER_00]: So let's start with what's the same because that's probably the easiest.
• 9:30
  [SPEAKER_00]: What's the same as it's still discovery, right?
• 9:32
  [SPEAKER_00]: You're still watching and looking for what applications are doing between each other or what two entities are doing between each other.
• 9:41
  [SPEAKER_00]: And I really think that's the easiest part.
• 9:44
  [SPEAKER_00]: It's still the same inventory and ownership and continuous coverage that you need.
• 9:48
  [SPEAKER_00]: It's really those fundamentals that we've had that are still in place.
• 9:52
  [SPEAKER_00]: What's different is it's behavioral and not static and the boundaries are fuzzy, right?
• 9:57
  [SPEAKER_00]: Before it used to be very specific, this person can do this thing.
• 10:01
  [SPEAKER_00]: It's not that way anymore.
• 10:03
  [SPEAKER_00]: So those boundaries have really gotten fuzzy, especially when they can start looking at your third party APIs that you can't scan or you can't see.
• 10:11
  [SPEAKER_00]: And you can only see them by their traffic.
• 10:13
  [SPEAKER_00]: And that's a change from where we were, and data is part of that asset now.
• 10:19
  [SPEAKER_00]: So understanding the data inside of a reg is needed in order to look at the assets risk or that edge is risk when the agent calls it.
• 10:29
  [SPEAKER_00]: And the bigger problem, and I think the one that we've been thinking about a lot is,
• 10:34
  [SPEAKER_00]: it changes in minutes, not really cycles, like it changes so quick because everything is just gotten so much faster.
• 10:41
  [SPEAKER_00]: So it's the same muscle, but it's new terrain.
• 10:45
  [SPEAKER_00]: The teams that do well will treat AI discovery as an extension of their API cloud discovery.
• 10:50
  [SPEAKER_00]: It's not a replacement, right?
• 10:52
  [SPEAKER_00]: It's additive to that, which is really what we've been focused on as well.
• 10:57
  [SPEAKER_01]: Yeah, having it alongside the API discovery application discovery that makes that just makes so much sense as you're expanding the infrastructure you're using as a company and then when you go about AI discovery obviously you come back with some results right how should organizations prioritize what they find is every AI asset equally risky and I think I could cherry pick for some of your answers maybe where you're going to go but I'm curious we have to say.
• 11:21
  [SPEAKER_00]: Understanding the behavior first of the agent what you want it to be able to do it what you don't want it to be able to is a good start and then looking at the edges which is what does each one call in the behaviors that are really important to look at risk at those edges and the data that's flowing through them and that's really where I've been focused for the last couple months is.
• 11:41
  [SPEAKER_00]: How do I take data that's flowing through transactions at real time and expose them for customers that can look at them to see if they're doing what they want them to do?
• 11:52
  [SPEAKER_01]: Excellent.
• 11:53
  [SPEAKER_01]: What does continuous discovery mean?
• 11:55
  [SPEAKER_01]: In a world where AI services can be deployed, connected, and changed in minutes, what does it mean to be in continuous discovery?
• 12:02
  [SPEAKER_00]: I think no longer can discover be a quarterly scan or a spreadsheet because by the time you've cataloged it, it's probably changed 10 or 15 times over.
• 12:11
  [SPEAKER_00]: So it's really all about always on, watching the traffic, watching for change in the applications, not just the existence of them there.
• 12:20
  [SPEAKER_00]: great that you have an application, but if you've done a release, what changed in that release and you need to know that real time, not at a quarterly scan.
• 12:28
  [SPEAKER_00]: And when you change an agent, what did they change?
• 12:30
  [SPEAKER_00]: What are the new libraries that they're using?
• 12:31
  [SPEAKER_00]: Do those have CVEs?
• 12:33
  [SPEAKER_00]: Understanding everything about the whole flow.
• 12:36
  [SPEAKER_00]: It's not just about we have somebody that does applications and we have somebody that does infrastructure and we have somebody that does data.
• 12:41
  [SPEAKER_00]: You have to do it all when it comes to AI.
• 12:44
  [SPEAKER_00]: It all needs to be in scope and viewed holistically
• 12:50
  [SPEAKER_01]: So things are moving so fast, you need to be doing this on a continuous basis, otherwise you're going to be left behind the days of quarterly scans for this particular thing are over.
• 13:00
  [SPEAKER_00]: Yeah, and inventory is, it's like a live stream, right?
• 13:02
  [SPEAKER_00]: Not a snapshot.
• 13:03
  [SPEAKER_00]: If you're discovery cadence is slower, then your deployment cadence, you're permanently behind.
• 13:09
  [SPEAKER_00]: So your discovery has to be faster than the deployments, which are getting faster and faster every day.
• 13:14
  [SPEAKER_01]: For sure.
• 13:15
  [SPEAKER_01]: Okay, Tim, I got one more question for you.
• 13:17
  [SPEAKER_01]: Once an organization has visibility into its AI footprint, what's next?
• 13:23
  [SPEAKER_01]: What are the biggest gaps in today's AI security programs?
• 13:27
  [SPEAKER_00]: observability, visibility, and then you need to be able to govern what's going on and understand the behaviors of the things in your environment.
• 13:35
  [SPEAKER_00]: So from inventory to enforcement, programs now list their AI, but very few can act up.
• 13:43
  [SPEAKER_00]: So great, I have AI, but can you, do you know what's going on with it and can you act on it?
• 13:48
  [SPEAKER_00]: Can you block an unsanctioned provider or an unsanctioned model?
• 13:51
  [SPEAKER_00]: Or can you cut off a misbehaving session in real time?
• 13:55
  [SPEAKER_00]: Can you make it so that you don't have PII leaving your egress?
• 14:00
  [SPEAKER_00]: So those are the kinds of things that you need to look for.
• 14:03
  [SPEAKER_00]: Runtime protection, prop injections, data leakage, model abuse, are all things that you need to be looking at after you have visibility.
• 14:10
  [SPEAKER_00]: And then audit, how can you prove that no PII is left?
• 14:14
  [SPEAKER_00]: and how are you going to make sure that the auditors understand it?
• 14:17
  [SPEAKER_00]: And then ownership, who owns what?
• 14:20
  [SPEAKER_00]: It seems simple, but with the speed at which we're going sometimes in a large corporation that's hard to figure out as who owns what.
• 14:27
  [SPEAKER_00]: So we've been spending a lot of time looking at not only observability, but also what to do after observability.
• 14:36
  [SPEAKER_00]: And it's really exciting to see what's after because that's really where you're making a difference in an organization.
• 14:41
  [SPEAKER_01]: certainly all that's very clear.
• 14:43
  [SPEAKER_01]: Tim, thank you for being on the show today.
• 14:45
  [SPEAKER_01]: I think that it's been really interesting digging into the discovery process around AI.
• 14:51
  [SPEAKER_01]: I hear you in saying identity is the most important thing to realize who does what and when and whether it's a person or an agent and you define shadow AI as living at the edge of productivity and
• 15:02
  [SPEAKER_01]: that AI discovery should be running alongside other regular discoveries in a continuous nature because the industry is moving fast and you don't want to find yourself as a business permanently behind which I really appreciate you strongly wording next.
• 15:16
  [SPEAKER_01]: I think that's really important.
• 15:18
  [SPEAKER_01]: The biggest gaps in business program for clear visibility, observability,
• 15:22
  [SPEAKER_01]: And from inventory to enforcement, businesses need to take this seriously.
• 15:26
  [SPEAKER_01]: Make AI discovery a regular part of their everyday operations.
• 15:30
  [SPEAKER_01]: I really appreciate you being on the show today.
• 15:31
  [SPEAKER_01]: Thanks for having me know.
• 15:33
  [SPEAKER_01]: Tim makes it clear that with the speed of the industry, AI discovery needs to be a part of businesses every day operation, continuously monitoring new tools and more importantly, who and what is taking action on their systems internally.
• 15:50
  [SPEAKER_01]: If you'd like to learn more about Walarm, you can visit Walarm.com, that's W-A-L-L-A-R-M.com.
• 16:00
  [SPEAKER_01]: And thanks again for listening.