Latest / Code Story | Startup Podcast for CTOs, CEOs and Technical Founders / The AI Control Loop: What's Missing in AI Security Today - with Craig Thomas of Wallarm

Transcript

The AI Control Loop: What's Missing in AI Security Today - with Craig Thomas of Wallarm · Code Story | Startup Podcast for CTOs, CEOs and Technical Founders

• 0:02
  [SPEAKER_01]: Hello listeners, today we are dropping another episode in our series The AI Control Loop.
• 0:07
  [SPEAKER_01]: Out Enterprise is governed the AI they've already deployed, sponsored by our friends at Walmart.
• 0:13
  [SPEAKER_01]: Walmart is the AI Control Platform for Enterprise AI, protecting every AI workload, API, and application in production, giving CSOs the governance they need and CIOs the speed they demand.
• 0:26
  [SPEAKER_01]: Organizations choose Wallarm for a complete inventory of APIs, AI agents, and AI apps using patented AI ML-based threat detection and blocking the operates at production speeds.
• 0:40
  [SPEAKER_01]: In today's episode Craig Thomas, Senior Solutions Engineer Wallarm, returns to the show to dive into why runtime behavior is the critical blind spot.
• 0:49
  [SPEAKER_01]: And what CISO should demand if they want to move from policy to control?
• 0:56
  [SPEAKER_01]: Craig, welcome back to the show.
• 0:58
  [SPEAKER_01]: Thank you for being in a return episode on code story.
• 1:02
  [SPEAKER_00]: Yeah, great to be back.
• 1:03
  [SPEAKER_00]: I know last time we talked about Rogue AI, so excited to dive in deeper about the threats and how to solve some of the AI problems today.
• 1:11
  [SPEAKER_01]: Absolutely really digging and really enjoying this the series we're putting together here title for today is what's missing in AI security today.
• 1:20
  [SPEAKER_01]: We're going to dive into runtime behavior being a critical blind spot and what CISO should demand and let's dive right into it.
• 1:28
  [SPEAKER_01]: So security teams are used to detecting incidents and responding after the fact, right?
• 1:33
  [SPEAKER_01]: We've covered that quite a bit in the series.
• 1:35
  [SPEAKER_01]: Why is that model becoming insufficient for AI driven systems?
• 1:39
  [SPEAKER_00]: Yeah, great question, Noah.
• 1:41
  [SPEAKER_00]: First of all, just kind of like the frame this whole discussion, right?
• 1:44
  [SPEAKER_00]: The traditional security really was built to answer one question, who gets access?
• 1:50
  [SPEAKER_00]: Firewall rules, identity access policies, network segmentation, all that's fundamentally controlling the gate.
• 1:57
  [SPEAKER_00]: But AI has to move that problem past the gate.
• 1:59
  [SPEAKER_00]: And so, yes, about detecting respond to systems.
• 2:03
  [SPEAKER_00]: Traditional hour assumes a human attack or moving it human speed.
• 2:07
  [SPEAKER_00]: You have time to detect, triage, contain that attack.
• 2:11
  [SPEAKER_00]: But as we talked about before and some of the other episodes, AI compresses that timeline dramatically.
• 2:17
  [SPEAKER_00]: Model connects will trade, manipulate, corrupt data in milliseconds versus hours, days or even weeks.
• 2:23
  [SPEAKER_00]: And so this incident with AI may not look like an incident, looks like normal API calls, normal outputs, and traffic.
• 2:31
  [SPEAKER_00]: I like to think about it about how incident response was designed.
• 2:34
  [SPEAKER_00]: Traditionally, someone breaches the perimeter.
• 2:37
  [SPEAKER_00]: They move laterally out over hours, days, weeks, your sim fires, and alert.
• 2:41
  [SPEAKER_00]: Sock analyst opens a ticket.
• 2:44
  [SPEAKER_00]: That mental model all assumes time.
• 2:46
  [SPEAKER_00]: AI just doesn't give you that time anymore.
• 2:49
  [SPEAKER_00]: And so if that prompt injection or that attack causes your AI agent to start dumping customer records to an external endpoint, that loop closes in seconds.
• 2:58
  [SPEAKER_00]: So by the time your sim would no more fire, the data's already gone and detecting a response, built for a world where attackers are slower than defenders.
• 3:06
  [SPEAKER_00]: But as Flynn, as we've talked about, AI flips that assumption today.
• 3:11
  [SPEAKER_01]: It sure does.
• 3:11
  [SPEAKER_01]: It moves so fast and these models that have been built in the past just really don't.
• 3:16
  [SPEAKER_01]: They really don't keep up and building on that when we talk about response today and enforcement often means actions like restarting pods or rotating credentials or shutting down services.
• 3:27
  [SPEAKER_01]: Things like that.
• 3:28
  [SPEAKER_01]: Why can those measures come too late even in an AI environment?
• 3:32
  [SPEAKER_00]: Those are stateless remediation, applied to stateful harm, but the damage is already done, it's already materialized.
• 3:39
  [SPEAKER_00]: AI agents often hold open sessions with those external services, so rotating a credential after the agent has already acted doesn't undo the action.
• 3:48
  [SPEAKER_00]: and so that's really what we have to fundamentally look at.
• 3:52
  [SPEAKER_00]: I like to equate it to restarting a pod is like locking a door after someone's already walked out with your files.
• 3:59
  [SPEAKER_00]: It's good, they're clean, you're good now, but it doesn't matter.
• 4:02
  [SPEAKER_00]: It really in that traditional sense that traditional web app, the worst case of a single compromise is usually bounded.
• 4:09
  [SPEAKER_00]: A one record touch,
• 4:14
  [SPEAKER_00]: that has that tool used that single compromise session.
• 4:17
  [SPEAKER_00]: I've already browsed your internal wiki, drafted an email, scheduled a meeting, queared your CRM.
• 4:23
  [SPEAKER_00]: All that happened before your orchestration layer or those pod restarts happened, right?
• 4:27
  [SPEAKER_00]: You can't unsend that email by rotating their credential or simply restarting that pod.
• 4:33
  [SPEAKER_01]: right on.
• 4:34
  [SPEAKER_01]: Okay, so if traditional response isn't enough, and I think we have established that for sure, why does AI behavior require controls that operate so much closer to runtime, and the engineer in me is being liked.
• 4:46
  [SPEAKER_01]: We're touching on this in different areas, but I want to be clear in why that's important.
• 4:50
  [SPEAKER_00]: It's really static controls.
• 4:52
  [SPEAKER_00]: These firewall rules, I am policies.
• 4:54
  [SPEAKER_00]: They can't evaluate intent.
• 4:56
  [SPEAKER_00]: And so a lot of looking at AI is really about that intent.
• 5:00
  [SPEAKER_00]: That semantic drift.
• 5:01
  [SPEAKER_00]: It's moved.
• 5:02
  [SPEAKER_00]: This threats move from who gets in to what happens after they're in.
• 5:06
  [SPEAKER_00]: So you're assuming these credentials are legit.
• 5:09
  [SPEAKER_00]: But then once they get in, that spreads.
• 5:12
  [SPEAKER_00]: So the shift really moves from traditional security being essentially that gate.
• 5:19
  [SPEAKER_00]: You check their credentials check.
• 5:21
  [SPEAKER_00]: They're good.
• 5:22
  [SPEAKER_00]: You check the traffic.
• 5:23
  [SPEAKER_00]: Yep, that looks good.
• 5:24
  [SPEAKER_00]: You let it through where you don't.
• 5:25
  [SPEAKER_00]: But once that AI agent with legitimate credentials, legitimate network access happens, your gate controls are done.
• 5:32
  [SPEAKER_00]: They're in.
• 5:33
  [SPEAKER_00]: They've already done their job.
• 5:34
  [SPEAKER_00]: So now that question, but it moves from what the agent does next.
• 5:38
  [SPEAKER_00]: Does it behave consistently with business intent that you desire?
• 5:43
  [SPEAKER_00]: Does it stay in scope?
• 5:44
  [SPEAKER_00]: Does it just keep
• 5:47
  [SPEAKER_00]: Or does it start doing things that look fine individually, but harmful in aggregate?
• 5:52
  [SPEAKER_00]: None of that happens at the gate.
• 5:54
  [SPEAKER_00]: Really, that's the runtime question.
• 5:56
  [SPEAKER_00]: So if you don't have controls that operate at runtime, as we've been talking about for AI security and API security for a while, right, if you don't have those controls that operate at runtime, you really have no controls for that class of the problem.
• 6:10
  [SPEAKER_00]: don't like to just be a scare-monger right there.
• 6:13
  [SPEAKER_00]: There are solutions and we're starting to see those solutions for how a CSO can gain this access and enable the business, but it has to shift from gate controls to runtime controls.
• 6:23
  [SPEAKER_01]: Right on, okay, so we're going the level down there when people hear runtime enforcement.
• 6:28
  [SPEAKER_01]: They may think of existing security controls, right?
• 6:31
  [SPEAKER_01]: What changes when enforcement happens at the kernel level in the runtime rather than only up the network identity or application layers?
• 6:39
  [SPEAKER_00]: Yeah, so those network identity layers to talk a little bit about already, but they see who and where, but they aren't really seeing what.
• 6:47
  [SPEAKER_00]: The application layer does see inputs and outputs, but no kind of intermediate behavior, what tools were called, what memory was accessed, what reasoning steps were taking.
• 6:57
  [SPEAKER_00]: And so shifting that to kernel level visibility, it actually sees those system calls, the file I owe the process behavior,
• 7:05
  [SPEAKER_00]: what the model is actually doing versus just what the model is called on the host and not just what it's saying at that API boundary.
• 7:15
  [SPEAKER_00]: And if you're only watching certain layers, you're only watching that front door aspect.
• 7:20
  [SPEAKER_00]: And so as you dive in the kernel layer, you get the aggregate of all these things.
• 7:25
  [SPEAKER_00]: And you need all those things at run time to truly see and shift and protect in real time the whole picture.
• 7:33
  [SPEAKER_00]: And then, like we talked about, that enables you leverage AI within your environments to get with the most from the AI that you're investing in for both the internal processes as well as customers.
• 7:44
  [SPEAKER_01]: Can you make that a bit more tangible for us, and I think for the audience, what does it actually mean to revoke or contain a compromised AI session without disrupting the broader deployment?
• 7:55
  [SPEAKER_01]: Because if you think linear and it's all part of the same, a similar workflow, and there's a process, if you cut a chunk in that workflow, kills the workflow.
• 8:02
  [SPEAKER_01]: So tell me a little bit more about that.
• 8:04
  [SPEAKER_01]: Make it more tangible for us.
• 8:06
  [SPEAKER_00]: Yeah, great question.
• 8:07
  [SPEAKER_00]: You've got a shift from these broad controls to being more scalpel-like, right?
• 8:13
  [SPEAKER_00]: So session level containment talks about terminating or sandboxing just a single agent's execution without taking down the model, the infrastructure, or adjacent sessions, right?
• 8:24
  [SPEAKER_00]: So you have to continue to enable the business while blocking that individual attacker or rogue AI agent from there.
• 8:33
  [SPEAKER_00]: So, look at it like killing a single process without rebooting the whole server.
• 8:38
  [SPEAKER_00]: But in order to do that, you need the instrumentation to identify the right process.
• 8:43
  [SPEAKER_00]: In practice, you revoke the session token or you freeze pending tool calls and that execution context.
• 8:50
  [SPEAKER_00]: You also need to preserve the session artifacts for forensic so you can see what has happened.
• 8:55
  [SPEAKER_00]: But you allow those other sessions to continue.
• 8:58
  [SPEAKER_00]: So, I was like that from an analogy perspective,
• 9:02
  [SPEAKER_00]: If you have a compromise of a count and an enterprise SaaS environment like Salesforce, if a single user session gets hijacked, you don't shut down for everyone.
• 9:12
  [SPEAKER_00]: You kill that session, you invalidate that token, you audit what that session touched.
• 9:16
  [SPEAKER_00]: AI needs those same parameters.
• 9:18
  [SPEAKER_00]: Primitives.
• 9:19
  [SPEAKER_00]: The problem is most organizations don't have them yet.
• 9:22
  [SPEAKER_00]: They kill the pod, which we talked about earlier, which kills everything, or they do nothing.
• 9:27
  [SPEAKER_00]: There's really no middle options.
• 9:29
  [SPEAKER_00]: So what real containment looks like, that I can identify the session that's misseth behaving, terminate exactly.
• 9:37
  [SPEAKER_00]: That execution context and preserve what it did for investigation.
• 9:41
  [SPEAKER_00]: Providing some feedback, we're going to talk about how do you provide feedback to developers, but provide that feedback?
• 9:47
  [SPEAKER_00]: But then leave everything else running.
• 9:49
  [SPEAKER_00]: So that's a technical capability gap for most teams right now that folks are looking at how do we do that?
• 9:54
  [SPEAKER_00]: But that's really the angle of being able to revoke or contain that session without breaking everything.
• 10:01
  [SPEAKER_01]: That makes sense.
• 10:01
  [SPEAKER_01]: Okay.
• 10:02
  [SPEAKER_01]: So when we move into this type of real-time containment and thinking about the risk for AI agents and them having access to things, how does this kind of real-time containment change the risk equation?
• 10:14
  [SPEAKER_01]: When AI agents have access to sensitive data external services, production workflows are essentially doing the work of a that you would that a human would do, how does the real-time containment change the risk?
• 10:26
  [SPEAKER_00]: Let's look at what it looks like without that containment.
• 10:29
  [SPEAKER_00]: That blast radius of a compromised agent is unbounded, right?
• 10:33
  [SPEAKER_00]: It has whatever access it was granted.
• 10:35
  [SPEAKER_00]: So if that user or that session has full access to go do whatever it wants, you're unbounded.
• 10:42
  [SPEAKER_00]: With containment that blast radius shrinks dramatically, it's bounded by detection latency, the faster you see at the less damage can accumulate.
• 10:51
  [SPEAKER_00]: And so this changes how you reason about those agent permissions.
• 10:54
  [SPEAKER_00]: You can grant broader access if you have confidence in your ability to contain quickly without that confidence, you're forced into over restriction, which then ultimately limits utility.
• 11:05
  [SPEAKER_00]: And I think it's really important reframe for CISOs thinking about AI age in architecture, right?
• 11:12
  [SPEAKER_00]: As a previous CISO and dealing with those day and a day out, right now the dominant strategy is least privilege taken to an extreme.
• 11:20
  [SPEAKER_00]: You give the age in almost nothing and hope it can still be useful.
• 11:24
  [SPEAKER_00]: Then the business comes back to either mad, that this age in isn't very useful, and you're the bad guy again, right?
• 11:30
  [SPEAKER_00]: That, so that works until the business decides it really needs that agent to actually be useful and now you're in a fight.
• 11:37
  [SPEAKER_00]: But the good news is if you have real containment capability, that really that calculus can change, can grant broader permissions because you're confident that if something goes wrong, you can catch it fast and the damage is bounded like we talked about.
• 11:51
  [SPEAKER_00]: Security stops being the ceiling on that utility and starts being the foundation that makes higher utility possible.
• 11:59
  [SPEAKER_00]: I really like to see so can now enable the business, provide the security, provide the governance versus just having to stalk the progress as you start to get the this additional visibility and confidence.
• 12:11
  [SPEAKER_01]: Right on, what are some examples of AI behaviors that organizations should be able to stop immediately when they move to your runtime enforcement?
• 12:20
  [SPEAKER_00]: Yeah, a couple jump demand right away, you know, a data movement outside the approved endpoint.
• 12:25
  [SPEAKER_00]: If a model is exfiltrating to unexpected destinations, you need to know when your data is leaving, right?
• 12:31
  [SPEAKER_00]: Because that becomes a real pain point and has some real legal and monetary consequences.
• 12:38
  [SPEAKER_00]: So that's one.
• 12:39
  [SPEAKER_00]: Another is prompt injection executing, especially when it alters the agent's stated goal or override systems instructions.
• 12:47
  [SPEAKER_00]: A tool that exceeds scope, but it's supposed to do A and B, but now it's doing C, right?
• 12:52
  [SPEAKER_00]: Or it's supposed to have red access only, but now it's making right into leaked calls.
• 12:56
  [SPEAKER_00]: You should be able to see and stop that immediately.
• 12:59
  [SPEAKER_00]: Another one similar to we, we've looked at for APIs for a while.
• 13:03
  [SPEAKER_00]: We've looked at even WAFs and DDoS, right?
• 13:06
  [SPEAKER_00]: Right anomalies.
• 13:07
  [SPEAKER_00]: You have sudden spikes in query volume, especially the sense of the systems that that suggests the agent being used as a proxy and so you need to be able to detect and block that.
• 13:19
  [SPEAKER_00]: And then, traditional security perspective lateral movement, if you have one agent accessing credentials or contacts intended for a different agent, now you certainly have a problem and you need to be able to see that and block it pretty quickly.
• 13:32
  [SPEAKER_01]: So you mentioned this a bit already where security teams don't want to create or limit the functionality.
• 13:40
  [SPEAKER_01]: So we have business leaders coming back to you and be like, hey, this isn't very useful because it's completely locked down.
• 13:46
  [SPEAKER_01]: Security teams don't want to be the bottleneck.
• 13:48
  [SPEAKER_01]: How do organizations balance strong enforcement with the need to keep AI development and deployment moving fast as fast as innovation can move?
• 13:56
  [SPEAKER_00]: for too often that a lot of organizations the security organization is operated kind of outside the bounds or outside the parameters that we ask developers to do.
• 14:06
  [SPEAKER_00]: So I'd start with policy as code.
• 14:08
  [SPEAKER_00]: Those enforcement rules can live in version control right alongside the agent code.
• 14:14
  [SPEAKER_00]: They're reviewed in your CACD pipeline before reaching production.
• 14:17
  [SPEAKER_00]: So take a similar approach you're taking to code with policy as well.
• 14:21
  [SPEAKER_00]: Observe before and force, right, that crawl walk around approach, start an audit mode, understand the baseline behavior.
• 14:28
  [SPEAKER_00]: Before writing those blocking rules, it helps avoid pulse pauses, slow development.
• 14:34
  [SPEAKER_00]: But then you have to come back to it.
• 14:35
  [SPEAKER_00]: You have to move to enforce.
• 14:37
  [SPEAKER_00]: You can't just leave it in observe forever.
• 14:40
  [SPEAKER_00]: Teared enforcement could be another one.
• 14:42
  [SPEAKER_00]: Similar to we've done with other blocking mechanisms in the past.
• 14:45
  [SPEAKER_00]: No, like Dev and Staging might have more permissed monitoring, whereas production has enforced policy.
• 14:51
  [SPEAKER_00]: Same tooling in place, just the different modes.
• 14:54
  [SPEAKER_00]: And then developer self service, I believe being able to allow teams to see their own agents behavioral profile, understand what it will block before they deploy, because one of the things that I think kills developer velocity and we've seen this over and over right is an enforcement.
• 15:11
  [SPEAKER_00]: But it's enforcement that nobody understands.
• 15:13
  [SPEAKER_00]: As a CSO and security professional, us all this too often, security blocks or flags things, but don't give that feedback back to the developers.
• 15:23
  [SPEAKER_00]: So the insight as to when and how and why.
• 15:26
  [SPEAKER_00]: And with that rule blocks your agent, you have no idea why you file a ticket, you wait you're frustrated, but if you can look at that dashboard that shows, hey, here's what your agent did in staging, here's which of those calls would have been blocked in production, here's why.
• 15:41
  [SPEAKER_00]: That's actually a developer accelerant.
• 15:44
  [SPEAKER_00]: All right, they catch issues before deployment, security team isn't the bottleneck or reactive after it's in prod,
• 15:54
  [SPEAKER_00]: And so the shift really then is from security as the team that says no at the end to security is the team that gives you the signal you need to be able to ship that code faster with confidence.
• 16:05
  [SPEAKER_01]: It really turns traditional stereotypes of security on its head and really enables.
• 16:11
  [SPEAKER_01]: things to move fast and be secure, which is incredibly important and really exciting, honestly, when you think about it.
• 16:17
  [SPEAKER_01]: Okay, I got one more question.
• 16:18
  [SPEAKER_01]: When organizations have the ability to discover, observe, and enforce AI behavior in real time, which you are illustrating is so important.
• 16:28
  [SPEAKER_01]: How does that change accountability at the enterprise level?
• 16:31
  [SPEAKER_01]: What does good governance look like from there?
• 16:33
  [SPEAKER_00]: You see this in modern EGRC tools.
• 16:36
  [SPEAKER_00]: You see this in modern security programs, right?
• 16:39
  [SPEAKER_00]: Governance becomes evidence-based rather than attestation-based.
• 16:43
  [SPEAKER_00]: It's not just a check mark once a year, once every three years.
• 16:47
  [SPEAKER_00]: When you just roll it out, you're not just asking, do you have a policy?
• 16:51
  [SPEAKER_00]: You're producing logs and evidence that show the policy was actually enforced.
• 16:56
  [SPEAKER_00]: So that accountability then, though it gets tied to specific AI sessions, specific agent identities, specific tool calls, not just hey, we use this model and those audit artifacts now are continuous, not a point in time when a border regulator asks you pull the timeline, not just a self assessment from six months ago.
• 17:16
  [SPEAKER_00]: It's also now limit the impact if there is an actual security event, right?
• 17:20
  [SPEAKER_00]: A story of her in the past is once you have that actual outestation based, even if you do have an incident, you can say how it was limited, how it was contained, right?
• 17:32
  [SPEAKER_00]: Those notifications don't have to go to all 5 million 10 million customers.
• 17:37
  [SPEAKER_00]: You can say, hey, these five customers were impacted.
• 17:40
  [SPEAKER_00]: So fewer notifications, lesser fines, even if there is an incident because you have that out of station in that limit.
• 17:47
  [SPEAKER_00]: And so then risk quantification becomes possible.
• 17:50
  [SPEAKER_00]: You can say this agent access this data 4,000 times in Q2.
• 17:55
  [SPEAKER_00]: Every access within policy rather than hey, we think our controls are good.
• 18:01
  [SPEAKER_00]: And so fundamentally, that changes the thing that changes most for the CISO is the conversation with the board.
• 18:08
  [SPEAKER_00]: Right now, the board asks, how do we know our AS systems are behaving appropriately?
• 18:13
  [SPEAKER_00]: The audit and stance are for most organization is, hey, we reviewed the design, we approve the deployment, we have a policy doc.
• 18:21
  [SPEAKER_00]: That's after station, right?
• 18:22
  [SPEAKER_00]: That's not evidence.
• 18:23
  [SPEAKER_00]: What government looks like when you have a closed control loop is I can pull a time frame of every action that eight that every agent took, every tool it called, every piece of data it touched.
• 18:34
  [SPEAKER_00]: And I can show you exactly where policy was enforced and where it wasn't.
• 18:39
  [SPEAKER_00]: So that's fundamentally a different conversation, and that's what moves AI security for a framework exercise to an operational capability.
• 18:49
  [SPEAKER_00]: So these, like to say, those good runtime controls enable broader AI adoption, not restricting it.
• 18:56
  [SPEAKER_00]: And this reframe lands well as he's so tired of being seen as blockers, and security becomes the foundation, not the ceiling.
• 19:04
  [SPEAKER_01]: Absolutely, Craig, that just makes too much sense.
• 19:06
  [SPEAKER_01]: I really appreciate you returning to the show today.
• 19:09
  [SPEAKER_01]: It's clear that businesses have to move to runtime containment.
• 19:12
  [SPEAKER_01]: I think we have nailed that on the head today.
• 19:14
  [SPEAKER_01]: The days of older slower models are over.
• 19:17
  [SPEAKER_01]: See, so it's half to acknowledge this blind spot and change the way they think about their operation.
• 19:22
  [SPEAKER_01]: You have to catch these issues fast and lower the blast radius.
• 19:26
  [SPEAKER_01]: And in doing so, you actually maintain speed and innovation rather than handwritten.
• 19:30
  [SPEAKER_01]: Enterprises can move this to an operational capability through these runtime controls.
• 19:35
  [SPEAKER_01]: I think you've made that very clear Craig really appreciate you coming back to the show and telling us what's missing in AI security today.
• 19:42
  [SPEAKER_00]: Thanks, Nellig.
• 19:44
  [SPEAKER_01]: It was great to have Craig back on the show to point out what's missing in AI security today.
• 19:49
  [SPEAKER_01]: And that is changing the models of security away from reactive to proactive at runtime.
• 19:57
  [SPEAKER_01]: If you'd like to learn more about Walarm, you can visit Walarm.com, that's W-A-L-L-A-R-M.com.
• 20:06
  [SPEAKER_01]: And thanks again for listening.