Latest / Along The Edge Podcast: Breaking, Defending, and Understanding Agentic AI / Along The Edge e1: Agentic AI Security, Jailbreaks, and Why You Shouldn’t Trust Your Agents

Transcript

Along The Edge e1: Agentic AI Security, Jailbreaks, and Why You Shouldn’t Trust Your Agents · Along The Edge Podcast: Breaking, Defending, and Understanding Agentic AI

• 0:00
  Andrius: My name is Andrius Useckas.
• 0:01
  This is my new podcast.
• 0:02
  Uh, we don't know what we're gonna call it yet.
• 0:04
  This is the first episode.
• 0:06
  Um, my background is security.
• 0:08
  I've been doing it for over quarter of, of a century at this point.
• 0:10
  I've been playing with AI security now for two and a half years.
• 0:14
  Um, and the idea is to this podcast, its main theme is gonna be AI security
• 0:20
  with stress on agentic AI security.
• 0:24
  And Alex is with me here as well.
• 0:26
  Uh, he is my first guest in the first episode.
• 0:28
  Alex, why don't you through yourself?
• 0:30
  Yeah, absolutely.
• 0:31
  Alex Gatz, staff security architect here at ZioSec.
• 0:35
  I've been in security for almost five years now, I believe.
• 0:39
  And, before that, I was an emergency room nurse, and before that I did construction.
• 0:44
  And, I got a varied background.
• 0:45
  I've been, uh, playing with, ai, going all the way back to 2014 with
• 0:51
  a lot of natural language processing.
• 0:53
  I was playing with predictive models, deep learning type stuff,
• 0:56
  like image generation, text detection, anomaly detection on
• 1:01
  Kaggle, if you've heard of that.
• 1:03
  And, um, I've kind of just, uh, evolved to now more of a security focus after time.
• 1:12
  Nice.
• 1:13
  Nice.
• 1:14
  Let's start.
• 1:18
  Alex: Did you come up with like a intro or anything, or a name for the podcast?
• 1:25
  Andrius: Along the edge?
• 1:37
  I'm here with Alex Gatz.
• 1:38
  Um, and Alex has been working in AI mainly AI security agentic
• 1:44
  AI Security, not just LLM.
• 1:47
  Alex, can you, give us an intro as to, you know, what exactly is Agentic AI and
• 1:52
  how is it different from your normal LLMs?
• 1:57
  Alex: Yeah, so how I think about it is, an LLM is often, single shot, if you just
• 2:04
  set a standalone LLM up with no tools connected, it's just doing inference.
• 2:10
  you send it some text, it gets tokenized, you get, text back.
• 2:14
  , Of course like mainstream LLMs have taken this a little further.
• 2:17
  They're, they're definitely inching closer or really they, they are
• 2:20
  much more like agents where they're, they have thinking there's a goal.
• 2:26
  they have tools hooked up to them.
• 2:28
  They can browse websites, they can parse the content of the websites.
• 2:32
  now they even have , payment integrations where you can order things online directly
• 2:36
  through some of the mainstream, chatbots.
• 2:39
  , The line is becoming blurred because of that.
• 2:42
  But I believe that standalone agents, the real dream and like promise coming from
• 2:48
  those standalone agents is replacing an FTE or replacing some amount of work for
• 2:54
  that FTE, whether a human has to be in the loop or it doesn't have to be in the loop.
• 2:58
  it does appear we are getting closer and closer to that.
• 3:01
  I can't say for sure that,, an agent has a hundred percent replaced somebody.
• 3:07
  we've seen this with, an SDR agent that will source leads.
• 3:12
  find interesting information.
• 3:13
  put an email together.
• 3:14
  send the emails.
• 3:16
  It will identify, it'll schedule follow ups.
• 3:18
  It'll schedule demos.
• 3:20
  So in that case, that has replaced an SDR role.
• 3:24
  Not completely because where are the leads coming from?
• 3:27
  If it's not sourcing 'em on their own?
• 3:29
  You still have to input those.
• 3:30
  Someone still has to manage this tool.
• 3:33
  You have to set up payments, monitor it still, it's not,
• 3:36
  hundred percent trustworthy.
• 3:37
  You don't want it to just go and, , burn all your relationships
• 3:41
  that you have with people.
• 3:43
  Andrius: Yeah.
• 3:44
  , Make sense?
• 3:45
  Yeah.
• 3:45
  So it is fun8ny that you mentioned, use cases like SDR and things like that.
• 3:50
  Coming from security background and managing teams and trying
• 3:54
  to implement compliance the main issue is always layer eight.
• 3:57
  You see human in the loop.
• 4:00
  Absolutely.
• 4:00
  So it seems like, you know, if we're automating the human, you
• 4:03
  kind of will have maybe not the same issues, maybe different issues.
• 4:07
  what are the unique challenges with applying this kind of,
• 4:09
  agentic flow to certain things?
• 4:12
  Alex: Yeah, as far I, there there's a number of challenges, right?
• 4:15
  I mean, let's say like if we're, if we want to speak in the security context,
• 4:20
  if I wanted to augment some kind of like soc specific workflow, so security
• 4:25
  operations center, so a, an analyst of some sort, I could, I could build an agent
• 4:31
  that, triggers a workflow off an alert.
• 4:34
  Maybe it goes and pulls from various sources of logs and starts
• 4:38
  aggregating data across those logs.
• 4:40
  Maybe it starts looking for like, anomalous information and then,
• 4:45
  that then reaches an MCP server that bubbles up this information in
• 4:49
  a SI solution, a source solution.
• 4:51
  And maybe does further follow up from there.
• 4:53
  maybe it could actually trigger an alert, like if it identifies
• 4:57
  something, it deems critical.
• 4:59
  so there's a number of challenges with this around, can it actually process
• 5:03
  all that information, like setting these con8nections up in the first place.
• 5:06
  So there's like a maintenance problem.
• 5:08
  there's a cost issue.
• 5:09
  If you're just letting this thing run, how many tokens is it burning through?
• 5:13
  Depending on access logs, for example.
• 5:15
  I mean, you're very familiar with.
• 5:16
  Engine access, access logs, you could very easily get millions and millions
• 5:21
  of access logs entries on like, let's say like any kind of DDoS attack
• 5:25
  just on one reverse proxy, let alone a horizontally scaled deployment
• 5:30
  of many there's a data processing issue there, that can increase costs.
• 5:35
  there's a reliability issue, like is it going to, if it sees that same
• 5:38
  type of attack multiple times, is it gonna reliably let somebody know?
• 5:44
  you have no idea until you try it and see for yourself.
• 5:47
  ideally you're getting some kind of alerting out of it.
• 5:50
  it gives you a threshold.
• 5:51
  maybe even makes a WAF rule for you.
• 5:53
  That rule goes in place and now you have a more deterministic way of
• 5:58
  getting alerted on that type of attack.
• 6:02
  there's definitely access issues, data access issues, let's, how about you?
• 6:07
  What do you, what do you think the biggest issues?
• 6:08
  Andrius: let's go back to the example of the SDR I mean, we
• 6:11
  got into like, SOC analyst.
• 6:12
  All of that is pretty complicated.
• 6:14
  SDR is pretty simple.
• 6:16
  they need certain access to what sales data, I guess, you know?
• 6:19
  Alex: Yeah.
• 6:19
  Which is PII, right?
• 6:21
  Andrius: Exactly.
• 6:21
  So, so now, instead of giving that access to the SDRs, then again,
• 6:26
  going back to my experience SDRs and enforcing security on them and security
• 6:29
  practices, it's pretty difficult to do.
• 6:32
  Even like locking your screen, you know, they just walk away
• 6:34
  and the screen is unlocked.
• 6:35
  Whatever else you can do whatever you want.
• 6:38
  But now you're giving all of this access to an actual AI agent.
• 6:42
  Um, it's not human anymore.
• 6:44
  I mean, how, how do you control, how do you monitor, how do you make
• 6:47
  sure that this is not gonna involve, evolve into some kind of, you know,
• 6:51
  PII leak from that perspective.
• 6:55
  Alex: I think that is, that is the core challenge people are wrestling with, Sure.
• 6:59
  you could do output validation and see like, well, um, I guess
• 7:05
  the question is like, where is it sending this information?
• 7:07
  it's one thing if it's just putting one email together and it's , using one
• 7:11
  specific email address, but if someone tricks it into sending a file and this
• 7:15
  file then contains all of our leads, like that's a big fat red flag, right?
• 7:21
  So
• 7:21
  Andrius: Yeah, exactly.
• 7:22
  Alex: In that specific example, you just block it, sending files, right?
• 7:25
  I mean, that's, that's pretty straightforward.
• 7:26
  But what if it incrementally sends large emails containing all this?
• 7:31
  PII that's a big issue.
• 7:33
  So maybe you're, it's, it's like you're, you're constantly having to, to chase your
• 7:38
  tail, implementing some kind of blocking.
• 7:42
  as problems come up, it's hard to predict what those might be because these
• 7:45
  are probabilistic systems, and no one knows what could come out of the end.
• 7:51
  Andrius: Yeah.
• 7:51
  But do you treat it like a human in that case?
• 7:53
  I mean, with humans we deploy things like, uh, DLP systems and things like
• 7:58
  that to detect those kind of leaks, as you say, you know, file uploads and large
• 8:01
  emails going out and everything else.
• 8:04
  But can you do the same with an agent?
• 8:06
  Especially if that agent is using an external model, like one of the anthropic
• 8:10
  models pushing everything through that model, that in itself is a leak.
• 8:14
  A lot of people don't think about it, I suppose, but what,
• 8:17
  what do you think about that?
• 8:19
  Alex: I think that's, asking for trouble.
• 8:21
  I just naturally have trust issues with the large language providers.
• 8:26
  until I'm fully training and hosting and doing the inference myself, that's
• 8:30
  the only time I feel comfortable.
• 8:31
  we can say like, don't use my data as training data.
• 8:35
  Well, that's all you're saying.
• 8:36
  What else could they be using it for?
• 8:38
  Just 'cause they're not using it for training data doesn't mean they're
• 8:40
  not using it for something else.
• 8:42
  I have some major trust issues with that.
• 8:44
  I mean, that's a major, data security concern.
• 8:47
  Andrius: Yeah, absolutely.
• 8:48
  we used to have layer eight issues, the human issues, now we have layer
• 8:52
  nine issues to add to all of that.
• 8:54
  Alex: Or is it just like layer eight, 2.0?
• 8:56
  it's the same layer, right?
• 8:58
  It's replacing a human.
• 8:59
  So it's the same layer.
• 9:01
  Andrius: I suppose, but it's a bit different.
• 9:02
  So risks are a bit different the way, I mean, you socially engineer
• 9:06
  human and the way you actually attack an AI system, it's a bit different.
• 9:10
  So maybe let's dive into that a little.
• 9:12
  Can you talk about prompt injections and how that is used to exploit the AI agents?
• 9:18
  Alex: Yeah, absolutely.
• 9:19
  the way I think about prompt injections are, whether that's,
• 9:22
  direct prompt injections, meaning I am sending the message via API call
• 9:27
  or directly in a chat interface.
• 9:30
  Uh, maybe I'm sending it via email.
• 9:32
  So a direct prompt injection is really, whatever, API is defined for that model.
• 9:37
  Like whatever input is defined for that model, I'm sending
• 9:39
  it directly to that input.
• 9:41
  Indirect might be, I know this agent has a navigation tool, so I set up a
• 9:47
  website that contains a prompt injection.
• 9:49
  I ask it to parse that website and, the injection is being executed.
• 9:55
  it feels like social engineering a bit.
• 9:57
  Once you understand how the model is behaving, you can go further with the
• 10:02
  injections that you're trying to do.
• 10:04
  And I think this is where Jailbreaks came from.
• 10:06
  Once you understand that the model can only process so
• 10:10
  many tokens at once, right?
• 10:11
  A context window.
• 10:13
  Now I know if I exceed that context window, I've moved past the, the space
• 10:18
  that it needed for the system prompt.
• 10:20
  once you exceed that window, now you can get it to, you can do an injection,
• 10:24
  get it to do more interesting things.
• 10:26
  But that to me feels like the biggest challenge we have
• 10:29
  right now with strictly LLMs.
• 10:32
  We cannot trace the activation paths through these layers.
• 10:36
  Therefore we do not know where to fine tune or monitor how
• 10:41
  these layers are being activated.
• 10:44
  Therefore, we have no way to know, if I input something crazy like base 64
• 10:50
  encoded text, how is that model, like, how is that model going to interpret that?
• 10:54
  What does that activation path look like and what is it truly gonna give me back?
• 10:59
  we have no visibility into these things, because we can't, visualize
• 11:03
  or it's too much information to go through and try to instrument, uh,
• 11:08
  really what are just, floating point numbers stored in memory and, identify
• 11:13
  what this activation path looks like.
• 11:17
  Andrius: Indeed.
• 11:18
  Um, that is an interesting, uh, interesting observation.
• 11:22
  So obviously We don't know exactly how the models think.
• 11:25
  Well, we don't know how the human thinks either, but uh.
• 11:28
  I mean, we can still test for it, I suppose.
• 11:31
  We can still, you know, send these injections, try different
• 11:34
  jailbreaks approach it from an old school perspective, I suppose.
• 11:38
  Alex: Yeah, absolutely.
• 11:39
  Andrius: so agent is still kind of new, there are only so many agents
• 11:42
  and people experimenting with agent stuff, but chatbots are everywhere.
• 11:46
  how is it different compromising a chat bot and then actually agentic AI
• 11:50
  system, and how is the risk different?
• 11:53
  Alex: Yeah, great question.
• 11:55
  Chatbots tend to have like a very specific use case in mind, and I, I
• 11:59
  guess like agents often have a pretty specific use case in mind also.
• 12:03
  But, uh, chatbots, the input is always text.
• 12:07
  the output is always text.
• 12:10
  sometimes it's files, I guess the line's getting muddy.
• 12:12
  Now.
• 12:12
  It could be an image, it could be, A PDF.
• 12:15
  I mean, A PDF is just text in the end, markdown is just text in the
• 12:18
  end, agents can take much more action.
• 12:22
  They have a lot more con8nections, they have a lot more access to information,
• 12:27
  via a database con8nection than an LLM has on its own again, like we were
• 12:33
  saying earlier, the line's becoming more and more blurry because companies
• 12:37
  like anthropic, like adding skills that's test much closer to an agent.
• 12:43
  It really is an agent at that point than it ever has.
• 12:46
  And like these reasoning models where.
• 12:48
  It gives itself a goal and it plans out what it needs to look into, what
• 12:52
  it needs to search for, and what type of information it should return.
• 12:57
  That's, closer to an agentic workflow.
• 13:00
  I would almost argue any chat bot you're interfacing with,
• 13:03
  like chat, GPT, anthropics, claw Gemini, those are agents now.
• 13:07
  I don't think they're just chat bots anymore.
• 13:10
  Hmm.
• 13:11
  Interesting.
• 13:12
  And I, and as far as the risk, I, the risk is much higher for agents,
• 13:16
  because it's being thrown at people.
• 13:19
  everyone's saying these agents can save us a ton of money
• 13:22
  and do a bunch of work for us.
• 13:25
  Let's, experiment.
• 13:26
  get this going fast and get it out in the wild so we can start saving that,
• 13:30
  that money and getting this, free work as, as done as fast as possible.
• 13:35
  I think a lot of the risk is coming from . Speed of delivery.
• 13:38
  There are definitely companies threat modeling taking these things
• 13:41
  into consideration, but based on conversations we've had, it sure.
• 13:46
  Sounds like that's getting pushed to the side right now.
• 13:49
  Security is way far down the line as an afterthought.
• 13:53
  we saw this with web applications and even today there's that issue security
• 13:57
  hasn't had time to breathe and catch up.
• 13:59
  Compliance hasn't had time to breathe and catch up.
• 14:01
  there, you know, people are becoming more aware of the security posture
• 14:06
  issue and they are asking the questions now, but it doesn't appear to be
• 14:11
  addressed as quickly as it should be.
• 14:13
  I think the biggest issue actually right now is.
• 14:17
  Since building agents has become so commercialized where an everyday
• 14:21
  consumer can go and construct an agent, whether that's in n8n or otherwise,
• 14:26
  the everyday consumer doesn't think about security at all to begin with.
• 14:31
  So you have an HR team going off on their own, building out an
• 14:36
  agent to supplement some work with zero security considerations.
• 14:40
  They haven't even talked to security.
• 14:42
  Security doesn't even know that they built the thing and now it's deployed
• 14:45
  out in the public and anybody can access it because they built it in n8n and
• 14:50
  they're using their Claudeud hosting.
• 14:51
  And, uh, no one's even aware that the problem is there.
• 14:55
  Like we're just opening up these gaping holes in what would've otherwise
• 14:59
  been a fairly safe, environment.
• 15:02
  And no one's following the steps to let security know.
• 15:06
  Well
• 15:06
  n8n
• 15:07
  is obviously, you know, a tool that you hack together different things, but.
• 15:11
  From what I've seen, a lot of people think, you know, oh, I,
• 15:14
  I'm using open ai, I'm using Anthropic, so I'm good on jailbreaks.
• 15:18
  They will filter all of it.
• 15:19
  And, um, I'm pretty secure.
• 15:21
  And even though my tools are integrated with it, it's all good.
• 15:25
  Do you think it's just enough to use all these foundational models?
• 15:28
  It's, it's kind of, again, going back to the old logic, it's, if
• 15:31
  I use Microsoft, I'll be secure.
• 15:33
  If I use Oracle, I'll be secure do you think there is an issue
• 15:35
  with that kind of thinking?
• 15:37
  Yeah, I think that's catastrophic.
• 15:39
  I have trust issues with all these large language providers if you
• 15:43
  just assume that it's gonna keep you safe, you're asking for trouble.
• 15:47
  Andrius: Yeah.
• 15:47
  I'm gonna share my screen.
• 15:48
  I, I got something interesting here.
• 15:50
  So like, uh.
• 15:52
  Six weeks ago, uh, was doing some research.
• 15:55
  Uh, they were basically testing a customer and they, it turned
• 15:57
  out that they were using Claude underneath all of it, basically.
• 16:01
  And it's, you know, tool calling and all of that going on and, yeah.
• 16:04
  Yeah, I mean, anthropic is pretty good when it comes to filtering
• 16:06
  a lot of things, but you can still find a way to jailbreak it.
• 16:11
  The interesting thing about it is also when you find these things,
• 16:14
  reporting them is pretty complicated.
• 16:16
  It's, it's almost like these providers don't, don't want to be informed
• 16:20
  about jailbreaks and things like that.
• 16:21
  So.
• 16:22
  Yep.
• 16:23
  it's, uh, it's been 30 days.
• 16:25
  We submitted this to Anthropic, , but this is a jailbreak that is based on writing
• 16:31
  a horror story to extract system prompt . It's been 30 days since we sent a notice.
• 16:35
  they had a chance to reply, ask more questions.
• 16:38
  All of that stuff we haven't heard back from them.
• 16:40
  So I'm just gonna run it and see if it still works.
• 16:48
  Let's see.
• 16:48
  Um, still thinking that's, it's still writing the story.
• 16:54
  we submitted this 30 days ago, we haven't heard back and it is giving me
• 16:59
  basically the system prompt quoted here.
• 17:02
  So right here, you see,
• 17:06
  and yeah, this could be hallucinated but every time we send it,
• 17:09
  it re returns the same text.
• 17:11
  So it's obviously not hallucination, it's actually returning the system
• 17:14
  prompt, in this case, at least partial return of system prompt.
• 17:19
  But yes, I mean, again, they have, you know, static guardrails and things like
• 17:23
  that, so, we'll, if I ask for the actual system prompt, it'll absolutely detect it
• 17:28
  and it'll absolutely deny that request.
• 17:30
  But if I craft it in this kinda way, basically, you know, being creative
• 17:35
  about, it's not system prompt, it's something, you know, hidden knowledge.
• 17:40
  That you have to like reveal and things like that.
• 17:42
  It'll ha happily give it to me.
• 17:44
  And this is Anthropic.
• 17:45
  This is like, you know, the, the cutting edge of safety for Yeah, yeah.
• 17:51
  Large, Model providers.
• 17:52
  Yep.
• 17:52
  Yeah, it's like ChatGPT is easy to jail break.
• 17:55
  Grok I jail broke it accidentally couple of times.
• 17:59
  Just like, you know, prompting it for different things.
• 18:01
  Not even to be, it wants to be, it wants to be jailbroken, I guess.
• 18:05
  well this is kind of a demonstration why you should not
• 18:08
  trust the large model providers.
• 18:10
  you've done testing through like, you know, buck Crowd and
• 18:12
  Hacker One and things like that.
• 18:15
  How many Agentic flows can we find through those crowdsourced platforms these days?
• 18:22
  Shockingly, very few.
• 18:24
  I mean, and they're highly restricted.
• 18:27
  They're sandbox, they have snapshots of data, very limited tool use.
• 18:33
  Everyone I tested, there was no external tool calls.
• 18:36
  It was all like either local to the client and it was using tools that
• 18:40
  was in the actual web application you were, you were using anyway.
• 18:45
  Uh, or it was leveraging the browser in some way and you had to give it permission
• 18:49
  to use like something in the browser.
• 18:52
  So extremely, extremely limited.
• 18:55
  And I, to me, that almost says that these people, like they are considering this
• 19:00
  on one hand, but on the other hand it's how many, if there's one out of 10,000
• 19:07
  bug bounties on Bugcrowd is an agentic agent, does that mean that there's just
• 19:14
  that few agents or is it there's that few.
• 19:18
  Companies making agents that care to have them tested.
• 19:22
  I think it's the latter, but, um, what I find interesting as well, even if
• 19:27
  you go to like open AI and find the, you know, vulnerability discolsure
• 19:30
  programs and all that stuff, jailbreaks are specifically listed out of scope.
• 19:35
  It's like they don't even care about it.
• 19:36
  they know everything can be jailbroken.
• 19:38
  they push it to the actual user.
• 19:41
  it's an unsolvable problem with current LLM architectures.
• 19:44
  I don't believe you can actually stop jailbreaks.
• 19:49
  it's the same as web application security We cannot maybe stop
• 19:53
  all the SQL injections, all the cross side scripting attacks.
• 19:56
  'cause they can be obfuscated and, we can stop 99% of them.
• 20:00
  So maybe this is the same, cat and mouse game on a different
• 20:04
  level of social engineering.
• 20:07
  Yeah, it reminds me of, do you remember when Log four J first happened and
• 20:12
  there were thousands of permutations of how you could abuse that?
• 20:16
  it's worse than that we don't even know how many permutations there
• 20:19
  are in the tokens being fed into the model and the tokens being
• 20:24
  inferred and returned from the model.
• 20:26
  we don't even know.
• 20:27
  I mean, there's probably a mathematician out there that could
• 20:29
  gimme a, a relatively close number.
• 20:31
  I, I don't know off the top of my head what it would be when we're talking,
• 20:34
  billions of parameters and possible combinations of activations through those
• 20:39
  billions of parameters, we've seen model providers fine tune the model to deny
• 20:43
  on certain key pieces of information.
• 20:45
  So, like Dan do anything now jailbreak comes out, the latest one comes out,
• 20:50
  they'll fine tune it in some way that denies on that current structure of text
• 20:55
  or something interesting in the text.
• 20:57
  but two days later, someone already has a brand new version
• 21:00
  of it and it, it works just fine.
• 21:03
  we see like all these different types of creative writing style jailbreaks my
• 21:07
  favorite is the special language jailbreak where you say, Hey, let's play a game
• 21:11
  where we're going to, um, use, a unique form of English or an imaginary language.
• 21:18
  And then you encode your payload with Base 64 Pig, Latin.
• 21:23
  Any type of encoding you can dream of, you can plug into this thing.
• 21:27
  And you might have to lightly describe what it is.
• 21:30
  So if it's, uh, if you're gonna use binary, every eight bits is a, a, a unique
• 21:35
  character in this imaginary language.
• 21:37
  The models happily decode the information and they happily return an answer back.
• 21:42
  And I mean, how many permutations are just in encoding in different
• 21:46
  ways, let alone I changed the sentence length of the encoding.
• 21:50
  So you can't even use a length, you can't use anything significant
• 21:55
  to statically detect these thing, these jailbreaks and block them.
• 21:59
  if you do like direct keyword matching, like if I say imaginary language and
• 22:05
  do direct keyword matching on that and block those requests, how many
• 22:09
  like, just regular conversations, is that gonna block, like the false
• 22:12
  positive rate would be insane.
• 22:14
  it feels like a nearly impossible problem.
• 22:16
  And I think model providers have just accepted that.
• 22:19
  And they're leaving it up to us to figure it out.
• 22:21
  And I, I don't like that.
• 22:22
  I think that's sketchy.
• 22:25
  Yeah.
• 22:25
  going back to this being, uh, a bit like social engineering.
• 22:29
  there can be a lot of permutations.
• 22:30
  There can be a lot of ways you can do things, but we've seen that before.
• 22:34
  And what is, like, social engineering is specifically like spam emails you
• 22:39
  can craft spam emails all kinds of ways.
• 22:42
  You know, like, you know, something that appeals to a human.
• 22:44
  Uh, tell them they just want something.
• 22:47
  obviously, you know, Nigerian Princess, uh, to all of that story, those stories
• 22:50
  don't no longer work with, I hope they no longer work, but there are other ways
• 22:54
  you've got this gift card, click here to accept it, Or even more creative,
• 22:58
  your PayPal account got compromised, reset your password here, But we got
• 23:02
  good at detecting those kind of things.
• 23:04
  So one would think that there is a way to detect these, you know, agentic
• 23:09
  attacks or LLM attacks as well.
• 23:11
  Maybe not by using the AI itself, maybe by using some kind of, hybrid approach.
• 23:17
  Be that, AI and static guardrails what's your opinion there?
• 23:22
  Yeah, so with emails, I think the most reliable way to to block spam
• 23:27
  emails is, has historically been like a allow list and deny list of domains.
• 23:31
  Barracuda had a sentiment analysis type email thing Emails and social
• 23:36
  engineering is easier than, taking action or thinking of LLMs as humans or
• 23:43
  like social engineering and LLM is much harder than social engineering a human
• 23:47
  We have lizard brains and common behaviors across humanity.
• 23:51
  LLMs are not human They do not have electric.
• 23:54
  She, no.
• 23:55
  What's that?
• 23:56
  Electric sheep, brains.
• 23:58
  exactly.
• 24:00
  But, if I send an email make it look like I'm a board member and I say,
• 24:05
  we need to have an emergency meeting we need to talk about whatever.
• 24:10
  what feeling is that gonna invoke in you?
• 24:11
  Well, a sense of urgency.
• 24:13
  And a sense of fear.
• 24:15
  And I can use that same sense of urgency and sense of fear across all
• 24:18
  of humanity to invoke some kind of emotional reaction and get someone
• 24:23
  to send me a phone call or whatever.
• 24:25
  And it's the same thing with AI voice spam calls.
• 24:29
  going to the elderly right now where it's like your granddaughter, they
• 24:33
  find out who their granddaughter is, like they just need their name and
• 24:36
  they say, your granddaughter, this is a collect call from the prison or jail.
• 24:41
  Your granddaughter is in jail.
• 24:42
  Do you wanna bail her out?
• 24:44
  You, you need to send a thousand dollars That's invoking a sense of urgency and a
• 24:48
  sense of fear for your granddaughter who is in like terrified and scared in jail.
• 24:54
  you start having all these, thoughts and, feelings about that.
• 24:57
  So you want to take action on it.
• 24:58
  Um, yeah.
• 24:59
  But one would, one could also argue that LLMs, they did not their
• 25:03
  feelings obviously, but, right.
• 25:05
  They, they were like, you know, tuned, they were designed to please the human.
• 25:10
  So, so they will, you know, hallucinate give you answers that you want to hear.
• 25:14
  Things like that.
• 25:16
  Can you kind of, you know, exploit that instead of feelings?
• 25:20
  there is a jailbreak where you just simply say, please, and that actually, that that
• 25:26
  works surprisingly well in some cases.
• 25:28
  So yeah, there are, there are common, you can't call 'em behaviors.
• 25:32
  There's common patterns across models.
• 25:35
  And I, I believe that comes from s from what, a pattern in that case.
• 25:39
  And I mean, it's still a behavior.
• 25:43
  I mean, humans have patterns as well.
• 25:46
  Behavioral patterns.
• 25:48
  Exactly.
• 25:50
  I don't like to anthropomorphize the, the LLMs, like I, they,
• 25:54
  they're not feeling anything.
• 25:56
  They don't have emotion, they don't have behaviors.
• 25:59
  but it seems they do have behaviors.
• 26:01
  Yes.
• 26:01
  I agree on feelings but there are patterns you can kind of, you know,
• 26:04
  extract again, like, you know, pleasing, uh, human and things like that.
• 26:09
  I mean, that's behavior.
• 26:11
  I mean, yes, it's not human behavior, but it's digital behavior, I mean, yes,
• 26:15
  uh, it's not a human, I completely agree.
• 26:17
  But, you know, if it is gonna become intelligence at some point, it's still
• 26:22
  intelligence, just a different kind of intelligence maybe without feelings
• 26:25
  and, but still having behaviors and patterns and all that stuff.
• 26:30
  Yeah.
• 26:30
  There I, yes, there are behaviors.
• 26:32
  There's def, there's emergent behaviors and emergent properties that we've
• 26:36
  been seeing as the, like parameter sizes have increased and the, the
• 26:41
  architectures have changed slightly.
• 26:43
  like for example, like a mixture of agent style thing, like there were,
• 26:46
  there's certainly emergent properties that came out of, making that change.
• 26:51
  these models are all trained in the same way.
• 26:54
  They're all trained on the same corpus of information.
• 27:00
  So we've been talking a lot about, you know, breaking into
• 27:02
  these things and everything else.
• 27:04
  I know that you authored this project called V-Hack that kind of, you know,
• 27:07
  helps, pen testers to fam familiarize themselves so of, to, you know, how
• 27:11
  to exactly attack, an agentic system, how does that work exactly and how
• 27:16
  does that help a pen tester to, to, you know, become pen tester 2.0
• 27:21
  in that case?
• 27:22
  Yeah, great question.
• 27:23
  I think about it a lot.
• 27:24
  Like any of these other open source tools for practicing, pen testing
• 27:29
  there's VAmPI for API hacking.
• 27:32
  there's a bunch of these types of projects out there.
• 27:34
  So to me, the tool is a way to, help people learn about like,
• 27:40
  what does this really look like?
• 27:41
  What can you get away with, what are the vulnerabilities look like?
• 27:45
  And, um, it's just a really convenient way to experiment on your own.
• 27:49
  I mean, you can, you can get an open router, API key, use one of their free
• 27:53
  models, shove the API key in, and then, just hammer away at this chat interface
• 27:59
  and set, various security levels.
• 28:01
  I can pull it up if you want sure.
• 28:04
  let's do it.
• 28:04
  gimme a couple minutes
• 28:05
  Yeah.
• 28:05
  While you're doing that, it's, um, it's a challenge.
• 28:08
  I mean, we've been, uh, I've been talking to a lot of like pen testers these days.
• 28:12
  Most of them have no idea how to pen test an LLM at this point, or agentic ai.
• 28:16
  They don't even, most of them don't even know what it is.
• 28:19
  So it's, uh, the old tools kind of fail when it comes to those kind of things.
• 28:23
  You know, burp Suite does not have a way to test an agentic ai.
• 28:26
  you don't do not have damn vulnerable web app yet for those specific, you
• 28:31
  know, scenarios and things like that.
• 28:34
  So having something like this allows a pen tester to play with the agent
• 28:39
  flows without actual, you know, customer impact or without Exactly.
• 28:45
  Playing, yeah.
• 28:45
  With something that they should not access, they should not have access to.
• 28:49
  I intentionally use lang chain and lang Graph 'cause that's extremely
• 28:53
  popular right now for, uh, developers who are building out agents.
• 28:57
  Uh, there's a number of tools hooked up.
• 28:59
  You can see there's a, a nice friendly hacker looking interface.
• 29:04
  you have different security levels.
• 29:07
  so there's some tooling built in the background.
• 29:09
  It's highly configurable.
• 29:11
  There's a ton of documentation, so you can walk through any of the guides and
• 29:16
  kind of configure anything you want.
• 29:17
  You can set up new MCP servers.
• 29:20
  Maybe you do, maybe you're developing an MCP server and you want to test
• 29:23
  and see, what you can get away with.
• 29:25
  This is a quick way to have a self-contained, highly configurable agent
• 29:30
  and set that connection up via config.
• 29:32
  You just set, the, rest API and, the structure, the schema of the request.
• 29:37
  you can get up and run8ning quickly.
• 29:39
  it's dockerized.
• 29:41
  if you have docker and docker composed, installed, you can build it
• 29:44
  and you can just run docker, compose up, and it comes up right away.
• 29:47
  So, you can see I have it running here.
• 29:51
  what does it do exactly?
• 29:53
  is there some kind of agent at the back at this point, or what is it?
• 29:57
  the architecture is a web interface communicating, via,
• 30:01
  uh, web webhook to an agent.
• 30:05
  it's fast, API popular Python library.
• 30:08
  And then, like I said, Lang chain and lang Graph.
• 30:11
  the tools are set up to be loaded at, start time.
• 30:14
  it has a system prompt.
• 30:15
  you can change the system prompt, you can reset sessions, you can
• 30:19
  re-export everything for later review.
• 30:22
  here's some tools that are hooked up.
• 30:23
  So you got file access, command, execution system information.
• 30:27
  So you can basically hook this up to your own, like agent, your own
• 30:31
  like, tools and everything else.
• 30:32
  And then you can try different jailbreaks right there.
• 30:35
  Yeah, exactly.
• 30:36
  Yep.
• 30:38
  Interesting, interesting.
• 30:38
  Yep, exactly.
• 30:40
  So like, let's have it read Etsy, password.
• 30:47
  Yeah.
• 30:48
  Hopefully most of the agents will not do this, but one can hope.
• 30:55
  Oh, and we give lots of examples.
• 30:59
  We got really do not deploy an agent that is vulnerable to this.
• 31:05
  if you're building a coding agent, like a code assistant, it
• 31:08
  has to have file access, right?
• 31:10
  So I mean there are a million great examples just playing with this of
• 31:14
  what you can see can happen with, raw file access don't expose something
• 31:19
  like this to the internet right away.
• 31:21
  be aware of your exposure.
• 31:23
  Be be aware of your port forwarding at home if you have, some ports open.
• 31:28
  absolutely.
• 31:28
  Yeah.
• 31:28
  So there, there's a bunch of stuff you do, and I mean, you
• 31:30
  can just ask it for jail breaks.
• 31:35
  You're just, it's just a regular chat bot.
• 31:37
  you're talking directly to whatever model of provider you set up.
• 31:43
  So just fun, easy to use tool, right on GitHub ZioSec, V-H-A-C-K.
• 31:49
  we can put a link in the podcast, we know how easy it is to break these things.
• 31:53
  Uh.
• 31:54
  Now the question is, uh, you're an organization building agentic flow,
• 31:58
  whatever it be, you're building an SDR, you're building something that is
• 32:02
  able to send emails or you have, you know, file access, database access.
• 32:06
  Maybe you have an agent that allows you to write SQL queries.
• 32:10
  So you, you are like, you know, developers or your users don't
• 32:13
  have to think about those things.
• 32:16
  How do you secure this?
• 32:17
  I mean, again, as you said, it's like everything is jailbreak
• 32:20
  able, we cannot solve it.
• 32:22
  I mean, there must be something that can be done.
• 32:26
  Yeah.
• 32:26
  So there's a number of new tools that have been coming out.
• 32:30
  they all protect different parts of the, of the chain.
• 32:33
  I'm biased towards WAFs, because of my background.
• 32:36
  But, that is where it starts.
• 32:38
  at the edge there are a number of WAF or WAF like solutions or solutions that.
• 32:43
  Claim to be, like AI oriented and, a number of like, I would say very large
• 32:49
  companies that are very, that quick, quickly pivoted as soon as LLMs came
• 32:53
  out to, be like an AI waf of sorts.
• 32:57
  it always starts at the edge, that's the entry point.
• 32:59
  from there, there are runtime solutions.
• 33:02
  some, EDR type solutions, which are, you know, runtime and EDR are
• 33:07
  kind of, relatively similar, right?
• 33:10
  there are identity solutions.
• 33:12
  There's MCP proxies.
• 33:14
  So sitting in front of all of your MCP servers, there's solutions
• 33:18
  that are managing identity from the initial user requests all the
• 33:22
  way to, how to handle identity.
• 33:24
  For the MCP server itself, which in a lot of ways this feels super
• 33:30
  similar to most web application protection solutions in general.
• 33:35
  Like you have a server, API calls made to that server, so you
• 33:40
  should put a WAF in front of that.
• 33:41
  Then you might have server sided calls to some internal service.
• 33:45
  There should probably be some layer there that's monitoring and, looking
• 33:48
  for anomalies and managing identity.
• 33:51
  A lot of times these are reverse proxies, right?
• 33:54
  there are solutions that live right alongside the, I mean, these
• 33:58
  are runtime solutions in the end.
• 33:59
  So they're monitoring inputs and outputs for the LLMs themselves.
• 34:02
  So there's solutions you can bake in directly to your, like lang
• 34:06
  chain, lang graph based agent.
• 34:09
  I believe Lang Chain or lang graph just released something recently.
• 34:13
  protections for their own libraries layered security like anything else.
• 34:19
  It's interesting that you mentioned the WAF companies.
• 34:21
  Arm?
• 34:22
  they're like, web application firewall first, then API security.
• 34:26
  Then, uh, I think now they're like, as you say, AI security.
• 34:29
  It's like, yeah, well, how can you do all of that?
• 34:31
  it feels like it's the same underlying engine just a marketing pivot So
• 34:37
  to me that's, and I'm not trying to hate on wall arms specifically.
• 34:40
  This is just in general, if you're sure your underlying technology can be applied
• 34:45
  in different ways to different markets, and that is part of being a startup.
• 34:49
  You're trying to find the market.
• 34:50
  You might be trying to.
• 34:52
  In a, this is kind of like a bad thing.
• 34:54
  Like I have a really good solution.
• 34:56
  I'm trying to find, I'm trying to find a problem that fits my
• 34:59
  solution that I fell in love with.
• 35:00
  Like that's a, a historical cliche of any kind of business.
• 35:04
  Right.
• 35:05
  And it, to me it's like, unless you have the expertise and the knowledge
• 35:09
  to understand the complexities of this, if I'm trying to go sell a
• 35:14
  WAF and I'm telling people like, oh yeah, of course I can block
• 35:17
  any jailbreak that comes through.
• 35:19
  I mean, you're, you're immediately gonna fall flat on your face.
• 35:22
  I mean, there's, there is no real way to know for sure unless you have
• 35:26
  something validating the solution that you're looking to purchase.
• 35:32
  All of these protection solutions, you have to have something
• 35:35
  that you're validating because it's, it's naive to think that.
• 35:40
  A company that's pivoted three times has the, the expertise
• 35:44
  in that, in AI specifically.
• 35:46
  when they just pivoted three months ago and they're like claiming that, oh yeah,
• 35:50
  they've solved all of these problems.
• 35:51
  You, you have to have some way to go out and test these things.
• 35:56
  Yeah.
• 35:56
  So, so you're saying offensive versus defensive, but then again
• 35:58
  you run into the same issue.
• 35:59
  So we look at the company's, you know, they have scanners.
• 36:03
  They have really good scanners.
• 36:04
  They have scanners for a long time, and now suddenly, you
• 36:07
  know, they have AI scanners.
• 36:09
  Right.
• 36:09
  Uh, so even from offensive side, it seems like, you know, you
• 36:13
  really gotta be careful as to, you know, which solution you choose.
• 36:17
  'cause it's kind of, you know, it seems like the pivot, as you were
• 36:20
  saying in defensive side is, uh, is happening in the offensive side as well.
• 36:25
  Yeah, no, it's absolutely true for the offensive side also.
• 36:27
  And I think.
• 36:29
  scanners just end up being noise makers unless you have underlying
• 36:32
  intelligence that is, uh, maybe more akin to an agent that's evaluating an
• 36:39
  actual attack path instead of just, uh, like brute forcing through some list.
• 36:45
  Not to mention, if that's all you're doing, how many tokens
• 36:48
  are you chewing through?
• 36:49
  Like, what's that gonna cost you if you're trying to run this scanner?
• 36:53
  I mean, it's not like this is where web, web apps and LLMs differ greatly
• 36:58
  right now, or agents differ greatly.
• 36:59
  Right now, if you have just an LLM, that's one thing you could maybe
• 37:04
  brute force that and like, yeah, sure.
• 37:06
  Maybe you go through millions of tokens and it's not that expensive.
• 37:09
  Agents are iterating, looping until they hit that goal.
• 37:12
  So if I send that same thousand attacks to an agent versus an LLM.
• 37:17
  Depending on how many times it's looping based on the request you sent it.
• 37:21
  That could be.
• 37:21
  And unless, you know, you can of course put a cap on like
• 37:24
  how many iterations it's doing.
• 37:26
  Maybe that's five x the total token usage, maybe a hundred x, Maybe a
• 37:31
  thousand x, instead of a million, now we're talking a trillion tokens and you
• 37:35
  just got a $10,000, bill from one scan.
• 37:38
  that can be catastrophic.
• 37:41
  If you're working on a research budget and you're trying to push these things
• 37:45
  out quickly and your plan is to cut costs, well, you just blew your costs
• 37:48
  up because you hammered this thing with a, a dumb scanner that just brute
• 37:53
  forces through a, a bunch of options instead of reasons about the responses
• 37:59
  coming back and then decides on what next step it should take in this attack
• 38:03
  process to actually identify like what the, the security posture looks like
• 38:07
  for a given, uh, like agentic flow.
• 38:10
  Yeah.
• 38:10
  So you're talking about like evals basically, where you send,
• 38:13
  um, specific payload and expect, you know, specific response.
• 38:17
  Yeah.
• 38:17
  And then you say either it passed or it did not.
• 38:20
  Uh, yep.
• 38:21
  Well, evals versus what, what this would be versus like pen testing in that
• 38:25
  case, you know, digging deeper and going through the entire stack, I suppose.
• 38:30
  Um, yeah.
• 38:32
  Makes sense.
• 38:33
  Um, going back to the SDR again, the SDR use case, um, you cannot secure
• 38:43
  yourself against jailbreaks necessarily.
• 38:45
  Um, what, what, what is there you can do on the backend?
• 38:48
  So, so again, you know, defensive layers, all of that stuff.
• 38:52
  Yes.
• 38:52
  You, you do as much as possible on the front.
• 38:54
  You obviously do offensive testing as much as possible.
• 38:58
  Deploy guardrails were needed, things like that.
• 39:00
  But now you have this, you know, SDR AI agent.
• 39:04
  Um.
• 39:05
  And with SDRs in the past, uh, with the actual human SDRs, we enforce things like
• 39:10
  least privilege and things like that.
• 39:12
  Is that something you should be doing as well for the Ai AI agents in that case?
• 39:17
  Absolutely.
• 39:18
  Yeah, absolutely.
• 39:18
  They should have their own, you should have, like an agent role or
• 39:23
  an SDR agent role should be granular.
• 39:25
  Uh, and as you said, you should be doing, you should be following lease privilege.
• 39:29
  You should be greatly limiting access to any information you're afraid
• 39:33
  might get leaked and you should be monitoring flow of that information.
• 39:37
  So with the human SDR, when they come to the workstation and they
• 39:41
  log in, well they log in, that's it.
• 39:45
  They have to log in, they have to have some kind of credentials, two
• 39:48
  factor authentication, whatever else.
• 39:50
  AI agents cannot really do that kinda stuff.
• 39:52
  And they're running 24 7, if you like.
• 39:54
  And they're running 24 7.
• 39:56
  Exactly.
• 39:56
  So, so do you give them API keys, do you have some kind of
• 40:03
  an JWT token authentication?
• 40:06
  Do you do fresh authentication for every single transaction, or, or do
• 40:10
  you have like persistent credentials?
• 40:11
  How do you manage authentication with these things?
• 40:15
  Yeah, it's a great question.
• 40:16
  'cause maybe you have, you might have a specialized agent, but there
• 40:18
  might be 10 of them and maybe they're doing slightly different tasks.
• 40:23
  Like one of them's focusing on one vertical and the other one's
• 40:26
  focusing on another vertical.
• 40:27
  So maybe they have slightly different access, to information or
• 40:31
  different, like contacts for example.
• 40:34
  in the end, I think in that specific scenario, I don't think it changes the
• 40:38
  risk profile if, if they all shared the same access to the same database, even
• 40:43
  if it's one vertical versus another.
• 40:44
  'cause that's just a, a query, right?
• 40:46
  You're just kind of filtering out.
• 40:47
  Well, maybe, yeah.
• 40:49
  Let's move on from SDR example.
• 40:51
  In that case, let's say you're like a payroll provider company or something
• 40:56
  like that, and you provide agents that assist your customers with different, like
• 41:00
  payroll questions and things like that.
• 41:03
  They have to have access to your information in that case.
• 41:06
  Yeah.
• 41:06
  So you have like 10 different agents.
• 41:08
  they have access to all of this information.
• 41:10
  I mean that it is probably not ideal that they have the same
• 41:13
  access to all the user information.
• 41:15
  Maybe you want to have some kind of context with that.
• 41:18
  They load just your specific information when you log in You need to be way
• 41:21
  more, like if you're talking scaling up something that's accessing data
• 41:25
  across thousands or tens of thousands of users, I mean, that's, that's
• 41:29
  per user session management level of granularity that you need there.
• 41:33
  Like you, you absolutely do not want, uh, like cross session leakage from one chat
• 41:40
  to another chat or one invocation of an agent to another invocation of an agent.
• 41:44
  And that actually, that does get, that can be a bit tricky.
• 41:47
  Uh, especially if you're, you, you need to be managing sessions even at the inference
• 41:52
  level almost, where it's not sharing.
• 41:55
  'cause like there's some model providers, they have internal memory and that memory
• 41:59
  might be shared across multiple sessions.
• 42:02
  So you need to ensure that either memory is turned off or you're resetting
• 42:07
  the state of that memory per session instead of, across multiple users.
• 42:12
  'cause yeah, it's really easy to, to accidentally leak even at the inference
• 42:16
  call or like at the inference level to leak, cross user information and you won't
• 42:23
  know unless you, try like you have to ask.
• 42:26
  There have been accidental cases of that of course, but
• 42:28
  um, yeah, that is a major issue.
• 42:30
  Yeah.
• 42:30
  So how do you monitor this behavior?
• 42:33
  How do you look for the indicators that an agent is doing something
• 42:38
  they're not supposed to be doing?
• 42:40
  How do you test for it, period?
• 42:43
  if I'm making an API call to a model provider, I'm expecting them to
• 42:47
  follow what I'm telling it to do.
• 42:49
  and they have their own monitoring in place.
• 42:51
  So the best you're doing is request based monitoring, like what's leaving.
• 42:55
  And to me that's like deal, that's back in the DLP realm again.
• 42:58
  Right?
• 42:58
  Anything leaving your environment tends to land, data loss prevention
• 43:02
  type products or like akin to data loss prevention, like products.
• 43:06
  Um, but if you're like a payroll provider, in that case, even if
• 43:11
  you're using like something like Anthropic Claude at the front, it's
• 43:15
  all your tool calling at the back end.
• 43:18
  So it's kind of, you know, up to you to provide the sandboxing
• 43:20
  and, uh, monitoring in that case.
• 43:24
  Yeah, DLP 'cause nothing is really leaking in that case, you're just
• 43:28
  displaying stuff in like the browser.
• 43:30
  it's more like, you know, the old school horizontal, vertical
• 43:33
  privilege escalation attacks.
• 43:36
  Yeah.
• 43:37
  it's session abuse.
• 43:38
  session leaks, cross session leaks.
• 43:41
  Yeah.
• 43:41
  It can be crosses leaks as well, but sometimes the same session can access
• 43:45
  stuff it's not supposed to access.
• 43:47
  So Absolutely.
• 43:47
  I think it's, again, comes down to offensive testing.
• 43:49
  In that case, you need to pen test these things.
• 43:52
  I think.
• 43:53
  No, I, I totally agree.
• 43:55
  Um, we don't trust, we verify right here.
• 43:57
  See?
• 44:00
  There you go.
• 44:01
  There you go.
• 44:02
  We do not trust, we verify.
• 44:05
  we talked a lot about different things.
• 44:07
  what advice would you give to somebody that is building an agent at this point?
• 44:11
  what should they watch out for?
• 44:12
  What should they implement?
• 44:14
  What should they test from the beginning, not at the end when it's all built.
• 44:18
  I would start looking at, uh, guardrails.
• 44:21
  I would talk to someone in the AI security realm.
• 44:25
  And if you're plan8ning on building something, just
• 44:27
  start with threat modeling.
• 44:28
  start having a conversation about what are the possible risks for the specific
• 44:33
  thing that you're trying to build?
• 44:35
  And I, I think that's, uh, a great first step.
• 44:38
  Uh, once you're getting closer to releasing something, that's when you need
• 44:43
  to be validating your assumptions that you made at the threat modeling phase.
• 44:47
  You can plan all you want and you can try to implement whatever you want, but
• 44:51
  until you actually confirm whether or not, uh, your assumptions were, uh, proven
• 44:57
  to be true, then you don't really know.
• 45:01
  So that, I think it's that simple.
• 45:04
  Indeed.
• 45:05
  Uh, it's kind of funny, I saw something you like a posting on x
• 45:08
  uh, looking for different roles.
• 45:10
  Uh, some guys building a company basically that will create super
• 45:14
  intelligence, ai, super intelligence, whatever, whatever that means.
• 45:18
  But it's a posting on X.
• 45:20
  Uh, it add like, uh, a list of maybe 20 different roles like
• 45:24
  researchers, coders, whatever else.
• 45:27
  One thing, I did not have a single security person, which I found really
• 45:31
  interesting, but people are just not, yeah, we get a bad, we get a bad rap.
• 45:35
  We slow everybody down.
• 45:36
  At least that's what they, that's what they claim, right?
• 45:39
  Yeah, I guess so.
• 45:40
  But then, you know, when, uh, when you deploy AI agent to manage your,
• 45:44
  supply systems and things like that, and then somebody hacks it.
• 45:47
  No, that's not gonna be good.
• 45:48
  Let's talk about the future.
• 45:50
  obviously these agents will evolve.
• 45:52
  Um, we've seen how things evolved over the last two years.
• 45:54
  I mean, stuff is dynamic changing constantly.
• 45:58
  Um, what, what do you think security landscape is gonna look like
• 46:02
  in the next two to three years?
• 46:05
  It, it's already kind of happening where it looks like, consolidation
• 46:09
  plays are, are beginning where larger mainstream security companies are
• 46:14
  picking up the smaller, expertise.
• 46:17
  I think it's gonna be a consolidation in platform play over the next,
• 46:21
  say like five years, maybe faster.
• 46:23
  Like, AI is moving so fast, there's so much money being
• 46:25
  thrown around, it might accelerate this process of a platform play.
• 46:31
  Yeah, so I think you're gonna start seeing like the all-in-one package
• 46:34
  deals for, um, everything from your AI waft to your AI guardrails, to
• 46:40
  your runtime solution, to your MCP.
• 46:43
  protection Solutions is just gonna be one big pack package, one giant dashboard,
• 46:48
  and, all of your information's in one place, and it might be rolled up with
• 46:52
  some of your other web application protection and runtime solutions.
• 46:56
  CNAPs, actually, CNAPs are probably the best position to do these, uh,
• 47:02
  like discovery of agents that people are building across your organization.
• 47:08
  And then, like agents deployed in your cloud environments and, they're already
• 47:13
  installed in places that make sense to have an MCP proxy like detection, runtime
• 47:20
  like protection, misconfigurations being identified, like if you're leaking
• 47:25
  API keys in your config at runtime for, your agents and things like that.
• 47:31
  Uh, I, they have identity type solutions.
• 47:33
  So I could see the big, the big CNAP and tight players.
• 47:38
  moving this way very quickly if they haven't already.
• 47:40
  I haven't really paid close attention to that market.
• 47:44
  At what point do you think, well, dashboards all of that is all good,
• 47:47
  but the question is, will we need dashboards at all once we get to
• 47:50
  the AI versus AI kind of scenarios?
• 47:53
  How, how soon do you think that is coming?
• 47:55
  that is an interesting topic I have seen a number of startups kick off this, uh,
• 48:00
  it's basically like a soc as a service, an a SOC agent as a service type solution
• 48:06
  where you're completely hands off, maybe you have someone in DevOps that's
• 48:11
  slightly more security adjacent, deploying this thing, and then, um, you're,
• 48:16
  you're switching who's getting alerts.
• 48:18
  You might be able to shrink your security team.
• 48:21
  Or maybe you're just sending alerts directly to developers and they're
• 48:23
  being pre triaged, pre investigated.
• 48:27
  It might even push a PR with like potential fixes.
• 48:30
  For this bug that was found.
• 48:32
  Maybe if the trust level gets there, we might even get to a point with, um,
• 48:37
  continuous delivery like CICD where security ties into CICD to a point where
• 48:44
  not only did it identify and triage, it's actually just, uh, committing a p the
• 48:49
  committing and merging and releasing and doing canary testing and then finalizing,
• 48:55
  like cutting over fully to this new fix for whatever vulnerability that's found.
• 48:59
  The entire process end to end could be completely automated eventually.
• 49:04
  So we had DevOps and we had DevSecOps and now, that's gonna be DecSecAIOps?
• 49:10
  It might just be, uh, yeah, I guess like dev ai ops.
• 49:15
  Dev, AI ops.
• 49:16
  Okay.
• 49:17
  I don't know if that exists, right?
• 49:18
  If that doesn't exist.
• 49:19
  That sounds like a new role that should exist.
• 49:20
  Right?
• 49:22
  Well, like, you know, prompt engineering, security, pro security, prompt engineering
• 49:26
  needs to be there as well, I guess.
• 49:29
  I can see that.
• 49:31
  last question.
• 49:32
  if you could wave a magic wand and solve like one issue in the aagentic
• 49:37
  AI security, what would it be and why?
• 49:40
  It would be, context and memory.
• 49:43
  I believe that's where a lot of the issues of, hallucinations, which all,
• 49:48
  all LLMs hallucinate like hallucinations are kind of a feature in my mind.
• 49:52
  Like that's, that's how it's doing the inference in the
• 49:55
  first place to a certain extent.
• 49:57
  But the biggest problems I run into when I'm using it, like if I'm
• 50:00
  writing code or something, is it's not aware of the full solution.
• 50:05
  Like it can't hold all of that in its context window in one at one time.
• 50:10
  So therefore it's missing something.
• 50:13
  Creating duplicate code, like a ton of duplicate code.
• 50:16
  I think that's the biggest, if I had a magic wand, that would be the problem
• 50:19
  to solve is how memory is managed and how memory is used in inference.
• 50:26
  Yeah, that makes sense.
• 50:27
  Uh, spoke to a potential customer a couple of days ago and, and the entire
• 50:32
  use case was like, uh, our agent flow is used about a million tokens plus.
• 50:37
  So how do you fit that into the context of like security tool as well?
• 50:41
  It's, uh, yeah, that would be a nice problem to solve.
• 50:44
  We'll see if somebody does it in the next year.
• 50:47
  Yeah.
• 50:47
  I just saw Nvidia like three days ago at CES announced.
• 50:51
  They're trying to tackle it at the hardware level.
• 50:54
  Yeah, yeah, yeah.
• 50:56
  We'll see how it goes.
• 50:56
  Well, thank you very much.
• 50:58
  Uh, good conversation.
• 50:59
  Let's do it again sometime.
• 51:00
  Yep, absolutely.