Latest / Agentic Conversations (formally mlops.community) / The Evolution of AI in Cyber Security // Jeff Schwartzentruber // #344

Transcript

The Evolution of AI in Cyber Security // Jeff Schwartzentruber // #344 · Agentic Conversations (formally mlops.community)

• 0:00
  This is what security analysts have always been about or the
• 0:02
  security team is. They know the truth of the
• 0:04
  organization. They know what the organization
• 0:06
  should be doing, what their system should operate like.
• 0:09
  Truth is going to be the like commodity of the future in a lot
• 0:14
  of respects because of how generative AI is attacking and
• 0:17
  eroding that level of truth to your point.
• 0:26
  We can start with give me the rundown on security.
• 0:30
  Bryce Oh, wow, That's like a it's a lot, but it's a it's fun.
• 0:34
  So cybersecurity, I guess we look at it linearly of I like to
• 0:39
  start off, especially from the AI capacity.
• 0:41
  It started from signatures. So a signature is, you know, a
• 0:45
  file hash or a specific runway, A malware will detonate.
• 0:49
  And looking at those patterns and signature for the longest
• 0:53
  time, you know, is static, but it's been the way and really one
• 0:57
  of the most efficient ways of identifying vulnerabilities or
• 1:00
  threats in an on your computer. So you download a piece of
• 1:03
  software, it had some small snippet of malware code in it.
• 1:07
  Now it's doing something nefarious.
• 1:09
  A security researcher would have uncovered that somewhere down
• 1:12
  the line published it to the rest of the security community.
• 1:15
  Different vendors from like antivirus or firewalls would
• 1:18
  have picked that up and said, hey, any time I see this type of
• 1:22
  signature, this type of identifier flag me, I want to
• 1:25
  know because this is this is a representative that this is a
• 1:27
  real threat or this could be a threat associated with this,
• 1:30
  this type of malware, for example.
• 1:33
  And that's really good like it's, it's the main it's one of
• 1:35
  the main sources of how we identify threats and clients
• 1:39
  devices or different areas. But where it starts lagging is
• 1:43
  that, you know, they can change malware, they can do a little
• 1:47
  mutation. So they have this one malware,
• 1:49
  they change a little bit of code in it and now it's a completely
• 1:51
  different hash or completely different signature.
• 1:54
  And so it's very easy with these static approaches, the signature
• 1:59
  based approaches to change how it's happening through like very
• 2:03
  frequently. So these these signatures are
• 2:06
  always changing. That's not really good.
• 2:07
  And so one thing that they did come up with is this concept
• 2:11
  really early on of like thresholding and it's really
• 2:13
  starting to move from the static approach of data to dynamic.
• 2:17
  And it's saying, well, you know, let's just use user logins.
• 2:22
  You're a user at my company and we work 9:00 to 5:00 and you
• 2:25
  always work nine to five. You're in finance.
• 2:27
  But if I see anyone logging in like 3 o'clock, what's strange
• 2:31
  to me like that's, that's just shows a, shows a threshold.
• 2:34
  So you put a threshold on like the times that people work,
• 2:37
  anytime a login happens outside that, you're like, that's
• 2:39
  strange. It's happening at 3:00 in the
• 2:41
  morning. I want to know about it.
• 2:43
  So it's kind of a little bit more dynamic, right?
• 2:44
  It's still static really in the terms of like what we talked
• 2:47
  about AI and anomaly detection, but it's starting to move more
• 2:51
  to that dynamic way of looking at security data and alerting on
• 2:55
  threats that happened in that way.
• 2:57
  These would be like policies. Type thing, exactly, exactly
• 3:00
  these certain policies, but based off thresholds and kind of
• 3:03
  heuristics that the security individuals know about the
• 3:06
  organization. But say you had an say an
• 3:08
  international organization and people are working all the time,
• 3:11
  different logins did a dynamic schedules.
• 3:14
  Well, the next natural evolution of that is to like put
• 3:18
  thresholds in that are dynamic based on the individual.
• 3:21
  And that's kind of where we get into profiling and anomaly
• 3:23
  detection. So in that and for a longest
• 3:26
  time when we talked about AI and security, it was anomaly
• 3:30
  detection using the baseline patterns of like traffic,
• 3:34
  authentications, user access and profiling it on a system or user
• 3:39
  basis. And anytime we deviated greatly
• 3:41
  beyond that, so say at 3:00, this user logged in or tried to
• 3:46
  login with three 300 passwords in less than a minute, you're
• 3:49
  like. Like that?
• 3:51
  Something fishy about that. Yeah, exactly.
• 3:53
  Maybe I should alert on that, right.
• 3:55
  So it has its, it has its like real purpose there because you
• 4:00
  can't really build a signature I guess could build a policy
• 4:02
  saying anything above like into that threshold.
• 4:04
  But this is where anomaly detection has really carved its
• 4:09
  strong suit in cybersecurity. And that's what we had for a
• 4:12
  longest time until kind of Jenny, I came in.
• 4:16
  Why did Jenny I destroy that pattern?
• 4:19
  So like when we're looking at the the previous data models
• 4:23
  coming out of that anomaly detection, just like looking at
• 4:25
  counts. So I talked about authentication
• 4:27
  counts. It's a very good data primitive
• 4:31
  to baseline. But a lot of the data we deal
• 4:33
  with is like log data or these unstructured data points that
• 4:38
  were always like really hard to parse or grab additional context
• 4:42
  out if and there and there's no normalization scheme I'd like.
• 4:46
  That's another thing that differs that makes it very
• 4:49
  difficult is that you have one vendor that uses this type of
• 4:53
  like common log format or this other type that uses Elastic
• 4:56
  common schema or CIM from Splunk.
• 4:58
  They're all using these different normalization schemes
• 5:01
  or log formats. And so if you're not having this
• 5:04
  like strong ETL pipeline to put them into a normalized thing, it
• 5:11
  becomes very hard to correlate amongst them.
• 5:14
  But and then to go around that, we're like, OK, well, we just
• 5:16
  want to look at IP addresses. So we'll regex everything out as
• 5:20
  IP addresses and bring in. But you know, it was, it was
• 5:24
  inconsistent and it was hard to work with.
• 5:27
  And we're really dealing with an NLP problem at some points as we
• 5:30
  got down the line like we do the signature base, which was really
• 5:32
  well known and robust and it still works really well.
• 5:35
  We had a nominate detection, but it was only really capturing
• 5:38
  patterns. And you know, if we if we talk
• 5:40
  about a large organization, if you wanted to profile everyone,
• 5:43
  you're trying to profile like thousands of users, hundreds of
• 5:49
  thousands of systems. And the cardinality at that just
• 5:52
  kind of explodes and makes it hard to maintain.
• 5:55
  Plus those systems create a lot of false positives.
• 5:58
  So, so I got to circle back a little bit here because you
• 6:02
  asked me about security. It's like such a a broad
• 6:04
  question and there's different maturity levels at which you get
• 6:09
  organizations. So if you're like a new
• 6:11
  organization and you're a start up, like you're really concerned
• 6:14
  about putting your product out there and you're going to take
• 6:17
  some risks because you can't afford to have that enterprise
• 6:20
  level security. But once you get to a certain
• 6:23
  level in security, you want to have the best degree.
• 6:26
  You want to have a lot of people, you want to have a
• 6:29
  dedicated security team. And at like the highest maturity
• 6:32
  when we go into any enterprise, they'll have something called
• 6:33
  the Security Operation Center and they'll have a team sitting
• 6:37
  behind computers ingesting all that security telemetry from
• 6:41
  firewalls and points. And they're taking a look at all
• 6:44
  those alerts. And anytime an alert comes in,
• 6:46
  they're like, well, I got to investigate this.
• 6:47
  This could be a real breach. And they risk associated with
• 6:49
  that, right? So once that's coming in and
• 6:53
  they're seeing that and if they get a lot of false positives,
• 6:55
  they're just drained, they're sucked down.
• 6:57
  They're doing a lot of actual work with no real value.
• 7:00
  It cost the organization and it's not really doing the work.
• 7:03
  So a lot of time if these anomaly detections, it was false
• 7:06
  positives. It was things that were maybe
• 7:08
  being too big and not tuned in. And so you had to spend a lot of
• 7:11
  time engineering those thresholds and those models to
• 7:14
  make sure they were really tuned in.
• 7:16
  So you're not fatiguing getting something called alert fatigue
• 7:19
  under the sock Alert fatigue. Is real man even in in any
• 7:23
  system right? But I can imagine how in this
• 7:25
  one, each whatever big, big problem that comes through, you
• 7:33
  think that this could be the downfall of our system you.
• 7:38
  You're, you're, you're exactly right.
• 7:39
  Like that's the, the attackers only have to be right once to
• 7:44
  get in. The defenders have to be right
• 7:46
  every single time. So we take with it, it's like
• 7:49
  that classic precision recall F1 score with cancer.
• 7:54
  It's like we cannot, we have that we have such high
• 7:56
  tolerances on what these models have to do and like how much can
• 7:59
  be the thresholds there that because if we let in one time
• 8:03
  and that causes a chain reaction, they've got
• 8:05
  persistence, they're able to extract data, they're able to
• 8:09
  enumerate and lateral movement inside the network.
• 8:11
  It it's, it's catastrophic in a lot of capacities, especially
• 8:14
  with the data. So we always got to be looking
• 8:17
  and investigating, but at a certain point like this is where
• 8:20
  risk comes in. An organization only has so much
• 8:22
  money to dedicate to security or to do these things.
• 8:26
  So you really have to optimize from an engineering perspective
• 8:29
  and an operational perspective these these systems to make sure
• 8:33
  you're not drowning in security. That's.
• 8:36
  A funny thing about this is that my first tech job ever was kind
• 8:41
  of in the security space, and it was a password management tool
• 8:44
  that we sold to manage service providers.
• 8:47
  And I remember I was on the sales side of the house, and it
• 8:50
  was so hard to convince someone who really didn't care about
• 8:56
  security why they should. Because it's just like, yeah,
• 9:00
  we're kind of flying by the seat of our pants.
• 9:02
  And we know that we should be doing things better, but we're
• 9:05
  not. And until we get hacked, we
• 9:07
  don't care. Right, right.
• 9:10
  And so that. That's where like risk really,
• 9:13
  it's unfortunate, but that's and I kind of want to talk about
• 9:15
  this later when we get into like a gentic of like how I think
• 9:18
  it's going to transform. But you're you're right.
• 9:21
  Like the the risk. Do you have the crown jewels are
• 9:25
  do you are you exposed significantly and do you carry
• 9:27
  PII? These organizations have to
• 9:29
  evaluate what their risk level is and you know, maybe they're
• 9:33
  like, we're willing to risk the entire corporation that we're
• 9:36
  not going to get hacked, but if we get hacked and we get sued,
• 9:39
  we're probably pretty much done. But like that's the risk you
• 9:42
  took going into that into that, right?
• 9:43
  So that's like another interesting point on, on
• 9:48
  security that also makes security AI and ML very
• 9:53
  difficult is that there is a bias, but it's like a personal
• 9:58
  bias from the organization and their risk.
• 10:00
  Like you, you can develop a model that works for one
• 10:03
  organization and, and their leader is like, yeah, that that
• 10:06
  works perfect for me, that meets my use cases.
• 10:08
  But you can go to another leader and they're like, I don't really
• 10:10
  care about that. That's not in my business line.
• 10:12
  That doesn't matter. So I want to, I want you to tune
• 10:15
  you this model to be over in this direction.
• 10:18
  Now I want you to it. So it it can vary.
• 10:20
  Like there's obviously some core commonalities.
• 10:22
  They're like, I don't want malware coming into my systems.
• 10:25
  I don't want, you know, AD dos attempt to happen.
• 10:28
  So how do I put these controls up?
• 10:30
  But it is that there is a personal, I don't want to say
• 10:34
  organizational personal kind of perspective that biases how we
• 10:38
  deliver security on a especially from E centre because we're
• 10:41
  delivering security operations for over 2000 customers across
• 10:46
  the world, right. And they all have different
• 10:48
  requirements, different businesses, different
• 10:51
  perspectives on securities, their security leaders
• 10:53
  respecting. So it when we talk about doing
• 10:56
  ML, it's like, yeah, we're covering these bases.
• 10:58
  But then we do see these edge cases come in that we got to
• 11:01
  like start tuning and be like, yeah, that makes sense.
• 11:02
  We want to incorporate that. Your concerns are legitimate.
• 11:05
  We can we can accommodate. It's it ends up becoming, to
• 11:08
  your point, how they feel about security in general.
• 11:12
  Yeah. Do you find it's a logarithmic
• 11:15
  curve type of thing where those last 10 percent, 5% of this risk
• 11:24
  that you want to take on, you need to throw more and more
• 11:27
  money at it continuously if you want to cover that last little
• 11:30
  part. So that's the trade off that
• 11:33
  folks have to make. It's not just like, OK, let's
• 11:35
  continue scaling our security and continue getting these
• 11:39
  returns on it. It's more like in order to cover
• 11:44
  this, we're going to have to really make a commitment and
• 11:47
  whether that's a commitment and resources or it's money that
• 11:51
  they have to weigh that so. Soprano's a really big deal,
• 11:54
  like a big thing that we kind of scope against and insecurity
• 11:58
  there. Is this for log analysis or
• 12:00
  something called long tail analysis, where it's one
• 12:02
  methodology where, OK, when I look at all my security logs, I
• 12:05
  want to only look at those events that are, like you said,
• 12:08
  very slim at the end because that's typically where an
• 12:10
  attacker will hide, right? He's like one off anomalous
• 12:13
  events, like it's called long tail analysis.
• 12:15
  And you just, you're like, I want to go over here and look at
• 12:17
  these ones because those are important ones, but you can,
• 12:20
  it's very much a Pareto type of thing.
• 12:22
  It's like, what do you do to improve security?
• 12:24
  Well, you know, one big thing is put multi factor on everything.
• 12:27
  Let's stop like you don't have have it tied to devices and
• 12:30
  start doing like there's some very common base like solid
• 12:32
  controls that we know about. So if we go into an organization
• 12:35
  like it gives a multi factor like, OK, we kind of know where
• 12:38
  your maturity value, where like your maturity level is in
• 12:40
  security. And we're like, you're going to
• 12:43
  gain a lot more from doing this. But if you have very sensitive
• 12:47
  data and like a lot of our customers do and and need these
• 12:50
  high levels of assurance and security, you got to start
• 12:54
  investing more because you can't lose once.
• 12:57
  Basically all the stuff you have to do when you get sock 2.
• 13:00
  Yes well, even beyond that though too, like sock 2 is just
• 13:04
  like, yeah, you want to get to sock 2 and that's like the right
• 13:06
  now everyone's trying to go for sock 2, but you know, go for
• 13:09
  those other things, ISO, PCI, there's so many more compliances
• 13:13
  and regulations and especially in this new market that we're
• 13:17
  going with with like agents taking a lot more network
• 13:19
  traffic up, agents taking a lot more compute up, being a bigger
• 13:24
  entity with inside a lot of people's networks going forward
• 13:27
  as well. But there isn't really too much
• 13:30
  regulation coming back from that at the time at the right now,
• 13:33
  like, you know, there's the EUAI act, which is like the best,
• 13:37
  like the the the crown jewel of this regulations, but states of
• 13:40
  kind of kibosh that California kibosh differ a bit.
• 13:44
  We don't know where this is going to be from an AI
• 13:46
  perspective, especially an agentic for security.
• 13:48
  And when those regulations come in to your point, they'll have
• 13:52
  to put more controls and start going down that to to fortify
• 13:55
  themselves on that. Yeah, like maybe let's talk a
• 13:58
  bit about that with the evolution of agents and how that
• 14:04
  has changed what security looks like.
• 14:08
  Because I know there's both sides of it.
• 14:10
  When I think about the gented use cases for security, I think
• 14:13
  about the agents and Gen. AI just being able to get you
• 14:22
  involved. Or when I think about agents and
• 14:26
  Gen. AI, I think about how there's
• 14:29
  now many more threat vectors if we're using agents.
• 14:34
  And my favorite one of these is I read a paper talking about the
• 14:39
  indirect prompt injection when the agent goes and it scrapes a
• 14:43
  website and then it gets prompt injected.
• 14:45
  So it can be someone is trying to use an agent normally like a
• 14:50
  good upholded citizen, and they still get screwed because the
• 14:58
  agent goes out and it scrapes a toxic website and gets prompt
• 15:01
  injection. So we're opening up these new
• 15:05
  vectors that we can be attacked on, but then we can also use
• 15:09
  agents to help us on the different things like you were
• 15:14
  mentioning earlier, like inspect logs or go and take action from
• 15:19
  different logs. If we see some specific or
• 15:21
  suspicious, we can now use an agent to hopefully try and patch
• 15:26
  things up quickly. Can you just talk a bit about
• 15:30
  your scope and what you've been seeing with agents out there in
• 15:33
  the wild in the security field? Absolutely.
• 15:36
  So when this first came out, we kind of understood how much game
• 15:40
  changer change this was going to happen and be used and also be
• 15:44
  used poorly with inside the enterprise.
• 15:48
  Like it came out so fast and everyone was adopting it very
• 15:51
  quickly and there wasn't much security control around and not
• 15:54
  much understanding. And we've been seeing the
• 15:56
  evolution of this field go so fast.
• 15:59
  And even from the security and academics, I were like this, we
• 16:02
  just can't keep up. There's not enough time to do
• 16:05
  solid traditional foundational security research out of the
• 16:09
  academia labs. I'm talking about more not so
• 16:11
  much of the, the corporate labs, but to, to keep up with the rate
• 16:15
  of which this is progressing. So we saw this effect coming in
• 16:19
  and to, to answer your question, want to talk about 3 essentially
• 16:22
  different buckets here. So we have the Jenny I for the
• 16:26
  Defenders. We have the Jenny I of how it's
• 16:28
  being used by the red team. And then we have the
• 16:29
  vulnerabilities that Jenny I is introducing to kind of like we
• 16:33
  point this indirect prompt injection attack.
• 16:35
  So when we, we knew this was happening early on, we spun out
• 16:39
  a lab called the E Center Labs that is focused on something
• 16:42
  called MDR for Jenny I. So manage detection response,
• 16:46
  but focusing on these new threat vectors that are coming out,
• 16:49
  like you said, indirect prompt injection attack.
• 16:51
  And when we're doing a research, the reason that's happening is
• 16:54
  because these agents for a lot of time are not being exposed
• 16:59
  their logs and their thinking patterns that have chains to
• 17:02
  downstream service customers. So they kind of work, you know,
• 17:06
  inside that black box, but even only your, you can, you can get
• 17:09
  traces if there if, if these service providers that are
• 17:13
  leveraging these services that are prone to prompt an injection
• 17:16
  attacks are not actually keeping traces of what those chains of
• 17:21
  thoughts are or what those law that service is doing, Then you
• 17:24
  have no visibility. You can't actually see where
• 17:27
  this indirect vulnerability is coming from.
• 17:30
  And in addition, these agents are kind of gaining more and
• 17:33
  more authority all the time and autonomy.
• 17:37
  So we're like, ah, they, they did this thing and we all, we
• 17:39
  get this is doing better than this person did previously or
• 17:42
  this, this system recruits, let's use it.
• 17:44
  And we're giving it the authority to go off and be more,
• 17:47
  but without the oversight. And it's a dangerous situational
• 17:51
  kind of criticality. So why what we, we saw, I was
• 17:55
  like, well, if you're going to implement any agentic or Gen.
• 17:59
  EI systems like you need a first have something called an LM
• 18:02
  gateway and we released one early on, which is essentially A
• 18:05
  proxy that exists between your organization or device and these
• 18:11
  downstream service providers. Like it could be a model or it
• 18:14
  could be a, a service provider that leverages an open source or
• 18:17
  commercial model that does some type of agentic or, or
• 18:22
  manipulation on top of those prompts.
• 18:24
  But we did a gateway so that we say every prompt that goes
• 18:27
  through, we want to record it and we want to record the
• 18:29
  responses. And there's a lot of power in
• 18:32
  just seeing that visibility. But what you don't get is what
• 18:36
  the agent and that agentic system is actually doing,
• 18:38
  because that's still stuck on the service providers side if
• 18:42
  you're not getting those logs as well brought back.
• 18:44
  So to that point, that autonomy and that lack of visibility is
• 18:50
  this most significant threat vector right now in my opinion,
• 18:53
  because how can you protect against something you can't see?
• 18:58
  And what's even kind of worse is that a lot of IP is tied up in
• 19:03
  these agentic systems of how they think.
• 19:05
  So if you have a Lang chain or something like that, you're
• 19:07
  like, this is the core IP of my organization, we're doing
• 19:09
  something unique versus everyone else.
• 19:12
  But if it's able to be prone to prompt interjection attack, you
• 19:15
  don't have strong guardrails, You haven't done the significant
• 19:18
  testing before being released because you've been more focused
• 19:22
  on production and not so focused on, you know, security
• 19:26
  requirements. Again, going back to that risk
• 19:28
  and maturity conversation, we have it, it, it, it's prone to
• 19:33
  happen a lot more and we're seeing a lot of that happen come
• 19:36
  in. So we're really all the
• 19:39
  development we've done it. We've we've been lucky to be at
• 19:41
  the forefront. I really am fortunate to to work
• 19:44
  with a lot of smart people, a lot of people that take security
• 19:48
  very strictly. And, and we've always been kind
• 19:51
  of been like, yeah, we need visibility, we need systems, we
• 19:53
  need rigor, we need to see what's going on.
• 19:55
  And luckily we've been staying ahead of of these threats.
• 19:59
  So that's like the one bucket there of.
• 20:02
  Of threats and more tax service that these agenda systems are
• 20:06
  getting. And I think it's only going to
• 20:07
  increase because we're constantly giving them more
• 20:10
  authority and we're using them more and more.
• 20:12
  It's going to become commonplace.
• 20:14
  So it's just makes sense that this will proliferate into a
• 20:17
  more larger and larger problem, which isn't, isn't, is, is
• 20:23
  detrimental to those analysts that have to look at the at
• 20:26
  those security events. So that's why we need that Jenny
• 20:29
  I that's why we've also been working on the Jenny I for the
• 20:31
  blue team be like, they have all this arc, they have this new
• 20:34
  attack ServiceNow. They're using Jenny I just like
• 20:37
  we're using it. The red team is and coming up
• 20:40
  and using it all forms of the kill chain.
• 20:42
  They're developing new malwares with it.
• 20:44
  They're using it for fraud a lot more.
• 20:48
  How? Do you keep up with this
• 20:50
  increasing tsunami of Gen. AI attacks and vulnerabilities?
• 20:55
  Well, we have to start developing systems on the blue
• 20:57
  side to combat that as well. Empower analysts to to respond
• 21:02
  faster and at scale. And then there's the third
• 21:06
  bucket, which is. Oh, and so I thought that, so we
• 21:09
  talked about the first buckle was the vulnerabilities that
• 21:11
  the, the LM systems and the Gen. AI systems give by themselves.
• 21:14
  There's that bucket. But I was, I, I briefly talked
• 21:16
  about the the third bucket being the threats, like how the
• 21:21
  attackers are using generative AI to inform their attacks.
• 21:25
  I'm not sure if you saw this paper that came out from
• 21:28
  Anthropic in August, but their threat intelligence research
• 21:31
  team put in an awesome paper because they're, they captured
• 21:35
  real case studies of different attackers leveraging clawed code
• 21:41
  to, you know, they saw that cloud code was being used in
• 21:43
  every form of the kill chain, which is amazing because it's
• 21:46
  able to assist the attackers. You know, previously where they
• 21:49
  didn't have skills or they need, they got stuck.
• 21:51
  They could like unblock themselves.
• 21:53
  Now they found in it, they found a, a person that was using cloud
• 21:57
  code to develop a ransomware as a service that was like
• 22:01
  completely at like beyond their skill level, just looking at the
• 22:04
  prompts, because they're following this prompt chain
• 22:06
  beyond their skill level, looking at their MD file and
• 22:08
  they put out their cloud MD files and show this is kind of
• 22:11
  like a boilerplate of what they're doing.
• 22:13
  So keep an eye out for it. And then they also saw which I
• 22:15
  think is the funniest, the North Korean employee hire fraud
• 22:20
  scheme like to get to this point.
• 22:24
  I talked about this at a high level.
• 22:25
  I'm glad you say this because I, I, this is kind of been in my
• 22:29
  opinion too, is that a generative AI?
• 22:32
  Its entire objective function is to be indistinguishable from
• 22:36
  human generated content. So by by it's like own latent,
• 22:41
  it's it's meant to be deceptive and on for and because it's
• 22:46
  deceptive, it lends itself to deceptive practices better.
• 22:50
  And so when we talk about that and do your point, what your
• 22:54
  Trump, what we really need to go for in the future, and this is
• 22:57
  what security analysts have always been about or the
• 22:59
  security team is they know the truth of the organization.
• 23:02
  They know what the organization should be doing, what their
• 23:05
  system should operate like. Truth is going to be the like
• 23:08
  commodity of the future in a lot of respects because of how
• 23:13
  generative AI is attacking and eroding that level of truth to
• 23:17
  your point. That is so wild, man.
• 23:20
  So yeah, there's there's very new attack vectors that are
• 23:24
  coming out, whether it is from the agents or it is the social
• 23:27
  engineering. I know that back when I was at
• 23:30
  that security company, the biggest threat to security was
• 23:37
  always humans, and it was human mistakes.
• 23:40
  They're making this, they're making like, I couldn't believe
• 23:43
  when I read this because this paper from a security because
• 23:47
  again, a lot of the time I spend I, I kind of have a 2 foot roll.
• 23:51
  I spent time in academia. I'm, I'm supervising graduates
• 23:54
  in the security and AI space. And, and we're academia
• 23:57
  typically comes out with a lot of it like novel vectors, like
• 23:59
  we knew about indirect prompt rejection.
• 24:01
  We knew about these security vulnerabilities and LMS. We, we
• 24:04
  know about a lot more coming in, but what's really hard is to
• 24:07
  catch these in the wild. When we look at some traditional
• 24:12
  security research, it's, you know, we found this, we found
• 24:17
  this in the wild, we found this, this signature in the wild.
• 24:21
  We build a report on it, we publish it and it goes out.
• 24:24
  But then for this ML stuff, it's not like it appears itself in
• 24:27
  the wild because the visibility is not there.
• 24:29
  It's not really well known. So we come up with these
• 24:32
  vulnerabilities, we come with these attack vectors, we publish
• 24:35
  them, you know, and we say that under this threat model, this is
• 24:38
  what could happen. But they kind of died.
• 24:41
  They typically died off for the most part because you can't find
• 24:44
  them. But after seeing, you know, a
• 24:47
  bunch of new papers and reports of indirect prompt injection
• 24:50
  attacks happening, seeing hugging face models poisoned by
• 24:55
  attackers. So data scientists are now being
• 24:57
  attacked more. And then now with Claude finally
• 25:00
  in these in these large LLM community sort of model
• 25:04
  providers saying this is the attackers and the nefarious ways
• 25:08
  we've been seeing our model deployed, you're like, OK, yeah.
• 25:10
  So finally everything we've been saying it is being used as being
• 25:14
  here. Here's some hard proof.
• 25:15
  And I love that paper so much because it's like bringing to
• 25:18
  the forefront these really unique attackers that we already
• 25:22
  knew. But you don't get this.
• 25:23
  You don't get a sense of how how they're being used into our
• 25:26
  North Korean one. They're they're some of the
• 25:30
  users couldn't even create a Git repo.
• 25:31
  They had to ask how to create a Git repo, and it's.
• 25:36
  It's amazing that that persists in my opinion.
• 25:39
  And we have yeah, 100. Percent, it's like there's
• 25:40
  there's a time delay between we see when we learn about
• 25:45
  something or when we hypothetically think something
• 25:47
  is possible. And then because we're we have a
• 25:50
  little bit of that shadow game going on and we don't get the
• 25:53
  data that Anthropic or these big labs have, then we can't see
• 25:58
  until they publish something that says, hey, yes, people are
• 26:00
  doing this and this is how they're doing it.
• 26:04
  That's right. That's right.
• 26:06
  What do you think the future looks like here for where we're
• 26:11
  going, how we're going? I got to talk about the Blue
• 26:15
  team a little bit here and and give some give some context on
• 26:18
  this because there is a new push right now.
• 26:23
  Agentic has been doing really interesting stuff.
• 26:26
  We've been seeing all these use cases.
• 26:27
  They're like, OK, LMS were you know, monoliths working by
• 26:30
  themselves able to do things, but you get these specific
• 26:33
  agents that are working in concert together and they're
• 26:36
  able to do a lot more power, right.
• 26:37
  And we're seeing that also on the investigation side.
• 26:40
  So going back to the SoC, because that's where you have
• 26:44
  these really skilled, highly qualified personnel sitting
• 26:48
  behind computers investigating this.
• 26:49
  So, you know, this malware detonated or this, this
• 26:52
  PowerShell command went off. What does that PowerShell
• 26:55
  command mean? What did it do?
• 26:57
  Did it affect any other computers?
• 26:58
  They're looking around the network to, to discern this.
• 27:00
  And it takes a, you know, a pretty skilled person to do
• 27:04
  that. There's obviously Tier 1, Tier 2
• 27:06
  and Tier 3 SoC analyst, but one thing that they're doing a lot
• 27:10
  of time, just like with any investigation, is collecting
• 27:13
  evidence, building that profile. And then at the end they compile
• 27:16
  a report and send it out to the customer being like, hey, this
• 27:19
  happened, but it wasn't a real, it wasn't it was a false
• 27:22
  positive. And This is why I give you the
• 27:23
  report or this happened, it was a true positive.
• 27:26
  This is what we did to kill the connection or quarantine the
• 27:30
  host. And this is and this is the
• 27:32
  report and this which need to do next.
• 27:34
  All that takes considerable amount of time and that's called
• 27:37
  meantime to detect and meantime to respond.
• 27:40
  And when you're seeing a flood of more signals come in larger
• 27:45
  attack service, we're seeing the economies like not scale like
• 27:49
  cybersecurity, just like most wars are a sign of attrition and
• 27:54
  economy where it's like we make it too hard for an attacker to
• 27:58
  attack a system or we don't have valuable data in the first
• 28:01
  place. We're going to have a less risk
• 28:03
  of like being attacked. They either be like, oh, that's
• 28:05
  too much work. I'm going to try to differ
• 28:06
  attack vector or they don't have anything.
• 28:08
  So why am I going to attack them?
• 28:09
  It's got no value this, but the economy of what this is going is
• 28:14
  our analysts are drowning all the time.
• 28:17
  They're getting that alert fatigue, they're getting
• 28:19
  bombarded. We need to scale up and it
• 28:21
  becomes this human generated problem.
• 28:23
  So what we want to do and what we are doing is leveraging these
• 28:27
  AI systems to front load some of that work that evidence
• 28:31
  collection. We can go out and do these
• 28:33
  automate, look up this IP it came from the signal came from
• 28:36
  sorry, this threat came from this IP.
• 28:38
  Go look up the reputation. Has anyone else seen this?
• 28:41
  Has this ever been seen any other where in the network?
• 28:44
  So it's able to front load a lot of this so that when it comes to
• 28:47
  the analyst, we're now giving them a report in their hand
• 28:50
  being like this is the work we did up to this point.
• 28:53
  The the agent did up at this point, use your expertise,
• 28:57
  review it, make sure it's good and then help.
• 29:00
  And then if we're if it's missing anything, they can go
• 29:02
  off and look at any other, you know, dots they need to connect
• 29:06
  to draw a conclusion. But we're greatly trying to
• 29:09
  reduce that mean time to detect and mean time to respond to
• 29:13
  account for this massive influx we're seeing from Gen.
• 29:16
  AI and the acceleration it's doing from the red from the
• 29:20
  attackers. It's fascinating to me how much
• 29:24
  of A parallel there is between this and what a friend of mine,
• 29:28
  Willem at Cleric is doing for root cause analysis.
• 29:32
  When your systems go out and you all are speaking basically the
• 29:37
  same language, just for two separate use cases, you're on
• 29:40
  the security side saying, hey, let's go and figure out what is
• 29:44
  happening and do as much of that rote work as possible because
• 29:47
  you don't want to have to switch between systems.
• 29:50
  You don't want to have to try and look up a did this IP.
• 29:53
  What's going on here? Is it nefarious?
• 29:54
  Is it not? You would love to have that type
• 29:57
  of information given to you where something or someone else
• 30:01
  has done it. And then you get to scan it and
• 30:04
  say, oh, here's what looks fishy or oh, no, this seems all right.
• 30:08
  And in that same vein, in the dev OPS sphere, Williams trying
• 30:13
  to do that and he's trying to say, hey, let's, if we see that
• 30:17
  there is some kind of outage, let's go and scan the Datadog
• 30:21
  logs and let's scan it. Let's look at Slack for any kind
• 30:24
  of messages that the Sr. ES have been pinging back and
• 30:26
  forth. And maybe there's a new policy
• 30:29
  or there's a new something or new roll out that happened that
• 30:32
  could be causing this issue. And these are like both ways
• 30:37
  that agents are helping, that kind of menial work that just
• 30:42
  you have to do but nobody enjoys doing.
• 30:45
  Yes, exactly. And to it to his use case and
• 30:48
  and similar, it's about reducing that time.
• 30:50
  So root cause analysis to reduce the time that outage is actually
• 30:54
  happening because it has a like direct business value effect,
• 30:58
  right. So vulnerability comes in,
• 31:00
  sorry, threat comes in, let's investigate and figure out if
• 31:03
  this thing is actually going to bring down the system.
• 31:06
  This is actually a legit threat and discern on it.
• 31:08
  So there I I see the parallels you're saying and it is exactly
• 31:11
  right. It's about using the the agentic
• 31:14
  systems to bring together correlate this data, help draw
• 31:18
  some maybe simple conclusions. So to the bet, to his point too,
• 31:22
  he's, are they comfortable letting the agentic system or a
• 31:27
  Genii system make a decision on a remediation action?
• 31:31
  Like that's a question. I, I, I'd be interested to hear
• 31:33
  from him because from a security standpoint, it's not there yet.
• 31:36
  There is a big push, especially coming off of RSA this year
• 31:41
  where it was agentic this and agentic that.
• 31:43
  And there's new startups coming out that are just being about
• 31:48
  pure agentic socks, like humanless socks.
• 31:51
  And that doesn't sit well. Like you can't put that
• 31:55
  liability and especially with these complex IT systems that
• 32:01
  require such deep level expertise to be able to bring
• 32:05
  this and give that power to an agent.
• 32:06
  Like again, giving that authority.
• 32:08
  Are you ready to give that authority to an agent system?
• 32:11
  It's just not there. You you need human in the loop,
• 32:14
  especially for these high criticality systems and.
• 32:17
  It feels like the agents are already so vulnerable.
• 32:20
  You're just adding another vulnerability if you're letting
• 32:22
  the agent make the decisions on that.
• 32:24
  I think for Williams, he is, but it's more on on the event of,
• 32:33
  hey, here's what I found. Do you want me to go and fix it?
• 32:37
  And that's when they'll go out. And so you get that human
• 32:42
  acknowledgement of here's what I think it is.
• 32:46
  I think I can fix it by doing these things and then boom, go
• 32:50
  for it. If I say, yeah, that that seems
• 32:53
  looks good to me, even though I do notice.
• 32:55
  And this is a fascinating, like human field of study that with
• 33:00
  my own use, whenever I get a suggestion, I just kind of
• 33:05
  blindly accept it and say, well, let's try.
• 33:09
  So that's not good if you're dealing with these very high
• 33:12
  risk use cases and it's in the security field or it's in like
• 33:15
  we're talking about an outage. Because it's just so cheap for
• 33:19
  me to say, yeah, like click a button and say, yeah, go do it.
• 33:23
  And cheap as in I don't have to use any mental power.
• 33:27
  Cheap is the perfect word. And I, and this is the so to the
• 33:31
  point of like, we don't want to give it authority there.
• 33:34
  We don't want to get at that point.
• 33:36
  This is going to demark the security or in this point, to
• 33:40
  your point, this, this DevOps approach where say you have a
• 33:44
  team of one person that has to look after a budding
• 33:47
  organization that is like a startup that's trying to do
• 33:50
  something. I can't afford a massive
• 33:52
  security sock, but this thing is cheap.
• 33:55
  It can give me something it can do.
• 33:57
  It's like another layer of defense.
• 33:59
  It might not be right all the time, but it's, it's right 80%
• 34:03
  of the time it's able to do something.
• 34:05
  Is it an acceptable level of risk for the cost that you're
• 34:08
  talking about? It's it is 2 sided.
• 34:10
  So I see you know, I think for the the security domain and if
• 34:16
  you talk to any security professional that's very
• 34:18
  diligent and serious about security, they're like no humans
• 34:22
  need to remain we own expertise. We've seen the agents failed too
• 34:25
  many times. But for that person that says I
• 34:28
  only have this small security budget, I have nothing to work
• 34:32
  with. I need to bring data.
• 34:33
  I need more security. I'm underwater, I'm drowning
• 34:36
  here. That's it, that's it.
• 34:38
  That's a viable alternative to de risk.
• 34:41
  The massive risk you have of having nothing kind of thing
• 34:44
  right. So to your point, cheap was the
• 34:46
  perfect a point because it's descaled this, but you know if
• 34:50
  can you afford and that kind of goes back to the risk part.
• 34:55
  Risk really pays. It is, it is.
• 34:57
  Is the risk worth the reward in this capacity of cost savings?