Latest / CXOTalk / SANS Institute: AI Agents Are an Attack Surface. Does your CISO know? | CXOTalk #910
Transcript
- 0:00The death by thousand cuts argument is actually true.
- 0:02You don't need to steal from ransomware attacks and millions
- 0:05of dollars. All you need to do is hit enough
- 0:07people with 300 a pop and do that at scale with SIM farms
- 0:11that are being found all across the US and all of a sudden the
- 0:13$300 per person times 10,000 equals the same as a single
- 0:17ransomware payout before. This kind of scalability has not
- 0:20been able to be achieved before AI.
- 0:22AI and autonomous agents are wreaking havoc with
- 0:26cybersecurity. Rob T Lee is chief AI officer of
- 0:31the Sands Institute and our Co host today is David Bray, chair
- 0:36of the Accelerator at the Stimson Center.
- 0:39Rob, why should we care about the AI life cycle and security?
- 0:43Like why does this even matter? You're on tapping a massive
- 0:47problem when we are end up starting to talk about the speed
- 0:50and the velocity of the tape would be achieved with
- 0:52autonomous agents. You look back just four or five
- 0:56months ago and you know clawed enough and clawed code being
- 0:59utilized and detected by Anthropic basically automating
- 1:03offensive capabilities in the attack inside organizations with
- 1:08minimal human in the loop to be able to make sure that
- 1:11hallucinations are minimized. Things that used to take months,
- 1:15weeks are now being accomplished in minutes and seconds.
- 1:19And it's and it's hard to wrap your head around about what that
- 1:22is specifically means for potentially detection.
- 1:24Being able to stop it and be able to see what's going to
- 1:28happen even six months after the fact.
- 1:31What the adversaries and capabilities of even an
- 1:33individual possibility of accomplishing inside a network
- 1:38much less attacking individual for scams.
- 1:41David sounds pretty scary. It does.
- 1:43At the same time, this is, I think every decade something new
- 1:46shows up on the scene. And this is what, you know, the
- 1:48challenge of our decade is exactly as Rob said, is the
- 1:51speed and scale of which generative AI is able to allow
- 1:56an attacker that might not be that sophisticated or possibly
- 2:00what would take a long lead time.
- 2:02They can piece together an attack and do the reconnaissance
- 2:04and actually execute. The attack is unprecedented, but
- 2:08it means we need to adapt and it means we need to, in some
- 2:11respects, find strategies that allow us to identify what we
- 2:15would call abnormal patterns of life happening in enterprises
- 2:18and adapting and responding faster.
- 2:21So I look at it as a sort of response of the attacker has
- 2:24evolved their strategies. It's time for the defenders to
- 2:26do the same. Rob, how have AI and autonomous
- 2:31agents change the nature of threats, change the nature of
- 2:35attacks? And as David was talking, I saw
- 2:38you kind of nodding your head in agreement.
- 2:41I'm a big history buff. Every single time there's a
- 2:43massive technological change, there is a massive change in
- 2:47tactics that need to follow, follow it and much less the
- 2:50strategy. You could go back to the
- 2:54introduction of the airplane, World War One, the tank,
- 2:57especially back then, you know, going back to the Civil War, you
- 3:00know, the the line defence with Napoleonic and what did we end
- 3:04up seeing then all the way to the aircraft carrier.
- 3:06Basically move this mindset into warfare, into what's going on
- 3:11cyber defence, cyber offense as a whole as a result of the
- 3:15speed, velocity, you end up taking a look at, you know, the
- 3:18essentially the blitzkrieg lap in World War 2.
- 3:20The same kind of mentality has occurred.
- 3:22Things that used to be planned out that would take weeks,
- 3:25months for an offensive campaign.
- 3:26We're now shortened to days. This type of mentality really
- 3:30created shock and awe to individuals trying to defend
- 3:33against it. That type of shock and awe is
- 3:35going to force a strategic and tactical redefinition.
- 3:39The way they were applying cyber defense currently.
- 3:42How fast can we ingest and leverage and threat
- 3:45intelligence, the tools, tactics and techniques that the
- 3:49adversaries are using and be able to leverage the
- 3:52capabilities that we have in front of us?
- 3:55AI changes not only the offensive capabilities, but the
- 3:59defensive ones. The hard part though, is being
- 4:02able to leverage his defensive ones in ways that are legal,
- 4:05able to reduce regulatory, you know, potential violations.
- 4:09But if you imagine this, you know, for example, of multiple
- 4:12different organizations get hit simultaneously, if they're
- 4:15ingesting those automated detected TTPS into it and then
- 4:19spread it out worldwide, we'd be able to have a chance to counter
- 4:24what the offensive teams are doing at the same velocity that
- 4:27they're accomplishing it. But right now, like even if you
- 4:30start, you know, saying how would that be accomplished?
- 4:32We have a hard time just sharing threat intelligence, handing
- 4:35pieces of paper over, or even calling each other on the phone,
- 4:38much less having a universal LOM that is disseminating TTPS
- 4:42worldwide. The technology's there.
- 4:45The question is, can we even use it to leverage against it in a
- 4:48proper cyber, cyber defense capability?
- 4:50That changed the tactics. And I'll add one first plus 100
- 4:55to Rob for one, pulling from history because I think that
- 4:57helps us address it. It actually helps you be a good
- 4:59stoic in the midst of whatever is happening.
- 5:01You can say themes and rhythms like this have happened before
- 5:06and we will adapt. And I think he's spot on that
- 5:08actually AI, well right now is currently super empowering
- 5:13attackers to his point, we could actually have an AI that again,
- 5:18if you look at what are the TTPS, what do the attack surface
- 5:21looks like? What are abnormal patterns?
- 5:23And having that shared, I'd go even one further, which is I can
- 5:27remember about 15 years ago, I was pushing for almost a public
- 5:31health approach to cybersecurity, which is in
- 5:33public health, we don't share the specifics, but we share what
- 5:36we're seeing in the different states because each of the 50
- 5:39different states have their own privacy laws and things like
- 5:41that. And one of the biggest barriers,
- 5:43I think, for sharing insights as to what's happening to one
- 5:47organization with many organizations is 1.
- 5:49They're worried about, you know, am I going to reveal sort of my
- 5:52own vulnerabilities? Am I going to reveal my
- 5:53intellectual property? What might be interesting is
- 5:56there is there's an approach called Federated learning.
- 5:58And Federated learning, instead of having the data be shipped
- 6:01somewhere, the algorithm comes and learns there.
- 6:03Imagine actually that we had an algorithm that you were willing
- 6:06to let learn on what your organization is seeing.
- 6:09And as a result, you get the benefits without the specifics
- 6:12or the data as to what other organizations industry.
- 6:15So maybe you're all in finance or maybe you're all in commerce
- 6:18or maybe you're all in transportation.
- 6:20You could actually say here are the current attacks as of the
- 6:23last hour that we have been seeing across the world in terms
- 6:27of what might be a tax surface activities that are being
- 6:30exploited. And as a result, you do that
- 6:32without having to send your data to anybody, and you're
- 6:34collectively more secure as a result.
- 6:37Rob, thoughts on how incident response and forensics must
- 6:44evolve, especially since AI can navigate breach networks, erase
- 6:50its own tracks corrupt evidence. So how are the rules of change
- 6:55or the technology has changed? So how how must incident
- 6:59response and forensics evolve? And as a result.
- 7:04There's so many different ways that we could potentially
- 7:06utilize the data that needs to be employed.
- 7:10You know, that capability of, you know, potentially be able to
- 7:13do sharing easier needs to be accomplished.
- 7:16And you know, instant response and threat hunting are now
- 7:20pivoting specifically on what you know and how fast you can
- 7:23know it. So instant response and be able
- 7:25to identify tax is crucial. So if you're looking for how
- 7:30they're pivoting across the network, how they're potentially
- 7:32doing data media exploitation, essentially looking for, I
- 7:37always go back to the Death Star plans inside a network so they
- 7:40could get it out or potentially embedding themselves to
- 7:43potentially cause havoc in the future critical infrastructure.
- 7:47For example, you know, salt and bull typhoon.
- 7:50The hard part here, I'm taking a step back and looking at
- 7:53response is somehow we need to achieve some sort of parity,
- 7:58meaning that as the offensive teams are able to move is
- 8:01increasing velocity, instant response needs to be able to
- 8:05match that. Here's a couple of challenges
- 8:07with that. I believe currently in 2026, and
- 8:10this could be different even six months from now, most executives
- 8:14are making decisions whether they're going to RSA and seeing
- 8:17these new solutions that are out there will have a hesitancy.
- 8:21By deploying a solution that is going to remove the human in the
- 8:25loop, offensive teams are going to feel more comfortable
- 8:29removing the human in the loop faster than you as a company
- 8:34that say, I'm going to take on that risk because if an answer
- 8:37response or detection tool says, hey, I found something, I'm
- 8:40going to block it automatically without a human analyzing that.
- 8:44Well, who's what happens at that situation?
- 8:47If they have, bad things could occur.
- 8:49So are you placing the organization at risk more
- 8:53definitively if you deploy these defensive tools that haven't
- 8:57been tested fully? I would be a little bit
- 8:59concerned right now not having a human saying, yeah, we should
- 9:02block that, turn off that node in order to prevent that attack.
- 9:06Offensive teams are still able to say, well, if it trips over
- 9:09something, not my fault, the cyber defenders can't take that
- 9:14risk. So as a result, we're not going
- 9:16to be able to achieve the same velocity.
- 9:18And that alone is going to create some of the parity issues
- 9:22that even if we match capability to capability, AI versus AI, how
- 9:26do you reduce that risk so we're able to match velocity versus
- 9:29velocity? I don't think we've solved that
- 9:31problem yet technically, because who's going to trust, hey, Rob,
- 9:36how do you know you're not tripping over something that's
- 9:38going to disconnect us from the Internet and cause problems?
- 9:42Technically we may be able to get there, but how do we know
- 9:45it's going to work effectively offensive teams if it trips over
- 9:48something and no big deal? And to Rob's point, I think he
- 9:52he underscored a very essential risk calculus, which is 1 The
- 9:58defenders want to keep the business running while also
- 10:02keeping it secure, whereas the attackers are, like you said,
- 10:05you know, we, we don't mind. And so because the attackers
- 10:10will be more likely to pull solutions that don't have a
- 10:13human in the loop, but we've got to figure out what are ways that
- 10:17can help the defenders move at the same speed.
- 10:20And I think the biggest barrier we have is despite all this talk
- 10:24about digital twins or or whatever, we're really not that
- 10:26mature in terms of enterprises. So that you could actually run a
- 10:30really rapid parallel simulation that says if my security tool
- 10:34opts to do this, what are not just the first order, but the
- 10:37second and third were a consequences that could happen
- 10:39to my enterprise. And as a result, I might be
- 10:42trying to secure it. But the next thing I know, I've
- 10:43actually blocked this application and now my customers
- 10:45can do business. The other thing I would also
- 10:48say, if I was a really sophisticated hacker, I would
- 10:51make it so that you did it to yourself.
- 10:53And as a result, you're now your security team is now dealing
- 10:56with the fact of not only have they actually caused the
- 10:59business to go offline, but now the company is upset at the
- 11:02security team. They're now demoralized.
- 11:05This would be hopefully something that a attacker
- 11:08wouldn't do, but maybe I've seen things in the past where you
- 11:11think demoralization is a tactic of certain threat actors.
- 11:15You can ask your questions and you should take advantage of
- 11:19this opportunity. When else will you have the
- 11:22chance to ask Rob Lee or David Bray pretty much whatever you
- 11:27want. So if you're watching on
- 11:29LinkedIn, pop your question into the chat.
- 11:31If you're watching on Twitter X, use the #cxo talk.
- 11:37And so on this point, Anthony Scruffignano, who's a very
- 11:41prominent data scientist, he's been a guest on CXO Talk a
- 11:44number of times, He makes the comment that one approach in
- 11:49attacks is for the bad guys to focus on a target that is
- 11:54already distracted in response mode.
- 11:57This is exactly what we have seen not just in cyberspace, but
- 12:02we've also seen in the real world where I get you distracted
- 12:06and focused responding to one activity.
- 12:08Maybe, you know, you've got something that looks like a
- 12:10hardware issue or software issue with your application.
- 12:13And the next thing you know, while you're busy focusing over
- 12:15here, I'm coming in and then going laterally or something
- 12:17like that. So to Anthony's point, what AI,
- 12:23generative AI in particular, now allows attackers to do is to
- 12:26actually plan something that's sophisticated, which did not
- 12:29used to be the domain of most attackers.
- 12:30It was really the domain of nation state actors.
- 12:33And again, it took them months both to do the reconnaissance
- 12:35and the coordination. Now, again, what I'm hoping is,
- 12:39is that we can actually evolve almost a science where on the
- 12:43defender side, we can quickly say I see these isolated events
- 12:46where they appear isolated, but they're actually part of a more
- 12:49coordinated campaign. And I know Anthony Scripignano
- 12:51has worked countering fraud and actually kept tackling
- 12:54fraudsters. And usually the challenge is, is
- 12:57often times you're seeing the tip of the iceberg of what is a
- 12:59most a much more deeper campaign.
- 13:01And so I think this is exactly why, one, what Sands is doing is
- 13:04so important and why we have to evolve what defenders do.
- 13:07Because now with both bots and agents, you could have 5 or 6
- 13:13fires going on at the same time. And the question is, while those
- 13:165 or 6 fires are happening, how do you know there's not a 7th 1
- 13:19quietly actually exfiltrating data out of your organization?
- 13:22Rob, thoughts on this kind of multifocal set of attacks can't
- 13:28campaign of attacks? A distraction.
- 13:31It's just the fact that most teams are exhausted and they're
- 13:34looking at the same, you know, screen every single day.
- 13:37And as a result of that, you know, it leans into one of the
- 13:40reasons why defensive agents, you know, could work 24/7 and
- 13:43never get exhausted will potentially be a game changer.
- 13:46The challenge though, is we're only seeing offensive teams
- 13:49leverage this against a single target right now.
- 13:51What happens when you start having agent to agents be able
- 13:54to coordinate and communicate with each other simultaneously
- 13:57across critical infrastructure hitting be able to hit power and
- 14:00water simultaneously? How do you have two teams at the
- 14:03same parity level on the defensive side to be able to
- 14:05prevent that? The shock and awe of that could
- 14:08occur there where you could have down to the millisecond
- 14:10execution across multiple different nodes, financial
- 14:13networks, supply chain, the critical infrastructure,
- 14:16electrical and hit boom, boom, boom.
- 14:19And you imagine what happens on criminal side ransomware attacks
- 14:23that they wait until you know multiple parts of a supply
- 14:26chain. You think the you know some of
- 14:28the things that happened in the past with ransomware.
- 14:30Imagine multiple different hospitals and insurance networks
- 14:33and pharmaceutical hit simultaneously A ransomware
- 14:36attacks. These types of things are
- 14:38definitively possible. And the scale of offensive
- 14:41attacks go larger when you go back to the distraction, the,
- 14:44you know, exhaustion, the limited teams, the workforce
- 14:48that we can't, you know, hire a bunch of more people here.
- 14:51The leveraging AI is the only solution.
- 14:53And we come back to that. You know, how do we balance that
- 14:57with what we're seeing, the offensive teams and how many
- 14:59potential attacks will need to occur before the risk calculus
- 15:03is so great that you're going to have to say we have to
- 15:06potentially embed ourselves utilizing some of these new
- 15:08technologies, remove some of the human in the loop and increase
- 15:11risk as a result of, you know, what can happen if we don't.
- 15:15It's it. I could go on for minutes and
- 15:17hours about this topic, you know, over a beer.
- 15:19But at the same time, this is possible and it likely is
- 15:22occurring right now. And it's a matter of time when
- 15:25it's, you know, a geopolitical incident occurs and they're
- 15:28going to do a show me session of how vulnerable that we all are.
- 15:33Let's talk about autonomous attacks, right?
- 15:37We have open claw and agents are the new, what?
- 15:43I don't know, the new refrigerators, everybody's got
- 15:46to have one and we're going to have thousands of them.
- 15:49This just makes the entire problem more complex as as Rob
- 15:55just said, right, they don't sleep.
- 15:56So who wants to jump in and let's let's talk about that.
- 16:00The good thing about his open claw and clawbot, the one thing
- 16:03you hear out of everyone's mouth initially is don't install it on
- 16:05your own system and it shows. People are thinking thoughtfully
- 16:09about turning on autonomous. Just go wreck havoc with my own
- 16:13data and it shows the risk calculus that what happened with
- 16:16security. You need AB and access to do
- 16:18anything useful to shut down attacks.
- 16:21But we're already seeing open claw and clawbot.
- 16:23People say don't install it on your own system with root access
- 16:26because of the fact it could cause havoc.
- 16:28These two things intersect and are not equal on the
- 16:30technological side. Spot on and I actually saw there
- 16:33was a story where someone was posting how supposedly they were
- 16:37in Ukraine. Maybe they were, maybe they
- 16:39weren't and they decided to install it on their system
- 16:42because it would make their home smart.
- 16:44Personally, I use whenever I see the word smart device smart
- 16:47home, I, I substitute the word hackable because that's really
- 16:50what it means when you say a smart device.
- 16:52And I don't know if I would do that in the middle of a active
- 16:55war zone because that's probably exploitable.
- 16:58So I do think to, to Rob's .1, there are cautions about be very
- 17:02cautious because these things will will possibly wreak havoc.
- 17:05The other challenge is, and this gets to the fundamental
- 17:07challenge of generative AI, which is generative AI is
- 17:10trained on past patterns, and while you can ask it to possibly
- 17:15remix those patterns in interesting ways, it will always
- 17:18be sort of tied to its past training set.
- 17:21It's not necessarily really good at novelty.
- 17:24In fact, when you try to present it with novelty, that's why you
- 17:26sometimes get something that's a pattern that doesn't actually
- 17:28exist, which they call hallucination.
- 17:30It's just it's just not something it's seen before.
- 17:33And so this is actually more of a challenge on the defender side
- 17:36because as we train these models to be defenders, we have those
- 17:4024/7 agents that are helping with defense.
- 17:43The challenge we've got is if a novel attacker really wants to
- 17:46exploit our attack surface, they will gain through various means
- 17:51insights as to what the agents are expecting to see or possibly
- 17:54may see and therefore come with something completely different,
- 17:58which is why is what Rob was saying earlier, we've almost got
- 18:01to have like an immune system response that says I've never
- 18:04seen this type of attack before. I'm going to try and rapidly
- 18:07characterize it. I'm going to try and figure out
- 18:08what the intent is. And then I'm going to percolate
- 18:10the lessons not just across my enterprise, but ideally it's
- 18:13refederated learning. I'm going to share those
- 18:15insights with others in my industry because this is where
- 18:18right now, the novelty of the offender attacking off inside is
- 18:24actually puts defenders at a disadvantage precisely because
- 18:27gender to the AI is limited to its past training sets.
- 18:30I have one thing to add to what you just said from a technical
- 18:32perspectives because it had a conversation just yesterday with
- 18:35one of my James line actually chatting about what we're
- 18:38talking about, our RSA keynotes, the attack surface with
- 18:42automated capabilities like Cloudbot and open cloud.
- 18:45When you end up thinking about it, imagine if you're saying,
- 18:47hey, go read my e-mail, essentially, you're basically
- 18:50setting yourself up for the ultimate spear phishing is
- 18:52they'll go click on every single link that's coming in to be able
- 18:55to see what it is. Well, the human at that point is
- 18:58now the security defense because the human can't click as fast as
- 19:01the automated capability. Here we've exploded the
- 19:05possibility of just the autonomous capability acting
- 19:09like a human exposing additional tax services because if you
- 19:13throw enough balls in the air, one of those is going to be hit
- 19:17by one of these autonomous agents.
- 19:19And it it's amazing when you end up thinking about just that
- 19:22alone, increasing speed of of, you know, you being able to
- 19:27respond to many emails and marketing and tweets and
- 19:29LinkedIn messages. Imagine basically you just throw
- 19:32enough spearfishes at that click capability.
- 19:36One of those is going to land and it's going to be done at
- 19:38speed. You know, human actually ends up
- 19:40being just because they're slow, ends up being a security
- 19:42governor accidentally. This points to the very
- 19:45challenge that while we need to use AI to help with our
- 19:48defenders that are already exhausted, that are already over
- 19:50capacity, AI itself is an attack surface itself.
- 19:54We we have indications. I wouldn't say confirmation, but
- 19:57indications. Some bad actors have identified
- 20:00when AI as being used for coding for vibe coding reliably
- 20:04hallucinates. And they have identified those
- 20:06reliable hallucinations and gone out and written a code library
- 20:10for the very hallucination that the code library has made it
- 20:13open source. What could be possibly wrong
- 20:16when you actually call a code library that didn't exist until
- 20:18a bad actor wrote it? So this gets to, not only do we
- 20:22need to use AI for defense, but as we're using in an enterprise,
- 20:26it itself is an attack surface, whether we're using it for what
- 20:29we're doing in enterprise OR we're using it to write code.
- 20:33If anything, cybersecurity professionals are going to be
- 20:35needed now more than ever. However, we've got to figure out
- 20:38how we up our game and also give them the headspace and and
- 20:41breathing room to deal with everything that's happening.
- 20:43So just if you can now expand this aperture into this, Anthony
- 20:49makes the comment swarms. AI swarms can also learn from
- 20:55how they fail and attack repeatedly in a more targeted
- 21:00way. So thoughts on AI swarms and
- 21:04then we'll take some questions. That gets to the challenge,
- 21:06which is I could inundate your enterprise if I know the
- 21:11capacity of the actual people to respond.
- 21:14And so I've seen swarms before, I've experienced swarms before.
- 21:17And so Anthony is spot on. And and like he said, given that
- 21:21they, they can do permeations and they can try to be adaptive,
- 21:25they only have to be right once out of 1000 or 10,000 attempts
- 21:28that they're doing. This also points to a lot of
- 21:32historical cybersecurity practices.
- 21:34And I know there's been a shift in, in the last five years, but
- 21:37a lot of historical cybersecurity practices relied
- 21:39on known CDs, critical vulnerabilities and exploits.
- 21:42And the signature, well, the challenges is if the swarm is
- 21:45changing its signature every second, that may not appear as
- 21:49ACV, that may not appear as something that a threat device
- 21:52is attacking. So swarms are definitely
- 21:54problematic. And it points to and I and I
- 21:56have a sense this is where Anthony's going to the question.
- 21:59We cannot just rely on past trainings.
- 22:01What we need is instead almost a algorithm that says I see what
- 22:05looks like non normal, non normalcy.
- 22:09It's almost like a second derivative of it.
- 22:11Because what we're looking for is.
- 22:12I don't know why I'm seeing the sort of abnormal pattern in my
- 22:16enterprise. It might be hardware or
- 22:17software, but it might be a sign of a swarm is trying to mess
- 22:20with me. Rob, thoughts on swarms?
- 22:23I feel we have not seen a specific or very many examples
- 22:28of swarms versus speed. And I flip that a little bit and
- 22:33I, when I go back to swarms, the swarm I tend to think about is
- 22:37the swarm of developers that land on a new capability.
- 22:40And I, you know, take a look at what happened with open cloud
- 22:42and cloud bot. Is that how long ago was this
- 22:45someone's basement project in, you know, weekend development?
- 22:50You know, Co work is a really another good example from cloud
- 22:53code to Co work being released was 10 days.
- 22:56When you end up having the development life cycle go from
- 22:58something that used to take years down into weeks.
- 23:01That is the swarm capability I'm looking at because instead of
- 23:04just one or two developers, you have hundreds of developers
- 23:08potentially iterating on code or even the code iterating on
- 23:11itself faster than we're able to potentially comprehend.
- 23:14Now to take that from, hey, there's a positive side to this
- 23:17is that even though you have capabilities to go from infancy
- 23:20to fully vetted capability within weeks.
- 23:23Cyber defenders can leverage the same capability because
- 23:27offensive teams tend to still be fairly tight held on their
- 23:31capability development. 3-4 developers.
- 23:33What if we flip that and potentially have hundreds, if
- 23:36not thousands of cyber defenders start working on capabilities
- 23:40for defense and able to take a clawed bot and make it into a
- 23:45very specific, you know, cruise missile that no one's ever seen
- 23:48before. Not just in something that took
- 23:51years to develop, but something that might take weeks and able
- 23:53to be deployed as a counter mechanism inside the networks.
- 23:56So what we're able to see on development speed, we could
- 24:00potentially use on a defensive mechanism for good.
- 24:03This changes the notion of patching.
- 24:05Instead of patching being something that takes months,
- 24:07patching could be done in weeks or even a day.
- 24:10Let's jump to some questions. This is really my favorite part
- 24:14where we get to hear from the audience and your question.
- 24:18So keep them coming. All right, so we have a question
- 24:22from Arsalan Khan on Twitter X who says how, how should
- 24:28businesses be held accountable or should they for cybersecurity
- 24:33issues? So where does the responsibility
- 24:36and the accountability the lie? Doing its response correctly
- 24:39still means you messed up. And what I mean by that is the
- 24:42fact that the incident occurred in the 1st place means that
- 24:46something went wrong, and even though the answer response team
- 24:50may have interdicted, it may have stopped the Death Star
- 24:53plans from being stolen. You don't get medals as a
- 24:56responder by saying hey, good job from stopping that.
- 24:59In the military you do. Even if an incursion happens, if
- 25:02you know, North Korea flies jets into South Korea, even though
- 25:05they encroached in the airspace, if we're able to launch missiles
- 25:08or something in there to interdict and stop them,
- 25:11everyone gets a medal. That does not happen in
- 25:13cybersecurity. The accountability and what our
- 25:16mentality, our culture is still has not changed to reward proper
- 25:21cyber defense. So accountability incentives are
- 25:24perverse. As a result of this.
- 25:27If you're a CSO and you do a good job, you detect the
- 25:30incident, the incident happened, you failed.
- 25:34That's the perverse incentives that occur today and is not
- 25:38going to change because AI is out there.
- 25:40Where's the metals for the instant responders, the sock
- 25:43teams that detect the incident PERT in the middle of the attack
- 25:47and then says, hey guys, we stopped it.
- 25:49They did not achieve their operational goals.
- 25:51They're still looked at from a, you know, liability risk and all
- 25:54this other calculus. You still.
- 25:55Failed if I could do a Hallelujah to what Rob just said
- 25:59because this is the challenge is we still have boards and we
- 26:03still have CE OS that think this is a preventative exercise when
- 26:07the reality is if you're going to be on the Internet, it is
- 26:10going to happen to you. What you want is exactly as Rob
- 26:13said you want a team that responds to it rapidly that
- 26:15actually tries to do their best to actually counter it and
- 26:18actually success is simply one, we detected it and two that we
- 26:21responded to it and we actually did our best to mitigate it.
- 26:23But that is, that is a mind shift take.
- 26:25And I hope, I mean, I'm, I'll do anything by the way, Rob, to
- 26:28help Sam's get that message out there.
- 26:30Because unless they want to unplug themselves from the
- 26:33Internet, bury whatever they're doing underground and make sure
- 26:35no human being touches it. These things will happen.
- 26:37And they will happen with increasing frequency, partly
- 26:39because of excited geopolitics, but also because of the
- 26:43capabilities that are out there. And if you're going to use AI,
- 26:46what I am trying to do is when I go to boards and I go to Cos and
- 26:50say, like if you're going to be adopting AI for enterprise to
- 26:53engage your customers or improve your internal operations, you
- 26:55should in parallel upgrade your cyber posture in parallel.
- 26:59Because that will actually make it that you're more responsible
- 27:02in your use of AI. While at the same time hopefully
- 27:04making it so that you can detect if bad bad actions are being
- 27:07sent your way because they will be sent your way.
- 27:09Let's jump to another question. And, and by the way, I have to
- 27:12say, hearing you both talk, I'm ready to hide.
- 27:16I'm ready to, you know, do the ostrich thing.
- 27:19I'll stay home. You know, I don't know, maybe
- 27:21put mattresses up against the window?
- 27:23I know someone that can help you get a Faraday cage if you want a
- 27:25Faraday cage. Yeah, but that's but I'll just
- 27:28like as a point you just made there as a parent, do you just
- 27:31like pull your kids in at home and say he never venture
- 27:33outside? You're going to wear wear your
- 27:35bicycle. There's a certain amount of
- 27:37continual risk no matter what you do that you have to assume
- 27:41you can't prevent any bicycle accident from happening.
- 27:44You just set the rules of the road.
- 27:46Just take a look both ways. Don't run with scissors.
- 27:48There's certain things that are acceptable risk.
- 27:50And unless you have that acceptable risk, there's no
- 27:52learning, there's no growth, there's no acceleration.
- 27:54So, you know, as a parent, you look at it, that is like, oh, I
- 27:57can't prevent all the bad things.
- 27:58But again, no ostrich head in the sand.
- 28:01It's a, hey, listen, we need to lean forward, especially in
- 28:03today's time business transformation.
- 28:05Everyone out there is trying to figure out how do I deal with
- 28:06AI? You just need enough of a
- 28:08parachute, enough of a rope around your waist to say I could
- 28:11take a certain amount of risk safely, but not without ever
- 28:16potentially falling it with 0 risk in the chasm.
- 28:19But it used to be that the risk came from getting hit by a car.
- 28:26Those risks remain. Only now we have this very
- 28:32large, ongoing, permanent additional attack surface
- 28:38because we live our lives online.
- 28:39Correct. And also because the good news
- 28:43is, so I tell people, if you have a smartphone, you have the
- 28:46capabilities to call anybody a moment's notice, hopefully with
- 28:48their permission, track them or any asset using air tags or
- 28:52other means anywhere in the world, again with hopefully
- 28:54their permission, and download satellite footage as recent as
- 28:5715 minutes ago at .25m resolution.
- 28:59You have the CIA in your pocket circa the late 1970s, early
- 29:021980s, and 2 billion people on the planet have it.
- 29:06So the the good news is we super empowered you to do things that
- 29:08are unprecedented. That also means that bad actors
- 29:11can do things that are unprecedented.
- 29:12I don't think we want to roll that back.
- 29:14It just means, again, we've got to figure out the fence in
- 29:16depth. And we've been here before in
- 29:18the 19 tens 1920s, there was this thing called TNT.
- 29:21TNT was being used by anarchists to do bad things.
- 29:23But we didn't go into lockdown mode.
- 29:26We figured out the fence in depth.
- 29:27And now it's less of a threat. And, you know, we continue to
- 29:29live our lives. So I think we can figure it out,
- 29:32but it's going to take new approaches.
- 29:34And I think part of the trouble with cybersecurity is both CI OS
- 29:37and SIS OS initially reported to chief financial officers, then
- 29:40later chief operating officers. And so it was seen exactly as
- 29:43Rob said, which was cybersecurity was a preventative
- 29:46function as opposed to it is, it is what you do as you're
- 29:49interacting with the world and doing commerce and doing
- 29:51whatever you do. And So what you want to do is
- 29:53have a team that does get the medals for rapidly detecting,
- 29:57rapidly containing and rapidly mitigating what will happen,
- 30:00because that's the nature of living in the world where.
- 30:02We have massive SIM farms that are out there.
- 30:04They imagine that instead of something that someone's hitting
- 30:07your kid or potentially you, your parents or your neighbors,
- 30:11that kind of scalability is now achievable with AI.
- 30:13So the death by thousand cuts argument is actually true.
- 30:16You don't need to steal from ransomware tax.
- 30:19You know, you know, millions of dollars.
- 30:20All you need to do is hit enough people with 300 a pop and do
- 30:24that at scale with SIM farms that are being found all across
- 30:26the US. And all of a sudden the $300 per
- 30:29person times 10,000 equals the same as a single ransomware
- 30:32payout before. This kind of scalability has not
- 30:35been able to be achieved before AI.
- 30:37Let's go back to some additional questions.
- 30:39And I'm still, I'm not convinced that the ostrich approach.
- 30:43I I still am thinking maybe I need to just simply hide and
- 30:45that's it. OK.
- 30:48But maybe by the time we're done with this conversation, I'll
- 30:51feel more confident that I can actually go out in the world.
- 30:53All right, This is from Chris Peterson on Twitter, XO.
- 30:58And I should also remember remind you guys that you should
- 31:02subscribe to the CXO Talk newsletter.
- 31:05Go to cxotalk.com now, subscribe and we will send you notices of
- 31:11upcoming shows. This is from Chris Peterson.
- 31:14And he says feeding AI tools good data often requires complex
- 31:21tool chains. How do we keep those tools and
- 31:25chains from becoming the attackers prize targets?
- 31:30You can't, it's the same thing as you know, UPS delivery,
- 31:33walking straight in the door and you're, you know, and UPS, you
- 31:35know, it's the same type of attack services you can't
- 31:39eliminate. The more you end up having a
- 31:42larger attack service. In this case, you take a look at
- 31:44what enterprise is now leveraging across, you know,
- 31:47simple Zoom sessions, what we're doing here from Teams and so
- 31:51forth. The likelihood of someone being
- 31:54able to penetrate that is becoming more prescient.
- 31:58I go back to the Chinese and North Korean threat actors that
- 32:01instead of hacking a network, they just get hired as an
- 32:03employee that's using your own tools at HR in humans and plus
- 32:08technicals like there's a send you a laptop that has, you know
- 32:11the keys to every access point in your network already set.
- 32:15This is going to take individuals to realize that even
- 32:18the TAC tools, the common management capabilities in our
- 32:22organization are able to be exploited and the attack surface
- 32:26for a single human, which is really interesting of it.
- 32:28Again, I was having a separate conversation last night just
- 32:30about this. What, 1020 years ago, actually
- 32:3320 years ago, a single individual would have 2-3 nodes,
- 32:36a phone, a computer, maybe a cable modem.
- 32:39Now you take a look at how many nodes are across my home, your
- 32:42home, you know 3050 nodes. You know with every single
- 32:45appliance in potential TV. The enterprise network size for
- 32:50a single individual happened 20 years ago is about the single
- 32:53enterprise size that a single individual has today.
- 32:56It's kind of mind numbing when you end up saying count up how
- 32:58many single devices that you individually have that are
- 33:01connected to the Internet. Every single one of those is a
- 33:03potential attack surface. Here is a question Reza Fatahi
- 33:08says there are good opportunities, I'm assuming for
- 33:11attacks and defense at the application and OS layers.
- 33:16Each angle is a different application, each layer is a
- 33:20different application. Any thoughts on that?
- 33:23Either of you want to jump into that?
- 33:25You're going to have to find ways to rapidly do independent
- 33:28verification, validation of what you're using as your tools,
- 33:31whether they're for data ingest, your applications, or things
- 33:34like that. And it has to be faster than
- 33:36what's currently been done. You can't just rely on published
- 33:40critical vulnerabilities. You really need for the things
- 33:42that are absolutely essential. You almost have to be constantly
- 33:46red teaming how an application or an OS might be used against
- 33:50you. And the good news is, again, AI.
- 33:52I've seen the actually instances.
- 33:54I won't name the name of the big tech company, but they did 2
- 33:57parallel paths for some of their cybersecurity functions.
- 33:59Unfortunately, they initially went down the path of replacing
- 34:01your cybersecurity people with AI, which I do not recommend, do
- 34:04not do. But they tried that.
- 34:06And then in another function for their mobile security, they had
- 34:09the AI focus on known critical vulnerabilities to free up their
- 34:12people, their cybersecure professionals to look for new
- 34:14exploits. Well, guess which team
- 34:15outperformed? It was the humans plus the the
- 34:17the AI that outperformed. Because you have to always be
- 34:20looking for new vulnerabilities. You cannot just rely on
- 34:23published known vulnerabilities. This is from Huawei Wang on
- 34:28LinkedIn and she says as a defender and as an attacker for
- 34:34technology readiness, do you see the direction of talent
- 34:38development moving towards what previously was called the 10X
- 34:42engine or a full stack engineer? So the idea of of talent and
- 34:47preparing talent to handle these issues.
- 34:51I go to leadership by vulnerability or vulnerability
- 34:53by leadership, depending on how you want to frame it, is that
- 34:55we're all learning. I'm learning.
- 34:56I, I had very little knowledge about AI two years ago and I'm
- 35:01struggling on a daily basis. And this is where from a talent
- 35:04management perspective, there's this idea that we need to hire
- 35:07people with AI experience. It's really hard to assess right
- 35:11now because when you by the time you hire them, their knowledge
- 35:14may be already out of date. So from a talent management
- 35:17perspective on things, I think it's more crucial than ever is
- 35:20trying to identify individuals that have a massive capacity to
- 35:24learn and learn Quick goes back into the old hacking mindset and
- 35:28what before was considered criminal.
- 35:31The original mindset was, I wonder if I put this thing and
- 35:34over here, do this thing, tie the wires together and take a
- 35:37step back. That is a fiddling mindset.
- 35:41And that's what hacking was considered.
- 35:43Even going back to the MIT days where they invented the term.
- 35:46That type of talent is hard to come by.
- 35:49And it's also hard to assess what I'm seeing organizations
- 35:53do. And it is a complete disconnect
- 35:56from and I I credit Kate Marshall on her sub stack
- 35:59mentioned this specifically you start seeing organizations using
- 36:03hackathons. We ended up seeing I think it
- 36:07was NVIDIA or Entropic. Just you wonder if you end up
- 36:10being the top competitor in this hackathon, not only are offered
- 36:14a job, you win a cash prize. It is going to assess someone's
- 36:17ability to work through problems to the problem solving
- 36:20challenges, to assess their mentality, to be able to show
- 36:22you're the right candidate. That is the only way
- 36:25organizations are going to be able to hire the right people
- 36:28with the massive technological change in the velocity that it's
- 36:31occurring at. David, this one seems tailor
- 36:34made for you and this is from Simone Joan Moore.
- 36:38And earlier you spoke about immune system response.
- 36:41She says what about the systems that are off, They're
- 36:44compromised or their DNA hasn't developed the signal or
- 36:49detection system for it. What do we do?
- 36:53You have to get instrument to your enterprise to have what's
- 36:55called patterns of light. What does normalcy look like for
- 36:57these systems, both in terms of their operations, but also the
- 37:01human use? And so then if a persona that
- 37:04looks like your CFO or your CEO, you know, at 9:00 PM on a Friday
- 37:09night suddenly seems to be accessing a finance system that
- 37:11is non normal. That might be.
- 37:14And again, you have to discuss to Rob's point, which is we have
- 37:16to have ways of defining what is reasonable risk calculus.
- 37:19You may want to stop that transaction until the chief
- 37:21information security officer actually has a conversation and
- 37:24says, was that really you CEO or CFO or was that some bad actor
- 37:28trying to exploit your persona to actually pull money out?
- 37:31And so, but that requires enterprises to 1 be using their
- 37:35logs and that instrumentation in a way to establish patterns of
- 37:38life. But there is precedent for this.
- 37:40The other thing I would say real quick and I'm and we haven't
- 37:43gone here, is that good. The good news is generative AI
- 37:46is really good at passing the Turing test.
- 37:47The bad news is generative AI is really good at passing the
- 37:49Turing test. I get concerned about audio,
- 37:52video, synthetic media being used to convince your humans to
- 37:57do a legitimate activity where nothing was really breached.
- 38:00It was just AI itself was used as a prod to get social
- 38:05engineering on steroids, to have things either money walk out the
- 38:09door or data walk out the door, etcetera.
- 38:10So generative AI itself is going to require us to have better
- 38:13ways of verifying that those workers and those people you're
- 38:16talking to really are legitimate.
- 38:18This is from Demetrius Giorgio who says looking into the
- 38:23future, when do you expect that AI agentic defenders will be
- 38:29able to be trusted as an experienced human without a
- 38:33human in the loop? And please answer for both IT
- 38:38and OT environments. Rob, you want to jump on that
- 38:43one? When you end up looking at the
- 38:44standard certification, accreditation and process of
- 38:48human beings go to your college degree, you get your
- 38:51certifications, there's, you know, assessments that there's
- 38:55someone end up going through in order.
- 38:57But again, that's the baseline. You could say you have the
- 39:00minimal qualifications to be able to do this job, but the
- 39:03trust is there's still some third party that is doing this
- 39:05assessment or I think the industry.
- 39:08And again, for anyone going to RSA in a few weeks, walk around
- 39:12the floor and the everyone's going to have this massive
- 39:15solutions. Just like you're saying.
- 39:16A Gentech defenders ask, how do you, how did they establish
- 39:19trust in that capability? Like what we, we have our own
- 39:22rubric. We went through our own battery
- 39:24exam. We here's our, here's what we
- 39:26did is I could go out there and say, Hey, my IQ is X.
- 39:30Well, how do we know that I'm a self assessing my IQ versus
- 39:34having a third party independent capability to establish that
- 39:36trust. The reason I use trust is all
- 39:38these different accreditation, certification, degree programs,
- 39:42assessments are all aimed to establish trust.
- 39:46Trust in the agentic defenders needs to have some sort of trust
- 39:49mechanism assessed by a third party, not software
- 39:52accreditation. All these other things does it
- 39:55handle reasoning equivalent to a third party human being?
- 39:59This is from Dan Turi and he says generally is industry
- 40:05accepting of integrated logging tools or is there a need for
- 40:10external immutable logging? It seems like we have rolled out
- 40:15tools without pause to validation and integrity.
- 40:20It depends on certain enterprises are more
- 40:22instrumented than others. The finance sector, they have to
- 40:24because they're regulated. I think what he's asking though,
- 40:27is is it sufficient to have a log where you don't have some
- 40:31immutability to it or not? And I would say yes, you want to
- 40:34have immutability to the log because as we know, some of
- 40:36these attackers, one of the things they'll do is they'll
- 40:38mess with the log or even destroy the log.
- 40:41And so the short answer is yes, on immutability.
- 40:45I do want to actually add to this question and, and, and tie
- 40:49it to what Rob also was saying though.
- 40:51I, I think we're going to find especially for agentic AI and I
- 40:55realized there are several companies plowing ahead with it.
- 40:57If something has a filter or a safety applied after compute.
- 41:02So post compute, I get very nervous about post compute
- 41:06agentic AI ever being 100% safe. And again, you can figure out
- 41:10contexts where it's acceptable to have that risk.
- 41:12I want to have more and more AI agents where I can have filters,
- 41:16boundaries, rules applied pre compute.
- 41:18Because I think the moment you let the machine compute
- 41:21something, just the nature of a hacker, an attacker is they will
- 41:24get it to do something you don't want it to do.
- 41:26So I think what I would encourage increasingly is look
- 41:29for people that are pushing solutions that are pre compute
- 41:32versus post compute. That's a nuance that unless
- 41:35you're computer scientists, it's not something people ask for.
- 41:37But I think this is the geek in me to saying pre compute is
- 41:40where we need to put more attention to.
- 41:42We have a question from Christopher Jablonski and Rob, I
- 41:47think this one is good for you. It's perfect for you.
- 41:51I'll only ask you, I'll read the question in a second.
- 41:54But Rob, to start answering relatively quickly because we're
- 41:58just going to run out of time. And this is from Christopher
- 42:00Jablonski. He says going back to how
- 42:03offensive team seem to seem to have a permanent competitive
- 42:07advantage as they are more comfortable with removing a
- 42:11human in the loop. By how much does this issue and
- 42:15other such asymmetries delay autonomous security from
- 42:20reaching reality? I think that's a reality for the
- 42:24near term and I actually think to a certain extent it is
- 42:28necessary because I think from a cyber defenders perspective and
- 42:31development in the industry, we've become a little bit
- 42:34assumption that we have some sort of parity that we're able
- 42:37to, you know, block and tackle what the offensive teams have
- 42:39been able to do for the past five to six years.
- 42:42Now. This is going to create some
- 42:44leverage in the industry and and hopefully a lot of innovation,
- 42:48they'll be able to match it. We're assuming that we're going
- 42:50to be able to leverage AI to AII go back into what the offensive
- 42:54teams aren't going to be able to do is create AI ecosystems that
- 42:57are going to be able to have defenders talk to one another.
- 43:00You even just take a look at Clonbot and some of the
- 43:02capabilities. It's already been able to
- 43:03leverage AI judicial systems for different agents, whether it's
- 43:07doing ability for AI agents to ask humans to do certain tasks.
- 43:11And again, that physical level task, you could have an AI agent
- 43:14who's your manager. Tell David, you know, hey, I
- 43:16need you to go and over to this system and unplug it from the
- 43:19wall. You know, even though it seems,
- 43:21you know, rash, but again, there's ransomware cases that
- 43:24that is literally what the response mechanism is to make
- 43:26sure that system is not infected.
- 43:28There are ways for innovation that are unexplored on the cyber
- 43:32defender side that this disparity, this asymmetry that's
- 43:35occurring is going to force. And I'm excited to see how this
- 43:38develops. Michael M throws a monkey wrench
- 43:42into all of this. OK, Michael M says sure all of
- 43:49this is true, but once you remove the human in the loop, AI
- 43:53errors are a greater problem and the bad guys simply have to hack
- 43:57the AI defender and convert it into a dual agent.
- 44:01Yeah. So that is spot on.
- 44:03I would say again, it depends. We we haven't dived into the
- 44:06different types of AI. There are AI methods that are
- 44:08more rigorous than say generative AI.
- 44:11But I will give the analogy and I think Rob's right, that in the
- 44:13short term, the advantage is on the offensive side, defensive
- 44:16heads up this game. But we do have US naval vessels
- 44:20where if the captain of the ship flips the switch, there is no
- 44:23human in the loop. If something's coming out that
- 44:25ship and it's a threatening the ship, the the the system will
- 44:28actually be fully autonomous in defending the ship.
- 44:30Now he obviously turns off that switch when he's not in a danger
- 44:33zone. He's not doing it in home port.
- 44:35But that's, that's that is a autonomous and AI have a
- 44:38different name. It's not generative AI that does
- 44:40successfully defend the ship. So I think what we need to get
- 44:43to is through a combination of more rigorous AI that's less
- 44:47brittle than generative AI, but also formal methods and other
- 44:50means like that. We can get something like that
- 44:52where if you decide you're now under siege, you want to flip
- 44:55the switch, then the autonomous defenses can go on.
- 44:58But right now, as we've already observed, we don't have enough
- 45:01knowledge of the second inserted and 4th order effects in an
- 45:04enterprise that it wouldn't be used against us.
- 45:07I believe on the defensive side, human in the loop is going to be
- 45:10there, and when it's not, there are two things that are
- 45:13simultaneously going to happen. You're going to be able to board
- 45:15a plane, look up in the cockpit and there's not going to be a
- 45:17pilot there. If you're comfortable with that,
- 45:20you're comfortable with Cyber Defense having no human in the
- 45:22loop. Second side to that, you going
- 45:24to the doctor, it says, hey, you need a surgery and you have the,
- 45:28you know, humanoid robot come in with a scalpel and Radius's
- 45:31date. You are you totally comfortable
- 45:33with that? When these two things happen
- 45:34near the same time, I believe human in the loop will be
- 45:37removed from Cyber Defense. We have some years to be able to
- 45:40get there. OK, I go back to what I said
- 45:42earlier. I'm going back to hiding once
- 45:44again because that seems like the most prudent response to all
- 45:49of this, all right? Have you gotten into a Waymo yet
- 45:51without a driver? You know, I live, I live in
- 45:54Boston, and no, I have never used a Waymo without a driver.
- 45:59Well, and I think real quick to the on the Waymo example, let's
- 46:02say Waymo's got everything working in the AI.
- 46:04We saw when the power went out, they stopped and that was not a
- 46:07cyber attack, it's just the power went out.
- 46:09I've also seen where there are foreign jammers and that can
- 46:13basically squelch any comms, any geolocation within a three
- 46:16kilometer radius. So yeah, the reality is the
- 46:19world is interconnected, but This is why it's important to
- 46:22have folks on the defender side. Let's very quickly go to another
- 46:25question. This is from Hater Souza, who
- 46:28says, OK, here's another monkey wrench.
- 46:33One of the current AI challenges is that there is no way to
- 46:39distinguish when AI provides the correct information or
- 46:44hallucinates. How can we boost the trust of
- 46:47the information provided and and also what role governments or
- 46:51entities should take? And let me split this up and
- 46:54Rob, maybe you can talk about the issue of boosting trust in
- 46:58the inter information from coming from AI systems.
- 47:02And David, very quickly you can talk about the role of
- 47:04governments and official entities in this.
- 47:08From a trust perspective, how do you trust it?
- 47:09Not going to do you know, hallucinations or anything else?
- 47:14Do you trust a human? I mean, I, you know, you employ
- 47:17someone, how do you know they're going to be 100% perfect in
- 47:20their answers? How do you know they're not
- 47:21going to read something and interpret it incorrectly?
- 47:24When you end up looking at prompt injection?
- 47:26It's no different than information warfare campaigns
- 47:28from, you know, Russia saying, hey, you know, if we have this
- 47:32massive groundswell of, you know, feeling on this one topic
- 47:38and then everyone believes that and that becomes the truth, how
- 47:41do you prevent humans from doing it is the same answer.
- 47:44How do you prevent agents from doing it?
- 47:46We have to start looking at these technologies as reasoning.
- 47:49They're able to be, you know, convinced of bad data.
- 47:52They're able to potentially interpret things incorrectly if
- 47:56you're not able to trust humans. And we have them employed
- 47:58everywhere, including in critical infrastructure, flying
- 48:01planes, going back to that example.
- 48:03And even bad data coming in there has resulted in bad things
- 48:06happening. There's very little trust that
- 48:09you're going to be able to ever assume in the system that every
- 48:12single if then loop 1 + 1 always equals 2 with AI and agents, 1 +
- 48:171 is going to equal 3 in many cases.
- 48:19Just like it happens, Freeman's mistakes will happen and we're
- 48:21not going to be able to get around it.
- 48:23If I can just say spot on to what?
- 48:25What Rob said, I, I often say, I define trust as the willingness
- 48:28to be vulnerable to the actions of an actor you cannot directly
- 48:31control. That could be a human, that
- 48:33could be an organization, that could be an AI.
- 48:35And what it shows is if you perceive competence, perceive
- 48:38benevolence, and perceive integrity, you're willing to
- 48:40trust. Well, the reality is, as Rob was
- 48:41saying, foreign actors can manipulate those perceptions so
- 48:44that you either do trust something you shouldn't or vice
- 48:46versa. But these are true for humans as
- 48:49well. So we should be approaching it
- 48:50the same way. We want to hold organizations
- 48:53and humans recognizing that they will make mistakes, but
- 48:55accountable if they make a mistake that actually results in
- 48:58harm or loss of life. I don't think you can have
- 49:01government regulate what is good information.
- 49:03That is a slippery slope to really bad things happening.
- 49:06I will give a shout out to Ellen McCarthy's Trust and Media
- 49:09Cooperative, which is trying to give people the tools to do this
- 49:11themselves or in communities. And then the last thing I will
- 49:14say, because I'll throw another monkey wrench into this.
- 49:16I have seen already the generative AI is really good at
- 49:19building psychological profiles of each of us thanks to our
- 49:22digital exhaust and figuring out if I present this information to
- 49:26David or Michael at this right time, I'm going to possibly get
- 49:29him to take a following action. That's not a cybersecurity risk.
- 49:32That's just a collective free society risk, which is we need
- 49:36to recognize that if we get our emotions up, if we get angry or
- 49:39we get fearful, is that a case where maybe AAI agent is trying
- 49:43to deliver information to mess with our emotions and get us to
- 49:46do a behavior? And that's where I tell people,
- 49:48always take a breath if all of a sudden you feel like your
- 49:50adrenaline just went skyrocketing.
- 49:52Rob, this is from David Batts. What factor should be evaluated
- 49:57to determine the need for a third party AI risk assessment
- 50:01really quickly please? Almost 100%.
- 50:04When are you not going to be able to?
- 50:05Are you to hire someone with no assessment of their skills,
- 50:08capabilities, knowledge? It's it's going to be 100%
- 50:10across the board. We're going to need to figure
- 50:12out some sort of trust assessment capabilities for AI.
- 50:15Right now it's a chat bot. We kind of think it was Google.
- 50:18Once we start letting autonomous decision making occur in your
- 50:21home, yeah, this is where you're definitely going to want to say
- 50:24who assess that thing to be able to make a decision even on like
- 50:27lower threat things that are occurring in the household is
- 50:32it's going to come down to there's going to it's going to
- 50:34need to exist somehow. You're going to want to know how
- 50:36does this thing know its skills, know its actions, and it's going
- 50:40to reason properly. Reza Fatahi comes back with very
- 50:43quick question. He says what are acceptable
- 50:49license latencies for a defense infrastructure for decision or
- 50:57deliberative process and for key decisions or deliberative
- 51:01deliberative process in terms of AP is any do either of you
- 51:06happen to know that? There's no magic number, but I
- 51:09think what it's showing it is, is what Rob has been emphasizing
- 51:11is we do need to up our game on the defender side.
- 51:14And I think it's about identifying 1.
- 51:16Is this something suspicious? But I don't think we're under
- 51:19siege. Are we now under siege?
- 51:20And then there we need to increase our op tempo.
- 51:22I don't think you can necessarily give a a specific
- 51:25latency number for the enterprise, but it does say you
- 51:28should be able to rapidly respond and mitigate what you
- 51:32perceive to be a tax at a much faster rate than you have.
- 51:36And that's going to require new playbooks, which is why I'm
- 51:38excited about the innovations that will happen on the Defender
- 51:41side in the year's end. Rob, what does a security
- 51:45culture look like in practice, especially given AI and
- 51:49autonomous systems and all the things that we've discussed?
- 51:53I don't think it's any different from any other career field
- 51:55that's out there. Everyone is running around with
- 51:57a lot of banks staring at AI. Is just going to replace my job
- 52:01on your skills or am I going to need to be able to maintain in
- 52:03the future the security cultures led by the managers?
- 52:07If the managers are not, you know, really emphasizing that AI
- 52:11is going to have a positive impact on you, your career and
- 52:14the business, then that's where the skilled security, cultural,
- 52:18cultural vulnerability actually exists, has to be LED.
- 52:22They have to be deliberate and they have to show that you're
- 52:24not at risk from learning AI and looking at a is to increase
- 52:28efficiencies in your organization.
- 52:30Otherwise, you're putting the organization at risk based off
- 52:32the offensive capabilities that exist today.
- 52:35David, looks like you're going to get the last word here.
- 52:38Organizations are building critical systems on data they
- 52:43can't fully verify and very often don't fully understand.
- 52:48What should boards of directors do and should boards treat data
- 52:53provenance as a fiduciary responsibility?
- 52:56David and very quickly please. I think though, I want to again
- 52:59double click on what Rob said, which is boards most
- 53:02importantly, even before they tackle data provenance, should
- 53:05be saying what type of culture are we creating for our
- 53:07cybersecurity professionals? If it's a shame and blame, if
- 53:10it's like how dare this ever happened, then I guarantee your
- 53:14company is not they exist in three or four years probably.
- 53:16If, however, it's the idea that this is, this is part of doing
- 53:19business. We want folks that are learning,
- 53:20we want folks that are adapting that include data Providence,
- 53:22but also how many people verify that the hardware ships really
- 53:25are what the hardware ships they are.
- 53:26That's a geopolitical reality too.
- 53:28That's not something that the board has to get into the sort
- 53:31of the knickers of cybersecurity professionals to do.
- 53:33But they should be saying what are we doing to learn and adapt
- 53:37and actually have the ethos that we have a culture that says it's
- 53:40about rapid detection, rapid mitigation of data issues,
- 53:43hardware issues for cybersecurity issues.
- 53:45All right. And with that, we're out of
- 53:47time. I mean, this conversation could
- 53:49go on. And I hope you will both come
- 53:51back and join CXO talk again so we can dive more deeply into
- 53:57these issues because we barely scratched the surface today.
- 53:59Oh. I want to do a Rob over a beer
- 54:01so maybe next time in first episode.
- 54:03We're going to pick ostrich ourselves 1st and then have the
- 54:05beer on the ground. Then we'll be feeling a lot more
- 54:08comfortable. Well, that sounds good.
- 54:11A huge thank you to Rob T Lee. He is Chief AI Officer of the
- 54:17Sands Institute. And to David Bray, who is chair
- 54:22of the accelerator at the Stimson Center.
- 54:26Gentlemen, thank you so much. And a huge thank you to
- 54:29everybody who watched. Before you go again, subscribe
- 54:33to the CXO Talk newsletter. We'll send you updates.
- 54:37You are now part of our community and we want you join
- 54:42us. And we have great shows, amazing
- 54:44shows actually, And we'll see you again next time.
- 54:46Take care, everybody. Bye bye.