Latest / The Road to Accountable AI / Harish Peri (Okta): When the Thing Accessing Your Systems Has a Brain

Transcript

Harish Peri (Okta): When the Thing Accessing Your Systems Has a Brain · The Road to Accountable AI

• 0:09
  Hi, I'm Kevin Werbeck, Professor of Legal Studies and Business Ethics at the Warden School of the University of Pennsylvania.
• 0:15
  For decades, I've studied emerging technologies from broadband to blockchain.
• 0:19
  Today, AI is promising to transform our world.
• 0:24
  But AI needs accountability, mechanisms to ensure it's developed and deployed in responsible, safe, and trustworthy ways.
• 0:32
  On this podcast, I speak with the experts leading the charge for accountable AI.
• 0:40
  When the thing accessing your finance or HR system isn't a person, but an agent with a brain that can be manipulated or just hallucinate, identity becomes a different kind of
• 0:51
  challenge.
• 0:52
  Harish Peri runs AI security at Okta, the access management and identity company.
• 0:58
  And he's seeing enterprises race to deploy AI agents that they can't fully see or control.
• 1:04
  In this episode, we talk shadow agents.
• 1:07
  kill switches, intent mismatches, standards, and more, as agentic security races to keep up with agentic deployment.
• 1:20
  Welcome Harish, thank you for joining me on the road to accountable AI.
• 1:24
  Fantastic.
• 1:25
  Thanks for the opportunity.
• 1:27
  Okta's a a company most people know about for digital identity and things like single sign on.
• 1:33
  How did the company get engaged in governance of AI agents?
• 1:37
  People know us for SSO, which is which is accurate in that that's where we started 17 years ago.
• 1:44
  But even before AI agents enter the picture, you know, our vision, bigger vision has always been getting the middle of people accessing technology.
• 1:53
  And as we we grew up in the age of the cloud where software moved to the cloud and became software as a service, and then identity and entitlements and access management really
• 2:03
  became a core pillar of security in general.
• 2:06
  So
• 2:07
  That's what that's what you know we've the journey, the bigger journey we've been on.
• 2:10
  And the new software is and will be agents, if you think about it.
• 2:15
  Like that's the new way in which people are going to interact with tech.
• 2:18
  And then that tech will interact with whatever systems are the back end.
• 2:22
  you know, the the the traditional front end, as you know, it might go away and be replaced completely by agents.
• 2:26
  And so it's really is an extension of our responsibility to say, if we were protecting people accessing tech, we're now gonna
• 2:34
  protect people accessing agents or agents accessing agents and subagents and all all the permutations in between.
• 2:42
  So it really is the natural next evolution of what we have always been up.
• 2:47
  At a broad level, what's different about thinking of that problem f in the agent context versus in the cloud context?
• 2:54
  I
• 2:55
  Yeah.
• 2:55
  The simple thing is um software doesn't have a brain.
• 2:59
  Agents have a brain.
• 3:01
  And I I use I say that maybe with a little tongue in cheek.
• 3:04
  You know, software, even robotic process automation, if you think about it, is just it's it's deterministic.
• 3:10
  It's if then, it's procedural.
• 3:12
  It does the thing that it's supposed to, give it an input, there's an output.
• 3:17
  With an agent, behind the scenes, it's really there is a large language model of some kind that is making a decision to say, should you know, a
• 3:25
  Based on what a user is telling me, should I actually allow or sorry, should what action should I take?
• 3:31
  There is a level of non-determinism that's involved in it.
• 3:34
  So the bar to make sure that an agent that has either been manipulated or has just hallucinated in general is not gonna the the bar for that to protect that agent from
• 3:45
  accessing the wrong resources or taking the wrong action is a lot higher.
• 3:49
  And so the non-determinism, while it leads to a lot of efficiencies.
• 3:54
  Um also leads to a higher degree of risk.
• 3:58
  And then what gave you the idea that from a business standpoint, focusing on these issues of aid and security was was something where there was an opportunity there?
• 4:06
  You know, it's it's it this is a one of the few cases where the market came to us overwhelmingly.
• 4:12
  Um it's it it at some point it would make for a good um case, actually, if in your world, where you know, pretty much maybe eight months ago, we started to see a massive influx of
• 4:26
  just questions from our our own customer base saying, Hey, we're starting to see the rise of agents in our workforce.
• 4:32
  We know that there is a security risk.
• 4:34
  Please help us.
• 4:36
  And so that was a perfect kind of conf convergence of our vision and what the market wanted to do as well.
• 4:44
  So it it's it's a it's a good time to be in the agent space overall.
• 4:49
  But at a general level, there you know, there's lots of cybersecurity companies and other companies that that have deep experts on the cyber side.
• 4:56
  How do the skill sets that you have in that that access management and identity world play into this context?
• 5:02
  Yeah.
• 5:02
  The agents are one of the f uh had the one unique characteristic that they they live on what's called the application layer, which means they're it's a kind of an app that could
• 5:13
  be running in the cloud, it could be running on your machine, it could be you know, built by a SaaS vendor that now has an agent.
• 5:20
  And you know, the brain I talked about being the large language model, that's one piece.
• 5:25
  The arms and legs, if you will, are the access hooks that the agent has into all the resources in your organization.
• 5:32
  You know, you hear a lot about MCP.
• 5:34
  That's one type of access.
• 5:35
  There's yeah, there are there are other ways in which agents can access the resources, but it's ultimately an access.
• 5:41
  And more importantly, it's an authorization problem.
• 5:44
  Is this agent allowed to execute this action or access this data at this moment in time on behalf of this user, given all available signals?
• 5:53
  So that very much lives in that application layer, more so than in the network layer or the endpoint.
• 6:01
  Piece or in broader cloud security.
• 6:03
  Now, the other components of the cyber stack are critical because they will, you know, they, they, when you put them all together, it it helps define a new type of zero trust
• 6:13
  where you can always trust but verify an agent's actions.
• 6:16
  You need signals from the other parts of the cybersecurity stack, but they all have to converge on identity and authorization.
• 6:23
  What does it mean then to focus on the application layer?
• 6:25
  What practically does that change in terms of how you design a solution?
• 6:30
  Well what what what I think that nothing well, some a lot of things change and then at the same time nothing's changed.
• 6:37
  Nothing changed a few few things change.
• 6:39
  What changes is if you're an organization know, creating your own agents, you do need to rethink through the the new types of security risks that can come into play.
• 6:51
  and it then it's it's there's a it's a combination of security security risks and also bad behavior.
• 6:57
  Security risks are things like prompt injection.
• 6:59
  Where, you know, like as a company, if you're exposing an agent to a, you know, to a to a user, likely there's going to be a chat interface.
• 7:08
  That chat interface is is essentially like opening up a new attack vector into your organization.
• 7:13
  And so prompt injection becomes a new kind of attack um vector, if you will.
• 7:18
  But also really thinking through are agents going to operate within their guardrails?
• 7:22
  Are they going to literally lose their minds or not?
• 7:24
  Is is also a very real consideration.
• 7:27
  So
• 7:28
  It changes the calculus of what security actually means if you're if you're building agents.
• 7:32
  Now, on the flip side, the things that don't change are still hygiene.
• 7:36
  You still need to ensure that, you know, at the at the identity level, things like API keys and access tokens don't just get, you know, publicly exposed.
• 7:46
  They're you know, because even if someone is not trying to manipulate an agent, an agent will try multiple execution methods in order to get his job done.
• 7:53
  And one of those could be let me just scan the file system and see if I can find an API key that's written in plain text somewhere and then proceed to delete a database.
• 8:02
  These have happened.
• 8:03
  These are real situations that have happened.
• 8:05
  And so data hygiene, access hy authorization hygiene, all are old issues.
• 8:11
  These have been around for 30 years.
• 8:12
  They become even more important now.
• 8:14
  So some things change and some other things are the same, but even more relevant than before.
• 8:22
  Do you have a working definition about what constitutes an agent?
• 8:25
  Does it matter?
• 8:26
  People use these terms in in very uh unclear ways often.
• 8:31
  Yeah, yeah.
• 8:32
  And I yes, that's that's a fair that's a fair point.
• 8:34
  I mean, agentic is a broader category of systems that can, you know, take actions or or make some decisions on their own that may not be a direct reflection of what they were
• 8:47
  prompted to do because they have some reasoning and some inference baked in there.
• 8:52
  but but the agents that I'm referring to are specifically um enterprise agents, these carry the biggest risk because you
• 9:00
  You know, if you have an agent that is accessing your finance system or your HR system or payments or ERP or your sensitive documents, and if it's an agent that multiple users are
• 9:10
  hitting at the same time, that creates a higher bar for security.
• 9:14
  I I would argue that if it's something simple where you're just asking, you know, Copilot or Google to say, summarize my email or move this data into PowerPoint and make it look
• 9:24
  good, that the risk is minimal because it's 100% within your envelope.
• 9:29
  It's when it leaves your envelope and has autonomy and also delegation that transcends who you are.
• 9:35
  That's where that's the type of agent that requires the highest level of uh risk, risk management.
• 9:42
  You mentioned customers coming to you looking for the these kinds of um security features.
• 9:47
  Uh what are you seeing out there in terms of both the maturity level for enterprises deploying agents and correspondingly the maturity level on governance?
• 9:58
  Yeah, it's it's weird.
• 10:01
  It's it's like a it's like a s simultaneous contradiction is what we're seeing.
• 10:06
  There is there are with without a doubt multiple, you know, many, many, many large organizations that are on the cusp of deploying agents into production.
• 10:15
  Uh at the same time, you know, the there's when you dig into it, it's like what that sometimes there's confusion around what exactly they're gonna be deploying to production.
• 10:25
  So it's a it's kind of like talking out of both sides of their mouth.
• 10:27
  But
• 10:28
  That's the truth about any major technology transformation, right?
• 10:31
  There's there's a pressure to do something, but then when you're actually at the point of doing that thing, it there's always some second guessing.
• 10:38
  There is one pattern though, which is the coding assistants are are everywhere.
• 10:44
  Cloud code, cursor, windsurf, um, codex.
• 10:47
  Those are those are massively proliferated throughout every enterprise right now.
• 10:52
  And we are we are seeing a uh common pattern of that being
• 10:57
  a risk in the sense of, you know, these assistants have access to your code base, your databases.
• 11:04
  It's quite sensitive stuff.
• 11:05
  And so it does require a much higher level of of authorization and access control.
• 11:11
  And you know, all of the companies that are behind those tools are working on guardrails and governance.
• 11:18
  So what what's the role that a company like Okta plays or or or the need for something above and beyond that?
• 11:24
  Well, it's funny.
• 11:25
  They are, but sometimes they're you know the guardrails that are being worked on are more in the realm of what I would call pure play AI.
• 11:33
  So it's to ensure the model doesn't lose its mind.
• 11:36
  Um the the reason you you know, the the reason there'll always be uh you know, n whether it's us or someone else, i i it has to do with independence and neutrality.
• 11:48
  And it has to do with the fact that you
• 11:51
  From a from a cybersecurity perspective, you do want to separate out the authorization control plane from where the agent is actually running.
• 11:59
  That gives you what's called a layered defense strategy because then it's multiple hops.
• 12:04
  Then an attacker has to break through in order to get to the gold mine.
• 12:07
  If you put it all in one place, you are concentrating a risk.
• 12:10
  If you spread it out, you're diversifying a risk.
• 12:12
  It's it's in in financial terms.
• 12:14
  And so that the the need for independent neutral control plane is is more important than ever.
• 12:20
  It's what
• 12:21
  We did for human identities when humans were connecting to SaaS, um, and still are, but it's what we are doing for agents connecting to resources as well.
• 12:29
  And so separating it out from the frontier models, from the Asian runtimes is is actually a a smart move from a cyber risk perspective.
• 12:40
  For someone who isn't as uh technically familiar, what exactly does an authorization control plane mean?
• 12:45
  Yeah, it's as it's as simple as what can my agents access and what can they do?
• 12:51
  That's that's as simple as that.
• 12:52
  So what can they access?
• 12:53
  Meaning, does every agent in your organization have a unique identity?
• 12:58
  And that matters because the regulators are gonna want auditing, logging, and and tracking of these agents at a transaction level.
• 13:07
  Um, you know, if you look at where the UAI act is going, Article 14 specifically calls for that, and that's gonna hit
• 13:15
  um all the companies in the EU very soon.
• 13:17
  Um so that level of like independent agent identity is critical.
• 13:22
  but the second piece is for a given agent, centrally controlling what they can access and the access levels is also, you know, important from a regulatory perspective.
• 13:30
  So that that's one part of authorization.
• 13:33
  But the second part, which is actually I think more exciting is what can they actually do at runtime?
• 13:38
  Which is this idea that when uh somebody prompts an agent to take to do something,
• 13:44
  One of the, you know, it's checking whether the intent of the user matches the actual action and checking for discrepancies is the future of conversation.
• 13:54
  That's what a control plane does.
• 13:55
  So for example, if you tell an agent, you know, clean this thing up, and what you meant was just move stuff into one one place, but they interpret it as delete, that's an intent
• 14:06
  mismatch.
• 14:07
  And so how do you have a control plane that can prevent that from happening before it's too late?
• 14:12
  So that th that's what I mean by an authorization control plane is really getting to a fine what we call fine grained, dynamic, context based authorization is the holy grail of
• 14:23
  where we're where we're pushing towards.
• 14:25
  Let's take that example, the the clean this up.
• 14:28
  That that's not uh a situation where the agent would necessarily wrong.
• 14:32
  It seems like that's something where the user is under specified.
• 14:36
  So it's it's it's it's essentially the similar to the alignment issues that the model developers are are having to wrestle with.
• 14:42
  So how would you address a situation like
• 14:45
  So a few different things.
• 14:46
  you would ensure that the one is at a at a coarse-grained level, the agent doesn't actually have broad-based rights into certain data sets.
• 14:59
  At a fine-grained level, you would introduce that context authorization I was referring to.
• 15:04
  And then you would be, as a builder of the agent, you would also introduce human in the loop to ensure that before some super sensitive transaction is executed, you are doing the
• 15:14
  final check of.
• 15:15
  Hi, I'm about to delete.
• 15:17
  Do you approve?
• 15:17
  It's like, wait, wait, wait, wait, hold on, stop, stop, stop.
• 15:20
  Now, human in the loop is tricky.
• 15:21
  If you introduce human loop everywhere, then it that you're not getting the benefits.
• 15:25
  Then you're just creating a really annoying intern that asks for permission for every action.
• 15:31
  Then it has no agency and therefore it is not an agent.
• 15:35
  So you know, but it's a combination of coarse grain, fine-grained authorization, and then also human loop to to secure these potentially dangerous transactions.
• 15:44
  And the answer to that problem you described for human in the loop is identify a a uh limited subset of situations where it's
• 15:50
  money, updating a stocks control database, um, changing an address, something that something that you know that a regulator might might complain about, you know, if you if
• 16:00
  you're in financial services or healthcare.
• 16:03
  but that's something that the designer of the agent needs to take into account up front.
• 16:09
  Back to the the first part that you were describing uh a minute ago.
• 16:13
  You talked about identity uh and and why that's important and and maybe required under under regulation.
• 16:18
  Uh how hard is it to identify agents?
• 16:22
  Ah, so it it's not actually it's not hard per se.
• 16:29
  It's more about how fast can you get there?
• 16:32
  So the in in you know in cyber terms, we talk about it as being on the edge, living on the various edges.
• 16:39
  So let me give an example.
• 16:41
  One way in which rogue agents creep into an organization is somebody just goes and on their work machine, they download clawed code or cursor.
• 16:53
  So now the question is how do you detect that?
• 16:55
  There are ways of detecting it.
• 16:56
  There are very, very robust endpoint, you know, softwares like CrowdStrike and Palo Alto and Janf for for the Mac universe.
• 17:07
  And so, you know, it it it's part of what we're look doing is integrating with those guys to say you can you literally live at the OS level.
• 17:15
  So you should be able to detect if somebody installed clawed code, that's been that's unauthorized.
• 17:20
  And so you flag that, you give it an identity, and then you ensure that you bring it under control.
• 17:24
  That's one edge is the endpoint as we call it.
• 17:27
  The other edge is the network.
• 17:28
  So there are, you know, vendors like Zscaler can that that literally live at the the the secure edge.
• 17:35
  They can sniff all the traffic as it's leaving going in and out of the company.
• 17:39
  And so there's patterns that can tell you if there's a genetic traffic or not.
• 17:43
  but again, once you detect it, what do you do about it?
• 17:46
  The key is then you have to give it an identity, right?
• 17:48
  It's like you find a rogue, then you register them in the system.
• 17:50
  That's kind of what the the whole another edge is the browser, where if somebody goes and uses their credit card to get access to a some SaaS vendor that's cre that gives them an
• 18:00
  agent, you can put a plugin in the browser to sniff that out and then register that.
• 18:06
  So the it's a multifaceted approach, but it's a little bit like law enforcement.
• 18:11
  Like you have to live at all the edges.
• 18:13
  Yeah.
• 18:13
  And then you gotta catch the crew, you know, catch the rogues.
• 18:16
  and bring into the system.
• 18:18
  That's uh that's so i it it it is actually a collaboration between all the cyber vendors in in the in the to to make this work.
• 18:27
  Do you find that organizations are aware of this?
• 18:30
  That this is kind of the shadow agent problem?
• 18:32
  And w what do they find when they they actually turn this on and they they see everything?
• 18:36
  So it's interesting.
• 18:37
  100% awareness that there is shadow AI running around.
• 18:42
  uh but when you actually double click into it, what you realize is uh 20% of agents carry about 80% of the risk.
• 18:51
  And so the question is you can discover every agent that's out there, but bringing those agents under control may get you no benefit.
• 18:59
  What mat that there's a certain set of agents, it and these are like the multiplayer enterprise agents, the ones that are
• 19:05
  Or the ones that are accessing sensitive resources.
• 19:08
  Discovering those and controlling those, even though they're small in number, they're big in risk.
• 19:13
  And the blast radius, as we say in cyber, in cyber to speak, is very high.
• 19:17
  And so the this conversation is shifting from find everything to find the very dangerous ones.
• 19:24
  And those need to be brought under control.
• 19:25
  It's it's quality over quantity of discovery.
• 19:29
  you know, so what about the problem that that agents can spawn other agents?
• 19:33
  How how do organizations maintain that identity in that environment?
• 19:37
  The notes from the field, if you will, are that's a that is a really good theoretical problem right now.
• 19:43
  Uh, but at scale, we don't see that happening yet.
• 19:48
  Now, the way to fix it though is if an agent is spawns a few ephemeral agents that are short-lived just to get some, you know, some specific tasks done.
• 19:59
  Uh if if that is known behavior, then you can design it where.
• 20:06
  The each type of subagent gets its own identity and there's the canonical representation has an identity.
• 20:10
  So you know that it is it is allowed.
• 20:13
  But let's say that that you you want the agent to just run free.
• 20:17
  Then what's important is you have authorization, where even if a subagent is spawned, when it's about to take an action, you have the final guardrail that says, excuse me, not
• 20:26
  allowed.
• 20:27
  And you log that.
• 20:29
  Right.
• 20:29
  So then the regulators are happy.
• 20:30
  So authorization, that's why.
• 20:34
  We are so gung-ho on authorization, it becomes very critical.
• 20:36
  Um, especially for the dynamic agents or the ephemeral agents.
• 20:41
  Um, sub agents that are known, that are part of the design pattern, that's easier.
• 20:46
  We just that's just A to A agent to agent as it's called.
• 20:50
  It's pretty, pretty straightforward.
• 20:52
  Right.
• 20:53
  What what's the you know biggest risk area that you're actually seeing out there i in organizations that they are
• 20:59
  It's the local, it's the desk, it's the laptop agents, the the coding assistants.
• 21:05
  That is the you know the the most clear and present risk right now because they are the the level of access they have and the power that they have is incredibly high.
• 21:18
  So that that needs to be detected and shut down or or detected and controlled right away.
• 21:24
  We don't want to shut it down.
• 21:25
  They're fantastic tools.
• 21:27
  You know, we're we're big
• 21:28
  fans of all the the coding assistants, but they have to be controlled properly.
• 21:33
  Otherwise there could be unintended behavior.
• 21:37
  And at at this point are those risks more in the the domain of the, you know, this accidentally deleted my production database or uh external kinds of threats like prompt
• 21:46
  injection?
• 21:47
  No, they can be prom projected because remember, these coding assistants are not just for developers.
• 21:52
  They're essentially giving you the power of the frontier model in a command line interface.
• 21:57
  And so, you know, you can you can imagine a world where somebody, you know, is is a victim of a phishing attack or some kind of social engineering where they accidentally, you know,
• 22:07
  uh obtain a markdown file that contains malicious instructions in there.
• 22:13
  Right.
• 22:13
  because it's all text.
• 22:14
  So
• 22:15
  You know, you you accidentally, you know, uh put in the wrong MD file into a skill that has been poisoned.
• 22:23
  So that's model poisoning, you know?
• 22:25
  Um, that is a very real risk.
• 22:29
  And it's not just in the realm of developers.
• 22:31
  Civilians that are using these frontier models this on your on your terminal are s are probably more susceptible to these risks.
• 22:41
  You mentioned a couple of standards already, like MCP and and A2A.
• 22:45
  D do we have the the standards that we need fully in this environment or are there missing pieces?
• 22:51
  No, no, no.
• 22:52
  Um MCP is is, you know, it's a really good wrapper on APIs and it allows any owner of resources to expose their tool calls, as they're as they're now called, in a in a
• 23:05
  standardized way.
• 23:06
  Great.
• 23:08
  A to A being another one for you know that helps with agent discovery and showing the agent card and all that's fine.
• 23:14
  But from a security perspective, there is there's still room for
• 23:21
  Things like universal agent identity.
• 23:23
  So in a in a future where every organization is creating their own agents and they're crossing trust boundaries, how does an what is an agent's passport?
• 23:31
  Or like every website has a unique IP that's resolved by the DNS authority.
• 23:37
  What is a agent's DNS resolution?
• 23:39
  Right.
• 23:39
  So there's that is a one standard.
• 23:42
  The other one is ah
• 23:45
  You know, if if you say if you truly want a uh agentic mesh where every organization's agents talk to every other organization's agents, um, identity has to transcend the
• 23:56
  identity issued by a single identity provider like us.
• 23:59
  You need a way for an agent to carry a cryptographically provable device or user-bound identity with what's called D Dpop demonstrated proof of possession, to say that you are
• 24:11
  actually acting on behalf of someone that crosses trust boundaries.
• 24:15
  So there are certain standards.
• 24:16
  Now now I'm gonna get very nerdy.
• 24:17
  There are certain standards like Spiffy, Inspire, or NTLS that you know, that that that can start to do that, but they need more widespread adoption.
• 24:27
  So we have by no means landed on the full set of standards that we need.
• 24:31
  It's it's still a work in progress.
• 24:33
  Yeah.
• 24:34
  Do you see that that universal Asian identity being a realistic possibility?
• 24:38
  Is it's it's partly a technical issue, but partly it's it's a just a very hard coordination issue to get everyone to agree.
• 24:46
  Yeah, yeah.
• 24:47
  Standards are hard.
• 24:48
  Standards um well, standards are easy to define and hard to adopt.
• 24:56
  and and we we are very familiar with that.
• 24:58
  We as a company, our our DNA is all about find a problem, solve it internally, figure out the common pattern, bring the others along, make a standard, then bring the industry
• 25:08
  along.
• 25:09
  Now, the last two steps take a lot of time.
• 25:12
  there's a standard that really led to the growth of identity called SAML.
• 25:15
  And SAML took many, many years to adopt.
• 25:18
  And sometimes I think we ourselves forget that it took us, you know, 10 years to get here.
• 25:23
  So we can push all the standards we want in in AI.
• 25:25
  And there's a right thing to do, but it takes time and and adoption and you know, um with AI, what's interesting is the the there's such a tremendous pressure to realize ROI and
• 25:39
  and positive outcomes that people are taking shortcuts.
• 25:44
  And one of the shortcuts is don't bother with the standard, just get it done somehow.
• 25:52
  Yeah, although it's interesting, you know, with uh not entirely the same thing, but with MCP, Anthropic put it out there and suddenly all their competitors adopted it and it's
• 26:00
  become fairly ubiquitous, at for those tool calls.
• 26:04
  Yeah.
• 26:05
  And and because it was a blocker to the core the the most core value that agents would provide, the security part of it will take longer.
• 26:14
  And this is always the the tension between innovation and security.
• 26:18
  Right.
• 26:18
  Like you want to innovate, but then as you innovate, you might introduce risks.
• 26:22
  But you know, developers will like, or whatever, don't bother me with all this.
• 26:25
  I just want to get this cool agent out there.
• 26:26
  Mm-hmm.
• 26:27
  You know, security guys get on our way.
• 26:29
  Kinda.
• 26:30
  So there's there's always that tension.
• 26:31
  Yeah.
• 26:33
  you mentioned earlier the the EU AI Act and some of the requirements that it's going to put into place.
• 26:37
  more generally, how do you see regulation impacting on these developments you've been talking about?
• 26:44
  Yeah.
• 26:44
  The the the biggest thing is gonna be attribution, um, or or or traceability, right?
• 26:51
  Or accountability.
• 26:51
  Come up with, you know, whatever big word you want.
• 26:54
  Is the idea that can you prove that an action that was taken by an agent within a company was on behalf of a user, if so, which user?
• 27:03
  And if so, why?
• 27:05
  And if it did happen, did you have all the right controls in place to prevent a potentially bad action?
• 27:10
  Like all proving all that is is gonna
• 27:12
  I I see that as being a big cottage industry for the auditors.
• 27:18
  and and the way you do that is the only system that can do that is the one that's in the what we call inline.
• 27:25
  It's inline in the flow from user to agent to resource, which is identity.
• 27:29
  So that so the the that's the trend that we one big trend that we expect to see from a regs perspective.
• 27:35
  The other one is kill switch, is this idea that can you halt this agent's activities instantaneously if breach is detected?
• 27:44
  Which also requires a central authorization control plane.
• 27:47
  Because then you have one it's the faucet.
• 27:49
  You just turn off the mains at the in the basement of the house and that agent is is shut off.
• 27:55
  Um, a lot of that though assumes that there is something like an audit requirement and in financial services we've got this whole infrastructure and you can't you know you can't
• 28:03
  file your quarterly numbers without getting them audited.
• 28:06
  Um, are we going to go to that point where there's mandatory audit requirements are gonna have to pull enterprises to to put all these things into place?
• 28:14
  Yeah.
• 28:14
  Like the way you normally the way you see it is um like MIST or one of the the government authorities will usually put out their position on it.
• 28:22
  And so we're actually working with them to formalize kind of what is the what are the right architectural considerations for agentic security.
• 28:32
  Those will then be adopted into states state specific and federal regs.
• 28:37
  Um and it might even, you know, it might end up being based on what the EU is doing.
• 28:42
  Because the EU is always
• 28:44
  at the forefront with regs.
• 28:45
  And so you can kinda see that in f you know, flowing into maybe SOX or NYDFS two hundred or wherever they are, um and and influencing those.
• 28:56
  But I think it's it starts with the with the definition of the right even frameworks and standards, which NIST plays a big role in.
• 29:05
  Mm-hmm.
• 29:05
  So one thing I want to ask you about is in the US, we don't have a lot of uh certainly federal AI regulation, but we do have a lot of litigation in the US.
• 29:14
  Uh and to what extent i is it possible that the need to attribute when things go wrong and someone is suing might drive companies to deploy some of these capabilities?
• 29:27
  Yeah.
• 29:28
  I I think I think I mean, without even being as I guess morbid as getting sued, uh we are we are definitely seeing that boards are asking their CEOs and and chief security officers
• 29:44
  to put agentic guard rules in place now because they're almost looking at it as an uh looking at it as an insurance policy.
• 29:52
  Now I I can't speak to how cyber insurance rates are changing and those are, you know, that wall is changing, but um, I don't think we have to wait for something legal to
• 30:02
  happen.
• 30:02
  Like this is this is a mandate from boards right now to pretty much, you know, for in the Fortune 500 to all of the like from I would I guarantee you this is a this is a topic in
• 30:12
  every security review that that that's happening in all those firms.
• 30:17
  So overall, how do you see this market evolving?
• 30:20
  You know, it'll follow some usual ebbs and flows of and it will it will stabilize.
• 30:25
  not not saying Gartner hype cycle or or pick pick your pick your sigmoid type graph, you know, for for this.
• 30:33
  Um I do predict though that this this can be bigger than uh cloud and and s and SaaS um overall, just because the the productivity gains are are phenomenal and then they
• 30:48
  compound.
• 30:49
  Because as they get better, they can make themselves better.
• 30:52
  And there is that whole like the true generative, you know, agents.
• 30:57
  so the the what that means is the pie overall of the identified economy is huge, which means all the sub players will all each of their pies will grow as well.
• 31:07
  And in particular with regard to the security piece of it, how does that uh world look to you as it goes forward?
• 31:14
  I think the I think that the the end a lot of cybersecurity is based on the some this thing called zero trust, which is this idea that you know you need to trust but verify at
• 31:25
  every at every um layer of the stack and then every system to system.
• 31:30
  But it was based on humans attacking humans.
• 31:33
  It was never based on humans delegating to agents that might take do crazy things or agents attacking agents.
• 31:41
  And so the the
• 31:42
  I think there's going to be a big toss-up for who's going to own the future of Zero Trust.
• 31:47
  The, the, you know, the signals are the same.
• 31:50
  They're coming from devices, from endpoints, from cloud management, from browsers.
• 31:55
  But who processes the signals, who makes the decision of increasing, you know, authorization guard rules or whatever in flight, that's going to change.
• 32:07
  So I think it's going to be a big toss-up.
• 32:08
  Um, and I do think there's there's going to be a
• 32:11
  kind of a a redefinition of of what what is zero trust architecture for agent tech look like?
• 32:18
  Harish, thank you so much.
• 32:19
  Really been a pleasure being with you.
• 32:20
  Yes, likewise, thank you so much.
• 32:24
  This has been the Road to Accountable AI.
• 32:26
  If you like what you're hearing, please give us a good review and check out my Substack for more insights on AI accountability.
• 32:32
  Thank you for listening.
• 32:37
  If you want to go deeper on AI governance, trust, and responsibility with me and other distinguished faculty of the world's business school, sign up for the next cohort of
• 32:46
  Wharton's Strategies for Accountable AI Online Executive Education Program.
• 32:51
  Featuring live interaction with faculty, expert interviews, and custom-designed asynchronous content.
• 32:57
  Join fellow business leaders to learn valuable skills you can put to work in your organization.
• 33:02
  Visit execed.warden.upen.edu slash acai for full details.
• 33:10
  I hope to see you there.