Latest / Expert Insights Podcast / #76. Exploring AI Safety and Organizational Privacy - (Sounil Yu, Knostic)
Transcript
- 0:03SPEAKER_01: Hello and welcome to the Expert Insights Podcast,
- 0:05SPEAKER_01: your go-to source for cybersecurity insights from industry experts.
- 0:09SPEAKER_01: I'm your host, Joel Wits.
- 0:14SPEAKER_01: Sunil Yu is the CTO of Gnostic.
- 0:17SPEAKER_01: In this conversation,
- 0:18SPEAKER_01: Sunil explores how we incorporate co-pilots into our work processes,
- 0:22SPEAKER_01: ensuring that they share the right information with the right people.
- 0:26He addresses what exactly need to know is and how to get the
- 0:29balance right with AI
- 0:31SPEAKER_01: guardrails,
- 0:32ensuring that productivity is genuinely enhanced without
- 0:35putting your organization
- 0:37SPEAKER_01: or your users at risk.
- 0:41SPEAKER_01: Thanks so much for taking the time to speak with us.
- 0:43SPEAKER_01: It'd be great to start with an introduction to yourself,
- 0:45SPEAKER_01: where you're from,
- 0:48SPEAKER_01: your job title,
- 0:48SPEAKER_01: your security background.
- 0:50SPEAKER_02: Yeah, so I'm Sunil Yu, and I'm the CTO at Gnostic.
- 0:53SPEAKER_02: And my background goes back 30 plus years in security.
- 0:56SPEAKER_02: Most recently, I ran security for a startup called Jupiter One.
- 1:00SPEAKER_02: Before that, I spent a year in venture capital.
- 1:03SPEAKER_02: Before that, I was the chief scientist at Bank of America.
- 1:06SPEAKER_02: And then before that,
- 1:07I spent several years as a consultant with a company called
- 1:10Blues Island Hamilton.
- 1:11SPEAKER_01: Right, okay.
- 1:12SPEAKER_01: Very big in the US government, right?
- 1:13SPEAKER_02: Yeah, we did a lot of interesting work that I can't speak about.
- 1:18SPEAKER_02: Yeah, okay.
- 1:19SPEAKER_02: But actually,
- 1:19SPEAKER_02: funny enough,
- 1:20I don't have to speak about it because it's already been leaked
- 1:23by certain
- 1:24SPEAKER_02: individuals.
- 1:26So what led you to Gnostic and what's exciting about what you
- 1:29guys are doing?
- 1:30SPEAKER_02: Yeah,
- 1:30SPEAKER_02: so what led us to Gnostic is actually,
- 1:32SPEAKER_02: I was trying to figure out how to solve a particular,
- 1:34SPEAKER_02: a real fundamental problem that we have in AI itself,
- 1:37SPEAKER_02: which is how do you understand how to benchmark,
- 1:40SPEAKER_02: or how do you define AI safety?
- 1:42SPEAKER_02: Safety is a,
- 1:43there's a lot of concerns around AI and how do we ensure that
- 1:46these systems are
- 1:46SPEAKER_02: built in a safe manner.
- 1:47SPEAKER_02: But what do you mean by safe?
- 1:48SPEAKER_02: Like there's no common universal definition.
- 1:51SPEAKER_02: Your definition may be different than China's,
- 1:52SPEAKER_02: which would be different than the US's or the Ukraine's or whatever.
- 1:55And so as I was trying to struggle to define that problem or to
- 1:59put my arms around
- 1:59SPEAKER_02: that problem,
- 2:00SPEAKER_02: I realized that there was an adjacent problem that's very similar,
- 2:03SPEAKER_02: which is how do you define what's appropriate within an organization
- 2:08SPEAKER_02: as it pertains to need to know.
- 2:10So need to know is a fundamental principle in security that
- 2:14undergirds all of access control.
- 2:16But the definition of need to know is not the same across
- 2:20different organizations.
- 2:21SPEAKER_02: And it turns out,
- 2:22SPEAKER_02: and I won't go into too much detail here,
- 2:23but there's an adjacency between the problem of defining what
- 2:27is safe and defining
- 2:29what are the right boundaries within an organization when it
- 2:31comes to what you
- 2:32SPEAKER_02: allow people to see or not see.
- 2:34SPEAKER_02: That
- 2:34SPEAKER_02: What I mean by that is when I talk about need to know,
- 2:37SPEAKER_02: in many organizations,
- 2:38SPEAKER_02: in the US at least,
- 2:39SPEAKER_02: you would say,
- 2:40SPEAKER_02: not everyone has a need to know for salary information.
- 2:42SPEAKER_02: So when people roll out a tool like Copilot inside their organization,
- 2:46SPEAKER_02: the number one question that is asked across the board is,
- 2:49SPEAKER_02: what is my boss's salary?
- 2:50SPEAKER_02: And unfortunately,
- 2:52SPEAKER_02: if you actually have access to that,
- 2:54SPEAKER_02: because it's hidden somewhere that you haven't found,
- 2:56SPEAKER_02: then Copilot comes back and says,
- 2:57SPEAKER_02: here's the salary information.
- 2:58SPEAKER_02: But I'm sorry,
- 2:59SPEAKER_02: you don't really actually have a need to know,
- 3:00SPEAKER_02: and Copilot doesn't understand that.
- 3:02SPEAKER_02: Okay, but what I just described is,
- 3:04SPEAKER_02: the culture of our country and our
- 3:08SPEAKER_02: organizations and corporate structures and so on and so forth.
- 3:11SPEAKER_02: But apparently if you go to some Nordic countries or places in India,
- 3:18SPEAKER_02: that is actually widely just known.
- 3:21And actually that's not only that,
- 3:22but in the US government,
- 3:23it's actually known.
- 3:24SPEAKER_02: That's transparent.
- 3:26So again,
- 3:26what do we mean by what's appropriate is different in every
- 3:29organization.
- 3:30SPEAKER_02: And so what we said is,
- 3:31SPEAKER_02: what we decided is,
- 3:32SPEAKER_02: it's a pretty,
- 3:33SPEAKER_02: understanding this basic principle of need to know,
- 3:36SPEAKER_02: is a core principle that we haven't actually written down anywhere.
- 3:40SPEAKER_02: It's not codified.
- 3:42And when we have large language models that are rolled out
- 3:45inside of these
- 3:46SPEAKER_02: different enterprises,
- 3:47SPEAKER_02: what rule set is it going to follow?
- 3:48SPEAKER_02: And if we haven't written it down, then it doesn't have that rule set.
- 3:51It does whatever it considers to be its best judgment,
- 3:53but what is that judgment?
- 3:55I would rather actually say,
- 3:56hey,
- 3:57large language model or copilot or whatever these tools are,
- 4:00SPEAKER_02: here are the boundaries for this organization.
- 4:02SPEAKER_02: Now that it's captured with its ability to adhere to those rules,
- 4:06SPEAKER_02: that's still in debate,
- 4:07but at least having the rules written down now gives us a basis
- 4:10to be able to say,
- 4:11let's try to get these systems to understand how to adhere to
- 4:16these rules and stick
- 4:17SPEAKER_02: to these rules when it comes to the information that it divulges.
- 4:21SPEAKER_01: Is it sort of training,
- 4:23SPEAKER_01: you know,
- 4:23you sort of roll this out to put in the safeguards and is it
- 4:25learning in real time
- 4:26SPEAKER_01: as well what some of the safeguards need to be in terms of,
- 4:29SPEAKER_01: I suppose there might be use cases that you don't even think about,
- 4:32SPEAKER_01: you know,
- 4:32SPEAKER_01: in terms of how users might engage with these AI tools.
- 4:34SPEAKER_02: Right.
- 4:35SPEAKER_02: So these safeguards,
- 4:35I'm not necessarily wanting the machines to dynamically figure
- 4:39out what these
- 4:39SPEAKER_02: safeguards are.
- 4:40SPEAKER_02: At the end of the day, there's still a human decision.
- 4:42SPEAKER_02: There's discretion that
- 4:45SPEAKER_02: humans have that machines don't have.
- 4:47And what we're really doing is using the opportunity for where
- 4:51the machine is not
- 4:52SPEAKER_02: sure,
- 4:52SPEAKER_02: okay,
- 4:53SPEAKER_02: or where it's ambiguous enough that it's worth asking the human to say,
- 4:57SPEAKER_02: hey,
- 4:58SPEAKER_02: use your discretion,
- 4:58SPEAKER_02: human,
- 4:59to tell us what the appropriate guardrail should be for this
- 5:02type of job function
- 5:04SPEAKER_02: and for this particular topic.
- 5:05SPEAKER_02: And with that decision,
- 5:07SPEAKER_02: now we start codifying that and saying,
- 5:09SPEAKER_02: within the culture of this organization,
- 5:10SPEAKER_02: these are the boundaries of how knowledge should be segmented inside the
- 5:14SPEAKER_02: organization.
- 5:15SPEAKER_02: Okay, now that organization's boundaries will change over time too.
- 5:19SPEAKER_02: Okay,
- 5:19SPEAKER_02: so again,
- 5:20SPEAKER_02: different organizations will have different boundaries,
- 5:21SPEAKER_02: but even within the same organization,
- 5:23SPEAKER_02: that will change over time as well.
- 5:25So what we're doing is the whole process of just keeping this
- 5:28update is a whole
- 5:29SPEAKER_02: effort in and of itself.
- 5:30SPEAKER_02: But the fundamental question is, does this need to even happen at all?
- 5:33SPEAKER_02: Is this a foundational need?
- 5:35SPEAKER_02: And we believe that it is a foundational need.
- 5:37SPEAKER_02: I love mental models.
- 5:38Mental models really help us just think through the problem
- 5:40space in a more systematic way.
- 5:42SPEAKER_02: And there's a mental model called the
- 5:45SPEAKER_02: D-I-K-W pyramid.
- 5:47SPEAKER_02: It stands for data, information, knowledge, and wisdom.
- 5:51And we're at the point where we are providing knowledge tools
- 5:54for knowledge workers.
- 5:55SPEAKER_02: So we've moved up this pyramid where now we're at the knowledge age.
- 5:59SPEAKER_02: We provide these knowledge tools.
- 6:00SPEAKER_02: What happens?
- 6:01SPEAKER_02: They start oversharing.
- 6:02SPEAKER_02: So our natural inclination is to say, well, let me go do data governance.
- 6:06SPEAKER_02: Surely the problem we have is a data governance problem.
- 6:09But the way that we actually apply those tools is really at
- 6:11these layers.
- 6:12SPEAKER_02: In other words,
- 6:13SPEAKER_02: we're trying to fix the problem at the data and information layer,
- 6:16SPEAKER_02: but yet we have a knowledge problem.
- 6:17SPEAKER_02: So now if I fix it at these layers,
- 6:19what happens is what we're doing is we're effectively squeezing
- 6:22this bottom part of
- 6:23SPEAKER_02: this pyramid.
- 6:23SPEAKER_02: And when I squeeze that bottom part, what I end up with is a dumb LLM.
- 6:27you end up with something that's less useful because you've
- 6:29just squeezed all the
- 6:31SPEAKER_02: useful information out of it.
- 6:32SPEAKER_02: But let's suppose it's perfect.
- 6:35SPEAKER_02: Even if it's perfect,
- 6:36SPEAKER_02: we still have a problem because if I can infer things,
- 6:39if I can infer layoffs from future building space and future
- 6:42equipment purchases,
- 6:43SPEAKER_02: I still have a problem.
- 6:44SPEAKER_02: Yeah.
- 6:44SPEAKER_02: So I have a knowledge level problem,
- 6:46SPEAKER_02: and if I tackle it at this layer,
- 6:49then I can restore the full set of content that the LLM can
- 6:52access and be able to
- 6:53tap the full set of knowledge that the organization really is
- 6:56trying to find as
- 6:56SPEAKER_02: well.
- 6:57SPEAKER_02: The question that we struggled with,
- 6:58SPEAKER_02: the question that we were working through is,
- 7:01SPEAKER_02: how do you control this?
- 7:02SPEAKER_02: Here we control it by looking at file permissions.
- 7:05SPEAKER_02: Here we try to control it by putting labels on it.
- 7:07SPEAKER_02: What is the mechanism for control up here?
- 7:10And the inescapable conclusion that we arrived at is we have to
- 7:14characterize need to know.
- 7:16SPEAKER_01: And that's what set us on the journey.
- 7:18SPEAKER_01: Yeah, fantastic.
- 7:19SPEAKER_01: And in terms of the bigger picture,
- 7:21SPEAKER_01: what people are asking the tool that you kind of need to know,
- 7:24SPEAKER_01: figuring out
- 7:25SPEAKER_01: I suppose the psychology of why they're asking those questions.
- 7:27SPEAKER_01: Is it malicious or is it more like you say, just curiosity?
- 7:31SPEAKER_01: And I'm curious how you're pitching it to businesses.
- 7:33Is it a security tool or is it HR or where does it sort of fit
- 7:36in that conversation?
- 7:37SPEAKER_02: An insider threat is certainly a concern,
- 7:39SPEAKER_02: but it turns out it's more than just the insider threat as a concern.
- 7:42SPEAKER_02: There's a lot of other like,
- 7:43SPEAKER_02: for example,
- 7:43SPEAKER_02: it's not an insider threat if I find out about harassment issues.
- 7:46SPEAKER_02: Right.
- 7:47SPEAKER_02: Right.
- 7:47SPEAKER_02: Right.
- 7:47It's just you don't want people to learn about who's been
- 7:50accused of sexual harassment.
- 7:51SPEAKER_02: That's not a traditional insider concern.
- 7:54SPEAKER_02: That's just like,
- 7:55SPEAKER_02: hey,
- 7:55that's we just kind of need to make sure that we don't leave
- 7:58that lying around or
- 8:00SPEAKER_02: on out of the financial results.
- 8:02SPEAKER_02: Or there are some things that are there are insider threats.
- 8:04SPEAKER_02: So let's say, for example, M&A information.
- 8:06You don't want people to have that because that could be used
- 8:09for trading or whatever.
- 8:10So there are traditional insider concerns,
- 8:11but there are just more just business harm concerns.
- 8:14SPEAKER_02: Here's an analogy that I've used.
- 8:15SPEAKER_02: You know, we care about our personal privacy, right?
- 8:18SPEAKER_02: And when we talk about our personal privacy,
- 8:19SPEAKER_02: it's usually like,
- 8:20SPEAKER_02: well,
- 8:21SPEAKER_02: our social security numbers or whatever else it is.
- 8:23But really what we are concerned about more about our privacy
- 8:26is not our data
- 8:27SPEAKER_02: privacy,
- 8:28SPEAKER_02: but it's a knowledge privacy.
- 8:29SPEAKER_02: Hey, what's your sexual orientation?
- 8:31SPEAKER_02: I'm sorry,
- 8:32SPEAKER_02: that's none of your business,
- 8:33SPEAKER_02: but might I be able to infer that from some of the tools that we have?
- 8:36SPEAKER_02: So there's that personal privacy part,
- 8:38SPEAKER_02: but then there's also this notion of knowledge,
- 8:40SPEAKER_02: the organizational privacy.
- 8:41SPEAKER_02: The organization, the corporation itself has this notion of privacy.
- 8:46And what we're trying to do is actually protect the privacy of
- 8:48the organization
- 8:49itself because there are certain parts of the organization that
- 8:52should be segmented
- 8:52off from other parts of the organization where they don't need
- 8:56to know about other
- 8:57SPEAKER_02: parts of what's happening here.
- 8:58SPEAKER_02: So that's what we're supposed to do.
- 8:59SPEAKER_02: And who typically owns the solution that you've found so far?
- 9:02SPEAKER_02: Yeah, so going back to the question of like, who's the buyer here?
- 9:05Again,
- 9:05security is a starting point,
- 9:07but really it's actually a business problem,
- 9:08right?
- 9:09SPEAKER_02: And that's where I was trying to get at.
- 9:10SPEAKER_02: It's the business who cares about this more than the security side,
- 9:15SPEAKER_02: but they care about it for two reasons.
- 9:17SPEAKER_02: One is a less compelling concern and the other one's more compelling.
- 9:19SPEAKER_02: The less compelling concern is the oversharing.
- 9:21SPEAKER_02: hey, we don't want this kind of stuff to get out to everyone.
- 9:24SPEAKER_02: But need to know can be expressed in two different ways of thinking.
- 9:27SPEAKER_02: One is need to know as a restrictive fashion.
- 9:29SPEAKER_02: You don't have a need to know, I can't share that with you.
- 9:31SPEAKER_02: But need to know can also be a permissive thing.
- 9:33You have a need to know for these things because that's your
- 9:37job function.
- 9:38SPEAKER_02: So let me get to things that you don't have access to
- 9:41SPEAKER_02: and get that to you.
- 9:42SPEAKER_02: So why do we want to roll out these AI tools?
- 9:45SPEAKER_02: The reason is, of course, for business productivity.
- 9:46SPEAKER_02: Number one reason is business productivity.
- 9:48SPEAKER_02: Knowledge tools for knowledge workers.
- 9:50How do you arm knowledge workers with,
- 9:52how do you equip knowledge workers?
- 9:54SPEAKER_02: You give them access to more knowledge.
- 9:55SPEAKER_02: Not constrain it, but actually give them more access.
- 9:58SPEAKER_02: What, if I understand your boundary, now I actually know what's in scope.
- 10:02And now let me get you the stuff that's in scope that you don't
- 10:04even know about.
- 10:05okay and that's truly transformative and i think in that view
- 10:09that that second view
- 10:10of let me get you things that you don't even know about the
- 10:13business side would say
- 10:15SPEAKER_02: this is why we roll out ai tools to begin with yeah we want that increase
- 10:19SPEAKER_02: productivity and
- 10:20if you don't know the boundaries are then uh either you don't
- 10:24you so we already
- 10:25live in that shrunken bottom part where we've already
- 10:28restricted access and again
- 10:30even that restricted access isn't perfect so you're going to
- 10:32overshare within that
- 10:33context because of the oversharing people want to squeeze it
- 10:35more but really that's
- 10:36not the right answer right how do you expand it so that
- 10:39includes things that you
- 10:39don't already have access to but you should yeah well how do
- 10:43you know what that is
- 10:43SPEAKER_02: it's your need to know boundary
- 10:47You're listening to the Expert Insights Podcast,
- 10:50home of leading cybersecurity conversations.
- 10:53Subscribe now to get insights from top cybersecurity leaders
- 10:56straight to your inbox.
- 10:58SPEAKER_01: How do you see the platform growing over like the next 12 months?
- 11:00SPEAKER_02: Sure.
- 11:01SPEAKER_02: Again,
- 11:01SPEAKER_02: with the foundation of understanding need to know,
- 11:03SPEAKER_02: we have a couple of different use cases.
- 11:05SPEAKER_02: The immediate use case is, let me help you find what's overshared.
- 11:07But there's a lot of other use cases once this foundation is
- 11:12established.
- 11:12SPEAKER_02: One,
- 11:13SPEAKER_02: it can be used for other platforms where you're using,
- 11:17SPEAKER_02: for example,
- 11:18SPEAKER_02: Notion or Slack or Salesforce.
- 11:20The same boundaries that you're trying to enforce exist in
- 11:25those same platforms as well.
- 11:27In other words,
- 11:27Nintendo can be violated in those or it can be permissive in
- 11:31those as well.
- 11:32SPEAKER_02: How do you enable that for all these other SaaS applications?
- 11:34SPEAKER_02: Second, there's, of course, agentic AI is like all the hotness right now.
- 11:37SPEAKER_02: But what boundaries are you creating for those as well?
- 11:40And what you'd want to do is to say,
- 11:41this agent is acting on behalf of this user.
- 11:44SPEAKER_02: What is that user's need to know boundary?
- 11:46SPEAKER_02: Right now, again, we have a one-size-fits-all boundary.
- 11:49SPEAKER_02: One-size-fits-all being nobody can ask about biological weapons.
- 11:52SPEAKER_02: Generally, I can understand why.
- 11:54SPEAKER_02: But inside of an organization, that doesn't make sense.
- 11:56SPEAKER_02: You can't say nobody should ask about employee misconduct.
- 11:59SPEAKER_02: Yeah.
- 12:00SPEAKER_02: Well, HR should be able to ask about that.
- 12:02SPEAKER_02: So you need to create these role or job specific boundaries.
- 12:05SPEAKER_02: And when it comes to these agents, we wanna do the same as well.
- 12:08SPEAKER_02: So we come up with a 80% correct set of rules around those boundaries.
- 12:13SPEAKER_02: Of course, we wanna get closer to 90, 95 out of the box.
- 12:17SPEAKER_02: for an organization.
- 12:18SPEAKER_02: So what we would do is over time,
- 12:20SPEAKER_02: as we have more customers where we can establish those boundaries,
- 12:24SPEAKER_02: we can then essentially say,
- 12:25SPEAKER_02: there's a cohort.
- 12:27SPEAKER_02: If you're in a particular cohort,
- 12:28SPEAKER_02: certain industries,
- 12:29SPEAKER_02: certain vertical,
- 12:30SPEAKER_02: certain size,
- 12:31SPEAKER_02: then your need to know boundaries will generally look like this.
- 12:33SPEAKER_02: And so as a starting point,
- 12:34SPEAKER_02: they would be able to get started really quickly and say,
- 12:37SPEAKER_02: okay,
- 12:37SPEAKER_02: great,
- 12:37SPEAKER_02: we'll accept yours as is,
- 12:39SPEAKER_02: and anything's on the margin,
- 12:40SPEAKER_02: we'll fix along the way.
- 12:42SPEAKER_02: But here's the thing that I think is more interesting.
- 12:44SPEAKER_02: It essentially creates this benchmark.
- 12:45OK,
- 12:46so your boundaries are generally,
- 12:48let's say you're a big financial services organization.
- 12:50SPEAKER_02: Let's say I'm Bank of America, which I was.
- 12:52SPEAKER_02: And with me is also Citibank and JP Morgan.
- 12:55SPEAKER_02: OK, but Wells Fargo is over here somewhere.
- 12:57SPEAKER_02: And it's not to say that that's
- 12:58wrong it's just to say that's unusual let's investigate further
- 13:02okay maybe they're
- 13:02doing something amazingly right by the way this is the exact
- 13:05same problem that we
- 13:06have with ai safety well who's to say this is right and this is
- 13:09wrong all i can say
- 13:10is facebook operates over here and the companies like facebook
- 13:14operate here
- 13:14SPEAKER_02: companies like amazon operate here companies like apple operate here okay
- 13:19SPEAKER_02: Again, which one's right, which one's wrong?
- 13:21SPEAKER_02: I'm not casting judgment.
- 13:22I'm just saying that if you're gonna be in the defense sector
- 13:26building autonomous
- 13:27SPEAKER_02: weapons,
- 13:28I would like for you to generally be in the party in the same
- 13:30cohort that treats
- 13:31SPEAKER_02: things like AI safety with a tremendous amount of concern and care.
- 13:35SPEAKER_02: And you're over here,
- 13:36SPEAKER_02: let's say with Apple,
- 13:37SPEAKER_02: let's say Apple is very conservative around how it uses AI.
- 13:41SPEAKER_02: First is, let's say, like a Facebook, which is a bit more loose.
- 13:44SPEAKER_02: You know,
- 13:44just if I saw a DOD entity or a military entity operating like
- 13:49a Facebook,
- 13:50SPEAKER_02: I'm not saying that's wrong.
- 13:51I'm just saying we may want to look into that and see what
- 13:55those practices are because...
- 13:57SPEAKER_02: that,
- 13:58SPEAKER_02: again,
- 13:58SPEAKER_02: it could be an advanced practice that we just didn't realize,
- 14:01SPEAKER_02: or it could be like,
- 14:02SPEAKER_02: okay,
- 14:02SPEAKER_02: we need to shut that down,
- 14:04SPEAKER_02: right?
- 14:04And so the same problem exists,
- 14:06whether it's AI safety or this concept of need to know.
- 14:09SPEAKER_02: What we end up with is cohorts,
- 14:11SPEAKER_02: benchmarks that allow us then,
- 14:13SPEAKER_02: one,
- 14:13SPEAKER_02: from a business standpoint,
- 14:14SPEAKER_02: be able to come out of the box with,
- 14:16SPEAKER_02: here's for you,
- 14:17we believe you're in this cohort with the need to know
- 14:20boundaries that we've
- 14:21SPEAKER_02: created.
- 14:21Let's see what your existing organization actually looks like
- 14:24and where the
- 14:24SPEAKER_02: violations are,
- 14:25SPEAKER_02: okay?
- 14:26SPEAKER_02: And that's the amazing thing, what we're doing.
- 14:28SPEAKER_02: Violations become so much easier to spot.
- 14:30Problematic issues become so much easier to spot because right
- 14:33now that's really
- 14:34hard to see because it's manifested at this machine level that
- 14:37we don't really
- 14:37SPEAKER_02: quite understand.
- 14:38SPEAKER_02: But when you start with this higher level understanding of need to know,
- 14:41SPEAKER_02: the violations become intuitively obvious.
- 14:43SPEAKER_02: Okay,
- 14:44so we create these cohorts and that allows us to build a note
- 14:47around just the
- 14:49SPEAKER_02: boundaries that we've created,
- 14:50but it also allows us to create these benchmarks that
- 14:53everyone's really wanting
- 14:54SPEAKER_02: anyway.
- 14:54Do you think we should be thinking of need to know as like a
- 14:56new category for want
- 14:57SPEAKER_01: of a better word in the cybersecurity?
- 14:58SPEAKER_01: You've got like DLP,
- 14:59you've got some of the more traditional ways that you would
- 15:02limit how users can
- 15:03SPEAKER_01: upload data or whatever.
- 15:04SPEAKER_01: Should need to know be thought of as a new way of doing that?
- 15:08SPEAKER_02: Well,
- 15:08SPEAKER_02: let me offer,
- 15:09SPEAKER_02: I eschew certain words,
- 15:11SPEAKER_02: but I'm going to use them just because it fits the mode,
- 15:14SPEAKER_02: right?
- 15:15We used to have firewalls that operated at layer three and
- 15:18layer four,
- 15:18okay,
- 15:19an OSI model.
- 15:20SPEAKER_02: They called them firewalls.
- 15:22SPEAKER_02: layer three and layer four.
- 15:23And then Palo Alto comes up with a next generation firewall
- 15:27that blocks at layer seven.
- 15:28SPEAKER_02: So if you're trying to block out layer seven,
- 15:30SPEAKER_02: it doesn't work so well at blocking at layer three and four.
- 15:33So we have current data governance tools that are trying to
- 15:36block at the data and
- 15:37SPEAKER_02: information layer,
- 15:38and they're not going to work for this new type of capability
- 15:41at the knowledge
- 15:42SPEAKER_02: level.
- 15:42SPEAKER_02: So is it next gen DLP?
- 15:45SPEAKER_02: or is it KLP, Knowledge Loss Prevention?
- 15:49SPEAKER_02: Again,
- 15:51I hate the word next generation because it doesn't really
- 15:53convey what the real
- 15:54SPEAKER_02: meaning is,
- 15:55SPEAKER_02: but you can see from this- The evolution,
- 15:56SPEAKER_02: yeah.
- 15:57Yeah,
- 15:57really,
- 15:57the point is that the model itself represents this next level,
- 16:01right?
- 16:01SPEAKER_02: It's next generation,
- 16:02SPEAKER_02: not because it's next generation,
- 16:03SPEAKER_02: but because now we've entered this new tier,
- 16:06SPEAKER_02: and that new tier needs a new set of controls.
- 16:08SPEAKER_02: And so anyway, is it a new category?
- 16:10Similar to,
- 16:11there's a quote which is,
- 16:12history doesn't repeat itself,
- 16:14but it rhymes.
- 16:14SPEAKER_02: So this is a rhyming aspect of it.
- 16:16SPEAKER_02: DLP, KLP.
- 16:18SPEAKER_02: Is it a new category?
- 16:19SPEAKER_02: Is a firewall and a next generation firewall a new category?
- 16:23SPEAKER_02: Or is it the same category, it just rhymes?
- 16:25SPEAKER_01: Okay, yeah, interesting.
- 16:26SPEAKER_02: What kind of excites you most about the platform and the journey?
- 16:29Well,
- 16:30so treating me to know as a first-class citizen turns out to
- 16:33solve a lot of problems.
- 16:35SPEAKER_02: It's so fundamental to security as a first principle,
- 16:38SPEAKER_02: and yet because it hasn't been written down,
- 16:40SPEAKER_02: because we thought it was too hard to write down,
- 16:42SPEAKER_02: we just never thought of doing it.
- 16:44But now these large English models and the AI systems enable us
- 16:47to actually write it down.
- 16:48SPEAKER_02: And once we write it down, it solves a lot of
- 16:51SPEAKER_02: challenges that we're going to have with AI.
- 16:53SPEAKER_02: But it actually undergirds,
- 16:54it solves a lot of other just core problems that we have in
- 16:57security overall.
- 16:58SPEAKER_02: Because once we have a need to know as a foundational principle,
- 17:01how we grant access or how we think about access or how we do
- 17:04access reviews,
- 17:05all that kind of stuff becomes so much easier because now it's
- 17:09something that we
- 17:09SPEAKER_02: understand in terms of the language of
- 17:12here's what this person's job function is and here's what they
- 17:15have a need to know for.
- 17:16The way that we currently manifest it is you have access to
- 17:19server XYZ ABC.
- 17:22SPEAKER_02: And you're like, why?
- 17:22SPEAKER_02: I don't know what that means.
- 17:24SPEAKER_02: And we had to derive it back.
- 17:25SPEAKER_02: But if we start from the beginning and say,
- 17:27let's use need to know as a foundational principle that we now
- 17:30have codified,
- 17:31SPEAKER_02: it creates a lot of new opportunities that we haven't even dreamed of.
- 17:34SPEAKER_02: But on the AI space,
- 17:37SPEAKER_02: it's exciting because now we have a way to anchor what these AI systems,
- 17:41SPEAKER_02: what rule sets we need them to adhere to.
- 17:43And that's exciting because then that opens up the whole
- 17:46opportunity for AI to do
- 17:47SPEAKER_02: more than just be a restrictive measure,
- 17:49SPEAKER_02: but rather that permissive measure I mentioned earlier.
- 17:51SPEAKER_01: Has the journey been fun, building the platform, building the team?
- 17:54You know,
- 17:55every startup has ups and downs,
- 17:57but it's been a steadily upward climb.
- 17:59SPEAKER_01: My final question, what's sort of one trend, one prediction?
- 18:02What have you got your eye on over the next kind of 12 months
- 18:05that our readers,
- 18:06SPEAKER_01: listeners should be aware of?
- 18:07SPEAKER_02: Well,
- 18:08SPEAKER_02: I hate to,
- 18:08SPEAKER_02: again,
- 18:09SPEAKER_02: say agentic AI is the hotness,
- 18:11SPEAKER_02: but let me break that down a little bit further,
- 18:13because I think it's important to recognize what is the real
- 18:16new problem that we're
- 18:17SPEAKER_02: going to face.
- 18:17And lo and behold,
- 18:18I have a mental model for that too,
- 18:20called the OODA loop.
- 18:21SPEAKER_02: which people may already be familiar with.
- 18:23SPEAKER_02: And the OODA loop has four major pieces, observe, orient, decide, act.
- 18:27SPEAKER_02: And what I've done is I've broken it out.
- 18:30SPEAKER_02: I've renamed it slightly.
- 18:31SPEAKER_02: So instead of observe, orient, I called it sensing and sense-making.
- 18:34SPEAKER_02: So you have sensing, sense-making, decision-making, and acting.
- 18:37And what we've seen is a progression where we let the machine
- 18:41do the sensing and
- 18:41SPEAKER_02: the acting,
- 18:42but we humans reserve the right to do the sense-making and
- 18:45decision-making.
- 18:46SPEAKER_02: Later on,
- 18:47SPEAKER_02: we let the machine do sense-making,
- 18:48SPEAKER_02: but we still reserve the right to do decision-making.
- 18:51Now we're at a stage where the machine's also going to do
- 18:54decision making as well.
- 18:55SPEAKER_02: And of course, one of the key questions is what could possibly go wrong?
- 18:59SPEAKER_02: There's a lot of things that could possibly go wrong,
- 19:00but the key question or key concern or key opportunity is
- 19:05what's the new way to
- 19:07SPEAKER_02: really arbitrate that?
- 19:08What are the right controls for decision making,
- 19:10autonomous decision making?
- 19:11SPEAKER_02: And that is...
- 19:14That is perhaps one of the more urgent concerns that we have
- 19:18because once a machine
- 19:20goes straight through this on its own without any human in the
- 19:24loop or on the loop,
- 19:25SPEAKER_02: again,
- 19:26SPEAKER_02: what could possibly go wrong?
- 19:27SPEAKER_02: There's a lot of things that could possibly go wrong.
- 19:29And we need to very quickly come up with the right
- 19:32decision-making guardrails.
- 19:33SPEAKER_02: And need to know is part of those decision-making guardrails.
- 19:36SPEAKER_02: Because what we're doing is we're giving a machine,
- 19:38SPEAKER_02: when you grant access to someone,
- 19:40SPEAKER_02: access to something,
- 19:41SPEAKER_02: these are the guardrails that are specific to that person.
- 19:44SPEAKER_02: And so there's a decision guardrail that we're creating.
- 19:47But there's going to be a lot of other decisions that these
- 19:49machines will make that
- 19:50SPEAKER_02: we don't even know how to express the language for.
- 19:52SPEAKER_02: And so we have to come up with the right language for those things.
- 19:55SPEAKER_02: And there's like no one that I know that's working that space.
- 19:58SPEAKER_02: Fascinating.
- 19:59SPEAKER_02: One of the controls that we expect is to have
- 20:02clear lines of authority and responsibility for that decision
- 20:07making action
- 20:08decision making that a system is going to do autonomously as
- 20:11well as like and the
- 20:12controls from here also apply to the next stage as well so you
- 20:15want controls to to
- 20:17have a stop button and a reverse button you want controls to
- 20:20ensure that the
- 20:21outcomes are deterministic and and all these sort of things so
- 20:24they're all additive
- 20:25controls and what i've done is just uh come up with a very
- 20:28systematic way of
- 20:29SPEAKER_02: thinking about what are those controls at each stage as i
- 20:33SPEAKER_02: consider these controls.
- 20:34SPEAKER_02: I look at the market, I'm like, they don't exist.
- 20:37And so there's going to be a huge opportunity space here when
- 20:40it comes to security.
- 20:41SPEAKER_02: I don't know if I would call it security, though.
- 20:43It's really around how do you properly gate decision-making,
- 20:47autonomous decision-making.
- 20:48SPEAKER_02: And that's not just a security problem.
- 20:50SPEAKER_02: That's, again, a business problem.
- 20:51SPEAKER_01: Thank you so much, guys.
- 20:52SPEAKER_01: Really appreciate it.
- 20:53SPEAKER_01: That was fantastic.
- 20:54SPEAKER_01: Thank you.
- 21:02This podcast was produced by Expert Insights,
- 21:05home of leading cybersecurity conversations.
- 21:08Subscribe now to get insights from top cybersecurity leaders
- 21:11straight into your inbox.
- 21:13Why not check out our weekly decrypted cybersecurity briefings
- 21:16and our Game
- 21:17SPEAKER_00: Changers conversations with innovative individuals.