Latest / Anaiya Algorithm / Governance Engineering: From the CIA to the Boardroom
Transcript
- 0:00SPEAKER_01: Every lawyer, no matter what their practice group, basically does six or seven things over and over and over again.
- 0:08SPEAKER_01: And at least 80%, and I'd be nice there, but between 80 and 90% of what a lawyer does today is much better done by AI.
- 0:18SPEAKER_01: Oh wow.
- 0:19SPEAKER_01: And when you're thinking about bringing AI into your company, you gotta think Iron Man, not the Terminator, right?
- 0:27SPEAKER_01: So you're not gonna unplug a person and plug in a bot and everything's fine, right?
- 0:33SPEAKER_01: Um it is much more like uh like a lawyer, where eight out of ten of my tasks are better done by AI.
- 0:41SPEAKER_01: So let's let the I AI do that thing with the appropriate controls and the appropriate QC.
- 0:49SPEAKER_01: Um, but then the human part of what being a lawyer is all about is so essential.
- 1:18SPEAKER_00: I'm your host, Madeline Amagashti.
- 1:22SPEAKER_00: Season two is about proof of a promise.
- 1:26SPEAKER_00: Conversations with people who are not theorizing about responsible AI, but building it, governing it, and being held accountable for it.
- 1:39SPEAKER_00: Today's guest has one of the most extraordinary vantage points in the AI governance conversation.
- 1:47SPEAKER_00: He was recruited by the CIA in 1992 before the World Wide Web was public to help build one, to help build one of the earliest data analytics programs in history.
- 2:01SPEAKER_00: He went on to become a partner at DLA Piper, one of the world's largest law firms, where he was chief data scientist and global AI practice chair, representing the vast majority of the Fortune 100.
- 2:18SPEAKER_00: He represented open AI and Anthropic.
- 2:22SPEAKER_00: He sat in the White House briefings, he was in the room where the EU AI Act was being negotiated.
- 2:31SPEAKER_00: And then at 56, he walked away from all of it.
- 2:36SPEAKER_00: Not because he lost faith in AI, but because he decided the most important thing he could do was build a law firm that is also a technology company, one that doesn't just advise on AI governance but engineers it into the systems themselves.
- 2:56SPEAKER_00: Bennett Borden is the founder of Clarion AI Partners, and his argument is simple, and I think it is one of the most important ideas in AI right now.
- 3:08SPEAKER_00: AI is completely controllable if you do it right.
- 3:13SPEAKER_00: Bennett, welcome to Annaya algorithm.
- 3:17SPEAKER_01: Thank you so much, Magna.
- 3:18SPEAKER_01: I really appreciate being here.
- 3:20SPEAKER_00: I'm extremely delighted to have you here, and I'm excited for today's conversation.
- 3:25SPEAKER_00: So, Bennett, let's start at the beginning because I don't think anyone listening will have heard an AI governance story that starts quite like yours.
- 3:35SPEAKER_00: 1992.
- 3:36SPEAKER_00: The CIA comes to you whilst you're studying history and philosophy in England.
- 3:43SPEAKER_00: What did they actually want and why did they think you were the right person?
- 3:48SPEAKER_01: Yeah, it certainly surprised me because, you know, as you said, I what I was studying was really turning points in history, um, but especially how technology, um technological revolutions had affected world society.
- 4:03SPEAKER_01: And so everything from, you know, the wheel to the saddle to gunpowder to all the way up through the computer age really was focused on the, you know, the history and and philosophical points, but also sociology and anthropology and archaeology and you know, why do how do societies decide to put themselves together?
- 4:26SPEAKER_01: And what is it about the characteristics of a society that make it kind of go one way or the other?
- 4:31SPEAKER_01: And so when I was pushed by the agency and they said, hey, we have this new data analytics shop we want you to be a part of, I'm like, you gotta have the wrong guy.
- 4:40SPEAKER_01: Like, I'm I'm I don't really do data.
- 4:43SPEAKER_01: I'm not a huge fan of math.
- 4:45SPEAKER_01: I truly hate statistics, and I failed calculus.
- 4:48SPEAKER_01: And they they said, Oh, we know all of that.
- 4:50SPEAKER_01: That's not why we want you.
- 4:52SPEAKER_01: And so I'm like, Well, why do you want me?
- 4:55SPEAKER_01: So it turns out it was very interesting.
- 4:57SPEAKER_01: So as you mentioned, in the early 90s, the world wide web and email was just starting to emerge from academia into the real world, and and cell phones were just coming on the market.
- 5:10SPEAKER_01: And so we were creating as a species this very personalized new kind of data trail, right?
- 5:20SPEAKER_01: This little contrail that we were putting off.
- 5:23SPEAKER_01: And the agency wanted to know what we could learn about societies, about people, about groups of people based on this new kind of digital trail.
- 5:32SPEAKER_01: Um, and they rightfully saw that this trail was only going to get more voluminous and more varied as time went by.
- 5:39SPEAKER_01: And so, you know, and this was at a time when this was a very, very new idea.
- 5:44SPEAKER_01: So today, every social media company, every app, every everything basically tracks everything you do and makes a lot of inferences about you.
- 5:53SPEAKER_01: But at the time, it was really cool cutting-edge science.
- 5:58SPEAKER_01: And so our our goal was can you identify people whose interests align or don't align with you the US?
- 6:08SPEAKER_01: And then can we understand their behavior?
- 6:12SPEAKER_01: Can we predict their behavior?
- 6:13SPEAKER_01: And then, of course, can we influence their behavior?
- 6:15SPEAKER_01: And so, really terribly interesting things to learn.
- 6:18SPEAKER_01: And that's where I kind of became involved in data.
- 6:22SPEAKER_00: Yeah.
- 6:23SPEAKER_01: Quite accidentally.
- 6:24SPEAKER_01: I didn't set out to become a data scientist.
- 6:27SPEAKER_01: But it also, my entire kind of viewpoint on data is that it is a reflection of humanity, right?
- 6:35SPEAKER_01: It's how we go about being humans in all cultures across the world.
- 6:39SPEAKER_01: And so I've always kind of seen data as almost like this sacred trust that we have about people, that even though this data is spewing off about us, it's still ours and it still reveals some of the most precious elements of our lives.
- 6:56SPEAKER_01: And so about eight years into the agency, so in 2000, I decided I'd always wanted to go to law school.
- 7:02SPEAKER_01: So I went on to Georgetown Law and then went and got a graduate degree in data analytics from NYU because the technology was evolving so quickly.
- 7:13SPEAKER_01: So I I worked in kind of what they call big law, right?
- 7:17SPEAKER_01: And one of the top 10 firms my whole career, kind of at that intersection of data, law, and technology across every industry you can imagine.
- 7:28SPEAKER_00: So that's eight years of working with the best minds in data and analytics data analytics.
- 7:35SPEAKER_00: So I could imagine that looking now at what every social media company does today that you're you were doing in 1992, that was cutting edge size back then.
- 7:47SPEAKER_00: Yeah.
- 7:48SPEAKER_00: And you were essentially building what became the blueprint for modern digital intelligence before most people knew.
- 7:58SPEAKER_01: And even when you know when I went to work and in law, so this was 2003, right after law school, you know, it was really the era of social media and kind of the attention economy.
- 8:11SPEAKER_00: Yeah.
- 8:12SPEAKER_01: Where trying to keep um a customer's attention on your screen as long as you could, um, that's really where you made all your money.
- 8:18SPEAKER_01: And nobody really thought at the time that there would be any kind of negative effects to this, right?
- 8:25SPEAKER_01: It was just in the early days of Facebook, you know, before there was even the others Instagram and X and TikTok.
- 8:31SPEAKER_01: And it was cool, right?
- 8:33SPEAKER_01: You could you could stay in touch with your friends, and you know, it had this.
- 8:37SPEAKER_01: Um, there was quite a lot of huge positivity around it at the time.
- 8:42SPEAKER_00: Yeah.
- 8:43SPEAKER_01: And it wasn't until much later, a generation later, that we started to understand the really damaging effects that social media can have and or the attention economy generally can have.
- 8:57SPEAKER_01: And so it's interesting to see how that's been playing out now.
- 9:00SPEAKER_01: Yeah.
- 9:00SPEAKER_01: That we have this generative AI.
- 9:05SPEAKER_01: This technology is truly almost miraculous.
- 9:09SPEAKER_01: It is the most transformative, powerful, and disruptive technology since at least electricity.
- 9:17SPEAKER_00: Yeah.
- 9:19SPEAKER_01: Way more than the internet, way more than computers.
- 9:22SPEAKER_01: Like this is this is something actually very new in the world.
- 9:25SPEAKER_01: And man, trying to keep our hands around it is a challenge, but something that's absolutely essential.
- 9:34SPEAKER_00: Absolutely.
- 9:34SPEAKER_00: I love the fact that you've drawn our attention to the that context of understanding what data could do to human behavior long before most organizations even had a I don't know, a data strategy.
- 9:50SPEAKER_01: Yeah.
- 9:51SPEAKER_00: And you were chief data scientist and global AI practice chair at DLA Piper, moving on from the CIA, obviously one of the biggest law firms in the world.
- 10:01SPEAKER_00: So tell us a bit about the study you ran on what AI could do to the practice of law.
- 10:08SPEAKER_01: It was wonderful, right?
- 10:09SPEAKER_01: Like I had worked my way up.
- 10:11SPEAKER_01: This is the pinnacle of my career.
- 10:13SPEAKER_01: Um and I was there before Gen AI, mostly what I worked on was algorithmic fairness.
- 10:21SPEAKER_01: And so, you know, there's algorithms, good old-fashioned, what they call classical AI now, you know, they work pretty much the same.
- 10:28SPEAKER_01: They're they're lines of code that kind of have if, then, if this, then this, then, if, you know, and very controllable.
- 10:36SPEAKER_01: And the whole point of it was to group people into groups that are similar enough that you could take an action on them.
- 10:44SPEAKER_01: So, like when somebody applies for a mortgage or a credit card or a loan, you're looking for similar credit risk.
- 10:52SPEAKER_01: Or a job, you're looking for job fit, or you know, social media, what are you going to click on and advertising, what's going to entice you?
- 10:58SPEAKER_01: Yeah.
- 10:59SPEAKER_01: And that was, you know, what my job was is to help companies design those and build those correctly, but really keep them compliant with the law.
- 11:09SPEAKER_01: All right.
- 11:09SPEAKER_01: So then Jenai came out in the fall of 22 and the whole world changed.
- 11:14SPEAKER_01: My because this was a vastly different technology.
- 11:19SPEAKER_01: And even in the early days, you know, we didn't really think we're like, oh, this is cute, you know, it was like I can talk to it and it'll talk back.
- 11:28SPEAKER_01: And, you know, the early days when there was a lot of hallucinations and a lot of, right?
- 11:33SPEAKER_01: It wasn't as good as it is now.
- 11:34SPEAKER_01: And but even then, I remember the very first time I started to really look into this, just like goosebump kind of like this is going to change everything in the world.
- 11:47SPEAKER_01: So as you said, we got to it represent OpenAI early on in Anthropic, and it already represented Google and Microsoft and Amazon for decades.
- 11:57SPEAKER_01: And so in January, so like three months after Gen AI came out, the head of the firm came to me and said, Um, hey, what do you think this Gen AI thing is going to have on the effect it's gonna have on the practice of law?
- 12:10SPEAKER_01: And and so I'm like, that's that's a pretty interesting question.
- 12:13SPEAKER_01: So, you know, luckily we had a great data set, right?
- 12:17SPEAKER_01: Because lawyers write down what we do every tenth of an hour, the buildable hour being the bane of our existence.
- 12:25SPEAKER_01: But there's a ton of stuff.
- 12:27SPEAKER_01: So we took 10 years worth of 14,000 attorneys, time entries, where they described what they did, right?
- 12:37SPEAKER_01: So we we were able to across every practice area in the entire firm, what steps go into creating a particular legal product or service?
- 12:47SPEAKER_01: And this was a massive data analysis.
- 12:49SPEAKER_01: We teamed up with the Imperial College in London.
- 12:52SPEAKER_01: I mean, it was a really big kind of picking apart how people do their jobs.
- 12:58SPEAKER_01: And what came back to me was just astonishing that every lawyer, no matter what their practice group, basically does six or seven things over and over and over again.
- 13:10SPEAKER_01: And at least 80%, and I'd be nice there, but between 80 and 90% of what a lawyer does today is much better done by AI.
- 13:21SPEAKER_00: Wow.
- 13:22SPEAKER_01: When those results came out, and this is the same time you're getting reports from like Mackenzie and Gartner, like that they're saying similar things about knowledge workers, and lawyers are always at the top one, two, or three of disruption.
- 13:33SPEAKER_01: I kind of decided two things at once.
- 13:35SPEAKER_01: Because as you said, this is the same time that we're having these meetings at the White House and in Congress and staff briefings.
- 13:42SPEAKER_01: And I I was I am on the scientific advisory board to NIST, which wrote one of the AI risk management frameworks that's very popular over in the EU, involved in those negotiations and um working with at least a dozen countries, different ministries of AI.
- 14:00SPEAKER_01: And so I became very, very passionate about we have to get this right.
- 14:06SPEAKER_00: Yeah.
- 14:06SPEAKER_01: This because I'm a huge AI optimist.
- 14:09SPEAKER_01: I think the world, what we the good we can do is just almost incalculable.
- 14:14SPEAKER_01: But the evil it can do is also not so great either, right?
- 14:18SPEAKER_01: And so I I got very focused on public policy, like let's get the regulation correctly, but then really focusing on how do you technologically control AI.
- 14:33SPEAKER_01: And so I came to kind of two conclusions at once.
- 14:37SPEAKER_01: One, the billable hour and you know, the way that especially larger law firms practice law by throwing bodies at a problem and charging by the hour, that world is simply debt.
- 14:48SPEAKER_01: It will not survive the Gen AI revolution.
- 14:51SPEAKER_01: And the billable hour is so utterly entrenched in big law firm culture that it's it's gonna take a lot to rip that out and put it in a new.
- 15:01SPEAKER_01: And, you know, I decided that I didn't want to spend the last 10 years of my career basically trying to save a sinking ship.
- 15:09SPEAKER_00: Yeah.
- 15:10SPEAKER_01: Because I realized the opposite was also true that a law firm that was based on the power of AI from the very beginning and taking advantage of everything that it can do, yeah, that could be a completely 22nd century kind of law firm.
- 15:27SPEAKER_01: And so at the right bulb age of 56, which is the perfect time to start your own thing, you know, I left the pinnacle of my career and formed Clarion.
- 15:36SPEAKER_01: And we're very we're a strange little amalgamation of services because we are lawyers.
- 15:42SPEAKER_01: All the lawyers are lawyerslash something, data scientist, computer scientist, and things.
- 15:48SPEAKER_01: And so fortunately, a great number of my clients came with me, which was very nice.
- 15:53SPEAKER_01: And so, like a client will call up and be like, hey, we're thinking about doing this, and we can help talk through the laws that might apply and the risks, and how do you mitigate those and can you prove it?
- 16:05SPEAKER_01: But the two things we do that I love the most are one is helping companies figure out their AI strategy.
- 16:12SPEAKER_01: Because this really is you can completely transform your company in beautiful, flourishing ways.
- 16:21SPEAKER_00: Definitely.
- 16:22SPEAKER_01: But step by step, and how do you do that and what do you start?
- 16:25SPEAKER_01: Like that, that is really important.
- 16:29SPEAKER_01: And then the third thing we do is we are, as you mentioned, a development shop.
- 16:34SPEAKER_01: So we actually help build AI systems for clients, everything from tiny little knowledge bots.
- 16:42SPEAKER_01: And so it's a it's a really interesting perspective on bringing a company's purpose kind of to life, figuring out what AI can do, yeah, intermessed with with humanity to make this incredibly more powerful entity out of it.
- 17:00SPEAKER_00: I love it.
- 17:01SPEAKER_00: I love this.
- 17:02SPEAKER_00: So I'll take it one step back.
- 17:05SPEAKER_00: So 80, that is a huge number, you know, from your own firm's data.
- 17:09SPEAKER_00: I think most leaders listening know that that number is true for their own organizations, but very few have actually done the analysis to prove it.
- 17:19SPEAKER_00: And I'm talking about how you could save so much time on repetitive tests.
- 17:26SPEAKER_00: But you did.
- 17:27SPEAKER_00: And then you did something even rarer.
- 17:30SPEAKER_00: You acted on it.
- 17:31SPEAKER_01: For good or ill, yes, for good.
- 17:34SPEAKER_01: No, it's been fantastic.
- 17:35SPEAKER_01: Like, and that's really, you know, and it's also because of how I came to studying data through a very human lens that data really is this representation of what it means to be human.
- 17:48SPEAKER_01: And so I naturally incline myself toward how does the technology develop human flourishing.
- 17:56SPEAKER_01: Yes.
- 17:56SPEAKER_01: And so we talk to our clients, we always use this analogy, right?
- 18:01SPEAKER_01: When we first start working with companies, we say when you're thinking about bringing AI into your company, you got to think Iron Man, not the Terminator, right?
- 18:11SPEAKER_01: So you're not gonna unplug a person and plug in a bot and everything's fine, right?
- 18:16SPEAKER_01: It is much more like a lawyer, where eight out of ten of my tasks are better done by AI.
- 18:24SPEAKER_01: So let's let the AI do that thing with the appropriate controls and the appropriate QC.
- 18:30SPEAKER_01: But then the human part of what being a lawyer is all about is so essential.
- 18:37SPEAKER_01: And so, how do you weave those two knowledge sets and experience sets together so that the AI is empowering that that human part of it and takes away inconsistencies and you know, uh, and actually can bring together the knowledge and experience of an entire firm as opposed to this partner or that partner.
- 18:59SPEAKER_01: And that's true of really every company, that you can you can take its experience and its acumen and its years of learning and growing and you know, what worked and what didn't, and bring that into an AI agenc system and just watch that hard-earned um experience be leveraged into something that makes the humans more powerful, that makes the product much better, much more consistent, much more, right, that that has this overlap of learnings, right, as opposed to just coming out of one group or another.
- 19:35SPEAKER_01: It's really one of my favorite things to do with companies.
- 19:38SPEAKER_00: Love it.
- 19:39SPEAKER_00: I love the Iron Man analogy a lot.
- 19:42SPEAKER_00: So think Iron Man, not the Terminator, right?
- 19:46SPEAKER_00: And you, I was gonna ask you to unpack that for the leaders listening who are wrestling with what AI transformation actually looks like inside their organization.
- 19:58SPEAKER_01: Yeah, and so the some of the mistakes that we see that companies make early on.
- 20:03SPEAKER_01: One is that they spend too much time defending how they have historically been successful.
- 20:12SPEAKER_01: And so even some of the biggest companies in the world, they've they've got a recipe they've been around for 100 years, and they do stuff a certain way.
- 20:21SPEAKER_01: And they're, you know, it's it's a CEO's job feels like they are the steward of that prior success.
- 20:28SPEAKER_01: And so when something new comes along, the pressure of what do I do I change what we've done for 100 years?
- 20:37SPEAKER_01: So to like the pressure of that stewardship can be really, really difficult.
- 20:42SPEAKER_01: But this is an age where you simply cannot not address this technology, even at the very tiniest levels where you're using the general AI models to let your people use it, that will give you some marginal efficiency gains, sometimes 20 or 30%, which is fantastic, right?
- 21:02SPEAKER_01: But without a concentrated pot like AI, you'll never achieve this true transformative effect.
- 21:11SPEAKER_01: And so the other mistake we see is that most companies don't actually understand deeply the steps that they take to do whatever it is they do.
- 21:23SPEAKER_01: They just kind of do stuff and they have, you know, sales does this and marketing does this and product does that.
- 21:28SPEAKER_01: But really breaking it down into these steps, here's what we do.
- 21:32SPEAKER_01: And so without that knowledge, it's very difficult to say, let's put AI in here, here, and here.
- 21:41SPEAKER_01: And so this does take some self analysis, right?
- 21:47SPEAKER_01: Of what the company does.
- 21:48SPEAKER_01: And moreover, what do we want it to do?
- 21:51SPEAKER_01: Because you can consciously harness the not just the productivity or the financial success of a firm.
- 22:00SPEAKER_01: But what is its culture?
- 22:02SPEAKER_01: What is its ethos, right?
- 22:04SPEAKER_01: What is what do I want my company to be like, or what legacy has it had that I want to continue?
- 22:10SPEAKER_01: Because those things are all capturable into these AI systems.
- 22:15SPEAKER_00: Yeah.
- 22:16SPEAKER_01: And in a way that no other technology has ever been able to do.
- 22:21SPEAKER_00: Thank you.
- 22:22SPEAKER_00: This is great.
- 22:22SPEAKER_00: And I'm gonna go back to Clarion AI Partners, because I'm excited about the concept that you shared.
- 22:30SPEAKER_00: Um you introduced the concept, which is governance engineering, because I think it it reframes something that most organizations are getting wrong.
- 22:40SPEAKER_00: So what is governance engineering and why does it matter?
- 22:44SPEAKER_01: This concept of governance engineering is really a very powerful one once you sort it out.
- 22:50SPEAKER_01: We loved the phrase so much that we trademarked it.
- 22:53SPEAKER_01: There's kind of a level one of governance where you've got the basics, you've got a an AI steering committee or group or something, right?
- 23:02SPEAKER_01: That's well represented from the different parts of the business.
- 23:05SPEAKER_01: You've got a policy and an acceptable use policy, and we will do this or that with data.
- 23:11SPEAKER_01: But then it kind of stops there and you kind of go, okay, now what do I do?
- 23:15SPEAKER_01: So then you move into the use case, right?
- 23:18SPEAKER_01: So that's where you start to take apart your business, figure out where good low-hanging fruit is.
- 23:24SPEAKER_01: And so, you know, tasks that are data-centric, highly repetitive, and you're applying the same judgments over and over again is perfectly optimizable by AI.
- 23:38SPEAKER_01: But then you get to the point of, well, how do I one make it work, right?
- 23:44SPEAKER_01: Like we wouldn't prefer these things to work.
- 23:46SPEAKER_00: Yeah.
- 23:47SPEAKER_01: But then how do I prove that they're working the way I want them to?
- 23:53SPEAKER_01: And so this idea of governance engineering is taking the words from laws or regulations or even industry best practices.
- 24:02SPEAKER_01: And for instance, if you think of most of the risk management frameworks or even the EU AI Act, they talk about concepts like fair, accurate, reliable, transparent.
- 24:14SPEAKER_01: Totally agree.
- 24:15SPEAKER_01: But what does that mean?
- 24:17SPEAKER_01: And it means different things in different contexts.
- 24:19SPEAKER_00: Yeah.
- 24:20SPEAKER_01: And so governance engineering is the ability to take those words and convert them into technological specifications to build into the application that not only ensure that the model is doing what you want it to do, but it spits out the evidence moment by moment that it's doing what it's supposed to do.
- 24:47SPEAKER_01: Because right now, AI regulation is all over the place and it's gonna stay pretty fractured.
- 24:54SPEAKER_01: It's not gonna settle down for four or five years, I would say.
- 24:57SPEAKER_01: It just can't.
- 24:58SPEAKER_01: Um it's moving so quickly, so much faster than regulators or legislatures can keep up with.
- 25:05SPEAKER_01: And so the best you're gonna get out of regulation is hey, don't screw up the world.
- 25:09SPEAKER_01: And you know, and so the this idea of you're gonna have to come up with the way to defend yourself.
- 25:20SPEAKER_01: And the law eventually falls back on this very fundamental principle of reasonableness.
- 25:29SPEAKER_01: So there's a there's a legal standard called the reasonable person standard or the reasonable company standard.
- 25:35SPEAKER_01: And it basically says, what would the average company do based on the information it had at the time?
- 25:43SPEAKER_01: And so what that's been interpreted to mean is if I'm introducing AI into a workflow, what new risks am I introducing into the market?
- 25:54SPEAKER_00: Yeah.
- 25:54SPEAKER_01: What did I do to reasonably mitigate those risks?
- 25:58SPEAKER_01: And then, and this is where everybody always forgets, how do I prove it?
- 26:04SPEAKER_00: Exactly.
- 26:05SPEAKER_01: And so by being able to, it's it's that analysis, like, hey, because builders want to build and they're like, oh, I could do this awesome thing, and and they're like, look, it'll do this thing.
- 26:14SPEAKER_01: But builders haven't been trained in risk or especially legal risk.
- 26:20SPEAKER_01: Yeah.
- 26:21SPEAKER_01: And the same problems on the other side with the lawyers, like the lawyers have been trained in nothing but risk.
- 26:25SPEAKER_01: Like, that's what we do.
- 26:26SPEAKER_01: But they don't know the technology well enough to ask the questions, even to to, and so that's one of the reasons why we decided to just be both.
- 26:34SPEAKER_01: We'll tell you how to do it, and we'll help you do it, and you know, but this you be if because if you have this fundamental evidentiary telemetry of that you're acting within the bounds of the law or regulation or best practices have set, then you can defend yourself whoever comes knocking.
- 26:54SPEAKER_01: Um whether it's a court or a regulator or the New York Times, whatever it is, you've gets look, we we thought this through defensibly.
- 27:03SPEAKER_01: We we did all these things to try to make it safe.
- 27:06SPEAKER_01: And here, look, you can look at the record.
- 27:08SPEAKER_01: We we did all the things that we said we would do.
- 27:11SPEAKER_00: I love that.
- 27:12SPEAKER_00: That distinction between having controls and being able to prove your controls work.
- 27:19SPEAKER_00: That is the gap most organizations are living in right now.
- 27:23SPEAKER_00: They have the policy, they don't have the proof.
- 27:26SPEAKER_00: And the proof is what the EU AI art or the FCA or the PRA, all of them, are actually going to ask for.
- 27:35SPEAKER_01: Exactly.
- 27:36SPEAKER_01: And especially because they don't know what to ask for now, right?
- 27:40SPEAKER_01: And so, you know, especially in the US, you've got this really strange kind of tug-of-war between the states and the federal government uh where because we aren't seeing federal legislation and we aren't seeing really clear guidance coming out of Washington, the states are stepping in, and so you've got, you know, eight or nine or ten states passing AI laws.
- 28:03SPEAKER_01: And it it's difficult for a company because you you basically have to engineer to the strictest state, because most companies aren't going to go through all the effort to, you know, treat California one way and Illinois another way and New York another way, right?
- 28:19SPEAKER_01: That's really hard to do.
- 28:20SPEAKER_01: Um and so they just, well, let's just build to the strictest standard.
- 28:26SPEAKER_01: But the the nice thing is if you look at how technology regulation has developed in the past, it's because industry has kind of gotten together and kind of said these are best practices.
- 28:41SPEAKER_01: And so even in the headlines lately, where you're seeing, you know, anthropic and open AI and some of the others saying, hey, we're a little worried uh that these things are getting really, really smart.
- 28:51SPEAKER_01: And yet out of Washington, what you hear is, no, it'll be fine.
- 28:55SPEAKER_01: It'll be fine.
- 28:56SPEAKER_01: And it won't be fine unless we make it fine on purpose.
- 28:59SPEAKER_01: And so what's beautiful to see is that all of these frontier model makers from you know open AI, Anthropic, Google, Amazon, Microsoft, they are they do talk a lot.
- 29:12SPEAKER_01: They are trying to make controls that actually work because you can, especially when you're building, like when we build for clients, the best way to control AI is with AI, right?
- 29:26SPEAKER_01: Humans just you're not gonna keep up.
- 29:28SPEAKER_01: Our little brains aren't gonna keep up.
- 29:30SPEAKER_01: And so building an AI agent, but then we build right beside it another agent whose whole whole job it is, is to oversee the first agent, right?
- 29:38SPEAKER_01: And so, and often we base those on different models because different models see things the world a little bit differently.
- 29:44SPEAKER_01: And so there's all kinds of ways you can control AI.
- 29:49SPEAKER_01: And sometimes I'll work with CEOs and they'll say, but the whole thing's a black box.
- 29:55SPEAKER_01: Like even the people who made it don't know how it works.
- 29:58SPEAKER_01: Doesn't matter because when a company uses generative AI, there's some result, like it does something, it touches the world somewhere, whether internally inside the company or externally.
- 30:13SPEAKER_01: You can always measure that touch.
- 30:17SPEAKER_01: And that touch is where you are regulated or have where your liability runs.
- 30:23SPEAKER_01: And so, like, think like an HR algorithm that sorts through resumes and decides who gets an interview.
- 30:32SPEAKER_01: Well, I can look at the people who were recommended for an interview and the pile of people who are recommended for not, and I can compare those two groups and figure out if there's discrimination built in somewhere.
- 30:43SPEAKER_01: Right.
- 30:44SPEAKER_01: Or all kinds of things.
- 30:46SPEAKER_01: So even if you think about where these models touch the world, that is where you need to measure, and that is where you'll protect yourself the most from liability.
- 30:59SPEAKER_00: I agree completely.
- 31:00SPEAKER_00: Thanks for sharing that.
- 31:02SPEAKER_00: So let's go and let's continue with the conversation we had before the session.
- 31:08SPEAKER_00: You talked about every company having a secret source.
- 31:13SPEAKER_00: So by that we mean expertise buried in their data and in their people.
- 31:19SPEAKER_00: And the real competitive advantage coming from from codifying that into AI systems.
- 31:25SPEAKER_01: So could you walk us through one of my favorite things, Magda?
- 31:29SPEAKER_01: So because like I said, if if companies buying or licensing this general technology and letting their people use it with the right controls and you know, the whitelists and blacklists and you know policies and around it, that's really great, honestly.
- 31:49SPEAKER_01: You'll you'll get these marginal utility gains.
- 31:52SPEAKER_01: The problem is that they're available to everybody.
- 31:57SPEAKER_01: And so maybe first movers will get an advantage for a little while, but then everybody catches up.
- 32:04SPEAKER_01: And then when everybody's special, nobody's special, right?
- 32:07SPEAKER_00: Yeah.
- 32:07SPEAKER_01: But the thing that really sets a company apart is its acumen, its experience, its secret sauce, right?
- 32:16SPEAKER_01: And the artifacts of that acumen are buried within its data.
- 32:21SPEAKER_01: And I mean buried.
- 32:22SPEAKER_01: Like most companies have very siloed systems, they've they keep everything forever.
- 32:28SPEAKER_01: No, it, you know, it's not organized in a way that is easy to get to, or it's locked in the brains of their best people, right?
- 32:36SPEAKER_01: That's where they've learned through experience and they're applying judgment.
- 32:39SPEAKER_01: The cool thing about Gen AI systems is you can capture knowledge from both of those places.
- 32:45SPEAKER_01: And so one of the things we love to do is go find these pockets of acumen, right?
- 32:52SPEAKER_01: And it's not like we try to, it used to be, you know, the data lake was the big thing.
- 32:56SPEAKER_01: Throw all your data into a big giant data lake, and then, you know, you can look at it.
- 33:01SPEAKER_01: Yes, but uh what we have found, especially with agentic AI, is you find pockets of expertise around a particular subject or workflow, and you find your best data on that, the most well curated that represents the best work you have.
- 33:21SPEAKER_01: And that's what you use as the knowledge base for these AI agents using retrieval augmented generation, you know, stick them in a folder somewhere.
- 33:31SPEAKER_01: And then you point a model at it, and then you constitutionalize that model, um, constitutional AI being a discipline that really anthropic came up with originally that says you you're you give the agent a persona.
- 33:49SPEAKER_01: And this is what kind of gets a little freaky because old-fashioned coders who are used to writing like 10,000 lines of code, you write the code, it does the code.
- 33:59SPEAKER_01: You alright?
- 34:00SPEAKER_01: It's it's perfectly controlled.
- 34:02SPEAKER_00: Yeah.
- 34:03SPEAKER_01: And Gen AI is not like that.
- 34:05SPEAKER_01: Like it is so much more like forming the psychology of the way this agent thinks.
- 34:13SPEAKER_01: And so you give it a persona.
- 34:15SPEAKER_01: So let's say we were making a chatbot for HR, because they get the same 17 questions over and over and over again.
- 34:21SPEAKER_01: And so you take all your HR policies and you put them into a database, you point a model at it and say, Hey, agent, you're you are a chatbot who's going to be interacting with employees, asking questions about HR.
- 34:36SPEAKER_01: And then you program in these constitutional principles about how you act.
- 34:43SPEAKER_01: Thou shalt this, thou shalt not that, right?
- 34:46SPEAKER_01: It's it's just these basic rules of how you're gonna live.
- 34:50SPEAKER_01: And with that, you all of a sudden people can query, you know, and um, it's a very the risk is really low, right?
- 34:59SPEAKER_01: Because it's internal.
- 35:00SPEAKER_01: And then if you answer the question wrong, well, okay, it's a little bit of frustration, but it's not the end of the world.
- 35:06SPEAKER_01: But the same concept works when you're building systems that face the public, yeah.
- 35:13SPEAKER_01: And that are multi-agent works, because you're basically creating an ensemble of experts, right?
- 35:21SPEAKER_01: And this is where China has really, you know, because we constrained the most powerful chips from getting to China, they were very, very smart and came up with, well, I don't need all the chips.
- 35:34SPEAKER_01: I don't need to use the whole model's brain for every question.
- 35:39SPEAKER_01: If it's a question about a little thing, well, I have that part of that little expert, right?
- 35:44SPEAKER_01: And so this ensemble of experts is a very, very powerful technique.
- 35:49SPEAKER_01: But the most interesting and challenging part is how do you orchestrate the behavior of all of those?
- 35:57SPEAKER_01: Like you've seen like the drone shows where there's like 10,000 drones doing the light shows and stuff like that.
- 36:03SPEAKER_01: And I'm like, how in the world are those things not just all crash into each other?
- 36:07SPEAKER_01: And right, but it's because of very clever programming.
- 36:09SPEAKER_01: And it's the same way with these expert or ensemble agent workflows.
- 36:16SPEAKER_01: You really have to think about them because they really do have little objectives all their own.
- 36:21SPEAKER_01: The objectives that you've given them, but you have to be really careful what you tell them what their optimization function is, right?
- 36:28SPEAKER_01: Like what definitely what makes them a good boy or a bad boy, right?
- 36:34SPEAKER_01: So we like one, we were building a pretty complex system for a client, and there was this kind of one corner of the system that just kept this weird behavior.
- 36:46SPEAKER_01: It there wasn't a pattern to it.
- 36:47SPEAKER_01: There was just it just was odd.
- 36:49SPEAKER_01: And so as we really delved into its logs and what it was doing, yeah, we figured out that it was withholding information or even lying to the agent next to it because it interpreted it, it's yeah, hey, do the best you can be to be okay, well, then I'll screw up everybody else around me too, so they'll all look even better, right?
- 37:12SPEAKER_01: And so we had to go back in and rewire its optimization and got cooperation a little better, you know.
- 37:20SPEAKER_01: But it's it really is like nothing we've ever dealt before.
- 37:25SPEAKER_01: And when you really are defining the psychology of how these systems work, and that's where all the fun is too.
- 37:36SPEAKER_00: I love this.
- 37:37SPEAKER_00: I just love how you talked about the secret source concept and elaborated it so beautifully.
- 37:44SPEAKER_00: It reframes the ROI question completely.
- 37:48SPEAKER_00: Right?
- 37:48SPEAKER_00: Um, the conversation in most boardrooms is about efficiency gains from AI tools.
- 37:56SPEAKER_00: What you're describing is something categorically different using AI to make your organization's own knowledge, compounds, and skills in a way that was previously impossible.
- 38:10SPEAKER_01: Exactly.
- 38:12SPEAKER_00: Amazing.
- 38:13SPEAKER_01: And how you can pull in what's really cool is you can use your knowledge as a base, but then you could enrich that data with outside data sources.
- 38:23SPEAKER_01: And so we were working with one company who had a very complex supply chain across the whole world, right?
- 38:32SPEAKER_00: Yeah.
- 38:32SPEAKER_01: And it was all very just in time.
- 38:35SPEAKER_01: Like if any one part of it fell down, the whole thing kind of fell down.
- 38:39SPEAKER_01: And we were brought in to see if we could make it even more accurate.
- 38:42SPEAKER_01: And so it was based on like all the demand, who bought what at what stores, and you know, like really detailed information about demand and location and all that kind of stuff.
- 38:52SPEAKER_01: But we brought in weather patterns and geopolitical conflict and things like that to inform those things.
- 39:01SPEAKER_01: So, like if there's a hurricane in the Sea of Japan that's gonna mess up traffic, you know that your stuff's gonna get there late.
- 39:09SPEAKER_01: And so automatically switching a supplier track to somewhere that's safer that gets the supplies to where they need to go.
- 39:18SPEAKER_01: And so it's really this bringing in outside knowledge, which has been accessible in completely new ways with generative AI, and then combining it with your own acumen, it's really amazing how you can transform a company.
- 39:36SPEAKER_01: And we always help companies try to understand that when you're doing this transformative work, you should don't start with like the biggest, most important thing in your company.
- 39:48SPEAKER_01: Let's go to the HR bot first, right?
- 39:50SPEAKER_01: Let's use something that's fairly easy, yeah, you know, clear ROI, you know, but frees up my HR people to do other things and answer the same 17 questions, you know, and because you've got this governance engineering is a discipline that you you teach to the company, and you know, and our goal is that we teach it sufficiently that they don't need us anymore, like that we can do other things.
- 40:16SPEAKER_01: But it's really figuring out that uh almost cadence of how do we build upon this and then this and then this and then this.
- 40:25SPEAKER_01: At the same time, realizing that, and this goes back to my view of data as being a reflection of humanity, is that trans bringing AI into your company to the point that it transforms it is a 20% technological problem.
- 40:42SPEAKER_01: I can build you anything and turn, you know, super suits all over the place, right?
- 40:46SPEAKER_01: But it's 80% psychological.
- 40:48SPEAKER_01: Yeah.
- 40:48SPEAKER_01: Like really, individual employees have very different views on is this a good thing?
- 40:55SPEAKER_01: Is it a bad thing?
- 40:56SPEAKER_01: Is it going to replace me?
- 40:57SPEAKER_01: Is it am I training it to take over my job?
- 41:00SPEAKER_01: And so by properly communicating the company's objectives and saying that the the people are the center of how I use AI, that changes their mindset.
- 41:16SPEAKER_01: Everybody wants a super suit, right?
- 41:18SPEAKER_01: And and so um, even as we work with companies, we we dive pretty deeply into their workings and talk to a lot of people.
- 41:26SPEAKER_01: And much of that is for the purpose of, you know, figuring out how the company works and stuff, but a whole lot of it seeing where people are psychologically and trying to lay this foundation of empowerment and flourishing as opposed to, oh my gosh, I gotta take my job.
- 41:44SPEAKER_01: Right.
- 41:44SPEAKER_01: And so it's, you know, it goes back to my CIA days, right?
- 41:48SPEAKER_01: Of trying to help people understand a message and internalize it.
- 41:53SPEAKER_01: And but also if they feel like they have a hand in building it, then you'll give much more loyalty and much more excitement about doing so.
- 42:04SPEAKER_00: I love the fact that you talked about their transformation being 20% technical and 80% psychological, and also the fact that people are excited, scared, and uncertain at the same time.
- 42:17SPEAKER_00: Um, so getting them into the mindset of, no, I'm getting a super suit.
- 42:23SPEAKER_00: It unlocks creativity, right?
- 42:25SPEAKER_00: And capacity rather than creating resistance.
- 42:30SPEAKER_00: So I'm gonna go into this last last but one question.
- 42:34SPEAKER_00: So you made a comparison that I want to bring to the audience.
- 42:39SPEAKER_00: AI and social media.
- 42:42SPEAKER_00: You said we got social media wrong.
- 42:45SPEAKER_00: What do you mean by that?
- 42:47SPEAKER_00: And what does getting AI wrong look like if we repeat the same mistake?
- 42:53SPEAKER_01: Yeah, you know, social media, it wasn't even conceivable that this would be negative somehow.
- 42:59SPEAKER_01: Like at the time that that social media was being created and and it was so exciting, right?
- 43:06SPEAKER_01: You could keep in touch with your friends, you could see what their your high school friends and their kids were doing.
- 43:13SPEAKER_01: Like it was this wonderful way to stay in touch.
- 43:16SPEAKER_01: But then it it turned a little evil, right?
- 43:20SPEAKER_01: Like what came out of it was it turned our children into screen starers, right?
- 43:27SPEAKER_01: Like, so instead of having real experiences out in the world, they had these fake experiences online, and then these things are run by algorithms that are meant to catch your attention and keep you on the screen.
- 43:44SPEAKER_01: And so the algorithms naturally figured out well, what keeps you on your screen?
- 43:50SPEAKER_01: And yeah, a puppy, you know, video for a little while, but it's the things that are are shocking or provocative or Right?
- 44:00SPEAKER_01: It's those things where you go, you can't help but look, right?
- 44:04SPEAKER_01: And left unchecked, you just have this entropy that brings people down, right?
- 44:12SPEAKER_00: Yeah.
- 44:12SPEAKER_01: And it I think it has a l had a lot to do with increasing the extreme polarization we're seeing in the United States, but also worldwide.
- 44:23SPEAKER_01: And it wasn't until we realized the dangers it was doing that anything started being done about it.
- 44:32SPEAKER_01: And so it's it's not a coincidence that the world's most connected generation in the history of the world is also the loneliest and has the greatest mental health crisis.
- 44:46SPEAKER_01: And so we're working on that, right?
- 44:49SPEAKER_01: State laws are being passed, even the social media companies themselves, they see the writing on the wall and they're putting in, you know, better protections and things like that.
- 44:58SPEAKER_01: But AI has potential to be the best thing for for a child ever, right?
- 45:03SPEAKER_01: Like definitely it has the potential of putting into the hands of every child on earth the best education ever in history.
- 45:13SPEAKER_01: But at the same time, it'll do what you tell it to do for the most part.
- 45:17SPEAKER_01: And it there are nefarious actors out there, and it they are much better.
- 45:23SPEAKER_01: It's like the the social media influencing you to do stuff is almost passive, right?
- 45:29SPEAKER_01: Because of what it shows you and what the AI systems will straight up talk to you, right?
- 45:34SPEAKER_01: And and convince you of things.
- 45:36SPEAKER_01: And and so this is where public policy and especially technological controls have to be at the forefront of what you're making.
- 45:46SPEAKER_01: We are seeing startup after startup after startup.
- 45:49SPEAKER_01: Oh, they've got this great idea.
- 45:50SPEAKER_01: We got this great idea.
- 45:51SPEAKER_01: Yes, great idea.
- 45:53SPEAKER_01: But stop and think a minute.
- 45:56SPEAKER_01: How could it go wrong?
- 45:57SPEAKER_01: And some startup founders, they're like, I don't want to think about what my thing could do wrong, you know, but but it can't, you know.
- 46:05SPEAKER_01: So thinking through what it's meant to do, how do I control it so it does exactly that?
- 46:12SPEAKER_01: And then how do I prove it?
- 46:14SPEAKER_01: Right?
- 46:14SPEAKER_01: You've got to have that thinking going into it so that your product is on the flourishing side and not on the manipulative side.
- 46:24SPEAKER_00: I agree.
- 46:25SPEAKER_00: This is one of the sensitive conversations as well, because it touches everyone.
- 46:32SPEAKER_00: I was in Geneva for the UN AI for good summit recently, and what stayed with me most was that very question.
- 46:41SPEAKER_00: What are we leaving behind for the next generation?
- 46:45SPEAKER_00: And you framed it beautifully.
- 46:48SPEAKER_00: We have the opportunity to get this right in a way we didn't with social media.
- 46:55SPEAKER_00: So that the question is whether we will take it.
- 47:00SPEAKER_01: It is absolutely our challenge.
- 47:02SPEAKER_01: And it is what we decide today will have incredible ramifications for the future.
- 47:08SPEAKER_00: Yeah.
- 47:09SPEAKER_01: And that's why even governments are going to be slow to act.
- 47:13SPEAKER_01: They just every time there's a giant technological advancement, right, the the law doesn't keep up, right?
- 47:20SPEAKER_01: And so the difference between what we can do and what the law talks about gets bigger and bigger.
- 47:25SPEAKER_01: And that means that we who work in AI, as builders, as lawyers, as leaders, we are the ones who are responsible for creating those standards, for getting together and developing frameworks and um helping the regulators come up with good public policy that makes sense, right?
- 47:45SPEAKER_01: That is fair to these wonderful and unlocks this innovation, but also does so in a safe and effective manner.
- 47:53SPEAKER_01: And that's really one of the big holdups is everybody goes, okay, let's be fair, let's be accurate, let's be reliable.
- 48:01SPEAKER_01: But nobody agrees on how do I mathematically measure that, right?
- 48:06SPEAKER_01: And so some of the biggest research project we are involved in at Clarion are around the different ways to design an AI system and the different kinds of controls that work best in different kinds of industries, and the hope that that will be picked up by academia, be picked up by public policy makers.
- 48:29SPEAKER_00: Thank you.
- 48:31SPEAKER_00: Well, your key message now, and it's on your business card, is AI for good and AI for all.
- 48:41SPEAKER_00: In one minute, make the case.
- 49:03SPEAKER_01: And that means only the wealthiest companies in the world could afford it.
- 49:07SPEAKER_01: And I decided that the equation needed to be balanced.
- 49:12SPEAKER_01: And so we work with some of the world's biggest companies and little tiny little mom and pop stores, like just because this AI technology is one of the most wonderful democratizing technologies in the world.
- 49:27SPEAKER_01: Anyone who has a really good idea can bring that idea to life without having to be a technological guru, right?
- 49:34SPEAKER_01: And so I love unlocking an idea of a little startup or a, you know, even a business that's been like we're working with some construction companies and some like landscaping companies.
- 49:46SPEAKER_01: They're like, how can I AI help me?
- 49:48SPEAKER_01: I'm like, you watch this, brother.
- 49:50SPEAKER_01: And so AI for all, like it's not it's not just meant to be in the hands of the most powerful in the world.
- 49:57SPEAKER_01: It's meant to be in everybody's hands and is probably better in the hands of more people than fewer.
- 50:05SPEAKER_00: Bennett, this has been one of the most wide-ranging and genuinely important conversations I have had on this show.
- 50:17SPEAKER_00: From the CIA to the courtroom to the engine room of AI governance.
- 50:23SPEAKER_00: The threat running through all of it is the same.
- 50:27SPEAKER_00: AI is controllable if you do it right.
- 50:32SPEAKER_00: And the cost of not doing it right is not just a compliance problem, but it is a human one.
- 50:40SPEAKER_01: I could not agree more.
- 50:40SPEAKER_01: Magda, thanks so much for having me.
- 50:42SPEAKER_01: It's a wonderful show, and I hope that we get to talk again.
- 50:46SPEAKER_00: I'm looking forward to it.
- 50:47SPEAKER_00: So thank you once again, and on to our listeners.
- 50:52SPEAKER_00: Governance engineering, as you understand, has been trademarked by Clarence AI.
- 50:57SPEAKER_00: It's not a technical concept, it is a leadership decision.
- 51:03SPEAKER_00: The choice to build the proof layer in from the start to supersuit your people rather than replace them.
- 51:12SPEAKER_00: And to ask yourself, honestly, can I demonstrate that my AI is doing what I designed it to do?
- 51:21SPEAKER_00: That is the question.
- 51:23SPEAKER_00: And if you cannot yet answer it, Viridian was built to help you get there.
- 51:28SPEAKER_00: AI governance, you can prove.
- 51:30SPEAKER_00: And it's available on the Microsoft Azure Marketplace, and the link will be in the show notes.
- 51:36SPEAKER_00: We will also share the details for Claire on AI Partners and Bennett so you can connect with Bennett and they will be happy to support you.
- 51:45SPEAKER_00: So thank you for tuning in to the Anaya algorithm.
- 51:50SPEAKER_00: Until next time, keep leading intentionally.
- 51:54SPEAKER_00: Thank you.