Latest / The Road to Accountable AI / Ray Eitel-Porter, Co-Author of Governing the Machine: The Confidence to Use AI

Transcript

Ray Eitel-Porter, Co-Author of Governing the Machine: The Confidence to Use AI · The Road to Accountable AI

• 0:00
  Ray, welcome.
• 0:01
  Great to talk with you.
• 0:03
  Likewise, thank you very much, Kevin.
• 0:06
  So you were the global lead for responsible AI at Accenture where you're still a senior advisor.
• 0:13
  What was your path to doing work in this responsible AI area?
• 0:19
  Well, through most of my career, if not all of my career, a key thread has been my fascination with technology and how technology can be applied to business challenges and
• 0:33
  social challenges.
• 0:35
  And I founded a startup software business at one point and have been involved in a number of different ways.
• 0:42
  But for the last sort of 12 or 13 years, I was with Accenture.
• 0:47
  first of all running their data and AI practice in the UK.
• 0:52
  Hard to imagine, but actually setting it up in the UK because 12, 13 years ago, there wasn't really a standalone practice.
• 0:59
  mean, people were working with data and they were doing analysis, but it wasn't, you know, uh a standalone practice.
• 1:07
  And after doing that for a number of years, we gradually started to see, and I'm sure you did too,
• 1:14
  you know, data privacy emerging as one of the concerns and then bias emerging as well.
• 1:20
  And those two were the first sort of red flags that alerted people, I think, to the fact that, yes, you can do fantastic things with data and analysis, but there are also a number
• 1:31
  of different risks and considerations that we need to take into account.
• 1:35
  And so gradually my path took me more in that direction.
• 1:38
  And
• 1:39
  I moved from being the UK lead for data and AI to being the global lead for responsible AI because I thought that, we'd need all the benefits of data and AI, but actually we can
• 1:52
  only achieve them if we do so in the right way.
• 1:57
  Early on, what were some of the hardest challenges of addressing those problems?
• 2:03
  I mean, the first one was that clients weren't interested, frankly.
• 2:07
  You know, if you went to a client and started talking about these issues, it was hard enough to get them to realize that actually data analytics could be helpful for their
• 2:16
  business.
• 2:17
  And then if you started talking about governance and privacy and bias and so on, that was a real tough sell.
• 2:24
  And it wasn't really until ChatGPT hit the scene that things changed significantly.
• 2:31
  uh And I think what happened then is obviously chat GPT raised the awareness of the potential of AI just in sort of social and personal use, but also immediately people
• 2:46
  started to see hallucinations and headline news of things going wrong.
• 2:51
  And so we had a lot of clients coming to us saying, we need this AI thing, but for heaven's sake, it's got to go well.
• 2:58
  We can't be the next company that is in the news because of XYZ going wrong.
• 3:06
  I think, so that's from the client perspective.
• 3:09
  think from the Accenture perspective, the biggest challenge, Accenture were a very early adopter or supporter of the responsible use of AI.
• 3:22
  But trying to build a compliance program across 750,000 people in 20 plus countries is a tough challenge.
• 3:31
  And trying to identify the ways to really capture everything in the net, I think that was the hardest piece.
• 3:41
  It's interesting the point you make, and I saw the same thing, that the surge in excitement about AI after ChatGPT was also the time that people started getting most
• 3:52
  concerned about the challenges of implementing it.
• 3:56
  So what did you see in terms of patterns of what clients were looking for to address those kinds of issues?
• 4:04
  Well, people began with principles, and I think that's the right place to start.
• 4:11
  If you don't have a North Star or a set of principles for the way you use AI, you can't build any kind of governance program behind it.
• 4:20
  So principles and policies were the starting point.
• 4:24
  After a while, that became relatively easy because a number of global organizations published principles, a number of companies published principles.
• 4:32
  it's not that difficult then to adapt them, you know, for your own organization.
• 4:38
  But then the challenge is actually in the implementation piece because you need principles and policies, but how many people read policies in an organization?
• 4:47
  Let's face it.
• 4:49
  Unless you have mechanisms that actually enforce them, then they're not going to achieve their objectives.
• 4:56
  And I think to be honest, that's still where many companies are today.
• 5:01
  It's how do you build those processes which are going to put in place the controls through your selection, training, development, release, monitoring of AI that will continue to
• 5:18
  make sure that it is delivering as intended.
• 5:23
  So what's different about the way you would think about doing an AI governance program now versus when you started?
• 5:32
  Um, so less emphasis on the principles and the policies more on the process.
• 5:38
  But the other thing that is changing is an increasing, interest in technical tooling and in potentially in a platform.
• 5:47
  So I think the maturity curve, if you like, for, for AI governance is once you are getting your head around having a process, you then, particularly if you're a larger organization,
• 6:00
  you realize you need some kind of.
• 6:02
  workflow control mechanism, let's call it, which is often then a platform.
• 6:07
  So it might be a standard workflow type platform like a service now.
• 6:11
  It might be one of the existing GRC platforms, although they don't tend to be perfectly well suited for AI issues.
• 6:20
  Or as you know, there are a number of specialist companies now that are offering AI, you know, platform governance platforms, which I think can be very helpful.
• 6:30
  And then
• 6:32
  Again, as you start to get into more assessment of actual AI systems, you get more interested in the tooling, know, specialist technical tooling to help you run particular
• 6:44
  types of assessment.
• 6:46
  If we're looking at gen AI, you're going to be doing red teaming.
• 6:49
  How can you semi-automate some of the red teaming?
• 6:52
  What are the guardrail type tools that you can use to start to try to put an envelope around the generative AI usage?
• 7:02
  Where are some of the gaps still in terms of the capabilities of that tool?
• 7:08
  Well, mean, agentic, think is still very much the jury's out on how we're going to work with that and control it.
• 7:15
  mean, totally see the power and potential of agentic AI, but also a minefield, I think, in terms of the risks that could come with it.
• 7:24
  um And even generative AI, I think organizations are still struggling to come to terms with the fact
• 7:33
  That here is a very powerful technology, but one which is not 100 % reliable.
• 7:40
  I think senior executives still struggle with this idea.
• 7:43
  They used to a world in which a software program has code that delivers an output.
• 7:49
  And if something goes wrong, if there's a bug, you can trace it back to a line of code and you can fix it.
• 7:53
  um Generative AI obviously is not like that at all.
• 7:57
  it is probabilistic nature of the outcomes.
• 8:02
  um
• 8:03
  I think organizations struggle with knowing how much testing is enough and how many guardrails are enough and how do we make a reasoned trade-off between obvious benefits but
• 8:17
  still some residual risk.
• 8:21
  So it's interesting, as you said, there's a desire for tools and if you've got tools, then you want to use the tool and say, just tell me what the mitigation is, what's the
• 8:29
  optimization and get the answer.
• 8:31
  But as you point out, sometimes with with GenerA, you just can't do that.
• 8:36
  So what's the way that organizations can actually get to results using the tools that are out there and the mechanisms like red teaming that address that problem?
• 8:47
  I mean, I think it is quite simply a question of doing enough, but the problem becomes defining for you as an organization, what is enough?
• 8:59
  And that of course is also very use case specific.
• 9:02
  So enough in a retail type situation may be very different to a health or a safety critical situation.
• 9:11
  So there are no general rules I think that you can apply.
• 9:16
  as to what is enough testing, know, what are enough in the way of um guard rails.
• 9:22
  And I've seen some organizations, for example, um in early release stages, particularly of, let's say a generative AI powered customer service assistant, actually having a
• 9:36
  combination of generative AI answers and deterministic programmed answers where
• 9:44
  they can detect if questions stray into particular areas where they're not comfortable that the Gen-A answers will stay enough on track and they get diverted off to
• 9:56
  old-fashioned, know, rules-based answers.
• 10:00
  And I don't think that's a bad approach.
• 10:01
  You know, uh if there are areas that are particularly high-risk or sensitive, then use other tools to control those.
• 10:13
  One of the big reasons I wanted to talk to you is that you've just published a book called, Covering the Machine with Miriam Bogle and Paul Donkhan.
• 10:19
  First, talk a little bit about how that project came about.
• 10:24
  Yeah, guess.
• 10:25
  I'm actually by by university training a linguist.
• 10:30
  So I've always enjoyed reading and writing.
• 10:35
  And I've been with Accenture, you know, a long time and advising companies and Accenture on building governance programs and increasingly seeing a real appetite on the part of
• 10:47
  organizations to use AI.
• 10:49
  But still this hesitation, this nervousness about
• 10:53
  Is it going to deliver the results that we expect or is it going to somehow go wrong?
• 10:58
  And when I looked around on the marketplace, I couldn't actually see any books that really set out a pragmatic blueprint, a roadmap of, know, steps one through N that you should
• 11:11
  follow in order to set up a governance program.
• 11:15
  And that's essentially what I've been doing for quite a few years in advising companies and Accenture.
• 11:20
  And so I thought, you know what?
• 11:22
  It's time for a change.
• 11:23
  I'm going to step back from my role at Accenture and have a go at trying to put down on paper my thoughts in this space.
• 11:34
  And after I've been writing for a while, I realized that actually I would really benefit from collaborating with somebody who is a trained lawyer, because clearly there are legal
• 11:46
  aspects.
• 11:46
  And I understand the legal landscape, but I'm not a lawyer by background.
• 11:51
  And Miriam brings that.
• 11:52
  legal, you know, focus to things.
• 11:55
  She also very helpfully brought a US perspective, which I think is very valuable, particularly given changes in landscape in the US and Europe, etc., you know, over the
• 12:05
  last 12 months.
• 12:06
  And then Paul, I reached out to Paul because he has a PhD in actually in agentic AI.
• 12:15
  It's interesting to think he did his PhD in agentic AI in the 90s.
• 12:19
  So there was
• 12:21
  Mm-hmm.
• 12:22
  happening back then, because I wanted somebody who was deeply technical to take the lead on writing the more technical chapters.
• 12:30
  And I think the book is a much better product from the fact that we all were involved in each other's chapters, you both of them are heavily involved in this space.
• 12:40
  And so that critical process of
• 12:44
  challenging and debating and brainstorming, think, was extremely helpful and actually very enjoyable too.
• 12:51
  What would you say is the key message that uh business people should take away from reading this book?
• 12:58
  I mean, I number one for us is that governance is not a nasty negative compliance thing.
• 13:05
  It's actually what gives you the confidence to use AI.
• 13:10
  No company that I've met wants AI to go off the rails.
• 13:13
  They don't want it to harm customers.
• 13:15
  They don't want it to go wrong and deliver the wrong results.
• 13:18
  And governance fundamentally is about putting in place the right assessments and controls so that you can
• 13:26
  happily deploy AI knowing that you've done everything possible to make sure that it's going to deliver what you intend it to deliver.
• 13:34
  And it's as simple as that.
• 13:38
  Is AI governance mature enough that you can give companies a step-by-step guideline now?
• 13:45
  I think it is.
• 13:46
  mean, look, AI governance will never stop maturing.
• 13:51
  Regulations change, technology changes, the way companies use technology changes.
• 13:56
  So it needs to be an evolving, you know, continual journey.
• 14:02
  But I think the core fundamentals, the foundations, if you like, I think you can put those in place and there is enough learning to understand, you know, what those should be.
• 14:14
  Learning from
• 14:15
  Governance more generally, not just for AI.
• 14:18
  Governance and risk control has been around for a long time in many different areas, so we can learn from that.
• 14:24
  And we have a decent amount of experience with governing data and then moving into governing analytics and so forth.
• 14:32
  And maybe to share one anecdote, I remember being in a meeting, was part of a panel discussion with a group of lawyers and general counsel.
• 14:44
  Um, very shortly after chat GPT was released.
• 14:48
  And one of the questions to the audience was, so, you know, what, what are you doing about chat GPT in your company?
• 14:55
  And the audience was kind of split.
• 14:56
  There were a number of companies that have said, we've banned it.
• 14:59
  know, we've locked it down.
• 15:00
  InfoSec have excluded it from the network, et cetera.
• 15:05
  Um, problem solved.
• 15:08
  and there were others that were saying, well, you know, we've updated this policy or we've done this training or whatever.
• 15:14
  And the key difference was the companies that had no AI governance or AI policy at all beforehand were the ones that had to resort to locking it down because they had no
• 15:25
  scaffolding to build on.
• 15:27
  The companies that already had some kind of AI and data governance program could relatively quickly look at it and say, okay, world's changed.
• 15:37
  We need to build in a bunch of new things.
• 15:40
  but we have something to work with.
• 15:42
  We have risk committees that look at data governance.
• 15:44
  We have procedures that people go through.
• 15:47
  We can add in some additional questions, tests, et cetera, and we can relatively quickly get on top of it.
• 15:53
  So that's why I think that putting in place that foundational governance now is so important, but you can then adapt as we see new developments.
• 16:03
  You start the book by going through a whole series of AI risks, nine different categories of risks.
• 16:09
  How do organizations avoid being overwhelmed and how can they prioritize which risks are most salient for them?
• 16:17
  Yes, I mean, I do always worry that we come across as very negative when we look at all the risks in this.
• 16:23
  But the idea is that that is hopefully a reasonably comprehensive list of risks.
• 16:28
  Not all will apply to all organizations by any means.
• 16:31
  I was actually giving a talk last night and for that particular audience, I selected four or five of the risks that I thought were most salient for those, you know, that industry
• 16:43
  and those particular companies.
• 16:45
  So I think it's a question of focusing on the way that you intend to use AI and then looking down that list of risks and saying, well, this risk would apply, this risk would
• 16:56
  not apply.
• 16:58
  The other thing, of course, is the context determines how impactful those risks could be.
• 17:06
  What could be the impact on people, environment, society, whatever, and if it is low impact,
• 17:14
  I wouldn't worry too much about the governance.
• 17:16
  Get on, innovate, as much as you can.
• 17:18
  It's in the areas of potential high impact, high harm that you need to pay particular attention.
• 17:26
  How should organizations get started down this path of building these governance processes?
• 17:34
  I guess the first thing you need to do is to appoint someone to lead it, as with any new program.
• 17:40
  You've got to have someone who's committed and actually wants to do this and drive the program.
• 17:45
  And that person also ideally needs to have some budget and they need to have the political clout to coalesce people around it because you can't do AI governance in a vacuum.
• 17:59
  You know, need technical people, you need legal people, you need InfoSec, you need risk, know, procurement.
• 18:05
  You've got to have a lot of different teams involved.
• 18:07
  So it really helps if the leader is somebody who can actually bring that coalition together across the organization.
• 18:16
  Then my starting point is actually what I call checkpoints.
• 18:21
  And what I mean by that is you want to be able to build an inventory or you want to be able to identify.
• 18:28
  where people are planning to use AI, because that's the point where you want to ask this question, is your proposed new product or service or whatever, either that you're building
• 18:38
  or you're buying, does it have AI in it?
• 18:41
  And if it has AI in it, is it going to be high risk?
• 18:44
  What are you actually going to do with that AI?
• 18:46
  And is it potentially high risk?
• 18:48
  Because then you can ask further assessment questions, et cetera.
• 18:52
  And I often find that's actually really
• 18:55
  difficult, you finding those checkpoints because you want to be as watertight as possible.
• 19:02
  So I often go after the budget allocation process because typically, you know, if you're going to either build, allocate people and time to build something, or you're going to buy
• 19:12
  in a third party system, there's probably some money attached to that, which probably needs an approval.
• 19:18
  And that may be a good point to try to identify where AI is going to be used.
• 19:23
  But in large organizations, multinationals, multifaceted organizations in particular, it's really tough.
• 19:30
  And nowadays even harder because you're getting, you know, low code, no code type ways that people can start using AI tools like Copilot that are open to potentially anyone in
• 19:43
  the organization.
• 19:44
  That's making it much harder to nail where AI is actually being used.
• 19:51
  So two questions to follow up on what you just said.
• 19:54
  First, you said start with hiring a person or designating a person.
• 19:58
  Isn't there a danger that if there's someone who's the responsible AI person, then everyone else in the organization says, great, now it's not my problem.
• 20:06
  They're thinking about it.
• 20:08
  Yes, and actually I think if you can get someone who is a business leader to take on the program, I think that's even more effective than, you know, classically we see CDO or
• 20:26
  maybe general counsel and sometimes CISO taking it on.
• 20:35
  In a perfect world, I would select someone who is actually business facing and a business lead because they are looking at how do I get value out of the AI?
• 20:46
  they, take a value first lens rather than a control lens.
• 20:50
  And a CDO might well, you know, have that perspective as well.
• 20:54
  Um, and I think that's a good way of framing the program rather than as a legal or compliance type program.
• 21:05
  Um, it also helps to avoid that problem of people saying, well, legal will take care of that.
• 21:12
  Or, you know, infosec guys will take care of that.
• 21:15
  And that's the slight danger with it being in CDO.
• 21:19
  There's a slight danger that business people say, Oh, well, that's a technical problem.
• 21:22
  You know, it's, it's, it's not our issue, but as, as you know, this is not just a technical issue by any means, this is very much a people and process and tech issue.
• 21:33
  And, and it's very much about those who are using.
• 21:36
  designing and using AI, understanding it.
• 21:39
  And those aren't just the people by any means.
• 21:42
  It's also an incentive question, right?
• 21:44
  That if responsible AI gets associated with uh slowing things down or this is about protecting ourselves and it's in a different bucket than things that are valuable for the
• 21:54
  organization, then ultimately that's a losing game.
• 21:57
  Yeah, absolutely.
• 21:59
  Absolutely.
• 21:59
  And interestingly enough, I'm seeing an increasing number of companies that, you know, are coming and asking for advice about how to shorten their governance process.
• 22:11
  Precisely that for that reason, because over time, for whatever reasons, the governance has kind of piled on.
• 22:18
  And then they now they look at it and they see it taking months to get use cases through governance.
• 22:24
  That's obviously no good to anyone.
• 22:26
  and will really upset the business folks.
• 22:29
  So actually, how can you streamline that to get it down to weeks or hopefully days?
• 22:34
  And I think a really important point there is getting people to build it in from the beginning and through the process.
• 22:41
  The worst thing you can do is have it as something at the end, because then if you find a problem, you go right back to square one, and that's painful, it's expensive, and so on.
• 22:52
  But if you can get people to adopt the idea of
• 22:57
  Asking these questions as they go through the process, it should just be a very simple, yeah, of course you pass at the end because you've done everything all the way through.
• 23:08
  Okay, and then the second thing that you said before was about high risk.
• 23:11
  And obviously it makes sense that some AI users are riskier than others and we should be most concerned with the ones that have that risk.
• 23:17
  But this is something, of course, that the European Union has struggled with as well in the AI Act, that anytime you have a category that says if something's high risk, then a
• 23:26
  whole bunch more applies to it, it's going to be difficult to draw those lines.
• 23:30
  And again, the incentives are going to be to say this is not high risk.
• 23:32
  So how can organizations in a practical way make those distinctions?
• 23:37
  Yeah, that's absolutely spot on.
• 23:39
  And actually, I think it even goes back to the question of definition of AI.
• 23:43
  So, you know, when you ask that question, does your product or service include AI, depending on how you define it, people will very, you know, people may decide to not
• 23:55
  classify their solution as having AI in it, in order to avoid having to go through the, you know, go through the assessment process.
• 24:05
  I saw one company take an interesting approach, perhaps a little time consuming for, you might not work for everyone, but they deliberately defined high risk, not by asking
• 24:20
  anything about the tools or the risk category, but they just asked people to describe what is your AI going to do?
• 24:31
  What is your service or your product you're designing going to do?
• 24:35
  And then that description, which could be filled in by any business person, didn't need a technical person to assess it, went then to a central team that looked at it and they
• 24:47
  decided, could this be potentially high risk or not?
• 24:51
  The idea was to make it very easy for non-technical people to complete the form filling, if you like, to enter the assessment process, but also to leave it to somebody else to
• 25:00
  decide, could this be higher risk or not?
• 25:05
  But I agree with you, it's a challenge, that definition point.
• 25:09
  Yeah, that's interesting.
• 25:10
  And it makes me think also that sometimes it's obvious what's high risk and sometimes it's not.
• 25:13
  So if we're talking about medical use case where the AI is going to be involved in life and death decisions, we can all see that's high risk.
• 25:20
  But if you've got something like, have we checked that we actually have the rights to the data that we're using training this model, we might be exposed to intellectual property
• 25:28
  liability, something like that.
• 25:29
  The people building the tool might not have any idea and that might be a massive risk for the company.
• 25:34
  Yes, absolutely.
• 25:36
  Yes.
• 25:37
  And you can't generalize by industry either.
• 25:41
  I remember um in one situation looking at it and we were talking about defense use cases and the immediate reaction was, well, everything in defense is going to be high risk,
• 25:51
  right?
• 25:52
  And then somebody said, well, what if they use AI to divide, to develop the menu in the canteen?
• 25:58
  That's pretty harmless, right?
• 25:59
  So you can't just say because it's the defense industry, it's necessarily high risk.
• 26:04
  It's very specific.
• 26:06
  So let's talk a little bit about regulation.
• 26:09
  As we're speaking now, the European Union is talking about delaying these high-risk requirements in the AI Act.
• 26:16
  The US under the Trump administration is very much taking a deregulatory approach on AI.
• 26:21
  The UK has a really interesting position in the middle.
• 26:24
  What's your general sense of where we are and where we're going in terms of regulation and policy influencing these decisions in companies?
• 26:33
  Yeah, I think it's a very interesting time at the moment.
• 26:35
  I mean, I guess first of all, on the EU, it's obviously not, it's not clear that the proposal will pass.
• 26:42
  There are a lot of complicated hurdles within the EU mechanism that it needs to go through.
• 26:47
  And there's quite a bit of opposition in various different places.
• 26:50
  So it may or may not happen.
• 26:51
  For that reason, I also think that most companies will not slow down their implementation of plans for the EU AI Act because you can't be sure that it's
• 27:03
  going to get delayed.
• 27:04
  And so you actually do need to be ready for next August.
• 27:07
  And you're not going to know for another number of months whether it's going to get delayed or not.
• 27:14
  I still see quite a number of companies looking at this is global companies looking at the EU AI Act and saying, we have to comply with that for Europe.
• 27:26
  So you know what we might as well adhere to that standard globally, because it's really tough.
• 27:32
  to try and have a different standard in a bunch of different countries.
• 27:36
  We may make some accommodations in particular geographies at a more local level, but set our global standard based on the EU AI Act and then give some latitude to local companies
• 27:48
  according to market circumstances, et cetera, to, you know, to, to deviate in certain situations, which is a little bit what we've seen with GDPR as well.
• 27:57
  You know, that kind of became a global standard.
• 27:59
  Of course, there were some
• 28:00
  variations, but that's broadly speaking.
• 28:03
  And also the UAI Act has been copied to a greater or lesser extent in quite a few different countries.
• 28:10
  You bits of it have been borrowed in quite a number of different countries in the global south and in other in other areas as well.
• 28:18
  I think if it hadn't been for the very, let's say, aggressive anti regulatory stance of the Trump administration, I think
• 28:28
  I would have been quite confident it would go in that direction.
• 28:30
  think given the current US framing, I'm not as confident, but I still think it will probably go in that direction.
• 28:40
  And the UK likes to be, say they're somewhere in the middle, but are we really significant enough as an economy that people are gonna change the way they do things because of a
• 28:52
  situation in the UK?
• 28:55
  maybe for some startups that are based here and a few industries that are local perhaps, but I'm not sure we're going to really have major influence on the way that the larger
• 29:04
  corporations move.
• 29:07
  So this leads to my last question, this discussion about how things are moving on the regulatory front.
• 29:13
  One of the challenges about writing a book around technology is takes a while to write it, then takes some time for it comes out and the world moves.
• 29:19
  So if you were starting writing this book today, would you do anything different?
• 29:26
  Well, one thing we did do to try and address exactly that concern was we wrote quickly and the publisher dramatically shortened their normal lead times, which was very helpful.
• 29:38
  I think we would include more on the tooling and approaches for agentic AI now because, the book was finished.
• 29:49
  Our first draft was finished in sort of February, March time, and then we were making
• 29:53
  small revisions, proofing, corrections, et cetera, through to about June.
• 29:57
  That was still pretty early days actually for governing agentic systems.
• 30:03
  And a number of things have come out, more papers, more research, a bit more practical experience since then.
• 30:10
  We still don't have all the answers by any means, but you would have a few more things to put in about agentic.
• 30:15
  Other than that, I don't think I would change much else actually.
• 30:19
  And we deliberately tried to...
• 30:21
  create a framework that we felt as best as possible would have longevity and that people could build on and would evolve with time.
• 30:32
  Ray, thank you so much for joining me and good luck with the book.
• 30:36
  Thank you very much, Kevin.
• 30:37
  It was a pleasure.
• 30:38
  Interesting discussion.