Latest / The Road to Accountable AI / Kay Firth-Butterfield: Using AI Wisely

Transcript

Kay Firth-Butterfield: Using AI Wisely · The Road to Accountable AI

• 0:01
  Okay, welcome to the podcast.
• 0:03
  Thank you.
• 0:04
  It's a pleasure to be with you.
• 0:08
  You are one of the pioneers of the field of AI ethics and governance.
• 0:12
  And obviously the field has changed a lot in more than a decade really, I think that you've been doing work in this area.
• 0:20
  How have your views evolved?
• 0:24
  I think that it's evolved in the fact that since I have been doing this, people have got PhDs in responsible AI.
• 0:42
  so, you know, when we started back in 2014, there was maybe two handfuls of people.
• 0:49
  But now there is a whole movement and people qualified around responsible AI.
• 0:57
  And as I say, with people with PhDs working on this area and in the olden days, we had to really get companies to sort of think about it and push them forward to think about it
• 1:17
  even more.
• 1:19
  And this was before Google and Microsoft and anybody had any sort of AI principles.
• 1:26
  um But now companies for good and ill understand that there is a whole movement around responsible AI.
• 1:37
  And of course, now we have the OECD, we have the EU AI Act, China has regulation around AI.
• 1:47
  44 states in the union have regulation around AI, but for how long?
• 1:53
  We don't know, of course, because of the embargo and the big, beautiful bill, if that passes.
• 2:01
  And we could talk more about that later, maybe.
• 2:05
  And so, yes, I mean, it's gone from a few of us to a worldwide movement with some legislation in place.
• 2:17
  In terms of the work that you do and what you do with companies though, has anything fundamentally changed now that it's part of this much broader movement?
• 2:25
  ah Yes, mainly because of the EU AI Act, I would say.
• 2:30
  I work with a lot of Fortune 100 and Fortune 500 companies and, you know, any multinational is having to is having to be cognizant of their liabilities under the EU AI
• 2:49
  Act.
• 2:49
  ah I think that, uh you know, there was a there was
• 2:55
  the movement around AI safety.
• 2:58
  And again, we can talk about how that's fractured.
• 3:01
  um But I would say that with small and medium sized businesses, we are still seeing difficulties in them really understanding the tools that are available to them and what
• 3:20
  the risks might be.
• 3:23
  And you know, I'm a lawyer by background and in fact, I'm still part of Doughty Street Chambers.
• 3:29
  And I was talking to my fellow barristers just yesterday and they were talking about the number of cases that they were seeing where briefs had been drafted with hallucinations
• 3:45
  around cases and also
• 3:49
  where documents had been deep faked.
• 3:53
  uh And so I think that uh we're seeing the outcomes of not doing it right, and they are potentially very bad for society and business.
• 4:11
  Let me just ask you one more follow-up on the first answer you gave.
• 4:14
  And this is probably something of a question against interest because I am an academic, but isn't there potentially a downside to the fact that now there people doing PhDs in
• 4:23
  responsible AI, that it's now a structured, established set of academic disciplines, you have consulting firms going in, and in some situations that can make it harder to really
• 4:36
  focus on what matters.
• 4:40
  don't think so.
• 4:41
  I think it's great that people see it as an academic discipline.
• 4:45
  I think that it gives it rigor.
• 4:48
  um In the past, there was a lot of discussion about is this management and business or is this law or is this philosophy?
• 5:00
  And there were a lot of philosophers in the original field and they felt that this word ethics
• 5:08
  They owned this word, ethics.
• 5:11
  And so probably we're not very excited about lawyers like me talking about it.
• 5:17
  But now I think we have, now that we've developed the field, we can see that in fact all disciplines need to be part of this deep conversation about what we want from AI and how
• 5:36
  companies are delivering it.
• 5:40
  Absolutely.
• 5:41
  I have the good fortune of being a professor in a department of legal studies and business ethics within a business school.
• 5:48
  most places don't have that interdisciplinary focus.
• 5:52
  So you spent a number of years at the World Economic Forum leading uh AI work and AI governance work.
• 5:59
  And much of that involved bringing different stakeholders around the world together.
• 6:02
  So what are some of the biggest challenges to gaining consensus in those kinds of?
• 6:08
  Oh, I think that if we're just, just whether you're talking about companies or countries, um I think one of the biggest issues was the fact that we were using this word ethics.
• 6:23
  And, and, and if you're working in an international setting, then different countries uh have different ethical perspectives.
• 6:34
  And so we
• 6:36
  We very soon found ourselves in the, whose ethics are we talking about um conversation?
• 6:43
  And so I really like the fact that we have moved to responsible or trustworthy AI.
• 6:51
  I happen to prefer responsible um because that released some of those tensions and enabled us to really say, well, okay, do we want to work?
• 7:03
  Do you as a country want to work?
• 7:05
  responsibly in the social contract between yourself and your citizens um and country and companies too, you know, do you want to be the person who says I'm not responsible?
• 7:19
  It's not and that as you being in a business setting, you'll know that that doesn't do your brand value very much good.
• 7:27
  So so I think that those those things were uh
• 7:33
  problematic to begin with, but I think we found ways to move forward.
• 7:38
  um I also feel that nowadays, just, when I'm talking to companies or countries, just say, why don't we use AI wisely?
• 7:51
  Because the use of the word wise takes away any of those...
• 7:59
  those judgments, you know, I'm not making a judgment about you.
• 8:03
  I'm not making a judgment that you are irresponsible.
• 8:07
  I'm just saying that it would be good for the bottom line generally, if you use it wisely.
• 8:14
  And, and that that I think has become very helpful as we as we navigate what we've got to do at the moment.
• 8:23
  But you you asked me about
• 8:26
  back in the days of the forum, you I was there, I left in 2023.
• 8:33
  And so I was there just as we hit the cusp of ChatGPT.
• 8:41
  But before that, everything was so new, and the conversation was so new.
• 8:49
  And so I think one of the other challenges was just to get people involved in the conversation.
• 8:55
  and the art of the possible um in that conversation.
• 9:04
  Today, though, we're at this point where countries, in part because of the implications of generative AI, are pushing forward very aggressively for deployment and towards AI
• 9:15
  supremacy or dominance.
• 9:16
  And this is certainly what you hear from the Trump administration, but also in the UK.
• 9:22
  I think there's a recognition of the economic development benefits of AI and desire to push it forward, and even in the European Union, despite the interest in regulation and so
• 9:30
  forth.
• 9:31
  m
• 9:31
  In that kind of environment where countries are fixated on removing barriers to AI deployment, how to make the case for uh responsible or wise use of AI when there's a
• 9:43
  suspicion that that's going to slow things down.
• 9:46
  Yeah, think it's very, I think it's hard.
• 9:49
  I mean, it's hard with companies as well, because they've all been told that you that you know, you need to move fast, break things innovate and, and, and responsibility or wise
• 10:02
  deployment will hold you back.
• 10:04
  But there is good evidence now that um actually, moving forwards breaking things um doesn't end up
• 10:15
  with you being successful necessarily.
• 10:19
  And so I was in Davos in last year, 2024, when the CEO of Accenture in the North America said that most of their work was undoing unsafe or unsatisfactory deployments of AI.
• 10:40
  And so you could say, and so just if we take some things that companies care about, you know, one, have you educated your employees to use AI wisely?
• 10:54
  And if not, then you have set yourself up a potential problem.
• 11:01
  And, um and so I, if you, if you are not, if you're just saying,
• 11:10
  OK, we're ignoring this and we're going to be using AI in every way possible, then you're looking at using agents and you haven't thought about responsible AI.
• 11:26
  That's a recipe for disaster.
• 11:29
  And I think the other thing that people are, I hope, beginning to notice is that um
• 11:37
  the public trust in AI is plummeting.
• 11:42
  So if we look at the Edelman Trust Barometer for this year, last year, the trust in AI plummeted by 22 percentage points.
• 11:54
  So it's now down in the 40s.
• 11:57
  So the majority of people don't trust AI whilst at the same time, countries are saying,
• 12:06
  Oh, know, and companies, we better move forward and break things.
• 12:09
  So there is this disconnect.
• 12:11
  And that's really important, I think, for the way that countries should be thinking about this.
• 12:19
  Because, you know, their citizens don't trust it.
• 12:24
  And yet they need to, they feel that they need to move on and release the constraints.
• 12:35
  And so really sort of thinking about the balance of GDP versus what your uh citizens want, I think is the missing link in this conversation.
• 12:55
  Why do you think we're seeing that plummeting in trust in AI?
• 13:00
  Why do I think...
• 13:02
  Sorry.
• 13:04
  Why do you think so many polls around the world show that ordinary people's trust in AI is dropping just at the time when we see all these amazing use cases?
• 13:15
  Yeah, think that I think there are two two major factors.
• 13:20
  One is, you know, most of the world went to it by number, went to the polls in twenty twenty four.
• 13:27
  And so I think there is a lot of conflating of social media problems, which are sort of old AI problems and have been around for a long time.
• 13:38
  But they came to the fore with particularly deep fakes in these elections.
• 13:44
  ah
• 13:45
  And I think the other thing is that suddenly people are getting worried that it will take their jobs.
• 13:54
  And as we see more and more layoffs, even in the tech sector, I think that that is really trickling down into the population at large.
• 14:09
  You talked about some of the incentives on companies to be responsible.
• 14:13
  uh For example, the Accenture case of having to undo lots of problems.
• 14:19
  Where do you think are some of the main areas where governments shouldn't rely on companies taking voluntary action where there need to be enforceable standards?
• 14:30
  Yeah, I think where there is um going to be something that actually affects the human, um we should be saying, okay, has this been designed with the human in mind or has it been
• 14:48
  designed with prophet in mind, for example?
• 14:53
  So let me give you some thoughts on that.
• 14:58
  First of all, you know,
• 15:00
  are there any jobs that we think humans should continue to do?
• 15:07
  Or can all of them eventually be handed over to AI or AI embodied robots in the blue collar sector?
• 15:16
  And so one of the things that I talk about is my own experience of getting breast cancer in 2023.
• 15:27
  And
• 15:28
  when my oncologist found out what I did, she said, wouldn't it be amazing if we could use AI to walk you through your journey with cancer?
• 15:38
  And I said, wouldn't it be amazing if I helped you to put AI in your back office and that gave you enough time to walk me, the human, through my human, potentially life-threatening
• 15:54
  illness?
• 15:55
  um And I would prefer that.
• 15:58
  I know many people say to me when I talk about this, my doctor's terrible.
• 16:03
  I much prefer an AI.
• 16:06
  But those are the sort of human choices about how we, that governments should be thinking about or companies should be thinking about.
• 16:16
  So if we look at healthcare, there's a lot of AI, there's a lot of potential business for AI in healthcare.
• 16:26
  And there are a lot of things that it can do really well, but you don't want it hallucinating in the healthcare system because people will die.
• 16:37
  It's different perhaps when it hallucinates in legal cases.
• 16:45
  One of the topics that you've spoken a lot about is uh the issue broadly of algorithmic bias and fairness.
• 16:51
  And uh you made the point that the data that's used to train many of these large language models and other AI systems is not representative uh based on gender, race, global south,
• 17:02
  and so forth.
• 17:04
  What can be done about that though, given that we have access and these companies have access to the data that's out there?
• 17:12
  Yeah, absolutely.
• 17:13
  And these are some of the big questions that I feel that our governments are not addressing.
• 17:21
  um And we should be having conversations about.
• 17:28
  So uh first of all, think the bigger issue is that, you know, we talk about how AI is going to make everybody's lives better and think and
• 17:40
  make business better.
• 17:42
  But actually, it's, you know, even if even if everybody who uses Facebook is using the AI tools of Facebook, willingly and knowingly, we still don't have that many billions of
• 18:01
  people using AI, and the benefits if or
• 18:07
  risks of AI are actually sort of now stuck in the global north because there are three billion people who are not connected to the internet.
• 18:19
  So none of their data is known to an LLM and they can't use it.
• 18:25
  So that's, you know, a good proportion of the world who aren't going to benefit from AI.
• 18:31
  So I think there's accessibility issues.
• 18:36
  as well as equality issues here.
• 18:39
  And what that means um is that, you know, if you get a disease that's prominent in South Africa, for example, then the data that's available through the LLM is not actually going
• 18:56
  to be particularly trained on that disease.
• 18:58
  So it's not as helpful as it is for white men.
• 19:03
  old white men who have heart attacks in America, because all the data comes from there.
• 19:09
  And so and likewise for for women, you know, the data is the data that that these machine that that the LLMs can use um is the data that's available.
• 19:23
  And so there's more data about men having heart attacks than there is about women.
• 19:28
  And so the the just the way that the
• 19:34
  AI that we use at the moment is created is fundamentally unequal.
• 19:40
  You say, well, what can we do about it?
• 19:42
  And I say, well, probably nothing.
• 19:44
  ah And that's part of the conversation that we should be having.
• 19:51
  And not just that we can't do anything about it, but that as AI creates more and more of its own data, it is creating its own data.
• 20:03
  at a faster pace than we humans are creating data.
• 20:07
  And so that actually means that we are caught in the past because all of the data is used to create its data is that historic bias data.
• 20:18
  So those are some of the conversations that we have to have.
• 20:23
  you know, in some countries, maybe having your own sovereign database and LLM might be an answer to some of this.
• 20:33
  But generally, if we don't have those conversations, we don't know that we're stuck in the past and our governments are simply saying, let's go, then we will see error compounded on
• 20:54
  error.
• 20:55
  So let me just give you an example of that.
• 20:57
  uh
• 21:02
  litigant produces 12 uh hallucinated cases.
• 21:10
  And then that goes to the other side, and they use chat GPT and they produce 12 more, is what's happening in in legal circles.
• 21:21
  And then they bat those around using and I'm using chat GPT just because it's the one that's known not to particularly single out chat GPT.
• 21:32
  They bat around that.
• 21:34
  Well, that then, those hallucinated cases then become part of the law that the LLM knows, even though they're completely wrong.
• 21:46
  So they will come up again and again and again.
• 21:49
  And there's no way of filtering them out, saying they're wrong and you shouldn't be showing them to anybody else.
• 22:01
  No, it's a huge challenge, but with hallucination at least, they're objectively wrong and no one would say that a lawyer should cite a case that didn't actually exist.
• 22:12
  We now have, especially the United States, a political environment where there's real pushback on efforts towards fairness and equity more broadly, seeing that as a kind of
• 22:24
  discrimination in and of itself.
• 22:26
  So...
• 22:27
  You talked about the arguments that became compelling for firms to adopt responsible AI in general.
• 22:35
  Are there ways to argue for this more inclusive vision of AI today?
• 22:42
  I so.
• 22:44
  As I say, I do a lot of work with companies and um most of them are getting up to speed with the EU AI Act or have already done so.
• 22:56
  And so I think in large companies, the words may be different, but the intent is the same.
• 23:07
  And that may be driven by Europe or it may just be driven by the fact that people understand that diversity makes better business units.
• 23:17
  um So I don't think that it's going to.
• 23:25
  I don't think that it's going to stop the responsible AI movement.
• 23:33
  What I am seeing is that some companies that haven't thought about how they don't need to think about how they set up with the EU are sort of feeling a bit adrift because there
• 23:50
  aren't laws that you can follow.
• 23:52
  So a lot of companies were following the
• 23:55
  Computative Colorado AI Act that looks really like the European AI Act, or they were following some states laws around hiring, for example.
• 24:07
  But if you cut out all of those, you effectively cut out all the models that companies can follow.
• 24:16
  And so they're left by themselves to try to find new, to try to navigate a course.
• 24:24
  That means that they don't lose their brand value because of some litigation.
• 24:34
  You sound pretty optimistic about the EU AI Act as a guiding light in this area.
• 24:40
  um Can you talk a little bit more about some of the issues given the implementation challenges, the debates over standards and codes of practice and the discussions now in
• 24:50
  the European Commission about streamlining and pausing and so forth somewhat in response to the US?
• 24:57
  Are you still optimistic that it's going to achieve its goals?
• 25:01
  I am optimistic that it's going to achieve its goals.
• 25:07
  I think that, you know, after Vice President Vance went to Paris, everybody went, oh But time passes and different relationships form and we will see.
• 25:28
  I think that the shock factor.
• 25:31
  um has had time to abate and conversations are properly being had.
• 25:44
  Going forward, what would you say is the biggest thing that's missing from the discourse or from the practice of responsible AI today?
• 25:54
  um I think that, uh as I've said, I think one thing is the discourse.
• 26:03
  um We are not talking about the future that we want as human beings or as citizens or as employees.
• 26:16
  We are merely having this steamroller roll over us.
• 26:21
  And so I think that's one thing that
• 26:24
  we urgently need to have.
• 26:26
  We urgently need to say, what is AI good for in terms of the human and look for good uses of AI.
• 26:38
  We are not having a sufficiently robust conversation about sustainability.
• 26:45
  We all saw Eric Schmidt go to Congress and say,
• 26:50
  up to 99 % of the world's energy by 2030.
• 26:55
  Should we not be having a conversation about that and whether AI is what we want using all of our electricity, especially when there are billions of people who still don't have
• 27:08
  electricity connected?
• 27:11
  We are hoping that AI will enable us to find answers to climate change.
• 27:20
  we need to have a robust conversation around that as well.
• 27:25
  So I think that the conversation that used to be around responsible use actually needs to open the aperture and think about um what is good for humans and what benefits do we want
• 27:41
  from AI and what can it actually deliver?
• 27:45
  And one of the things that I actually believe
• 27:49
  that all companies should be doing is giving their employees a 101 class on AI.
• 27:57
  They don't have to make them into AI scientists.
• 28:00
  They just have to help them understand how to use the tool wisely.
• 28:05
  And that will actually protect the companies from their employees not using it wisely.
• 28:14
  and will enable those employees to see new ways of using it that they can use in their job and make themselves more productive.
• 28:23
  um I think that when you actually have your hands on it, it's less scary.
• 28:30
  um so, um and of course, then if you educate employees, employees are actually citizens.
• 28:41
  Say you end up
• 28:42
  with a much more educated conversation around these tools.
• 28:51
  Where do we have those conversations that you're talking about though?
• 28:54
  Because it's in a company's interest to maximize their profitability, which includes treating their employees well, but also if there's efficiency of laying people off, then
• 29:06
  companies are incentivized at least to do that.
• 29:09
  So if we depend on them to educate their workers as citizens, isn't that asking too much?
• 29:15
  Well, I think it's the only place that we can see it being done, especially if the states are prevented from, so in the big, beautiful bill, the um states are going to be prevented
• 29:30
  from enforcing laws on AI that they already have passed and prevented from passing new laws on AI for 10 years.
• 29:42
  That's...
• 29:42
  uh
• 29:43
  that essentially means that it takes the conversation away from the states and the states had been having good conversations in my view around the balance between regulation and
• 29:57
  innovation and leaves that conversation with companies.
• 30:05
  And you talk about
• 30:07
  Companies are built to make profit.
• 30:09
  Of course they are.
• 30:10
  And we all depend upon making profits because we invest our pensions in them.
• 30:15
  So what I'm merely saying is that actually if you educate people about AI, you get a proper discourse and you get a proper discourse in your company.
• 30:28
  You don't get such bad uses of it.
• 30:32
  You probably won't get shadow AI.
• 30:34
  You know, the using of the tools badly.
• 30:37
  on uh personal devices.
• 30:42
  And uh so you get an educated workforce, it's going to be safer for your company, and it's going to be more productive.
• 30:52
  They are going to be able to think of new ways of using AI.
• 30:56
  And there's a Spanish bank called BBVA, which also works in Latin America, and they've been gradually
• 31:06
  giving all of their employees access to an LLM and just seeing what they can build.
• 31:15
  And they've been building really interesting, innovative ideas for the company.
• 31:24
  Fantastic.
• 31:25
  We need to wrap up.
• 31:26
  Okay, thanks so much for your insights.
• 31:28
  It's been a pleasure talking with you.
• 31:30
  Likewise, thank you.