Latest / The Road to Accountable AI / Kevin Werbach: Why You Should Care About AI Accountability

Transcript

Kevin Werbach: Why You Should Care About AI Accountability · The Road to Accountable AI

0:01
Welcome to the Road to Accountable AI.
0:03
I'm Kevin Werbach, chair of the Department of Legal Studies and Business Ethics at
0:07
the Wharton School of the University of Pennsylvania.
0:10
I've been studying emerging tech my entire career, with a focus on the complex
0:14
interactions among technology, law, and business.
0:18
On this podcast, I'll interview leaders in the fast developing area of responsible,
0:22
safe, and trustworthy artificial intelligence, or as I like to call it,
0:27
accountable AI.
0:29
You'll hear from
0:30
practitioners, academics, and policymakers working to address the real-life
0:34
challenges of making AI systems that serve human needs while meeting important legal
0:40
and ethical obligations.
0:42
My goal is to dig into the state of practice among leading-edge organizations,
0:47
as well as current public policy developments around the world.
0:51
With the rapid development and deployment of AI, we're well past the point.
0:55
where high level aspirations, statements of principles, and thought experiments are
1:00
sufficient.
1:01
No one wants to see AI systems that discriminate against people from certain
1:05
groups, for example, but how do organizations actually tackle the
1:09
challenges of algorithmic bias?
1:12
Just how much is AI really a black box?
1:15
And what can be done to make it more explainable and interpretable?
1:19
What are the best practices for handling the data that powers AI systems?
1:23
and building models that are robust to the inevitable challenges over time?
1:28
Is there anything that can be done about the dangers of deepfakes or concerns about
1:31
intellectual property violations with generative AI systems?
1:35
And just what are the legal and regulatory obligations that those building and
1:40
deploying AI systems need to address?
1:44
These are the kinds of questions that thoughtful people in the AI space are
1:48
thinking about and trying to address today.
1:52
Some have been doing so for years, deploying increasingly sophisticated
1:55
approaches.
1:57
On this podcast, I'll engage with the experts to give you actionable insights
2:01
for this important set of challenges.
2:04
For this initial episode, I'd like to provide some context for how I think about
2:08
the concept of accountable AI and the major issues that you should be focusing
2:13
on in this area.
2:14
And the key question of why your organization or you as an individual
2:20
should put the time and effort in necessary to do this work.
2:25
I also periodically post my analysis and commentary on Substack at acco I hope
2:32
you'll consider subscribing there for updates.
2:35
So why should you care about accountable AI?
2:40
In mid 2023, the Pew Research Center surveyed Americans on their views about
2:45
AI.
2:46
The results were quite frankly, pretty grim.
2:49
For those of us excited about AI's potential, despite the massive coverage
2:54
around ChatGPT and other powerful new generative AI platforms and a decade
2:58
before that of spectacular deployments of deep learning technology, a majority of
3:03
respondents said they were more concerned than excited about the increased use of AI
3:09
in daily life.
3:11
More concerned than excited.
3:13
Only 10% said they were more excited than concerned.
3:17
Now,
3:17
Some of this is a reflection of general worries about the power of big tech
3:21
platforms.
3:22
That didn't start with AI, that won't end with AI.
3:25
But one of the most significant aspects of the current AI boom is that it's not just
3:29
Google and Microsoft and similar big companies that are investing in or
3:33
deploying AI solutions.
3:35
AI has applications for virtually every industry, virtually every company.
3:41
Already, almost every substantial firm that you can think of.
3:45
is already at least experimenting with either generative AI, predictive models,
3:51
or both.
3:52
And AI is being incorporated into tools for functions like hiring, marketing, data
3:57
analysis, and customer support that are used by even the smallest companies.
4:02
Not to mention foundational productivity tools like word processors, email clients,
4:06
and spreadsheets.
4:08
According to the conference board, a majority of all US workers are already
4:13
using generative AI.
4:16
Often without the approval or knowledge of their employers, of course.
4:20
Now, one possibility is that AI has no serious problems, and people are just
4:26
unreasonably fearful about the dangers it might cause.
4:30
Even if that's the case, concerns about AI will reduce adoption, create customer
4:35
backlash, and lead companies to hesitate rather than innovate.
4:39
It will also lead to overly restrictive policies and laws.
4:43
And the reality is that...
4:45
The worries about AI are not unfounded.
4:49
For years, there have been countless examples of AI systems that, for example,
4:53
show racial and gender disparities in recognizing faces, recommending people for
4:57
jobs, and issuing credit.
5:00
The recent wave of generative AI popularized the phenomenon of
5:03
hallucination, where chat bots confidently relate facts that are not factual.
5:09
Deepfakes and other AI-generated misinformation threaten the integrity of
5:13
our elections.
5:14
and provide opportunities for harassment and crime.
5:17
Autonomous cars crash, and AI-driven systems for health insurance and public
5:22
assistance unfairly deny benefits.
5:25
We've seen it already now, time and time again.
5:30
The Organization for Economic Cooperation and Development, the OECD, which is an
5:34
intergovernmental organization, operates what they call an AI incident database
5:38
that tracks reported examples of AI failures and controversies.
5:42
It now includes nearly 8,000 incidents.
5:46
And while the database goes back to 2014, more than three quarters of all those
5:51
incidents are since the start of 2023.
5:54
This is a big problem.
5:57
And that's just incidents that are reported somewhere that make it into the
6:00
database.
6:00
There are surely many more that don't even get tracked.
6:05
You'll notice I haven't even mentioned concerns about super intelligent AI
6:09
turning on humanity.
6:11
While there are important questions around the capabilities of so-called frontier
6:15
models, the problems I'm talking about are not hypothetical future possibilities.
6:21
They are happening right now, today.
6:24
And if there's one prediction I'm confident about, it's that the rapid
6:28
adoption of AI around the world is not going to slow down and it's going to raise
6:33
the stakes further.
6:35
Already, 18% of companies in the S&P 500
6:38
mention AI in their 2023 annual reports as a risk factor.
6:44
And that probably should have been higher.
6:47
So if you're interested in AI, whether as a developer, a manager, an investor or a
6:52
user, you need to be aware of AI's problems and limitations along with its
6:58
benefits and possibilities.
7:01
And you need to understand what can be done to address those problems and
7:04
limitations.
7:06
The bad news is there's no magic bullet.
7:09
The technical tools for trustworthy AI are out there.
7:12
They continue to improve, but you can't just plug in a fairness or privacy module
7:17
and be done with it.
7:18
Accountable AI necessarily involves business decisions, ethical decisions, and
7:24
considerations of trade-offs.
7:26
It also requires organizational processes and culture so that the available
7:31
responses are actually implemented.
7:35
Let me say a little bit about why I decided to launch this podcast.
7:39
As I said, I've been working in the area of technology policy for nearly 30 years,
7:44
when I served as counsel for new technology policy originally in the 1990s
7:50
at the U.S.
7:50
Federal Communications Commission.
7:52
That was in the early days of the commercial internet.
7:55
And back then, less than 50 million people worldwide were online.
8:01
The vast majority of them were in the U.S.
8:03
and almost everyone was dialing up for access over their phone lines.
8:07
Most of today's internet firms like Google and Facebook hadn't even been founded.
8:12
The iPhone wouldn't be launched for another decade and streaming video was a
8:16
futuristic fantasy.
8:18
Yet even then, we could see the important public policy challenges the internet
8:22
raised.
8:23
It was clear that government and the private sector would have to deal with
8:27
questions like privacy, intellectual property, consumer protection, and
8:32
assigning legal responsibility for harms.
8:34
I worked with the White House and other federal agencies on the first governmental
8:38
policy framework on e-commerce, which set the tone worldwide.
8:44
Ever since then, I've looked for emerging technology trends and inflection points
8:49
when technologies suddenly accelerate their adoption.
8:53
That is what has drawn me to AI.
8:55
My degree is in law, but for two decades, I've worked at Wharton, one of the world's
8:59
top business schools.
9:01
So I focused on the confluence of legal, regulatory, and ethical questions and
9:06
their relevance for business.
9:08
Along the way, I've written books on gamification and blockchain, written on
9:12
topics ranging from wireless spectrum sharing to network neutrality to China's
9:16
data-driven social credit system.
9:18
And the reality, as I've found, is that public policy is messy.
9:24
What laws are adopted and the gray areas they leave and how they're interpreted are
9:29
all complicated.
9:30
human-driven phenomena.
9:32
The same is true about how organizations behave.
9:35
Companies, even the biggest and most powerful ones, are communities of people,
9:39
making countless decisions that call for judgment.
9:43
The organizations that do the best have an understanding of the issues and a
9:46
perspective on macro trends, but also cultures that reward both creative
9:51
innovation and doing the right thing.
9:55
So that brings me to AI.
9:57
When I look at AI today, I see incredible promise.
10:00
both for economic efficiency and other financial benefits, as well as in
10:03
improving human lives.
10:05
Some of that promise is already being realized, but there are huge challenges
10:09
and uncertainties.
10:11
Companies and governments are responding differently to those challenges as they
10:14
always do.
10:15
Microsoft, Google, Salesforce, and Meta are all big tech companies, but they have
10:19
very different cultures if you know them.
10:23
And the culture of say Microsoft today is completely different than it was 20 years
10:27
ago.
10:28
Similarly, you can't...
10:29
really compare the legal environment for technology in the US, Europe, and China or
10:35
anywhere else without understanding the broader political and economic
10:39
differences.
10:40
The field of AI has been around since the 1950s and has gone through several boom
10:45
and bust cycles where technology seemed poised for broader adoption but never
10:49
quite realized its potential.
10:52
This time though, something is different.
10:54
A set of technologies known as deep learning, combined with the massive
10:58
amounts of data, now accessible through digital platforms, and the massive
11:02
collective computing power of the cloud, suddenly demonstrated remarkable
11:07
performance on tasks like image classification and machine translation in
11:11
the early 2010s.
11:13
The big tech platforms like Google, Microsoft, Amazon, and Alibaba began
11:17
investing heavily in AI, as did countless other companies.
11:21
Recognizing the power back then,
11:24
of that machine learning revolution and the larger potential of data-driven
11:27
techniques, the Wharton School, where I'm a professor, created a cross-disciplinary
11:32
concentration in business analytics.
11:34
This was around 2015 that this got off the ground.
11:38
I was part of the committee that developed that program.
11:41
And as we were discussing it, I made the point to my colleagues that even as we
11:44
taught our students the methods to harness the power of AI and analytics, the
11:49
techniques for what to do to build these kinds of systems,
11:53
We also had to address the significant challenges that were raised.
11:57
We had to address the ethical and societal problems with AI and other algorithmic
12:02
systems, as well as the range of legal and regulatory issues that might be relevant,
12:08
even if there wasn't yet specific legislation.
12:11
To their credit, all of my colleagues agreed with me, but none of the others had
12:15
the expertise to teach a course on those topics.
12:18
And so I remember distinctly, they all looked at me and said,
12:22
OK, what are you going to do?
12:24
And it turned out at the time there were very few courses on ethical data science
12:28
anywhere.
12:30
And as far as I could tell, none existed in a business school.
12:33
So I created one called Big Data, Big Responsibilities, that launched in 2016.
12:39
And it highlighted issues such as accuracy, privacy, fairness, and
12:44
manipulation with all forms of algorithmic systems, including AI.
12:48
And since then, I've taught students about
12:50
the evolving set of ethical and legal challenges as the technology continues to
12:55
develop while following important market evolutions in my research.
13:01
Three things have happened that convinced me to launch this podcast on top of the
13:07
other research and teaching that I'm doing.
13:09
The first is we've had another inflection point with the maturation of generative
13:13
AI.
13:14
The launch of ChatGPT in late 2022 was in some ways just another step on a long...
13:19
technical journey.
13:21
It was based on foundations that have been set several years before in academic work
13:27
and publicly accessible work at companies like Google, and then wave after wave of
13:32
GPT technology developed by OpenAI.
13:36
But in other ways, ChatGPT was a shot heard around the world.
13:41
It was the starting gun for a massive global race to implement generative AI far
13:47
more broadly and deeply than ever before.
13:49
the capability was just too great to ignore.
13:54
It turns out though that generative AI raises virtually all of the legal and
13:59
ethical issues that were already on the table with other machine learning
14:02
approaches and adds a few of its own.
14:06
One of the topics I plan to discuss with some of my guests is where accountable AI
14:10
challenges are unique to generative AI.
14:13
For example, with concerns about copyright protection, about...
14:17
works that are created by these AI platforms or the training data that's used
14:22
to scrape and train them, and where the issues are equally relevant to the
14:28
established predictive and discriminative AI systems or to other kinds of analytics
14:33
that might not even deserve the label of AI.
14:37
That's the first major development, the rise of generative AI.
14:40
The second one that convinced me to do this podcast is on the positive side of
14:44
the ledger.
14:45
For years, I would teach students about what could go wrong with AI and often did.
14:50
And there are so many examples of those problems that students would very quickly
14:58
say, okay, you've convinced us, professor, what do we do about it?
15:02
How do we respond?
15:04
What are the best practices that companies are using to at least mitigate those
15:09
harms?
15:11
And it turned out for a long time, I didn't have a good answer.
15:14
There were always too few examples about how the best companies are actually
15:20
thinking about these challenges and how are they putting into place systems and
15:25
mechanisms and policies for governance of AI, for implementing these systems
15:32
responsibly, safely and ethically.
15:35
Again, AI systems don't usually cause harm because they're deployed by evil heartless
15:39
people, although there are some exceptions.
15:42
The dangerous consequences usually, though, are unintended.
15:45
For a long time, though, there was only vague guidance for thoughtful
15:49
organizations and individuals who wanted to address those problems.
15:53
Now, fortunately, that's changed.
15:54
There are now many organizations taking the lead to build accountable AI programs
15:59
and others providing services like AI audits that can help firms do so.
16:06
We still got a long way to go on the road to accountable AI, but the path is now
16:10
well-marked.
16:11
by those who want to follow best practices and understand their options.
16:14
This podcast will bring you some of the most prominent and sophisticated thinkers
16:18
addressing these challenges in real organizations.
16:21
The third major development is that AI is now a big topic among regulators,
16:25
legislators, and other government agencies throughout the world.
16:29
As I said, there were important legal concerns for AI even before there were any
16:32
AI specific laws, but now there's more than ever to be aware of.
16:36
I was in the US government when the first battles were fought over regulating the
16:39
internet in the 1990s, and I've engaged in many tech policy debates since then.
16:44
AI, I am convinced, is the next massive wave of technology, legal, and regulatory
16:51
development.
16:52
You need to understand what is happening and what is likely to happen and how it
16:56
might cover whatever organization or activity you're involved with.
17:01
So these are the kinds of activities and the kinds of issues.
17:05
that I'll explore with my guests on the podcast.
17:08
My goal is to focus on the practical.
17:11
How do companies with effective accountable AI efforts structure their
17:15
programs?
17:16
What are the most important steps to take and the most important missteps to avoid?
17:20
What will the real implications be of the AI legislation and regulatory activities
17:26
that are under development?
17:28
In a recent post on my sub stack, I listed five reasons for investing the time and
17:33
resources necessary
17:34
to engage successfully in accountable AI.
17:37
So that's the last thing that I'd like to leave you with in this introductory
17:41
episode.
17:42
What I find interesting is that in many ways, these arguments that I'm going to
17:46
give you are the same ones for why companies should engage in ethical or
17:51
responsible business behavior more generally.
17:55
For many years, Wharton has had a required class for MBA students on ethical and
17:58
legal responsibility.
17:59
When I started teaching it,
18:01
I was worried that it would be difficult to convince our students to take these
18:05
issues seriously.
18:06
They're business students, of course.
18:08
But what I found was that if presented properly, these are topics that students
18:12
relish the opportunity to consider because they know they'll face difficult questions
18:16
as they move throughout their business careers.
18:19
And they realize that these are not things that you will necessarily understand how
18:22
to address just by learning the other kinds of skills that are necessary in
18:26
business.
18:27
The trick is not to pretend that law and ethics are simple.
18:30
that I or anyone else can always tell you the right answer, that good people always
18:34
do good things, or that bad things always come from bad people.
18:38
Instead, it's essential to understand that the legal context and ethical frameworks
18:43
that develop throughout history provide some context for developing a thoughtful
18:50
approach that makes sense in the particular unique context that you are
18:54
going to find in your business situation.
18:57
And then with those frameworks,
18:59
you can learn from the experience of considering different examples if you're
19:04
able to work through those case studies.
19:06
And all of that's relevant to AI as well.
19:08
All of that is part of the foundation for having the conversations that I'm going to
19:12
have on this podcast.
19:14
The five arguments very quickly for investing in accountable AI are that it's
19:18
the right thing to do, risk, trust, business benefits, and regulation.
19:24
The first rationale starts from the realization that actions have
19:27
consequences.
19:28
Yes, computers are just machines.
19:30
And we call AI is just a bunch of math running on those machines.
19:34
But it's a big mistake to leap from there to the idea that any technology can be
19:38
neutral or immune from human influence.
19:41
And we know AI systems aren't perfect, but it's a big mistake to leap from there to
19:48
the idea that because, I screwed up, you know what, I read something twice.
19:55
All right.
19:56
He's doing so great.
19:57
Let me go back to him.
19:58
Hopefully he can probably just cut that out.
20:00
There's a line that's here twice.
20:15
Yeah, there was two sentences that were repeated twice.
20:17
I hope I'm not boring you to tears here, but it's mostly that.
20:20
All right, let me go back to being a...
20:27
The first rationale stems from the realization that actions have
20:31
consequences.
20:32
Yes, computers are just machines, and what we call AI is just a bunch of math running
20:37
on those machines.
20:38
And we know AI systems aren't perfect, but it's a big mistake to leap from there to
20:43
the idea that any technology can be neutral or immune from human influence.
20:49
If something goes wrong, those building or deploying the system bear some
20:54
responsibility.
20:55
some moral responsibility and in some cases some legal responsibility.
20:59
Do you really want to be the one though, who is deploying a system that spews hate
21:05
speech, discriminates against marginalized groups, manipulates people, or abuses
21:10
their personal information?
21:11
Sure, there can be disagreement on where to draw the lines between profit and
21:14
purpose.
21:16
But I'm very convinced that no one goes to work trying to do evil.
21:22
So thinking about doing the right thing.
21:25
is an inherent feeling that most people have in organizations, and they want to
21:30
know that they are taking at least some reasonable, acceptable steps to address
21:36
the concerns and the problems with their AI systems.
21:39
Second reason is that those AI risks can have direct negative consequences for
21:44
firms that are deploying AI.
21:46
There already have been many serious controversies, as I've talked about, about
21:49
AI implementation.
21:51
So for example, just in the area of bias and discrimination.
21:54
More than a third of the respondents in the Data Robot 2022 survey on the state of
22:00
AI bias said their firm had already experienced incidents.
22:03
And of those, 62% lost revenues, 61% lost customers, and 43% lost employees.
22:12
Those are real serious measurable impacts to companies from failing to engage in
22:18
accountable AI.
22:19
And as the use of AI increases, the awareness of those uses increase.
22:23
the numbers are only bound to go up.
22:26
Third reason is trust.
22:28
Everyone talks about the crisis of trust in today's world.
22:31
I even wrote a book about trust in the context of blockchain technology a few
22:34
years ago.
22:36
AI promises to further undermine trust in institutions thanks to developments like
22:40
manipulation, misinformation, and deepfakes.
22:43
But it also faces roadblocks when people are unwilling to trust AI systems.
22:48
This trust gap for AI is corrosive.
22:50
It means AI won't be adopted when it should be, it won't deliver the full range
22:53
of benefits when it is.
22:55
And that in turn leads to the fourth reason to care about accountable AI.
22:59
If, as I've said before, those who fail to implement AI responsibly risk negative
23:04
consequences, those on the other side will reap the benefits.
23:07
There are costs both in hard dollars for compliance and indirect cost in sometimes
23:12
slowing innovation and deployment, but those investing in competencies of
23:17
accountable AI
23:18
will reap even more significant benefits.
23:20
They can differentiate themselves from competitors, build a strong message to
23:24
their employees and customers, and develop the knowledge and the skills and the
23:29
experience in monitoring and measurement that will improve overall performance of
23:35
their AI deployments.
23:37
And then finally, there's regulation, which I've already talked about.
23:40
For being the legal Wild West that we talk about,
23:46
there's actually an awful lot of AI regulation out there.
23:49
And frankly, it's a misnomer to say that AI today is totally unregulated.
23:55
It's actually subject to a whole lot of existing law and regulation, and it's one
24:00
of the most active areas of regulatory development and investigation around the
24:04
world.
24:05
The European Union, China, and several US states have already passed AI-specific
24:09
laws.
24:10
Other countries such as the UK and Japan and the US
24:14
at the federal level have issued significant policy documents, and a host
24:17
of regulators are engaging in enforcement actions on algorithmic bias and privacy
24:21
and more under existing legal authority.
24:25
Regulation puts compliance obligations on firms, which, frankly, most of them are
24:30
ill-prepared to meet.
24:31
The 2023 KPMG AI Risk Survey found that 84% of respondents
24:38
said they believe independent audits of AI models will be required within four years
24:43
by regulation.
24:44
Yet guess what percentage of those firms felt like they were ready?
24:48
Only 18% thought they had the internal expertise to audit their AI system today.
24:54
So this is just one of many examples that the firms and the individuals who start
25:00
now, who think about and learn about the problems and concerns and limitations
25:06
around AI,
25:07
who work through the case studies and who study the organizations that are
25:12
developing these kinds of accountable AI structures, all those are the
25:15
organizations that are going to be best positioned for success.
25:20
So hopefully, if you weren't already convinced of the need for accountable AI,
25:23
you are now.
25:25
But one last thing, what exactly does accountable AI mean or any of the related
25:29
concepts such as AI governance, AI ethics, responsible AI, trustworthy AI, AI risk
25:34
management or AI safety?
25:35
There's a whole lot of terms.
25:37
I offer my perspective in more detail in some of my posts at acco But at a high
25:44
level, it's fair to say all of these refer to the same basic family of concepts, but
25:51
there are particular differences in emphasis.
25:54
I prefer accountable AI because accountability means asserting
25:59
responsibility over perspective and actual harms.
26:03
then connecting that responsibility to concrete prevention and mitigation
26:07
strategies, and at the same time, working towards standards and processes to
26:12
integrate them into business operations.
26:14
It's more than just the general sense of responsibility.
26:18
It's that entire circle of addressing it in a structured and thoughtful way.
26:24
Financial accounting and auditing are the foundations of trust, by analogy,
26:27
underlying the modern global economic system.
26:31
I think we will see accountability as essential to bringing the necessary levels
26:36
of trust to AI.
26:38
Of course, all systems of accountability have flaws and gaps.
26:41
And when you get down to the ground level as someone building or deploying an AI
26:44
system, there are many questions about how to implement such approaches.
26:48
This is where to me things get interesting.
26:51
How much should your accountable AI efforts, for example, be centralized in a
26:54
group of experts versus pushed out throughout the organization?
26:59
What...
26:59
makes an audit of an AI system effective, or a red teaming exercise for adversarial
27:04
testing of a large language model?
27:06
How can you measure fairness?
27:08
There are actually several different tests and they're often incompatible.
27:12
Where are you willing to trade off?
27:13
Interpretability of your models for performance and beyond the technical
27:16
questions, what does it take to create a truly effective culture of accountability
27:21
around AI?
27:23
These are the kinds of questions I'll be asking the experts I interview in
27:26
subsequent episodes.
27:27
You'll hear from executives at major companies, including big tech firms
27:31
developing AI, as well as those in other sectors implementing it, from policymakers
27:37
involved in some of the key legislative and regulatory processes, and influential
27:41
researchers, academics, and lawyers analyzing key developments in this area.
27:47
These experts on the front lines of accountable AI will offer practical
27:50
real-world insights for implementing effective programs and their perspectives
27:54
on the evolving regulatory and governance landscape.
27:58
AI raises an array of issues, and there are major new developments just about
28:03
every week.
28:04
Implementing AI systems effectively and doing so with attention to legal and
28:08
ethical considerations is a journey.
28:10
That's why I call this podcast, The Road to Accountable AI.
28:13
The good news is we're on a road to somewhere, and there's much to see and
28:18
talk about and think about and understand along the way.
28:22
Please join me on the journey.