Latest / The Road to Accountable AI / Reid Blackman: Aligning AI with Ethics in the Real World

Transcript

Reid Blackman: Aligning AI with Ethics in the Real World · The Road to Accountable AI

0:01
Reed, welcome to The Road to Accountable AI.
0:05
Thanks for having me.
0:07
Tell me a little bit about how you got into this field of AI ethics, because
0:12
relatively speaking, you've been doing it for quite some time.
0:15
That's good.
0:16
Relatively speaking, I've been doing it forever.
0:19
Yeah, I mean, there's sort of a grander story, I suppose, but to sort of keep it
0:22
brief, I was a philosophy professor specializing in ethics.
0:26
I had the idea for an ethics consulting company many, many years ago.
0:30
I didn't see any kind of market for it.
0:33
Fast forward, I heard some things.
0:35
I thought, okay, maybe there's a market for this actually.
0:37
And so I left academia and I started an ethics consulting company.
0:40
with the intention of making it focus on emerging technologies and perhaps AI in
0:45
particular.
0:46
I didn't know how quickly the AI space would grow.
0:49
Thankfully, it grew quite quickly.
0:51
And so I started advising and consulting on how to do AI responsibly, ethically, at
0:58
least not negligently.
1:00
That's the goal here.
1:02
What does the academic training philosophy bring you in this kind of applied field?
1:07
Well, before people called it responsible AI, they called it AI ethics.
1:10
We're talking about the ethical implications of AI, ethical, reputational,
1:13
regulatory, and legal risks.
1:15
If you want to know something about AI ethics, you better know something about
1:18
AI, and you better know something about ethics.
1:20
And I know a bunch about ethics.
1:21
That's what the PhD and the 10 years of being a professor and the whatever it was,
1:25
15 plus years of academic research and publishing do.
1:28
So when people make certain kinds of claims about responsible AI or AI ethics,
1:33
frankly, a lot of times they just don't have a good grip on the ethics of things.
1:37
And so they say things that are false, misleading will lead them down wrong
1:40
paths.
1:41
And so having that kind of experience, understanding, frankly, expertise under my
1:45
belt means that I can navigate the conversation really well.
1:48
I'll give you one quick story to sort of put some meat on the bones of that.
1:52
When I very, when I was really just early transitioning from academic life to my own
1:57
company, I joined IEEE, that's the standardizing organization for engineers.
2:03
because they were working on this project called Ethically Aligned Design.
2:06
And it was about how do we align, what do they call it?
2:09
Autonomous and intelligent systems with our human values or something along those
2:13
lines.
2:14
And I got myself onto this committee that was in charge of figuring out what are the
2:18
guidelines or what's the advice around research and design of AI from an ethics
2:23
perspective.
2:23
And I got on the first call and I was anticipating not being able to say
2:28
anything because I thought these are a bunch of engineers.
2:30
They, I don't know.
2:31
I don't really know anything about AI.
2:32
I'm not an engineer.
2:33
I'm not a computer scientist.
2:34
I'm not a mathematician.
2:35
I'm an ethicist.
2:36
So I expected it to be a sort of meeting in which I just sat back and listened to
2:41
the experts.
2:42
What I learned within the first few minutes is, oh, this isn't some advanced
2:49
engineering discussion.
2:50
This is first semester, first year students talking about ethics.
2:55
They don't know anything.
2:56
They're engineers.
2:57
And...
2:58
What wound up happening to my sort of amazement is I wound up essentially
3:01
leading the entire conversation, just leading because they were just saying
3:06
things that, you know, my first year, first semester undergraduates used to say
3:10
about ethics and they were just fundamentally confused.
3:12
And so I went in there and I sort of led the conversation and had to clear up a
3:16
number of confusions, answer a bunch of questions, direct the discussion in way
3:20
that would actually be fruitful.
3:22
And so it turns out that it's really the ethics expertise that's more useful than
3:26
the AI expertise in a lot of these conversations.
3:28
Well, you know, not necessarily from that specific example, but what are some
3:32
misunderstandings that people tend to have if they don't have some real background?
3:36
Well, one would explain, I think, the transition from talking about AI ethics as
3:40
was the standard to talking about responsible AI as has become the standard
3:43
among corporations.
3:44
They just don't know what the heck to do with ethics.
3:47
The number one misconception, which I opened my book, Ethical Machines, with
3:51
this is they just think it's squishy, it's subjective, it's just a matter of opinion,
3:54
it's how you feel about stuff.
3:56
And if you think of ethics as just how you feel about stuff, you know, I like
4:01
chocolate, you prefer vanilla.
4:03
I like genocide, you prefer peace.
4:05
If we think of it along those ways, then of course you're not going to get your way
4:10
to actionable steps to identifying and mitigating the risks.
4:13
So unless you think about ethics as something that's more just touchy -feely
4:17
stuff, you're not going to actually get your way to designing a framework,
4:22
designing policy, designing procedures, designing metrics, designing training.
4:28
Otherwise you just tell everyone to consult how they feel about stuff.
4:31
That's not, that's silly.
4:33
And so,
4:34
Getting rid of that misconception, articulating that ethics is or at least
4:38
can be conceived of as being perfectly objective, and that there are ways of
4:42
doing ethical inquiry that are responsible versus irresponsible, that there are ways
4:47
of articulating it in concrete ways that lead to action, that's a phenomenally
4:51
useful perspective that most people simply don't have.
4:55
Let me just play devil's advocate a little bit more, though.
4:58
People talk about responsible AI, but nowadays people also talk about things
5:01
like AI governance or AI risk management or AI safety, which are all terms that
5:06
don't necessarily have that kind of ethical loading.
5:10
Do you think that's misguided?
5:12
Should we still be talking about ethics or are these different things?
5:16
So I've argued, this is a point I make in my book, that we shouldn't use the term
5:20
responsible AI, that the term responsible AI is a term that was invented by
5:24
corporates essentially, so they don't have to say the word ethics because they don't
5:27
want to deal with it.
5:28
Incidentally, this is in line with some advice that I got early on when I was
5:33
still a professor.
5:34
I was talking to an executive at Starbucks and she said, not unreasonably, you might
5:40
not want to say that you're doing ethics consulting.
5:41
People don't like the word ethics.
5:43
They don't get it.
5:45
And I said, no, no, I'm just going to own the word ethics then.
5:47
If that's the answer, I'm just going to own the word ethics.
5:50
And I changed it incidentally to ethical risk because I thought risk was a language
5:53
that businesses understand.
5:55
Now, look, there's no principled reason to oppose the term responsible AI.
6:02
It's not as though that's completely bonkers from a ethical or a conceptual
6:07
perspective.
6:07
That's reasonable.
6:08
I think though, this has been my experience that...
6:12
What happens when people use the term responsible AI in practice is that they
6:16
throw in ethics -y stuff into the responsible AI bucket, but then they throw
6:20
a bunch of other stuff in as well.
6:21
So they'll say, yes, responsible AI is ethics and safety and regulatory
6:25
compliance and engineering excellence and cybersecurity.
6:29
It's just all the stuff.
6:31
And then what happens is they say, okay, now let's address the responsible AI
6:35
bucket.
6:36
And then the experts in the room are experts on things like...
6:40
engineering excellence and cyber security and regulatory compliance.
6:43
And so what do they do?
6:45
They prioritize all of those things to the exclusion of talking about the ethics
6:51
stuff.
6:51
And so I think that they miss out on huge swaths of risk because they've decided
6:55
let's just, you know, we're focused on responsibility now and these things within
6:58
those buckets.
6:59
And so I think again, while in principle there's nothing wrong with it,
7:03
pragmatically or in practice, what happens is that ethical risks get pushed to the
7:07
sidelines.
7:09
Yeah, it's fascinating.
7:10
I mean, I'm a professor of legal studies and business ethics, and we have for a
7:14
long time had a lot of the same conversations more broadly in terms of,
7:18
you know, what does it mean?
7:19
How should we talk about it?
7:21
You know, what's more effective practically in terms of getting businesses
7:24
to change their behaviors?
7:25
Do you think these conversations that we're having about AI are really just a
7:30
microcosm of the larger issues for business?
7:32
Or are there some things about AI that are really distinctive in these discussions?
7:38
Probably a little bit of both.
7:40
You know, I think that...
7:43
Sorry, just a quick...
7:44
I'm getting some feedback.
7:45
I can hear myself a little bit.
7:46
Is it coming out of your speaker and back into your microphone?
7:50
I could put my earbuds in.
7:58
Yeah, I'm not hearing anything, but.
8:00
Yeah, it's probably on my hand.
8:03
OK.
8:05
It's coming out of me.
8:07
Yes, it might be.
8:08
Let me just.
8:14
this connects up with the headphones.
8:16
But yeah.
8:27
Can you say something?
8:29
Sorry.
8:30
Okay, yeah, now I'm hearing you in the earbuds.
8:32
Okay.
8:32
Okay.
8:34
Let's let's try that.
8:36
So now, now I can hear you in the earbuds.
8:37
Hopefully that'll.
8:38
All right.
8:39
I can't even see your earbuds.
8:41
That's good.
8:41
I guess I can see them barely, but if I'm, if I'm really looking.
8:44
So, okay.
8:45
So let's say you asked the question and obviously this look at edit out, you asked
8:48
the question, is this sort of microcosm for ethics and business or is there
8:52
something special about AI?
8:53
Yep.
8:54
Um, so it's a little bit of both.
8:58
I think that one reason why the ethical risks of AI are much discussed is because
9:04
it's of the nature of the beast that is AI.
9:07
machine learning that certain kinds of risks are quite likely to be realized
9:11
despite the best intentions in some cases of the people designing the AI.
9:16
So we get things like biased AI and black box AI, we get privacy violations and
9:21
those kinds of ethical breaches are not usually the result of bad actors, of bad
9:25
apples.
9:26
And so since we've got this powerful new tool, people want to use it for innovation
9:31
to drive revenue, to drive internal efficiency, et cetera.
9:34
But it comes with these associated risks.
9:36
We naturally get this kind of conversation around risks.
9:41
I also think that there's probably some contingent story to be told about why
9:45
people are paying particular attention to this technology in ways that people don't
9:50
pay attention to ethical breaches by corporations more generally in a way.
9:54
So perhaps there's some story to be told about how the people in the technological
10:00
space, they tend to be younger, they tend to be more activist -y.
10:04
And so you get the overlapping diagram of being a technologist and being sort of
10:10
activist -y, social media active, social justice warrior.
10:13
And since you get that overlap of the diagram, you get as a matter of sort of
10:17
historical contingent fact, lots of attention on the ethical impacts of AI.
10:22
Yeah, it's interesting.
10:23
And obviously we've seen a lot of evolution in terms of adoption of AI and
10:27
awareness of AI in the last several years.
10:30
Have the kinds of questions that companies ask you or the things that they bring you
10:34
in for changed?
10:37
I wouldn't say that they've changed radically.
10:38
I think that the frequency with which the conversations are had, the frequency with
10:42
which people want help with this sort of thing has increased over the years.
10:46
There are certain kinds of risks that have popped up very high on the radar as of
10:50
late, specifically since ChatGPT came out.
10:53
Very much increased concerns around cybersecurity or the security integrity of
10:58
their data.
10:59
That's been a major issue, of course.
11:00
And then, of course, the hallucination issue that ChatGPT or other models will
11:05
output just false.
11:06
false statements.
11:08
So I think, you know, if I said, you know, what are the big risks that have come out
11:10
lately?
11:11
It'd be the cyber risks and the hallucination risks.
11:14
The bias risks that are associated with generative AI, those have been around
11:18
since we've been talking about AI ethics at all.
11:20
So that's not really new, but it's perhaps exacerbated or, or especially now that it
11:24
comes in visual form, it's captured the imagination of people a bit more, but
11:30
nothing has fundamentally changed.
11:32
I don't think it's just awareness has grown.
11:34
It's just that awareness has grown.
11:38
So in terms of generative AI, you mentioned the hallucination potential.
11:41
How should we think about that from an ethics standpoint?
11:48
You're on mute now, it looks like.
11:52
Sorry, I muted myself by accident.
11:53
Not about.
11:56
The hallucination problem is not tremendously, no, I was thinking something
12:00
else.
12:03
The hallucination problem is a problem both for the technologists and for
12:08
ethicists.
12:09
So, one, everyone wants accurate AI.
12:11
Nobody wants things that are said that are false.
12:14
That makes it useless.
12:16
So, one, there's just a sort straight -up technical problem.
12:18
It only veers into the lane of the ethicist when it's giving outputs that are
12:23
ethically significant or can have an impact on, say, the livelihood of people.
12:27
So, if it's hallucinating about where you should put your money, let's say it's
12:31
acting as a financial advisor,
12:33
or it's hallucinating about what the appropriate diagnosis is, or it's
12:36
hallucinating about whether you can combine these two kinds of drugs when
12:39
you're asking about your prescriptions or something along those lines, then the
12:43
ethical impacts are serious.
12:44
So there's a way in which, you know, I think about the ethics of hallucinations,
12:48
I think I think this, as something akin to the ethics of bridge building.
12:52
You know, when you build a bridge, you don't want the thing to fall apart,
12:56
otherwise it would be a useless bridge.
12:58
Of course, if it falls apart, if the technical aspects are all out of whack,
13:01
then so too will the ethical impacts be out of whack, because then people will be
13:05
on the bridge and it'll collapse and they'll die.
13:06
So that's really ethically bad.
13:07
But it's really fine.
13:09
It's not, if you like an ethical problem as such to solve, it's more like there's
13:14
these quasi -technical problems like outputting false information that has or
13:20
can have serious ethical impacts.
13:23
But there are these questions with generative AI about what's the appropriate
13:27
response, what might be toxic or improper, culturally insensitive, or biased in ways
13:34
where reasonable minds can differ.
13:36
So if a company brings you in to figure those things out, how would you advise
13:42
them?
13:43
So there's a couple of things to say here.
13:44
I mean, the first thing to say is that usually my company, myself, my company is
13:50
not brought in at that really granular level of we've got this particular model
13:55
for this particular use case.
13:56
And we want to know what are the standards that should govern this particular model
14:00
that happens.
14:00
Sometimes it's starting to come up a little bit more precisely because of
14:04
particular applications of LLMs for exactly the reasons that you've just
14:07
stated.
14:08
Usually what they're looking for is how do we even, how do we get our organizational
14:11
hands around this?
14:13
more generally.
14:14
So how do we create policy around this kind of thing?
14:16
How do we create the procedures and augment existing workflows for our say
14:21
data scientists and product managers to look for these kinds of risks, to flag
14:24
these kinds of things?
14:25
How do we put together the appropriate risk board or committee so that there's
14:29
someone to whom they can elevate those kinds of concerns?
14:31
How do we create the grounds on which that committee is going to make decisions about
14:36
what the appropriate benchmarks are for sufficiently safe or sufficiently
14:40
appropriate?
14:40
Something along those lines.
14:43
As for how those substantive decisions get made, there's roughly two kinds of things,
14:48
two kinds of tools that I like to reference.
14:50
One is an ethics statement or an AI ethics statement.
14:53
Some people are going to call them responsible AI statements.
14:56
Those are really, if you like, high level articulations of values connected to
14:59
guardrails of action.
15:00
So because we value privacy, we'll never do X.
15:03
We'll never sell your data to a third party.
15:05
Because we value accountability, we will integrate people's
15:12
compliance with our AI ethics program into their performance reviews.
15:15
We'll put ownership of the AI ethics program in the hands of a C -suite
15:20
executive, that sort of thing.
15:22
So there's those high level policies that are, I'll give you one more, because we
15:27
value justice or fairness, we will vet our AI models for bias at each and every stage
15:33
of the AI life cycle.
15:34
So these are sort of commitments of action tied to values that get at least some
15:39
procedural guardrails in place.
15:41
and some, some substantive actions that are ruled out after that.
15:45
I mean, even getting that as a big deal and implementing those procedures, that's
15:49
a big deal.
15:49
So forget about, you know, as an act, as at least as a former academic and you, you
15:55
know, you want to deal with those really tough cases.
15:56
Let's deal with these really tough cases where there's a lot of gray area.
16:01
How do we decide what's appropriate for this LM?
16:03
But the truth is, and I would love for my, for most of my customers and my clients to
16:08
be like, yeah, let's help us with these.
16:09
really hard, tough gray area cases.
16:12
But that's, that's not where their heads are.
16:14
Their heads are, can we just, that's running.
16:17
Gray area cases is running.
16:18
We're just learning how to walk here.
16:20
We're barely crawling at the moment.
16:22
How can we crawl safely without like falling off the table?
16:25
And so, sorry, I'm going on a bit of a ramble.
16:29
So one is there are those kinds of conversations about really concrete models
16:34
and what constitutes appropriate.
16:35
One thing to do is to have general kinds of policies.
16:38
The second thing to do is to have what I call ethical case law.
16:41
So this is when you articulate certain kinds of cases that you would say this
16:45
would be inappropriate.
16:46
This would be not appropriate.
16:48
Sort of get internal alignment on those being what I like to call the ethical
16:52
nightmares.
16:53
And then when you have a particular novel case, like should it be allowed to say X
16:57
to Y, let's say something with a certain political content to a general audience,
17:04
you can compare it to your ethical case law, internal ethical case law and see
17:07
whether...
17:08
that provides sufficient guidance.
17:09
And then of course, there's going to be deliberation among the ethics board or the
17:11
risk committee, whatever.
17:13
The thing that I was backing up to say too soon, I sort of interrupted myself is the
17:17
truth of the matter is while the hard cases are sexy and they're interesting,
17:21
especially to academics, they are not where most companies are because most
17:25
companies are dealing with the not crazy gray cases.
17:29
You know, maybe an open AI is dealing with those gray cases, a Google, a Microsoft,
17:35
but a financial services company.
17:37
Insurance industry, health care and life sciences, food and beverage, apparel,
17:44
retail, they're not dealing with those crazy situations.
17:48
They're just hiring people, right?
17:50
They're using AI software to hire people.
17:52
How do we not do that in a discriminatory and biased way?
17:55
So I don't know, this is my sort of, okay, I'm going to stop talking now, but this is
17:57
my very long -winded way of saying most companies don't deal with those kinds of
18:02
tough cases.
18:03
Yeah, no, that was great.
18:04
I there was a lot of real good meat in what you put out there.
18:08
So let me just unpack some of it.
18:11
If companies are just starting down this path, what's the most important thing for
18:16
them to do first?
18:18
I always like to say that sort of table stakes is a call it a risk assessment or a
18:22
gap analysis or AI ethical risk maturity assessment.
18:26
I think most organizations don't sort of know where they are and what they have in
18:30
relation to what those risks may be.
18:32
So looking to see what existing governance structures do we have that cover this sort
18:37
of thing?
18:38
What, what do we lack?
18:39
What kinds of policies do we have that cover this thing?
18:41
What policies do we lack in most cases?
18:44
If a company has done literally nothing on the responsible AI front,
18:48
then you might not even need to do that gap analysis risk assessment maturity
18:53
assessment.
18:53
You know, you've got nothing.
18:54
And now you've got to start with an AI ethics statement, which is going to
18:59
articulate what those standards are and the procedures that are associated with
19:02
those standards.
19:03
And frankly, if you can, the metrics that are associated with those procedures, I
19:07
like to draw a really clear line from values of a business or sorry, AI ethical
19:12
risk values to procedures and the metrics that are used.
19:17
to track compliance and impact of those procedures.
19:22
So that's a good first step.
19:26
And then sort of part and partial that is then to start building a customized
19:29
framework.
19:30
When I say framework, I mean something somewhat specific.
19:33
It's going to include policies, additional governance structures, governance bodies,
19:38
like an ethics board, it's going to include metrics, might include procedures,
19:44
might include race matrices, that sort of thing.
19:48
Yeah, so it's one of things I really love about your book, Ethical Machines, that
19:51
it's rooted in serious ethics, but it's very pragmatic in terms of these kinds of
19:56
things that you're talking about.
19:57
But how much do companies really differ?
20:01
You're talking about customization, but I think there's probably no company would
20:04
say, well, our values, we don't care about privacy, or our values, our discrimination
20:08
is OK.
20:09
So where is it really going to be customized?
20:12
There's actually quite a bit of customization to be done.
20:13
It's actually most of what we do is the customization bit.
20:17
There are lots of things that vary.
20:20
So for instance, you're correct.
20:22
Two different organizations, a hundred organizations are going to say we value
20:25
privacy.
20:26
What that means is going to vary quite a bit across organizations.
20:30
Some will say we'll never sell consumer data to a third party.
20:34
That's part of what our commitment to privacy is.
20:36
Other companies will say, no, no, no.
20:38
We're going to sell that data.
20:39
That's what we do.
20:40
It's a major revenue source for us.
20:42
So we respect people's privacy, which means we will always ensure that we're
20:46
going to aggregate and anonymize that data and use various kinds of privacy
20:49
preserving techniques like Q anonymity.
20:52
But that's, um, K anonymity, but, uh, that's, uh, you know, that's very
20:59
different from saying we're not going to sell the data at all.
21:01
There are differences in what the governance structures have to look like.
21:04
So look, some organizations are structured like partnerships.
21:08
And some are, if you like the sort of the classic pyramid of CEO on top and then
21:13
everyone below that.
21:15
When you have an organization that has sort of like a CEO on top and the CEO
21:18
says, this is how it goes.
21:21
This is a gross oversimplification, but more or less, okay, if you've got the buy
21:25
-in of the CEO, that's more or less how it's going to go.
21:27
If on the other hand, you have say a giant consultancy spread across dozens of
21:31
markets where there's not that kind of top down change.
21:37
because each you need to get buy -in from each of the partnerships, each of the
21:41
heads of each of the partnerships.
21:42
Now governance looks totally different because there's not, if you like, one
21:45
board to rule them all.
21:46
They might say like, yeah, that's good for you guys in the US, but we're over here in
21:50
the EU.
21:51
And so we're going to do things, we're going have to do things a little bit
21:52
differently over here.
21:53
Our standards are going to be different here.
21:55
Some people want to apply the same standards across all markets.
21:59
So yes, we're in EMEA, we're in the EU, we're in the US, you know, North America,
22:03
et cetera.
22:04
And we're going have the same standards across the board.
22:06
Others say, no, no, no, we're going to have different standards for different
22:09
markets, different risk thresholds for different markets.
22:11
We're going to roll this out in different ways.
22:14
So we're going to roll it out by market, for instance.
22:16
So first we're going to roll out this program in the U .S.
22:18
Then we're going to expand to North America, and then we're going to expand to
22:22
the U and then to EMEA, et cetera.
22:24
Others say, no, no, we're not going to roll out by market.
22:27
We're going to roll out by role.
22:28
So first we're going to roll it out to all the data scientists.
22:30
I don't care where geographically they happen to be.
22:33
And then we're going to roll out to HR.
22:35
And then we're going to roll it out to marketing.
22:36
And so there's different ways of, well, that's going to be customized.
22:40
Who's going to sit on the board?
22:42
What kind of authority does the board have?
22:44
Can the ethics board be overruled by a more senior executive?
22:47
That's it.
22:48
Some organizations can say, no, it can't be overruled.
22:51
That's the final say.
22:52
Other will say, yeah, it can be overruled by say the CEO or something along those
22:56
lines or a chief data officer.
22:57
Chief data officer is not actually on that board.
23:00
So there's a tremendous amount of, what does training look like?
23:04
What are risk?
23:04
We already do risk matrices in our organization.
23:07
And the way that we do risk assessments is different how our peers do risk
23:11
assessments.
23:12
We want responsible AI or AI ethical risk assessments that cohere with, that
23:16
harmonize or integrate with how we already think about risk in our organization, how
23:19
we already do those risk assessments.
23:21
And same with metrics.
23:23
We want metrics that mirror the kinds of risk metrics that we already have at the
23:28
enterprise wide or department level, et cetera.
23:30
So there's a tremendous amount of customization that needs to be done.
23:34
That's why there's the NIST AI assurance or whatever it's called framework.
23:39
And it's great.
23:40
It's super smart.
23:42
It's totally generic though, necessarily.
23:45
And so you give that document to, here's a really smart, generic framework.
23:50
And there is a tremendous amount of work left to be done.
23:55
But are there right -
23:59
of understanding what the organization is and customizing the AI ethics.
24:05
So I think there's reasonable and unreasonable.
24:07
And I think this is a place where we sort of can think about it in terms of the
24:10
political spectrum.
24:11
I think that there are companies who are sort of on the left side of the spectrum,
24:14
take a Patagonia.
24:15
There are companies on the right side of the spectrum, take a Hobby Lobby.
24:18
And we sort of think, yeah, that's fine in the corporate world having that.
24:21
I mean, most anywhere you think about it.
24:22
Having that kind of diversity of values, that's acceptable to us.
24:26
There's beyond the pale, right?
24:27
So you don't want KKK values anywhere around that would be beyond the pale.
24:32
So when are there right and wrong answers?
24:33
I think that...
24:34
There are some wrong answers certainly, but there's lots of right answers that are
24:37
mutually incompatible, just like Patagonia's stance on some issues is going
24:41
to be incompatible with the stance of, say, a hobby lobby.
24:45
That said, there's also right and wrong answers as to, hey, does doing this
24:49
actually help us to achieve our goals?
24:51
Is this means sufficient to achieve our end?
24:54
And then there are certainly right and wrong answers.
24:57
They'll say, oh, we'll just do this.
24:59
We'll just tell everyone x.
25:01
And we'll say, that ain't going to do it.
25:03
You could.
25:04
That's not what's going to happen.
25:05
You could tell them that, but that will definitely lead to a lot of things falling
25:08
between the cracks.
25:09
We highly recommend against that.
25:11
So I think there's right and wrong and reasonable and unreasonable in the realm
25:15
of what the values are, what the standards are, what the procedures are.
25:19
And then there's right and wrong in terms of, are these means sufficient for
25:22
achieving our ends?
25:23
So you gave an example of can the CEO overrule the ethics board?
25:28
How would you help a company think through what the answer is?
25:32
Well, I mean, it's this.
25:34
Look, if you really want to take the ethics of this seriously, then you have an
25:38
ethics board, cross -functional ethics board, well -trained, both in, if you
25:44
like, ethical deliberation on the kinds of issues they're going to face, but also
25:48
well -trained in understanding, because usually it's this sort of board that's
25:53
also going to oversee the rollout, compliance, and impact of the program,
25:57
well -trained in how to oversee a program like this.
26:00
And if you really want to prioritize ethics and trustworthiness, etc., then you
26:05
say the decision by the ethics committee is the final one.
26:08
That's it.
26:09
What they say, if they say this is a no -go, then it's a no -go.
26:13
And that's really good if you want to prioritize more than, you know, not maybe
26:16
not more than anything, but very highly prioritize something like long -term
26:20
trustworthiness of the brand, then you do that.
26:24
That said, I think we need to be honest here and say, there's going to be cases
26:28
where doing the unethical thing is going to be really profitable.
26:31
There are sort of AI ethicists out there who say, oh, the right thing is also the
26:37
profitable thing.
26:37
And so we're like, no, that we're true.
26:39
We wouldn't have to talk about ethics.
26:40
We'd be done here because they would already do the profitable thing and then
26:43
the ethics that would fall out of that.
26:44
So there's clearly conflicts.
26:47
And so if you say, listen, if you want to give yourself some wiggle room where
26:52
the...
26:53
The reward is so great, the ROI, say bottom line ROI is so great that you think
27:01
the CEO should have the ability to override the ethics committee.
27:05
That's not, it's not crazy.
27:07
It's a little bit dangerous in the short term anyway, but it's not a crazy view to
27:12
hold.
27:13
In fact, I think in some cases it's quite reasonable to say, you know, yeah, this is
27:17
a business at the end of the day and we're willing to take bigger risks.
27:21
It's not.
27:22
It's it's it's gonna tell me another context.
27:24
I mean, there are certainly cases in which CEOs choose them morally abhorrent for
27:27
some short -term profits.
27:28
That's a bad thing.
27:30
But should it be sort of, if you like, structurally disallowed by the
27:33
organization?
27:34
Probably not in many cases.
27:36
If I'm looking at a company and trying to assess how serious they are, how effective
27:42
they are in terms of what they're doing on AI ethics, are there things that I should
27:45
look for, their tells?
27:56
Let's start with that, sure.
27:57
I'm a customer or I'm someone just trying to evaluate the company from the outside.
28:01
You'll have no idea.
28:02
No clue.
28:03
The people, the sort of statements that they put out are phenomenally superficial.
28:07
They're not going to make their policy public.
28:10
They're usually not going to, you know, when we work with our clients, we train
28:14
them how to do auditing, how to ready themselves for internal and external
28:18
auditing of their AI ethics program.
28:20
And in principle, they can make that stuff public.
28:22
Number one, most companies are not there yet.
28:24
They haven't done enough.
28:25
They haven't done enough to audit.
28:27
They're still designing the program.
28:29
Some are at that audit stage, but most are not.
28:32
And two, most of them are not going to be public about it.
28:35
It just puts a target on their back.
28:36
In the early days, this doesn't happen anymore, but five years ago, companies
28:42
were way more resistant to being public about what they said about AI ethics or
28:46
ethics generally because they felt that if we say that we really care about ethics,
28:50
we're putting a target on our back for journalists to catch us.
28:53
So if we say we care a lot about AI ethics and then something goes haywire, we're
28:57
going to get skewered.
28:59
by the journalist.
28:59
So let's just shut up about it.
29:01
Keep quiet.
29:01
We'll do this stuff internally, but let's not tell everyone about it.
29:05
That's changed over the last five years.
29:06
Now it's more like, oh, whoa, everyone else is putting out AI ethics statements.
29:10
We better put out an AI ethics statement to keep up.
29:12
So that's good.
29:14
But the statements are, by and large, phenomenally superficial and mean nothing.
29:17
The average citizen, they don't know anything.
29:20
Even if they look at the privacy policy of the organization, I've seen many examples
29:25
of paper tigers, many examples where,
29:43
do you measure success in terms of the work that you do with organizations?
29:48
How do I measure success?
29:49
That's a good question.
29:50
I mean, you know, if they're just doing the things that we're telling them to do
29:54
more or less, I mean, when I say what we tell them to do, we're giving them
29:56
options.
29:57
We say, look, you can build your program this way or that way.
30:01
You know, for instance, we say, you can build an AI ethical risk program or
30:04
responsible AI program, or you can build a digital ethics program.
30:07
The digital ethics program will include AI, but it's also a bit broader than that
30:12
to include other kinds of technologies, including ones that we don't, either
30:16
applications of AI,
30:18
that we don't have yet, new kinds of AI we don't have yet, new kinds of technologies,
30:23
new applications, new technologies.
30:24
So let's say blockchain, quantum, AR, VR.
30:29
Ideally, they would all say, oh yeah, digital ethics.
30:32
It's the broadest, it's the most comprehensive, it's the most proactive way
30:36
we can address this.
30:37
These issues aren't going away, they're only growing.
30:39
So we say, look, digital ethics is ideal.
30:42
But sometimes they say, listen, we'd love to do that.
30:46
We don't have the internal buy -in for that.
30:48
What we have, we have board support for building an AI ethics program.
30:54
That's what we're not going to go back to them and say, you know, we want to do
30:56
something bigger or we want to do something broader.
30:58
That's where our buy -in is.
30:59
That's the budget we have.
31:01
That's where our focus is.
31:02
That's what our priorities are.
31:03
So that's what we're going to do.
31:05
Is that a success or a failure on my view?
31:08
It's still a success.
31:09
You know, it's not the ideal, but it's, it's a success.
31:12
What's another success if they actually move on to implementation?
31:16
There are.
31:17
plenty of organizations that will start to write an ethics statement and then just
31:20
stop.
31:20
That's a failure.
31:22
You've got to get to the point of implementation.
31:24
You've got to get to the point of training your people.
31:27
This is, this is crucial.
31:29
There's got to be that, if you like role agnostic training, the training that
31:32
everyone across the organization gets, then there has to be role specific
31:35
training.
31:36
There has to be updates or augmentations of workflows of say data scientists,
31:41
people in HR, et cetera.
31:43
There has to be documentation.
31:44
You know, there's got to be all these.
31:46
things where it actually impacts the day -to -day operations.
31:50
So what we like to say to our clients is, okay, you've done all this, blah, blah,
31:53
blah, now we got to get the implementation, because you can write all
31:56
this stuff, you can have this policy, but that doesn't mean that the data scientist
31:59
knows what they need to do tomorrow at 9 a .m.
32:01
when they get to work.
32:02
This needs to be made clear.
32:04
And if they don't get to that, that's maybe not yet a failure, but it's not yet
32:08
a success.
32:10
One thing that's changed in this area in recent years has been that there's been an
32:15
upsurge of regulation and legislation and discussion and passage of things like the
32:20
AI Act in Europe.
32:22
How does that impact on what you do or on what companies actually do on the ground?
32:27
Because on the one hand, now there's going to be all these requirements.
32:30
On the other hand, if this just becomes a check the box for compliance, then it
32:34
might actually be counterproductive.
32:35
So what do you see?
32:37
I think it's too early to tell.
32:39
It's still TBD.
32:40
Yeah, there are regulations out there.
32:42
You know, New York City came out with a regulation not so long ago.
32:44
I think it passed this past summer that all HR tools, sorry, all AI tools involved
32:52
in the HR hiring decision -making process need to be audited by a third party.
32:58
And that's not being enforced.
33:01
I have no idea if it's had any impact at all.
33:04
The EU AI Act.
33:06
promises to do more That's plausible, especially if you're in the EU.
33:11
Of course, there are lots of companies in the US who operate in the EU So they'll
33:15
have to make sure that they're compliant as well It's gonna do something.
33:18
I think it's just frankly it's too early to tell How much of an impact it's going
33:22
to have one thing that I that I anticipate happening could be false The vast majority
33:28
of owners of this kind of these kinds of problems within organizations are on the C
33:33
suite on the tech side of the house
33:35
So a chief data officer, a chief analytics officer, sometimes a chief information
33:38
security officer or a chief information officer, chief digital officer, blah,
33:43
blah, blah, blah, blah.
33:44
It's the risk and compliance people, not so much involved.
33:48
We ask, we say, look, it's great.
33:50
We need to bring them to the table.
33:51
We need them in these conversations.
33:53
Legal is pretty standard to say, well, bring them to the table.
33:55
But usually the risk and compliance people are not spearheading it.
33:59
So I think operationally we'll probably see a change where it gets.
34:03
less in the hands of tech people and more in the hands of those risk and compliance
34:09
people, or at least if it's less in the hands of the data people, the risk and
34:13
compliance people will have to get more involved because that's their job.
34:16
So I think we'll see more of that.
34:17
That in principle will be helpful because those chief data officers, what do they
34:22
know about risk and compliance?
34:23
They're not risk and compliance folks.
34:25
They're engineers, they're data scientists, they build cool shit.
34:29
That's not what they do.
34:31
And so I think that having...
34:33
trained risk and compliance people probably owning the program or at least
34:38
playing a significant role in it will increase adoption.
34:42
Will it be mere box checking?
34:45
I don't know.
34:47
Probably the beginning is my guess.
34:49
There's also a way in which companies tend to overestimate what they're actually
34:52
doing.
34:53
So we'll ask companies, oh, so are you doing X, Y, and oh yeah, yeah, yeah.
34:56
You're checking for bias?
34:56
Yeah, yeah, yeah, yeah.
34:57
We check for bias.
34:58
And then we sort of start digging around and we find out, oh, not really.
35:02
I mean like...
35:03
The thing that they're doing is so minimal.
35:05
And so there's also a level of self -deception, or at least ignorance, about
35:10
how robust what they're doing actually is.
35:12
So, you know, I'm optimistic, but it's a cautious optimism.
35:17
I wouldn't be shocked if it turned out to be box -checking.
35:21
And is there anything else that you see potentially coming in the next few years
35:26
that's going to have a major impact on AI?
35:31
I mean, there's two kinds of things, broadly speaking, that could have the
35:33
impact.
35:34
The, um, and they're, they're compatible.
35:35
One is a big disaster.
35:40
Think of like Cambridge Analytica, but for AI.
35:42
And so if you get that kind of thing, that will help.
35:45
And, uh, help.
35:47
I mean, you know, that'll be the silver lining.
35:49
That'll be a cause of people thinking more seriously.
35:52
That might happen.
35:53
It wouldn't surprise me.
35:54
But the other thing is just, you know, death by a thousand cuts.
35:58
It's so.
35:58
there's this AI incident and then this one and then this one and then this one and
36:02
this one, neither one of which is a massive crisis, but the aggregate should
36:06
be terrifying, I think, eventually to people in the C -suite and the board
36:10
responsible for protecting the brand or their organization.
36:12
So I just think it's going to be the case that we're seeing increased adoption and
36:17
as you get increased adoption, you get increased deployment and increased
36:20
deployment means increased probability of things going ethically sideways.
36:25
And so you get more things going ethically sideways and they're going to be
36:27
published, et cetera, et cetera.
36:29
On that note, by the way, increased adoption, there are some people who say,
36:34
oh, generative AI, it's been a year already, or it's been a year and two
36:38
months, whatever.
36:39
We haven't seen the massive change yet.
36:41
And I just think, just chill, just hold on.
36:44
We're talking about the most cutting edge technology out there.
36:46
They're working on it.
36:47
They're figuring it out.
36:49
Most of our clients are running dozens, and in some cases, hundreds of pilots
36:55
using generative AI.
36:56
They're just trying to figure out how this thing works.
36:58
They're trying to figure out.
36:59
Does it solve a business problem for us?
37:00
What business problem does it solve?
37:01
How do we use it responsibly?
37:03
So, you know, for all the talk about, oh, it's been, it's out here.
37:08
No, not really.
37:09
Open AI released their stuff a year and a bit ago.
37:12
Google a little bit less than that.
37:15
Then there's all these other, you know, llama and anthropic.
37:19
Just wait a second.
37:20
We're going to see, I would, unless things go oddly haywire, we're going to see lots
37:27
of adoption, lots of deployment.
37:28
in 2024, let alone 2025.
37:32
And with the increased adoption, we're going to see more bad incidents.
37:37
All right, well, more bad incidents, but hopefully also more good work for people
37:41
like you.
37:42
Yeah, I mean, I should say, you know, this comes against the backdrop of being, I
37:46
don't know, not a pessimist about the technology.
37:48
I'm not a catastrophist.
37:49
I mean, there are some people out there who are screaming that the sky is falling
37:52
and everyone's going to die and this is a disaster and it's not ready for prime time
37:57
and it never will be.
37:58
I just think that's sort of stupid.
38:00
I mean, I've seen use cases that work and they do amazing things.
38:03
There are cases, I was recently talking to an organization yesterday that said they
38:07
implemented a generative AI solution for certain kinds of...
38:11
search and summarization of a process that used to take 40 hours and now it takes
38:16
four.
38:17
And they are happy with the results.
38:18
They're happy with the metrics.
38:19
They're happy with how it's level of accuracy.
38:21
They're happy with the level of risk that there is in the particular use case that
38:24
it is.
38:26
Just give it, there'll be more.
38:27
That's a well -funded, you know, organization with lot who has lots of
38:30
power to throw out these kinds of problems.
38:32
Others don't have that kind of power or they're slower.
38:34
They want to see what others are doing.
38:36
But yeah, I'm not a catastrophist.
38:38
I think that there's tremendous good opportunity.
38:41
And yeah, there's lots of opportunity though to make sure for people like me
38:44
that to help organizations make sure they don't go off the rails.
38:48
Reed, thanks so much for this conversation.
38:50
It's been really instructive and fascinating.
38:52
my pleasure.