Latest / Health Ethics Podcast / Societal Informed Consent in the Age of AI

Transcript

Societal Informed Consent in the Age of AI · Health Ethics Podcast

0:08
Welcome to Health Ethics, a series of conversations with leading experts about ethical considerations
0:15
associated with health and healthcare, both locally and globally. I am Brian Pilkington,
0:20
Professor of Bioethics in the Department of Medical Sciences at the Hackensack Meridan School
0:25
of Medicine. This series, like its predecessor, which focused on the COVID
0:29
nineteen pandemic, relies on the idea that challenging ethical issues are best addressed by
0:34
folks from diverse backgrounds practically reasoning together, and complements the Health Ethics series with
0:41
its panel of leading experts from medicine, nursing, politics, economics, philosophy
0:47
and the law. Thank you and enjoy the episode. Welcome everyone to the
0:58
Health Ethics Podcast. I am Brian Pilkington, Professor of Bioethics at the Hackensack
1:03
Grinian School of Medicine, and it is my great pleasure to be here with
1:07
Brian Green. Brian is sorry, Brian Patrick Green. Brian is the Director
1:11
of Technology Ethics at the Marklist Center for Applied Ethics at Santa Clara University.
1:17
He also teaches AI ethics at Santa Clara's Graduate School of engineering. Folks interested
1:22
in that. I've probably read Brian's work, as he's written a ton.
1:25
His work focuses on AI and ethics, technology ethics and corporations, ethics and
1:30
space exploration and use, the ethics of technological manipulation of human beings, the
1:34
ethics of migration and adaptation towards risky emerging technologies, which we'll get too shortly,
1:41
and various aspects of the impact of technology and engineering on human life and
1:45
society, including the relationship of technology and religion. Brian is the author of
1:51
the book Space Ethics, co author of Ethics in the Age of Disruptive Technologies
1:56
and Operational Roadmap, and co author of Ethics into Technology Practice, along with
2:01
a ton of other stuff. And we'll be sure to link on the show's
2:04
page. So, Brian, with that long introduction, thanks so much for
2:07
being here. I appreciate it. Thanks so much for inviting me on the
2:10
show. So you recently wrote a really interesting piece and you've sort of talked
2:16
about this in other context, but this idea of societal informed consent, So
2:22
folks from all walks of life listen to the podcast. But we have a
2:24
lot of sort of medical medical ethics kind of people and medical students engaged.
2:29
So I'll push you to medicine in a moment, but can you give us
2:32
some background, generally on what led you to being interested in this topic before
2:38
I grow you on societal consent? Sure. So I've been teaching ethics to
2:43
engineers for I guess about fifteen years now, and one of the earlier articles
2:49
I discovered mentioned this idea of societal informed consent. I believe it was an
2:52
article by Carl Mitchem. The title of the article was a philosophical inadequacy of
2:58
Engineering. Now he was specific we be talking about engineering doesn't have a goal
3:02
in the same way that say, law or medicine has a goal. Engineering
3:07
seems to be much more instrumentalized as far as a field of practice goes.
3:12
But one of the things he talked about in there was the fact that we
3:16
have these great, big, new technological systems that we implement into society,
3:22
and nobody ever gets asked whether they want that or not. I mean,
3:24
the users of it might have to sign an end user license agreement or some
3:28
other form of contract to be using a particular kind of technology. But if
3:32
we're having our entire society completely transformed by technology. There's a lot of bystanders
3:38
who don't get asked about this. This same idea also shows up in a
3:43
kind of classic engineering ethics textbook called an Introduction to Engineering by Mike Martin and
3:47
Roland Schinzinger, and they have a whole chapter which is on technology as a
3:52
social experiment and the fact that technology really has this experimental element to it and
3:58
we never know what's going to happen. And so what I argued in the
4:01
article that I wrote was there had been a recent fatality in San Francisco where
4:06
a bystander had been run over by a self driving car. And you know,
4:13
if you're using an app or that's using or otherwise using a self driving
4:17
car, at some point you clicked a box that said you accepted the risk
4:20
that goes along with that. But bystanders don't. And so we're fundamentally changing
4:25
the way society operates by making people take risks that they didn't sign up for.
4:30
Now, this has been kind of standard throughout technological history. New technologies
4:35
come and we're all exposed to these new risks that we didn't sign up for.
4:41
But the question is, maybe we're getting to the point now where we
4:44
need to be thinking more about the societal level of risk. If we're talking
4:47
about something that has a societal level of effect of risk, then we probably
4:51
need to be thinking more about consenting on a societal level. That's a helpful
4:58
on the quick tour through engineer ethics, I think is a good background,
5:01
especially for a medical student, some of whom have interests in this area.
5:05
So that's that's excellent. So say a little bit more if you can,
5:09
about societal and formed consent and how that might go. I mean, you
5:15
sort of mentioned the idea of practicality and impracticality and the piece and so we'll
5:20
go there. But what, Yeah, how would this work? It's a
5:23
lot of folks. Yeah, and it's international too, right, so it's
5:28
not like it's a simple thing. One thing that you could do is you
5:30
could say, we're a representative democracy. We delegate this power to the legislators
5:36
of our country, the government basically and the executive branch and as well as
5:42
the court system to say what's an alliable technology or not? Is this technology
5:47
too risky? Are the lawsuits going to be too big? How's the liability
5:51
going to work? Maybe technology companies don't want to create something if they're going
5:56
to be particularly liable for it. So you could look at it from that
6:00
perspective, or you could look at it from a more authoritarian perspective, which
6:03
is that in this particular regime, then the government says what's right and everybody
6:10
just has to deal with it. And very often that could be exposing people
6:13
to surveillance or you know, a police state, or all sorts of other
6:16
sorts of things that we and more liberal democracies might find to be unacceptable.
6:21
But I think one of the most important things about a liberal democracy is that
6:27
that we as citizens do have a say in how our country is supposed to
6:30
operate. We have a say in terms of what we do economically, in
6:33
terms of what we what we buy, in terms of what contracts we sign.
6:40
We're free to sign contracts or not sign them as we see fit.
6:43
We're free to elect the politicians that we like. And yet still in at
6:46
the same time, this idea of the self driving car coming down the road
6:50
and just running you over, when nobody said that was an okay thing to
6:55
have happened. It's certainly not the individual who signed up for it or who
7:00
did not sign up for it, I should say, certainly not the individual
7:02
who gets run over. They probably, you know, unless they have the
7:06
app. I don't know what the end user license agreement looks like on the
7:10
app. Maybe the app says if you sign this app, then you're okay
7:13
with being a bystander who's run over by a vehicle. Maybe it does say
7:16
that. I don't know, and certainly I know that in self driving cars,
7:21
Tesla has a very long end user license agreement that lists out many,
7:27
many cases of failure modes of how the car can go wrong, and it
7:30
says that when you sign that form, you are responsible as the driver for
7:34
every single one of those failure modes that you're supposed to prevent those from happening.
7:39
And that's gotten Tesla off the hook for a number of accidents that it's
7:42
autopilot has caused. So these issues are out there. It's not really clear
7:46
how we overcome the situation other than that we have to have a really big
7:50
societal conversation about that. We don't have the social institutions to have a proper
7:56
conversation, and so the question is how do we Is there some way that
7:59
we can and develop these institutions? Is there some way that we can have
8:01
this conversation so that it's more than just saying, oh, the government will
8:07
take care of it, or the court system will take care of it,
8:09
or the companies will take care of it. It needs to be something that's
8:13
bigger than that. It needs to be a more large scale conversation. But
8:16
once again, it's difficult to imagine exactly how to implement it, which is
8:20
probably the reason it hasn't yet been implemented. What that suggests to me is
8:24
a model of this language is loaded but personal responsibility. So both someone says
8:31
by any of the societal consense stuff is nice. But as long as everyone
8:37
reads the end user agreements, which no one does. But as long as
8:39
you do that and it's on you, isn't that easier And doesn't that make
8:43
more sense than having society, you know, trying to find some way create
8:48
these new institutions, get everybody involved, as long as each person we already
8:54
have systems in place wherein if in the self driving car case, you know,
9:00
I'm on the hook for the Tesla, even though it's Tesla's AI mistakes
9:03
or whatever. If I'm on the hook, then we don't need this law.
9:07
This new or revived or newly applied notion of consent. Yeah. I
9:13
mean what you bring up is a really fundamental point, right, don't We
9:16
still have personal responsibility here. If society changes around us, then our responsibility
9:20
is to adapt. But I think the question here is that you're dealing with
9:24
technologies that are kind of at a fundamentally more social level in the first place.
9:30
So, for example, if one person has a cell phone, first
9:33
of all, you don't have anyone. I guess you can call people on
9:35
regular phones, but you wouldn't have another cell phone person to you know,
9:37
communicate with. But you know, there's a qualitative difference between having one cell
9:43
phone and having one hundred million cell phones and having people drive with cell phones.
9:48
So, for example, I love to give this example to my students.
9:52
I was right outside of Santa Clara University one day and there were there.
9:58
It's a three lane road, three lanes on each side, so six
10:01
lanes total. The three lanes on my side had a red light. I
10:03
was three cars back, so there were two full rows of cars in front
10:07
of me. The light turns green, nobody moves the first row of cars.
10:13
Everyone was looking at their cell phones. I look at them all.
10:15
Oh, they're all looking at their cell phones. The second row of cars,
10:16
everyone was looking at their cell phones. And after we sat there for
10:20
about fifteen seconds, I just, you know, tapped on the horn lightly
10:22
woke everybody up again, get them out of their phones, and got them
10:26
moving. There were six people who were all on their cell phones, who
10:28
all had not responded to the green light. This is at a major intersection
10:33
where you know, it is a major intersection. It's you know, six
10:35
lane road at a two lane road on the other side, but lots of
10:39
pedestrians crossed there because it's right between the university and the train station, and
10:43
so this is a dangerous place for people not to be paying attention. We
10:48
fundamentally changed the way our society operates when we introduced smartphones into our society.
10:54
It's you know, it's been fantastic. It's given us so many new abilities,
10:58
but it's also raised a lot of risks that nobody is a society really
11:01
brought up. I'll give an example from space, because I think this is
11:05
a particularly notable one. In my books on space ethics, I talk about
11:09
in societal and formed consent several times in there in different context one of which
11:13
has to do with space concept called planetary protection. So planetary protection basically has
11:20
to do with forward and backwards contamination in space. If we send something to
11:26
the Moon and bring it back, we don't know if it's contaminated. So
11:28
the first marsh or the first astronauts who came back from the Moon, they
11:33
were put in an isolation chamber for a little while until we decided it was
11:39
safe to bring them back out because we didn't know if they were contaminated.
11:43
Now we know the Moon is pretty much sterile, but Mars is not.
11:46
It's potentially not sterile. It could have microbes on it of various sorts.
11:50
Whether it's bacteria or things that are similar to bacteria, we don't really know.
11:54
It could be quite different. If we are sending stuff to Mars.
11:58
We don't want to damage mars Is ecosystem by transporting a bunch of Earth bacteria
12:03
or things there. We also don't want to damage Earth's ecosystem when we bring
12:07
stuff back. And this is an example of any person you know. Let's
12:13
say it's this guy who starts a rocket company and he starts sending rockets to
12:16
Mars, and you know, at some point the rockets might bring stuff back.
12:20
What if he brings back something that is a microbe that's actually damaging to
12:24
Earth's environment. It's not likely maybe that'll be a disease exactly, but it
12:28
could change the way our plants grow, or it could be a you know,
12:33
damaging in some way to the ecosystem of the Earth. The rest of
12:37
us didn't sign up for that, right, So if you're talking about things
12:41
that are really at the level of civilization, you know, they're risky scenarios.
12:46
That's that's maybe a less risky scenario, but it depends really on what
12:50
the microbe is. We have lots of examples of invasive species on Earth that
12:54
are extremely, extremely costly, costing trillions of dollars a year to the economy
12:58
because of you know, damaging agriculture and things like that. There are worse
13:05
examples from space. Also. There's a group called MEDI, which is messaging
13:07
extraterrestrial intelligence, and MEDI wants to send messages out there, broadcasting from Earth
13:13
saying hi, aliens, we're here, come say hi to us. We're
13:16
friendly. And of course, you know, if they send a message out
13:20
there and the aliens aren't friendly, and they'll say, oh, here's an
13:22
easy target, let's go destroy them. And in that case, you know,
13:26
a few people made a decision that then puts the entire population of Earth
13:30
at risk. It seems like maybe we should have a say in that,
13:33
and it's not just these few people who should do it. So there there
13:37
are lots of examples of how these technologies kind of make it. Because of
13:43
the quantitative change in power that they give us, it yields a qualitative change
13:50
in the kind of consent that we need to give. So it's one thing
13:52
for one person to accept, you know, an operation, get informed consent
13:56
for having a medical procedure, for example. It's another thing to have all
14:01
of society, the entire planet perhaps put at risk, whether it's from contamination
14:05
from space or because somebody managed to contact aliens. You know, it actually
14:11
doesn't matter whether the aliens exist, right, we don't know what the probability
14:13
is of aliens existing. We could say it's zero. There still should not
14:18
be people putting us at risk, you know, because nobody signed up for
14:22
this. Fundamentally, we need to once again, like I said, we
14:24
need it some way to have a conversation on these sorts of issues, whether
14:28
it is through our government, whether it is through other means in society.
14:33
Maybe it's maybe universities are one of the ways that we could do this for
14:35
example. There are options for possibilities, but we really need to once again
14:43
kind of focus on this issue because technology, like I said, it's gone
14:46
from quantitative to qualitative changes. It's one thing to be able to radio you
14:52
know, other people around the world. It's another thing to radio another planet
14:56
or another civilization. Or it's one thing to move my between Europe and North
15:01
America or between Asia and South America. It's another thing to move microbes between
15:05
planets. And so if we're talking about once again artificial intelligence, you know,
15:11
sorry, just adding more and more things on top of this. When
15:13
it comes to artificial intelligence, we're going to be raising questions of once again
15:20
controlling the entire society's information ecosystem, which is how social media does this right
15:24
now or search engines this prioritization of what should people be reading, what should
15:31
they not be reading, And when we do that, we have the opportunity
15:35
to greatly affect how people view the world just by filtering their information differently.
15:39
And we've seen this at place so much in terms of YouTube, in terms
15:43
of news, in terms of Twitter and Facebook and Instagram. There's just lots
15:50
and lots of examples of how we've started applying artificial intelligence in these very large
15:54
ways which you're having outsized consequences, bigger consequences than people expect. A compelling
16:03
case with the example. Before I turn to medicine and AI. One general
16:07
question that still lingers here is the sort of standard technology ethics question around expertise.
16:15
So there are folks who understand all this stuck stuff a heck of a
16:18
lot better than I do. I mean, I know, when you drive
16:22
your car to the mechanic, they change out the mouse right on the wheel,
16:26
and that's what gets the car going again. And then when your mouse
16:27
gets tired, you go back to the mechanic and they put the new new
16:30
mouse in. So I'm not the right person. I don't have the expertise
16:33
to sort of say what cars should do. How would I even think about
16:37
the latest new emerging technology. How would I think about AI? So is
16:41
there a problem for societal and formed consent if folks don't know enough to properly
16:48
give the consent? Yeah, this is a horrible question. Horrible is and
16:52
it's very difficult to not yet you can punt, you can your questions very
16:56
good. So yes, how how can you actually how can you be informed?
17:03
Right? How can you be informed enough to give consent? If you're
17:07
not an expert. This problem shows up with machine learning researchers themselves. They'll
17:11
create a new model and they don't understand exactly how it operates the whole idea
17:15
of machine learning is that the model figures it out right. We humans,
17:19
we have this incredible ability to create something that we can't fully understand. It
17:25
gives outputs that we find to be useful, and yet we don't necessarily see
17:29
how you know, we can't explain what it's going on in there. It's
17:33
not very easily transparent. You can do all these you can do a lot
17:37
of research on it. Actually, these machine learning models are pretty amazing in
17:40
that way, in terms of the of the ability to research and say,
17:44
oh, this neuron does this, and this neuron does that, and if
17:47
you can adjust the weights between them, then you change the way other things
17:51
operate. But one of the one of the things that these large language models
17:56
have actually been designed to do is to or at least this is one of
18:00
the things that the engineers designing them have designed them hopefully to be able to
18:04
do at some point would be to look at their own neural network, or
18:07
look at another machine learning model's neural network, and then explain it to you
18:11
in language that you can understand. So when you say, hey, why
18:15
did they get this output that seems a little strange, the model can look
18:19
at it, analyze it and say, oh, that's because the weights here,
18:22
you know, And if it's a very sophisticated person, maybe it'll give
18:26
you a very mathematical, you know, solution or explanation of why that happens.
18:32
You know, it's neuron this in this layer and the weights between these
18:36
vectors are off, or something like that. And on the other hand,
18:40
if you're talking to a regular human being, you'd be like, oh,
18:41
the you know, that's just the way the model works. You might not
18:47
be actually be able to simplify it in a way that a normal person can
18:49
understand in a useful way. And so when we're talking about that, then
18:53
we have to once again think about informed consent is not only a matter of
19:00
a legal document where you can waive it in front of someone and say sign
19:03
this here and you know, I've heard stories of you know, some informed
19:07
consent papers are two pages long, some of them are twenty pages long.
19:11
It all depends on what the organization is that's building it, what their legal
19:15
liabilities are, what they're trying to protect against, and those sorts of things.
19:19
Ultimately, we really have to ask these sorts of questions and think to
19:23
ourselves. We have to trust each other at some point, right, we
19:26
have to trust other human beings, and we have to trust the technology that
19:30
we're not being put at risks that we shouldn't be put at. But that
19:36
question, you know, once again, how do we how do we How
19:38
do the technology developers know what an acceptable risk is if they don't actually have
19:42
a conversation with the people that they're exposing to the risk. And once again,
19:47
so this is kind of where you get to this institutional question, how
19:49
can we talk to each other in a way so that we can start building
19:52
up this sort of trust, because once again, it's it's one thing to
19:56
do a medical operation where you have to sign a piece of paper. It's
20:00
another to put the entire earth population at risk because of something risky that you're
20:04
doing. But at least having a conversation about it beforehand, we'll tell the
20:08
technology developer what the expectations are, and then if they follow those expectations,
20:15
if it becomes a problem and goes to a court case or becomes some other
20:18
sort of problem, they'll least be able to say, we were trying to
20:22
do what the conversation said we should do. We were trying to fulfill the
20:26
will of society, if you want to think of it that way. It's
20:30
always good to have at least one unfair question in the conversation. So no,
20:33
I appreciate that the answer, and I think there's only so much everyone
20:37
can know and do. But that idea that in order to program things and
20:41
develop things in the right kinds of ways, that we you know, the
20:45
conversation's helpful there, and you point into a huge topic which I will not
20:49
throw at you. We don't have time for. But it's just considerations of
20:52
trust, right, and how much trust is there if we you know,
20:56
and you've written on and to folks at Google, you know, the explain
20:59
of ability problem and other things that I think sort of raise some of these
21:03
considerations. So let's move a little more directly to medicine and AI and sort
21:07
of any reflections you might have on societal consent in that area. And so
21:12
you wrote a piece recently which caught my eye. Should AI require societal informed
21:18
consent? And so in that I'm going to quote from you here, Should
21:23
we as a society use AI in medicine knowing that it may allow patients to
21:27
die? If medical professionals might ask for consent from the patient for the use
21:33
of AI in some but not all cases, how do we decide which ones?
21:37
And so that's the quotation, and it comes after other examples. Right,
21:40
you've already laid out the sort of self driving car consideration. Even if
21:44
overall things are positive, we're gonna have some bad results. What do we
21:48
do so help us think through this? Yeah, I think I think the
21:52
standards of keeping the best interests of the patient in mind are really important here.
21:57
Once the end, it's gets so much to trust because currently there are
22:00
some pretty perverse incentives, you know, honestly in medicine, in terms of
22:06
cost cutting, in terms of protecting the institution from liability and deflecting liability perhaps
22:12
onto individuals in the system who are not going to have as much institutional support.
22:18
Whether it might be individual doctors, or it might be individual patients or
22:22
you know, various other or you know, it really depends on how these
22:26
kind of legal structures are put together. But we really need to consider how
22:34
things can go wrong, I guess would be the way to look at it,
22:38
because there's always an incentive whenever anybody is working in a system where money
22:45
is at stake, which is the way that our medical system fundamentally provides treatment
22:51
but also punishes, you know, people if they make a mistake, you
22:55
get rewarded. If you do a good job, they give you hopefully more
22:57
money or you get more contracts or things like that. And you also are
23:00
liable if something goes wrong, but it really depends on how the contracts are
23:03
structured as to how that liability operates, which is, you know, somebody
23:10
as somebody who doesn't really would like to be able to trust people to just
23:14
do the right thing. It's it's not my ideal way to think about the
23:18
situation. But if we're thinking about a company that's making an AI for medical
23:25
purposes, you can be guaranteed, absolutely one hundred percent certain that they're going
23:30
to have a corporate lawyer on their side who's going to say if this system
23:33
makes a bad choice, we're limited in terms of our liability in some in
23:38
some way, somebody's going to try to control that that loss of money if
23:44
something goes wrong, because mistakes, as you said, mistakes will be made,
23:48
and it might be a lot safer than what humans do. Right now,
23:52
that's one of the fundamental questions at stake here, which is that maybe
23:56
humans are ninety eight percent accurate, Maybe the AIS to be ninety nine percent
24:00
accurate, in which case you should go with the higher standard of care.
24:03
However, if if you still make individual doctors liable for using AI clinical decision
24:11
support and say this is ultimately the doctor's choice, they should, they'll be
24:17
more accurate, so they'll probably do better on average. But if you're the
24:21
unlucky one and you're stuck with following the AI, but the AI made a
24:25
mistake and you're the one who's liable for it and not the AI, then
24:29
that's a big problem. And so you have to think about how the contractor
24:33
structured between an individual doctor and the medical system that they're working with, and
24:37
the way the contracts or structured between the medical the hospital or the medical system
24:44
and the companies that are producing the technology. There's some really difficult issues there
24:52
because when people start trying to defend themselves in that way, then it starts
24:56
becoming less trustworthy looking from the outside, because then you realize, oh,
25:00
maybe it's not me as the patient, Maybe it's not my best interest or
25:04
at stake here. They only care what happens to me insofar as they're not
25:10
responsible for hurting me, which is not the same thing as actually wanting to
25:14
help your patients become healthier. So this trust issue is is just it's unavoidable,
25:21
unfortunately, and the more changes that we make to a system and the
25:29
faster that we make them, the less capable of is trust of keep keeping
25:33
up with that, because we've seen a whole bunch of this come out with
25:36
recent large language models. Do they create unbiased outputs, so they create very
25:41
very biased outputs or misleading or false outputs. Folks look at those and they
25:45
realize, oh, this is not trustworthy. But if you know that those
25:49
large language models are going to be making choices that affect the way the world
25:55
works, then you've basically decided that you're going to live in a system where
26:00
you can't trust what's going on in the system, and that that kind of
26:04
contamination of trust, that erosion of trust, really can be destructive of the
26:11
way society operates. How would we resolve resolve disagreement given that folks have different
26:17
levels of knowledge, folks are vulnerable in different kinds of ways. Society,
26:22
at least certain society, certain pockets are polarized. And you had that excellent
26:26
point at the outset that this isn't a US specific consideration, it's global,
26:30
right, So now we've got global health ethics considerations on the table. So
26:34
this really has turned into a massive question. Let me shaak it down.
26:38
Any insights or suggestions or reflections since you've been thinking about this a bunch for
26:45
how to have the conversation knowing folks come from really different walks of life.
26:49
Yeah, and this adds to the intractability of the issue. I'm going to
26:55
use another example from space. So back when the Apollo program was saying going
27:00
to set humans on the Moon, they're going to you know, touch down,
27:02
and then they're going to come back and and all these sorts of you
27:04
know, wonderful, exciting things. There were certain groups of people around the
27:08
world, including Native American tribes and other groups of people in other nations who
27:14
said the moon is a sacred object and humans should not touch it. And
27:18
this actually became a pretty big issue. This issue came up in the news
27:22
again just a few weeks ago because there was going to be a craft touching
27:26
down on the Moon that contained cremated remains of people who wanted to have,
27:32
you know, their ashes scattered on the moon basically, and once again they
27:34
said, look, the moon is a sacred object. Don't scatter human remains
27:38
there. That's not the right way to do it. That's, you know,
27:41
desecration of a sacred place. And as an outsider, you can look
27:47
at this really cynically and say, look, we have the power and you
27:49
don't, and we're going to do it. Whether you accept this or not.
27:52
That's the bad solution, that's that's not the right way to do it.
27:56
The right solution is to listen to them and say, we see that
28:02
you have your perspective here. We need we want to respect your perspective as
28:07
much as we can. Is there some way that we can do this more
28:10
respectfully? And that opens up the conversation. Now there's a possibility at the
28:15
other side will say, look, there's no way you can do as respectfully
28:18
and we're not going to talk to you. We're just going to try to
28:19
stop you, in which case you can't have a conversation about it. But
28:25
it might be the case that they say something like, yes, actually,
28:29
maybe there is a way for us to for you to do it more respectfully.
28:33
For example, don't leave you know, waste or junk on the moon
28:36
or something like that, or or when you do this, just you know,
28:40
be respectful, or or could you read this, you know, maybe
28:41
maybe they have a prayer or something like this, say this prayer as you
28:45
before you get out of a vehicle or something like that. There might be
28:48
ways that you can do this that are respectful of their culture. And a
28:52
lot of the times when you're when you're dealing with these cross cultural issues,
28:56
it openally comes down to respect right, which is which is that a more
29:02
vulnerable group of people have been harmed by a more powerful group of people historically
29:07
speaking, and they would like that harm to stop, and so they would
29:10
like to set up a way to communicate between these groups so that they can
29:17
be more fairly heard. They want to have their voice heard because they haven't
29:19
been listened to for a long time, and they would like to know they're
29:22
being heard in terms of the actions that the more powerful group is going to
29:26
be taking. And so once again, this involves a conversation, and it
29:32
involves thinking about what are the steps that you can take where not everybody's going
29:37
to always get what they want all the time, but hopefully this can be
29:41
done in a way that's at least respectful. That there's an idea called moral
29:45
traces, which is that even when you decide to do something, you need
29:49
to still remember the folks who disagreed with you and what their moral concerns were.
29:53
Can you still do this in a way that is respectful and acknowledges those
29:57
other opinions and funnamentally tries to maintain the relationship with them, because so much
30:03
of contemporary culture and trust is built on relationships, and yet we've been endangering
30:10
these relationships by not listening to people, by not talking to people, by
30:15
not taking actions that demonstrate respect. Just to bring it back to the more
30:19
medical ethics side of things, one of the nice things about medical ethics is
30:22
that it does have this long history of informed consent going on for decades now.
30:27
Of course, it's not a perfect history, and there are some very
30:30
very bad things that happen that cause this to become an important concern in medicine.
30:36
But because medicine has this history built into it in a lot of ways,
30:38
it's more advanced than other areas of society in terms of having this conversation.
30:44
Where it's not advanced and where progress needs to be made is once again
30:48
thinking at the more social level, the interpersonal level, taking it away from
30:52
just one person signing a consent form to thinking about this is our entire society?
30:56
How can we have everyone in society effectively sign a consent form because they're
31:03
being exposed to something that they might not want to be exposed to. And
31:06
I want to keep us going with some of the international considerations here one because
31:11
it's super interesting. And once we get into societal and formed consent, where
31:15
do we draw those boundaries? Right? So the global health line or the
31:18
global sort of social consideration I think is right? Are there groups, organizations?
31:25
Are there places to go? How are people thinking about this sort of
31:30
idea but on a global scale as opposed to just a community one. Yeah,
31:36
there's actually a really interesting grant that I'm part of. It's being run
31:41
in France at the Catholic University of Lyone, and it involves ten universities around
31:45
the world, you know, university in Africa, South America, several in
31:48
Europe and in Asia also. And the basic idea for this grant is to
31:55
get people talking about AI and how it affects democracy and education and healthcare.
32:01
The name of the grant is New Humanism in the Age of Neuroscience and AI,
32:07
and this fundamentally it's about starting this conversation if we're going to have AI
32:14
and neuroscience and other technologies that are really fundamentally transforming how we envision ourselves as
32:20
human beings. And this is a global conversation once again, because everybody in
32:23
the world is going to be affected by these new technologies. Is there some
32:27
way that we can start collecting data that's important global data from people all around
32:32
the world to see how they would either like this technology to be used or
32:37
what their concerns are and their worries are about the technology. So specifically,
32:42
when you're talking about healthcare, for example, people are very concerned that it's
32:45
going to interfere with a doctor patient relationship. That this doctor patient relationship is
32:51
almost kind of like a sacred thing, which is that there's meant to be
32:54
trust, there's meant to be a relationship there which is really built on the
33:00
patient knowing that the doctor has their best interest in mind and wanting to help
33:04
them. And this, you know, this idea, you know, in
33:07
medical ethics goes all the way back to Hippocrates, and you know, goes
33:12
out all across history. And yet we know that of course it's it's not
33:15
something that we can just assume will happen. It has to be something that's
33:17
developed and really reinforced. And we know that technology. One of the major
33:22
things technology does is cut costs. And where did costs come from? They
33:27
come a lot from wages and from time actually spent communicating with people. And
33:32
if you can reduce the number of medical providers and increase the amount of contact
33:37
that people are having with AI, for example, that might make your medical
33:40
system a lot more efficient, but it's not going to make it a lot
33:44
more patient friendly. It's going to make people unhappy with the kind of medical
33:47
care that they're getting. And that's just one example of some of these ideas
33:52
that have come out so far. Now once again getting towards this very very
33:55
automated medical system is still a ways in the future, but we're on the
33:59
trajectory all ready, and so the question really is if we're already on this
34:04
path, the question is where do we stop? What is the right place
34:07
to stop? And we need to think about once again gathering this data because
34:13
it might be that folks in some countries in the world are very happy with
34:15
very automated healthcare while other people in other parts of the world don't want automated
34:20
healthcare at all. So that kind of variability, that's a legitimate variability,
34:24
right. We don't have to have everyone in the world have exactly the same
34:28
thing on this issue, but we do at least need to know what people
34:31
want, what differences exist, and how we can really make sure that technology
34:37
is serving people and not misserving them or serving them in a really inappropriate and
34:44
degrading sort of way that they don't want. And I think there's really good
34:47
opportunity there with this grant, but it's just one grant. There are opportunities
34:52
there for other people around the world to start having these answering the questions also
34:58
and then starting to aggregate the day. It's one thing to have a conversation
35:01
like this at the level of a hospital or a hospital system. It's another
35:06
thing to have it as a global conversation. And that's, like I said,
35:09
really something that we need to be thinking about here. Just to reiterate,
35:12
legitimate variability is allowed. People can have very different desires that they want
35:17
to have fulfilled by AI, but the way the system could be developed could
35:22
be the complete opposite of that, where everybody gets the same thing whether they
35:25
want it or not. This is where we have to start asking what really
35:30
is the right way to do this and can we get some sort of some
35:34
sort of legitimacy based on making sure that we talk to people and find out
35:37
what they actually want. Yeah, understanding what preferences, desires, needs are
35:44
around the world the thing is the first step before we can really go in
35:46
and think critically about how these how the systems get rolled out. One of
35:51
the interesting things about an informed consent is that you fill out the piece of
35:54
paper and then the thing happens to you. But the thing with societal informed
35:59
consent is that it would be better if you could you're going to change society
36:04
and then give consent for it. No, you still have to do that.
36:07
The signing up for consent before it happens. But you don't actually know
36:10
what the technology is going to do until the technology is created and released in
36:14
society. So the informed component of that is extremely difficult to imagine, which
36:20
is why the technology should be developed with society in mind in the first place.
36:24
That's why it's so important to have these conversations around the world, getting
36:28
everyone's data first, what people actually want first, then developing the technology,
36:35
then having it make its impact on society after you've kind of taken that information
36:40
in, because otherwise it's just not possible to get the informed consent before it
36:44
happens. You need to have the conversation before the impact starts becoming too great.
36:49
It needs to be incorporated in that In that sort of way, I
36:51
mean, in some ways, since you're talking about medicine, have the sort
36:54
of surgical decision assist AI ethics question in my head, I mean, the
37:01
one of the arguments sort of in favor of the human surgeon is that,
37:06
well, the human surgeon can in a really serious way understand the values deeply
37:12
of the patient that you know, she has been working with, and so
37:16
when the myriad of decisions, you know, the decision points arises she can
37:21
navigate those well, but it's because she has that data in the first place.
37:24
So even on a societal informed consent scale, yeah, this makes uh,
37:29
this makes a lot of sense. Brian, thank you so much for
37:31
the time. Appreciate you being here. Is Brian Pilkinson. I've been speeing
37:35
with Brian Patrick Green, director of Technology Ethics at the Marklist Center for Applied
37:38
to the Ethics at Santa Clara University, and also, as we mentioned,
37:43
the author of Space Ethics, Ethics in the Age of Disruptive Technologies, and
37:46
Operational Roadmap, Ethics and Technology Practice, and all other excellent technological ethics books.
37:52
So, Brian, thanks so much for the time. I really appreciate
37:53
it. Thank you so much. This podcast series is a production of the
37:58
Health Ethics Lab, and I'm grateful to colleagues in the Department of Medical Sciences,
38:01
the IHS Library, and the Hackensack bridy In School of Medicine for making
38:06
this collaboration possible. To learn more about our work or to engage with the
38:09
series content, reach out to me Brian Pilkington at Brian dot Pilkington at hmh
38:16
N dot org.