Latest / Ethical Machines / Season finale: A New Ethics for AI Ethics?

Transcript

Season finale: A New Ethics for AI Ethics? · Ethical Machines

0:01
[SPEAKER_01]: Before we dive into today's episode, I want to say that this is the last episode of season two, and we'll launch season three sometime in mid to late September.
0:08
[SPEAKER_01]: So you won't hear from me at least not on podcast form for the next month and a half or so, but don't worry, not going away season three coming mid September.
0:18
[SPEAKER_01]: That's the first thing to mention.
0:20
[SPEAKER_01]: The second thing is actually just to talk about today's episode.
0:23
[SPEAKER_01]: It's sort of fitting to end this season with talking to someone who has been in the field of technology ethics for just about as long as anybody else.
0:31
[SPEAKER_01]: So today we're talking to Wanda Walla.
0:33
[SPEAKER_01]: He was the twenty twenty to twenty twenty four Carnegie Hero Senior Fellow at the Carnegie Council for Ethics and International Affairs.
0:41
[SPEAKER_01]: He's also a senior advisor to the Hastings Center and a scholar at the Yale University Interdisciplinary Center for Bioethics where he chaired technology ethics studies for eleven years.
0:51
[SPEAKER_01]: He recently wrote a book called A Dangerous Master, how to keep technology from slipping beyond our control.
0:56
[SPEAKER_01]: And he's also the author of a book called Moral Machines, not to be confused with my own ethical machines.
1:02
[SPEAKER_01]: Moral Machines teaching robots right from wrong, which he wrote back in twenty ten, you're with that book with Colin Allen.
1:08
[SPEAKER_01]: And finally, the World Economic Forum appointed Mr. Wallick Coacher of its global future council on technology, thousand policy for the twenty sixteenth, twenty eighteen term.
1:18
[SPEAKER_01]: So this is someone who has been in the space for a very long time.
1:22
[SPEAKER_01]: He's got lots of interesting things to say.
1:25
[SPEAKER_01]: You can think of our conversations having three parts.
1:27
[SPEAKER_01]: In the first part, we talk a lot about the history of tech ethics or technology ethics.
1:33
[SPEAKER_01]: Why he used to use the phrase and still does use the phrase machine ethics?
1:37
[SPEAKER_01]: Why he doesn't like the phrase value alignment.
1:39
[SPEAKER_01]: That's the first part of the conversation.
1:42
[SPEAKER_01]: In the second part of the conversation he turns from his sort of distaste for the way the technology tends to talk about ethics to the ways in which he has problems with.
1:52
[SPEAKER_01]: the standard ethical theories and how you would apply them to technology ethics.
1:56
[SPEAKER_01]: So he's thinking in particular of deontology and utilitarianism.
2:01
[SPEAKER_01]: Deontology is roughly the view that the right thing to do is whatever the rules say you want to do is a very rough characterization, but it's fine for our purposes for today.
2:09
[SPEAKER_01]: And utilitarianism is the view that you should do that, which maximizes the greatest amount of good for the greatest amount of people.
2:15
[SPEAKER_01]: That's again,
2:17
[SPEAKER_01]: very rough articulation of what you tell Tern is in amounts to, but that's good for our purposes.
2:20
[SPEAKER_01]: So the second part of the conversation is why he rejects those theories as being helpful in the tech ethics base.
2:26
[SPEAKER_01]: And then finally, he talks to me about his own views, what he calls trade-off ethics and synodethics.
2:31
[SPEAKER_01]: But the whole point of his introducing them is that he thinks that they are ways of talking about ethics and doing ethics in a way that's more conducive to actually making
2:43
[SPEAKER_01]: decisions when it matters, when you're designing and deploying AI in other technologies.
2:50
[SPEAKER_01]: Okay, that's what this episode is about.
2:52
[SPEAKER_01]: Again, it's the last one of season two.
2:54
[SPEAKER_01]: If you miss me, you can come see me at substactreadblackment.substact.com.
2:59
[SPEAKER_01]: You can email me at EM at readblackment.com.
3:02
[SPEAKER_01]: And I don't know.
3:03
[SPEAKER_01]: That's about it for now.
3:04
[SPEAKER_01]: Let's go talk to Wendell.
3:19
[SPEAKER_01]: So there's a standard line that we need to get AI to align with our human values.
3:24
[SPEAKER_01]: And so it's all about we are, if you like, morally, socially, a certain way.
3:28
[SPEAKER_01]: And we have to get AI to come port to the kinds of beings that we are.
3:32
[SPEAKER_01]: You've got this claim that strikes me as maybe intentional with that, or it's the opposite of it, or maybe it's not relevant at all, just misunderstanding.
3:38
[SPEAKER_01]: Your claim is that humans need a moral upgrade.
3:42
[SPEAKER_01]: So I guess I want to understand, what, what do I have that claim right and to what would a moral upgrade amount to?
3:47
[SPEAKER_00]: Okay, these aren't exactly two different things.
3:51
[SPEAKER_00]: I've been working on what machines need to be able to be sensitive to moral considerations, in fact, are then that into their choices and actions and then make moral decisions.
4:05
[SPEAKER_00]: And that originally,
4:07
[SPEAKER_00]: twenty years ago when I first got into the subject was called machine ethics and that feels still you didn't get into it you started to create I'm among those who created it yeah yeah so the fascinating thing is what you are talking about is value alignment and that came out of some of the leaders of AI who did not like
4:31
[SPEAKER_00]: machine ethics, they were not convinced by the philosophers getting into this field who actually had created it before they were even thinking of the issue and therefore they brought in this term value alignment.
4:48
[SPEAKER_00]: as a way of putting something that was non-philosophical that they, they, I researchers were going to solve and fit much more into their ballet work.
4:59
[SPEAKER_00]: And I think that this is jumping around to little historical things that happened.
5:04
[SPEAKER_00]: But around two thousand and fifteen, I can be in the first workshops over three years.
5:11
[SPEAKER_00]: that brought those leading AI researchers together with the leading philosophers in what was known as machine ethics, but also other fields like engineering, ethics, and resilience engineering, and a bunch of things.
5:26
[SPEAKER_00]: And in any case, the AI researchers want to go their own way, and they created this term value alignment, which caught on very much.
5:35
[SPEAKER_00]: with the AI researchers.
5:37
[SPEAKER_00]: So if you talk to AI researchers about this question, they think of it as the value alignment problem.
5:43
[SPEAKER_00]: We philosophers who had been thinking about this problem thought that was awfully anemic.
5:51
[SPEAKER_00]: values about as anemic term.
5:54
[SPEAKER_00]: As you can come up with, we thought they didn't understand ethics very well.
5:59
[SPEAKER_00]: Correct.
6:03
[SPEAKER_00]: But the problem is they took a lot of the air out of what had already been a philosophical project as to whether or not we could make
6:13
[SPEAKER_00]: machines sensitive to moral considerations, in fact, are them into their choices and actions.
6:19
[SPEAKER_00]: So when you hear that term, it's probably coming from a researcher who thinks they're going to solve what has been called the value alignment problem.
6:32
[SPEAKER_00]: I've never been convinced that anything has happened yet in the value alignment problem that is worthy of strong consideration.
6:44
[SPEAKER_00]: And I've been a bit missed really that the project of creating moral machines or ethical machines is a certain similarity between the titles of the books we've written even if they're separated by a fifteen or twenty years.
6:59
[SPEAKER_00]: But that I think requires something more.
7:03
[SPEAKER_00]: It requires a degree of sophistication that we have not mastered.
7:09
[SPEAKER_00]: So when we wrote that book, we went through existing ethical theories and talked about whether they were computationally tractable, whether they could be implemented in machines.
7:21
[SPEAKER_00]: But we also ran into what's been a broader problem in ethics for a long time.
7:25
[SPEAKER_00]: Is that people don't agree that they're different ethical theories, that individuals champion different ethical theories,
7:34
[SPEAKER_00]: Most of which fall into these two baskets of the ontological or rule-based theory, which is the Ten Commandments or whatever, your set of rules has to be.
7:46
[SPEAKER_00]: And the only thing you have to do to be good is to follow the rules.
7:51
[SPEAKER_00]: But that got upset in the
7:56
[SPEAKER_00]: Oh, I think it was the eighteenth century.
7:57
[SPEAKER_00]: I keep getting my dates a little confused, but Jeremy Benson was a century here and there.
8:03
[SPEAKER_00]: Yeah.
8:03
[SPEAKER_00]: So the real, for him, the issue was not the rules, the issue was what were the consequences of your action?
8:10
[SPEAKER_00]: And he took the action that had the most positive consequences.
8:15
[SPEAKER_00]: And all that's done is set off this war between people who subscribe to either a particular set of rules or subscribe to consequentialism for the next three hundred years.
8:26
[SPEAKER_00]: When I got into this universe, it was clear that people who were really thinking deeply about ethics, they would bring both systems of thought into play.
8:35
[SPEAKER_00]: But neither system of thought
8:37
[SPEAKER_00]: gave you an algorithm, gave you a system that was going to be a convincing rule.
8:44
[SPEAKER_01]: Yeah, some of them are not even meant to be like indirect utilitarianism is, look, the best, the right thing to do is whatever maximizes happiness, but don't go around trying to make that calculation yourself.
8:53
[SPEAKER_01]: It's not meant to be a decision guide.
8:55
[SPEAKER_01]: It's meant to be a theory about what makes right.
8:57
[SPEAKER_00]: Exactly.
8:58
[SPEAKER_00]: So the problem was that we really didn't have an ethical theory.
9:04
[SPEAKER_00]: that would survive either from machines.
9:07
[SPEAKER_00]: And arguably, we may not have an ethical theory that suffices for humans.
9:13
[SPEAKER_00]: And I believe we do not have ethical constructs, ethical theories that suffice for humans if they're going to deal with some of the more
9:24
[SPEAKER_00]: devilishly difficult dilemmas that were confronting over the neck right now really, and over the next hundred years or so.
9:31
[SPEAKER_00]: And these go all the way from problems of do you embrace enhancements?
9:37
[SPEAKER_00]: or which enhancements do embrace?
9:40
[SPEAKER_00]: Should we have lethal autonomous weapons systems?
9:42
[SPEAKER_00]: Should we have weapons systems that can make life and death decisions?
9:45
[SPEAKER_00]: And can we even stop that from being developed regardless of what the consequences might be?
9:52
[SPEAKER_00]: So that set me thinking, if I were to come up with a new ethical theory, what would that be?
9:58
[SPEAKER_00]: And I knew that it had to do a number of things.
10:01
[SPEAKER_00]: One it had to deal with the problem that for many of our ethical decisions, more than one value come into play, many values may come into play, and they might conflict.
10:11
[SPEAKER_00]: And they are prioritized differently by different people.
10:14
[SPEAKER_00]: And you don't have all the information you need anyways to make an appropriate decision.
10:19
[SPEAKER_00]: So what kind of ethics?
10:22
[SPEAKER_00]: looking at ethics, looking at science, what kind of method would make it possible to work through those devilishly difficult problems?
10:32
[SPEAKER_00]: And I came up with two complementary ethical theories, one that I call trade-off ethics, and the other that I call a silent ethic.
10:42
[SPEAKER_00]: And trade-off ethics is specifically about the traditional social ethical decisions we make.
10:52
[SPEAKER_00]: And what it says is a version of consequentialism, but it can integrate any kind of values you want into it.
10:59
[SPEAKER_00]: And it basically says, look at all the possible options you have.
11:05
[SPEAKER_00]: But don't just look at which of those options maximizes benefits.
11:11
[SPEAKER_00]: Look at the determinants from selecting any of those options.
11:16
[SPEAKER_00]: And look at whether you can ameliorate those
11:19
[SPEAKER_00]: harms.
11:20
[SPEAKER_00]: And if you can manipulate those harms, and that gets factored into the choice you're making.
11:25
[SPEAKER_00]: So it's not enough to take the greatest good for the greatest number.
11:29
[SPEAKER_00]: If in fact what that's going to do is lead to an atom bomb being dropped on Europe or something like that.
11:36
[SPEAKER_00]: I mean, that's just going to just carry me example.
11:38
[SPEAKER_00]: So that's trade-off ethics.
11:40
[SPEAKER_00]: And I think since I started talking about this a number of people are launching on to that because I think that's, yes, that is part of what's wrong.
11:49
[SPEAKER_00]: We don't have ethical systems that can work through competing claims or we don't have ethical systems that really want to moderate the deficits.
11:58
[SPEAKER_00]: So that's the kind of the reason logical analytical way of going forward.
12:07
[SPEAKER_00]: And the
12:08
[SPEAKER_00]: What I call the silent ethic has more to do with what might be traditionally thought of as virtue ethics.
12:16
[SPEAKER_00]: Because virtue ethics was more about the character of the individual, making the decision.
12:21
[SPEAKER_00]: And if you have a virtuous person making a decision, then that's what's right and good.
12:26
[SPEAKER_00]: And I called it a silence ethic because it wasn't about specific virtues like courage or right reason.
12:35
[SPEAKER_00]: Right reason can be taken care of with trade-off ethics.
12:39
[SPEAKER_00]: But it was more about that the individual knew that they were in the right place and the right time in the right way.
12:48
[SPEAKER_00]: And it basically says, when you're making a decision, make that decision, which moves your mind toward a quieting of thought and thinking.
13:01
[SPEAKER_00]: And a quieting of thought thinking has been a major thing since the late sixties, but it's also in sports medicine.
13:10
[SPEAKER_00]: It's in all kinds of thoughts about creativity and art.
13:15
[SPEAKER_00]: There in a certain sense, people are
13:19
[SPEAKER_00]: in their body, they're engaged in the moment when their mind is quiet.
13:24
[SPEAKER_00]: Sure, what the substance use tradition created was an acquiring of a quiet mind.
13:31
[SPEAKER_00]: And I've actually reversed it for ethics.
13:33
[SPEAKER_00]: So this is because anybody who tried to meditate, let's say, or getting sports ethics, they very quickly learned that no technique was going to actually quiet your mind.
13:44
[SPEAKER_00]: It was just going to refresh your thoughts.
13:46
[SPEAKER_00]: So you couldn't find your mind.
13:48
[SPEAKER_00]: You might actually naturally would come to rest as what shall we say, as the byproduct of having dealt with the challenge at hand, so that you are now present because you adult with whatever was a distraction, that distraction coming up to you in a mental form.
14:09
[SPEAKER_00]: That's the silent effort, but these two don't go apart because somebody can repress thought and think they'd silence their mind.
14:17
[SPEAKER_00]: So these are really two sides of a coin.
14:20
[SPEAKER_00]: One is about how the individual knows they're in the right place at the right time in the right way.
14:25
[SPEAKER_00]: And the other is how do we work through that difficult dilemma?
14:32
[SPEAKER_01]: Okay, so we've got this problem where we want to try to figure out how do we get machines to take moral consideration into account.
14:40
[SPEAKER_01]: Old school, deontology, utilitarianism or consequentialism is not going to do the tricks.
14:44
[SPEAKER_01]: We've got these other things.
14:46
[SPEAKER_01]: How does this contrast with the claim that we talked about earlier, which is that humans need a moral upgrade?
14:54
[SPEAKER_01]: Well, that amount, too.
14:56
[SPEAKER_01]: And why do they need an upgrade?
14:57
[SPEAKER_00]: The moral upgrade in this, you brought in this word upgrade.
15:00
[SPEAKER_00]: I haven't usually called this an upgrade, but I have called it an evolution in human moral intelligence.
15:07
[SPEAKER_00]: Sure, we can do that too.
15:09
[SPEAKER_00]: And all I'm saying is that if you look at it deeply, maybe you'll come out in the same place that I did, that these are both principles, processes that represent the thinking
15:25
[SPEAKER_00]: the embodiment of moral intelligence when it has to deal with circumstances where values can flip, where you've got, devilishly difficult problems where you've got a lack of information or blind spots.
15:43
[SPEAKER_00]: So these are both techniques, one being more a reasoning technique and the other being more an internal psychological processing.
15:56
[SPEAKER_01]: Okay, okay, so I think I understand let me I'm gonna make an analogy tell me if this completely misrepresent your view from the vicinity So suppose you think okay, there's all these people I'm just like my people now There's all these people different speaking different languages wouldn't it be great if we also have the same language let's invent a common language call that Esperanza or whatever is that what is Esperanza and then Esperanto I was in the vicinity
16:18
[SPEAKER_01]: Esperanto, and that's going to be like universal language, which we should all communicate.
16:23
[SPEAKER_01]: So take that kind of scenario.
16:24
[SPEAKER_01]: And then I think what you're saying is, we've got machines operating in the way that they're operating, we've got humans operating in the way that they're operating.
16:31
[SPEAKER_01]: What if we came up with a universal moral language, so to speak?
16:34
[SPEAKER_01]: Although it's not just language, it's the reasoning that goes behind the moral decisions that we make.
16:39
[SPEAKER_01]: And if we could come up with an articulation of how human, how any entity ought to be reasoning,
16:46
[SPEAKER_01]: It can apply for humans.
16:47
[SPEAKER_01]: And so as humans get a sort of more upgrade by reasoning in this new kind of way.
16:52
[SPEAKER_01]: And we can also give it to the machines because it's somehow or other operationalizable machines.
16:57
[SPEAKER_01]: And so that's what the mission is.
16:59
[SPEAKER_01]: So that's why you're talking about these two kinds of ethics that are opposite sides of the same coin.
17:03
[SPEAKER_01]: It's like a common sort of a common moral denominator on how to think about
17:08
[SPEAKER_01]: What's morally right and wrong, how to come to those conclusions, what more deliberation looks like and so on.
17:13
[SPEAKER_00]: Is that capture, but I differ a little from how you explicated.
17:17
[SPEAKER_00]: Okay, yes, I don't think of it as about.
17:21
[SPEAKER_00]: in agreement about a reasoning as I look at it as a methodology which can take us toward a morally acceptable conclusion.
17:32
[SPEAKER_00]: So I'm looking at it more as you might look at science.
17:35
[SPEAKER_00]: People who understand science understand that science is not the conclusions that scientists have made.
17:42
[SPEAKER_00]: It's a methodology that can drive you toward
17:47
[SPEAKER_00]: demonstration of what is that fact or what is still ambiguous and what might actually drive further science.
17:55
[SPEAKER_01]: Okay, but let's assume two things though.
17:57
[SPEAKER_01]: I take it that scientists when they're engaging in the scientific method are trying to conform or just conform a hypothesis so that eventually it can receive the honorific of being a full-blown theory.
18:08
[SPEAKER_01]: And that's just a hypothesis anymore.
18:10
[SPEAKER_01]: But when we're engaged in ethical or moral deliberation, trying to figure out what's the right thing to do here, what's the wrong thing to do?
18:16
[SPEAKER_01]: When we're being, as you put it, sensitive to various moral considerations, we're not trying to come to, I don't think, maybe you want to put it this way.
18:22
[SPEAKER_01]: A hypothesis, let alone a theory about what's right and wrong.
18:26
[SPEAKER_01]: We're trying to figure out right here right now, when this particular situation should I, ziggers ag, is it getting morally acceptable or is it getting the moral thing to do?
18:34
[SPEAKER_01]: And so it's not quite a methodology for, it's not like the scientific methodology was coming up with a,
18:40
[SPEAKER_00]: theory, it's no longer, but the science builds on the past and has created a language.
18:46
[SPEAKER_00]: So we now have identified a certain number of chemicals that find their way onto the periodic table.
18:56
[SPEAKER_00]: And the thing has happened with ethics.
18:59
[SPEAKER_00]: The problem with the ontology, for example, is not that most of these values are wrong.
19:06
[SPEAKER_00]: In fact, most of them are pretty much agreed upon.
19:10
[SPEAKER_00]: It's just that when you get into complex decision-making or even sometimes simple decision-making, people prioritize them differently when more than one of the changes of it.
19:21
[SPEAKER_00]: Consider that a few years ago, it became apparent that we were going to need some principles for AI ethics.
19:32
[SPEAKER_00]: And about a hundred by some cons, a hundred and fifty or two hundred different groups
19:38
[SPEAKER_00]: assembles and they put together lists and the lists were by and large they were deanthropological lists and not only that they were more or less the same list it's very little difference between the two leading lists one that came out of the OECD and the other that came out of UNESCO and there is not much difference between them in the Beijing principles
20:02
[SPEAKER_00]: There's only one glaring difference actually, which is the baked gene principles.
20:07
[SPEAKER_00]: Don't list human rights as a valued that is paramount.
20:13
[SPEAKER_00]: even though the Chinese government assigned the international declaration of human rights.
20:20
[SPEAKER_00]: Instead, they bring in, and why does it?
20:23
[SPEAKER_00]: It's not tradition, but it's something like that.
20:26
[SPEAKER_00]: It's basically honoring the culture, which is a much more conservative principle.
20:32
[SPEAKER_00]: But otherwise, it's basically the same list.
20:36
[SPEAKER_00]: So the problem wasn't that there was a lot of disagreement.
20:39
[SPEAKER_00]: There may be some disagreement.
20:41
[SPEAKER_00]: Conservatives and some conservatives in America, some of the mega ones may hate aspects of the international declaration of human rights.
20:50
[SPEAKER_00]: But most people who are not putting their political agenda first, acknowledge that the international declaration of human rights and human rights in general, and doing no harm as a paramount principles that we all agree with.
21:05
[SPEAKER_00]: The problems become in the application of that, and my concern is the application to particularly difficult challenges.
21:13
[SPEAKER_00]: I'm not talking about whether your culture's sexual preferences are well embrace gay marriages or not.
21:20
[SPEAKER_00]: I'm talking about these kinds of issues that I know and we'll basically tear our societies apart if we don't come up with some new framework for dealing with them.
21:33
[SPEAKER_01]: So I'm trying to wrap my head around all this is the ideas of them like all right.
21:38
[SPEAKER_01]: Look, we're sort of on the same page, at least enough on the same page, on a basic list of principles or rules or values or whatever it is.
21:45
[SPEAKER_01]: Where we start to differ is number one in the interpretation of those principles.
21:50
[SPEAKER_01]: So we might all agree, we should respect people, but we disagree about what amounts to respecting someone versus disrespect to someone.
21:56
[SPEAKER_01]: And then we have disagreements about how to apply the set of principles to a given case, because different people will emphasize different
22:04
[SPEAKER_01]: aspects of a particular situation, they'll be more wedded to one principle over another.
22:10
[SPEAKER_01]: And so we just agree about how to actually apply them in a consistent, let alone accurate way.
22:15
[SPEAKER_01]: And so then you come in and say, look, we better figure this out because it's going to be really important for how we conduct ourselves, particularly in the edge of AI.
22:24
[SPEAKER_01]: particularly because of how we're going to program, create, design, build, guide.
22:29
[SPEAKER_00]: It's not only one aspect, but I bring an AI into play.
22:33
[SPEAKER_00]: There are many other things that come into place.
22:36
[SPEAKER_00]: So how can humans and AI work together in this kind of framework is one of the things.
22:42
[SPEAKER_01]: Okay, look, here's how we take things usually go or away things go in legal cases, right?
22:47
[SPEAKER_01]: We've got the constitution, we have the declaration of independence.
22:50
[SPEAKER_01]: We've got human rights, blah, blah, blah.
22:52
[SPEAKER_01]: We've got case law and what you do in a new case is you take the various principles, the various
23:00
[SPEAKER_01]: deliberations that show up in case law, the conclusions that show up in case law, you apply to new cases and blah, blah, off you go.
23:06
[SPEAKER_01]: There's not when you do that kind of thing, you couldn't do it while you could do it poorly.
23:10
[SPEAKER_01]: I think everyone knows that.
23:12
[SPEAKER_01]: But when you do it well, it's not obvious that you're doing it well by virtue of following a procedure, such that following procedure means that you are doing it well.
23:22
[SPEAKER_01]: It's rather something like, as you put it earlier, a sensitivity to different in this case legal considerations,
23:29
[SPEAKER_01]: The different things that are relevant to irrelevant, the extent to which they're relevant to irrelevant, the extent to which certain features of the situation are important or not important, so on and so forth.
23:37
[SPEAKER_01]: But you do that with a kind of, I don't know, for lack of a better phrase, intuitive sense of what's important, what's not, what's relevant and what's not relevant, as opposed to following some explicit procedure.
23:49
[SPEAKER_01]: So if we know that we can do well in that kind of legal deliberative case, without while there are legal conflicts, right, there's legal, there's the rights, the legal rights of different people come into conflict, we've got to weigh these kinds of things.
24:02
[SPEAKER_01]: And we can do without having an articulate or discrete or something like that methodology for doing it.
24:09
[SPEAKER_01]: Why should we think that we need it in the moral case?
24:12
[SPEAKER_00]: If we can do it without a discreet methodology fine, I'm just saying we aren't doing it.
24:17
[SPEAKER_00]: We're filling it that.
24:19
[SPEAKER_00]: And the kinds of challenges that are coming down the pike are ones that we don't seem to have a satisfactory way of dealing with.
24:27
[SPEAKER_00]: So I'm trying to point what I think the direction is.
24:30
[SPEAKER_00]: to deal with this in a satisfactory manner.
24:34
[SPEAKER_00]: And I've got nothing against intuition that might actually play in a different sense.
24:39
[SPEAKER_00]: There's certain things that I feel that I don't have to question.
24:43
[SPEAKER_00]: I know that this is the name of the game.
24:46
[SPEAKER_00]: And I'm quiet about it.
24:47
[SPEAKER_00]: I'm not questioning.
24:49
[SPEAKER_00]: But other people may have questions about where I'm standing.
24:53
[SPEAKER_00]: And therefore we may still have to go through the trade-off ethics, conversation.
25:00
[SPEAKER_00]: But if I know that I'm in the right place in the right way at the right time, I'm not likely to move from there until somebody points out something to me is, will you miss this?
25:11
[SPEAKER_00]: And you're less stable than you think you are.
25:13
[SPEAKER_00]: You're less encompassing than you think you're being.
25:17
[SPEAKER_00]: And that's fine.
25:19
[SPEAKER_00]: That's part of the moral dialogue.
25:21
[SPEAKER_00]: But I think what it comes to intuition or it comes to what I will call wisdom or your understanding
25:27
[SPEAKER_00]: Many of us can see things pretty quickly in black and white terms, and I don't mean black and white, it's absolutely here.
25:34
[SPEAKER_00]: I mean that they understand this is wrong.
25:38
[SPEAKER_00]: It's not very complicated for me to go through a lot of things that our president is doing right now, but are just wrong.
25:48
[SPEAKER_00]: Presuming, preserving the union and the constitution itself or the betterment of all the people is the bottom line of what you're trying to do.
25:59
[SPEAKER_00]: But I may have to get into a pretty intense dialogue with other people who don't see it as wrong to find out why they don't see it or to find out whether my sense of what's right and wrong is inappropriate in this case.
26:19
[SPEAKER_00]: or understanding the case at hand.
26:21
[SPEAKER_00]: So I think I'm right, but I'm actually talking about something totally different.
26:27
[SPEAKER_01]: Okay, so let me give you, so here's always see the jury that you're trying that you're articulating.
26:32
[SPEAKER_01]: It's something like we've got to come up with this sort of methodology to engage in the appropriate weighing of conflicting values, principles, rules, whatever.
26:45
[SPEAKER_01]: That's if you'll accept one.
26:47
[SPEAKER_01]: And then step two is once we come to a kind of agreement on that, then we apply it to how we develop AI.
26:54
[SPEAKER_01]: Do I have that right?
26:55
[SPEAKER_00]: I'm less concerned about this in terms of how we develop AI.
27:00
[SPEAKER_00]: I think yes, there are issues in the development of AI, such as
27:06
[SPEAKER_00]: Do we develop lethal autonomous weapons and deploy them or do we put in place some kind of international binding framework?
27:16
[SPEAKER_00]: We can't be going down that road and anybody who is going down that road is not working in the interest of humanity at the whole.
27:25
[SPEAKER_00]: So yes, but a lot of those things, I think are relatively clear, but not absolutely clear, because they haven't been brought to an end.
27:34
[SPEAKER_00]: So I can say I feel absolutely clear that lethal autonomous weapons are a bad idea, or autonomy in machines making life and death decisions about humans.
27:49
[SPEAKER_00]: has no place given where we have arrived in terms of the intelligence and morality of the machines.
27:59
[SPEAKER_00]: And in terms of where that may take us over the next, fifty-hundred thousand years, that is just untenable.
28:08
[SPEAKER_00]: It's untenable if the survival of humans is a concern to you.
28:13
[SPEAKER_01]: I'm losing this thread a bit because I thought that
28:17
[SPEAKER_01]: We're trying to figure out, among other things, number one, how do we engage fruitfully in ethical deliberation and moral deliberation?
28:26
[SPEAKER_01]: And secondly, how do we get machines to engage in something like moral deliberation or sensitivity to moral considerations or whatever it is?
28:35
[SPEAKER_00]: And because that's a tough problem, we sort of, I think we're going a little too fast because we're conflating the two things, okay?
28:45
[SPEAKER_00]: One is,
28:48
[SPEAKER_00]: We don't have actually any good theory about how to get machines to make morally good decisions.
28:55
[SPEAKER_00]: Yeah, agreed.
28:56
[SPEAKER_00]: We just don't have it.
28:57
[SPEAKER_00]: So to some extent, part of what I'm arguing when it comes to machines is you cannot rely on their decision-making ability, except perhaps some very confined
29:17
[SPEAKER_00]: vertical markets where there's not likely to be much overflow.
29:23
[SPEAKER_00]: A context of fixed predictable overflow from their decision making to other realms that you can't predict.
29:29
[SPEAKER_00]: Now, oftentimes, that's less true than it seems.
29:33
[SPEAKER_00]: And that's one of the problems we have with machines.
29:36
[SPEAKER_00]: taking decisions within sensitive control architecture.
29:40
[SPEAKER_00]: But in principle, I think that we just don't have it.
29:44
[SPEAKER_00]: And we only have it in pretty confined cases.
29:47
[SPEAKER_00]: We have what in moral machines we called operational morality, which is basically the morality that its program didn't have the machines by the designers of systems because I can more or less anticipate all the situations of machine.
30:02
[SPEAKER_00]: can get into.
30:03
[SPEAKER_00]: But then there's this next level which is functional morality comes a little bit more difficult.
30:09
[SPEAKER_00]: So now we have algorithmic bias built into the machines because of the data it was trained on.
30:17
[SPEAKER_00]: Now you may have to introduce some kind of ethical algorithm or some kind of mechanism that's either going to filter that out or point out when that's occurring.
30:32
[SPEAKER_00]: either that or you need humans working with machine or so sensitive that they'll recognize, who understand the whole field so well that they'll recognize when bias is occurring, but that's getting harder and harder.
30:43
[SPEAKER_00]: That's less of what we can rely upon.
30:46
[SPEAKER_00]: So I think in the meantime, the ethical decision making part still belongs with the humans.
30:55
[SPEAKER_00]: But that doesn't mean, and it may not be even humans individually.
31:00
[SPEAKER_00]: When we're talking about devilish political problems, I've been in dialogues for myself for twenty-five years about enhancements.
31:10
[SPEAKER_00]: We have to finish that conversation.
31:12
[SPEAKER_00]: Down to the station has a lot of ways to go.
31:15
[SPEAKER_00]: Flossers never finish a conversation.
31:17
[SPEAKER_00]: But it's not just a silly stuff.
31:18
[SPEAKER_00]: We're going to finish the practical conversation because it's going to come up with each different kind of drug for example or neural link or whatever we're talking about.
31:29
[SPEAKER_00]: how this falls within ethical realms.
31:33
[SPEAKER_00]: So we're still laying out the framework through which we might be able to make evaluations as these cases come up.
31:40
[SPEAKER_00]: And it's a collective conversation.
31:42
[SPEAKER_00]: It's not one person.
31:44
[SPEAKER_00]: It's a significant portion of the society, which has been engaged in that reflection.
31:49
[SPEAKER_00]: I don't know how significant a lot of people have just opinions, but they couldn't really
31:54
[SPEAKER_00]: It could really go and give you a concrete explanation, but the ones who are reflecting deeply are glasses and enhancement or not, and is that enhancement that justifies a lot of other enhancements or it's just something different.
32:09
[SPEAKER_00]: These conversations are going on.
32:11
[SPEAKER_00]: I'm not talking about this simple thing about whether your teenage daughter should sleep with her boyfriend or not.
32:16
[SPEAKER_00]: I'm talking about these big issues that are going to die the direction of humanity for as long as we can keep them humanity going.
32:29
[SPEAKER_00]: So that becomes a difficult thing.
32:30
[SPEAKER_00]: So what I think is we're involved in really what is collective problems solving, or joint problems solving, but the machines can be a significant player in that joint problem solving.
32:46
[SPEAKER_00]: For example, one of the reasons we humans, I mean, we humans don't engage in consequentialist calculations, even if we say we're consequentialist.
32:55
[SPEAKER_00]: because we can't go through all the processes and we don't know where the stopping point is.
33:01
[SPEAKER_00]: Do I look at the cherry and other other consequences?
33:07
[SPEAKER_00]: So we don't really engage in that.
33:10
[SPEAKER_00]: an intuition pump that I have an intuition that this will cause less harm.
33:15
[SPEAKER_00]: And then we go with that.
33:17
[SPEAKER_00]: So it's not like we have a, we have bounded morality.
33:22
[SPEAKER_00]: We have a, we have bounded cognition to go back to her Simon, who got an,
33:29
[SPEAKER_00]: who got a Nobel Prize for bounded rationality, and it's one of the fathers of AI, saying, we can just contain so much in our mind at the same time.
33:40
[SPEAKER_00]: And therefore, we are not able to look at all the options.
33:44
[SPEAKER_00]: And we tend to go with that one, which kind of intuitionally is satisfying, what says word.
33:50
[SPEAKER_00]: You're feeling it.
33:51
[SPEAKER_00]: So it feels good.
33:53
[SPEAKER_00]: But the fact is, with machines, it could be a little bit different.
33:56
[SPEAKER_00]: I could ask these machines get more and more sophisticated, and he can begin to have these kinds of dialogues, well, LL and saying, okay, which option should I consider in this case?
34:10
[SPEAKER_00]: Now look at each of those options and what are the trade-offs?
34:14
[SPEAKER_00]: What are the possible consequences that might be undesirable for that option?
34:20
[SPEAKER_00]: I don't think the machine effectively has been able to make the final decision, but if I had a lot of that information laid out for me in a dialogue process, I might get a pretty good idea.
34:33
[SPEAKER_00]: which of those options, yeah, which of those courses of actions leads to a societally appropriate response or framework.
34:43
[SPEAKER_00]: And it may be the two or three of them do, or maybe it's only one, but it's certainly not in the realm of there's an absolute right and an absolute wrong.
34:53
[SPEAKER_01]: Is that part of what you mean when you say we need a moral upgrade that we couldn't use machines to aid us in moral deliberation?
35:02
[SPEAKER_01]: So you could fight in a situation, you go to your favorite LLM claw to whatever.
35:06
[SPEAKER_01]: Use tell the situation that you're in, blah, blah, blah, blah.
35:09
[SPEAKER_01]: And it tells you if you like the moral pros and cons of this course of action versus that course of action.
35:14
[SPEAKER_00]: I'd have to think about what you just said there because I don't know if you have gotten that far yet.
35:20
[SPEAKER_00]: In other words, I think of an evolution in individuals moral intelligence, which you're calling an upgrade.
35:28
[SPEAKER_00]: I'm just saying, I think that's the next step in ethical understanding, but whether it's now in combination with the machines in upgrade in moral decision making, yeah.
35:42
[SPEAKER_00]: as I try and bring in your language for it.
35:45
[SPEAKER_00]: Yeah, sure.
35:46
[SPEAKER_00]: Because I think we are moving into realms where the machines can be helpful in illustrating the options, but they can't be helpful in making the decisions.
35:57
[SPEAKER_00]: They don't have the cognitive framework to make the decisions.
36:01
[SPEAKER_00]: Let me go back to moral machines, two thousand and eight, but book comes out in two thousand nine ten.
36:08
[SPEAKER_00]: It doesn't really matter.
36:09
[SPEAKER_00]: But after we went through the exercise of looking at moral theories and whether they were computationally instantiable.
36:18
[SPEAKER_00]: We sort of, well, does that get us there?
36:22
[SPEAKER_00]: And even if you could instantiate, let's say consequences, which would be the easiest one, actually, because it says just calculate.
36:30
[SPEAKER_00]: Great, it's great for the great number.
36:32
[SPEAKER_00]: Presuming you could define good and, you know, what was great.
36:36
[SPEAKER_01]: Yeah, you got a conventional goods, which could be a good one.
36:38
[SPEAKER_00]: All these, I mean, we get into the world of different forms of consequentials.
36:43
[SPEAKER_00]: And then we're at the river.
36:44
[SPEAKER_00]: But the point is, even in principle, if any one of them could be infected, would that be adequate?
36:51
[SPEAKER_00]: And the problem we realized was there's all these things that humans have that we take for granted when we talk about moral decision-making, that you cannot take for granted when you're talking about machines.
37:07
[SPEAKER_00]: because you're building a moral decision making and an artificial moral agent from the bottom up.
37:13
[SPEAKER_00]: You can't take for granted consciousness.
37:17
[SPEAKER_00]: You can't take for granted sentence.
37:19
[SPEAKER_00]: You can't take for granted being embodied.
37:21
[SPEAKER_00]: You can't take them for granted moral emotions.
37:24
[SPEAKER_00]: You can't take for granted a theory of mind.
37:27
[SPEAKER_01]: Let's go to this.
37:28
[SPEAKER_01]: Where do you think the research needs to go at this point?
37:34
[SPEAKER_01]: When we're talking about the intersection of ethics and AI, where do you think the most important work is yet to be done?
37:38
[SPEAKER_00]: I think the most important work is sort of out there, but not being, not having succeeded yet.
37:46
[SPEAKER_00]: And that's the responsible implementation of AI.
37:51
[SPEAKER_00]: Most of what's happening right now is irresponsible implementation.
37:56
[SPEAKER_00]: And so then the problem is how do you get responsible implementation?
38:00
[SPEAKER_00]: First of all, you realize that AI isn't some simplistic thing where you just plug in this machine.
38:06
[SPEAKER_00]: It gives you answers, you ask questions and it gives you answers.
38:10
[SPEAKER_00]: It's a technology that is pretty tricky.
38:14
[SPEAKER_00]: And if you don't understand what you can rely upon and what you can't rely upon and what the system will do, then you're going to publish a brief that has sites cases that don't exist.
38:25
[SPEAKER_00]: So I think that's the hard work right now.
38:30
[SPEAKER_00]: And the other hard, the hard work is controllability.
38:33
[SPEAKER_00]: But controllability is already there in the sense that you want to know that if the AI is taking an action, you have some sense of why it's taking the action and you can stop it.
38:46
[SPEAKER_00]: But the most of the controllability research is directed at our official super intelligence.
38:52
[SPEAKER_00]: It's just how do we ensure that we don't get runaway AI.
38:56
[SPEAKER_00]: So I think controllability, safety, really responsible deployment of systems.
39:04
[SPEAKER_00]: And in a certain sense, those are all human projects.
39:08
[SPEAKER_01]: Okay, so bottom line, it's just like you think we've got enough of the ethics figured out for now, such that in principle, we should be able to make moral machines.
39:21
[SPEAKER_00]: There's a gap between what we know.
39:22
[SPEAKER_00]: We can make more on machines.
39:25
[SPEAKER_00]: I think we can work through the difficult decisions as to whether where we want to deploy the machines can be done responsibly.
39:37
[SPEAKER_00]: That's not the more machines.
39:39
[SPEAKER_00]: More machines is that the machines themselves have the faculties.
39:44
[SPEAKER_00]: And machines have some faculties.
39:46
[SPEAKER_00]: There's been low budget research, which is put forward pretty simplistic issues to log language models and
39:55
[SPEAKER_00]: Some people say the responses are better than some humans.
39:59
[SPEAKER_00]: I mean, there's actually some humans, but on all dimensions, including virtue, I read that and I almost jumped out of my seat because virtue isn't that a machine tells you what is the virtuous thing to do, virtuous that you have the will and intention to do it.
40:19
[SPEAKER_00]: That has to be it.
40:19
[SPEAKER_00]: Yeah, yeah, you are.
40:21
[SPEAKER_00]: Sure.
40:21
[SPEAKER_00]: And I'm sure I can know what the virtuous thing to do to I don't do it.
40:25
[SPEAKER_00]: That doesn't make me virtuous.
40:27
[SPEAKER_01]: Sure.
40:27
[SPEAKER_01]: That makes you continent on air.
40:28
[SPEAKER_01]: So that was okay.
40:30
[SPEAKER_01]: All right.
40:31
[SPEAKER_01]: We'll have to keep fighting the good fight.
40:33
[SPEAKER_01]: So thank you.
40:34
[SPEAKER_01]: This has been fun.
40:35
[SPEAKER_01]: No, this has been great.
40:36
[SPEAKER_01]: Yeah.
40:36
[SPEAKER_01]: Thanks so much, bundle.
40:37
[SPEAKER_01]: Really enjoyed it.
40:40
[SPEAKER_01]: Thanks so much everyone for listening and now let me thank the people who made this podcast possible.
40:45
[SPEAKER_01]: So first of all, we have the Lotto Alexander.
40:47
[SPEAKER_01]: He's been with me since day one.
40:49
[SPEAKER_01]: He showed me even how to start recording a podcast in the first place.
40:52
[SPEAKER_01]: He does editing stuff.
40:53
[SPEAKER_01]: He's been fantastic and he does really everything I ask him to and more.
40:58
[SPEAKER_01]: I've never read anyone else who says so much.
41:00
[SPEAKER_01]: Yeah, I don't know how to do that, but I'll figure it out.
41:02
[SPEAKER_01]: What more could you ask for in a producer?
41:04
[SPEAKER_01]: Next up, Jennifer Strong, my podcast guru, I don't know what else to call her.
41:08
[SPEAKER_01]: She just advised me on how to design a podcast, how to grow it, and she's responsible for if you like this kind of season two relaunch and everything around it.
41:17
[SPEAKER_01]: So thank you to you, Jennifer.
41:19
[SPEAKER_01]: Next up, Meg Marco, responsible for all the excellent artwork.
41:22
[SPEAKER_01]: Before I did the artwork, I didn't really know what I was doing.
41:25
[SPEAKER_01]: We put it together, it worked well enough.
41:27
[SPEAKER_01]: But I think Meg really did a phenomenal job bringing everything together.
41:31
[SPEAKER_01]: Garrett Lane, sound engineer, that I frankly don't deserve, he's just too good for this podcast.
41:37
[SPEAKER_01]: Don't tell him I said that though.
41:38
[SPEAKER_01]: Next up, Benjamin Colton, he's a professor of philosophy who has helped me source guests identify who's interesting, who has things to say.
41:47
[SPEAKER_01]: et cetera, et cetera.
41:47
[SPEAKER_01]: So without him, I would have anyone to talk to.
41:50
[SPEAKER_01]: And then finally, Ashley Wasselenko, who helps with all manner of administrative things, making sure that the podcast guests find their way to the podcast and schedule the recording, get their bios and headshots, and really just keeping the show on the rail to me organized.
42:04
[SPEAKER_01]: Thanks to all of you, really appreciate all your hard work.