Latest / Gil I Am / The Worst Case Scenario for AI

Transcript

The Worst Case Scenario for AI · Gil I Am

• 0:00
  Speaker 1: The generalgun experience. So this is logical. And the problem is,
• 0:05
  Speaker 1: like I said, when I've talked to Mark and Dreesen
• 0:08
  Speaker 1: and many other people, they think this is just fear mongering.
• 0:13
  Speaker 1: Will be fine, This is worst case scenario. Will be fine.
• 0:17
  Speaker 2: It is worst case scenario. But that's standard in computer
• 0:20
  Speaker 2: science and cryptography and complexity and computability. You're not looking
• 0:23
  Speaker 2: at best case. I'm ready for a best case, give
• 0:25
  Speaker 2: me utopia. I'm looking at problems which I likely to happen.
• 0:29
  Speaker 2: And it's not just me saying it. You have Nobel
• 0:32
  Speaker 2: Prize winners during award winners all saying this is very dangerous.
• 0:37
  Speaker 2: Twenty thirty percent pedom. This is standard in industry. Thirty
• 0:41
  Speaker 2: percent is what surveys of machine learning experts are giving
• 0:44
  Speaker 2: us right now.
• 0:46
  Speaker 1: So what is worst case scenario? Like how could AI
• 0:50
  Speaker 1: eventually lead to the destruction of the human race?
• 0:53
  Speaker 2: So you can asking me how I would kill everyone? Sure,
• 0:56
  Speaker 2: and that's a great question. I can give you standard answers.
• 1:00
  Speaker 2: I would talk about computer viruses breaking into maybe nuclear facilities,
• 1:05
  Speaker 2: nuclear of war. I can talk about synthetic biology and nanottack,
• 1:09
  Speaker 2: but all that is not interesting. Then you realize we're
• 1:11
  Speaker 2: talking about superintelligence, a system which is thousands of times
• 1:15
  Speaker 2: smarter than me. It would come up with something completely novel,
• 1:18
  Speaker 2: more optimal, better way, more efficient way of doing it,
• 1:21
  Speaker 2: and I cannot predict it because I'm not that smart. Jesus,
• 1:27
  Speaker 2: That's exactly what it is. We're basically setting up an
• 1:31
  Speaker 2: adversarial situation with agents which are like squirrels versus humans.
• 1:37
  Speaker 2: No group of squirrels can figure out how to control us.
• 1:41
  Speaker 2: Even if you give them more resources, more acorns, whatever,
• 1:44
  Speaker 2: They're not going to solve that problem. And it's the
• 1:46
  Speaker 2: same for us. And most people think one or two
• 1:48
  Speaker 2: steps ahead, and it's not enough. It's not enough in chess,
• 1:52
  Speaker 2: it's not enough here. If you think about AGI and
• 1:55
  Speaker 2: then maybe super intelligence, that's not the end of that game.
• 1:58
  Speaker 2: The process continues. You'll get super intelligence creating next level AI.
• 2:02
  Speaker 2: So superintelligence plus plus two point zero three point zero,
• 2:06
  Speaker 2: it goes on indefinitely. You have to create a safety
• 2:08
  Speaker 2: mechanism which scales forever, never makes mistakes, and keeps us
• 2:13
  Speaker 2: in decision making position so we can undo something if
• 2:16
  Speaker 2: we don't like it.
• 2:17
  Speaker 1: And it would take super intelligence to create a safety
• 2:20
  Speaker 1: mechanism to control superintelligence.
• 2:22
  Speaker 2: At that level, and it's a catch twenty two. If
• 2:24
  Speaker 2: we had friendly AI, we can make another friendly eye,
• 2:27
  Speaker 2: so like aliens send us one and we trust it,
• 2:30
  Speaker 2: then we can use it to build local version, which
• 2:33
  Speaker 2: is somewhat safe.
• 2:36
  Speaker 1: Have you thought about the possibility that this is the
• 2:39
  Speaker 1: role of the human race in that this happens all
• 2:43
  Speaker 1: throughout the cosmos, is that curious humans who thrive on
• 2:48
  Speaker 1: innovation will ultimately create a better version of life.
• 2:54
  Speaker 2: I thought about it. Many people think that's the answer
• 2:58
  Speaker 2: to Fermi paradox. It is also now a group of
• 3:02
  Speaker 2: people looking at what they call a warfy successor basically
• 3:06
  Speaker 2: they kind of say, yep, we're going to build superintelligence. Yep,
• 3:09
  Speaker 2: we can't control it. So what properties would we like
• 3:11
  Speaker 2: to see in those systems? How important is it that
• 3:14
  Speaker 2: it likes art and poetry and spreads it through the universe?
• 3:17
  Speaker 2: And to me, it's like, I don't want to give
• 3:20
  Speaker 2: up yet. I'm not ready to decide if killers of
• 3:23
  Speaker 2: my family and everyone will like poetry. I want to
• 3:26
  Speaker 2: We're still here, We're still making decisions. Let's figure out
• 3:29
  Speaker 2: what we can do well.
• 3:30
  Speaker 1: Poetry is only relevant to us because poetry is difficult
• 3:34
  Speaker 1: to create and it resonates with us. Poetry doesn't mean
• 3:38
  Speaker 1: jack shit to a flower.
• 3:40
  Speaker 2: It's more global to me. I don't care what happens
• 3:42
  Speaker 2: after I'm dead. My family is that all the humans
• 3:45
  Speaker 2: are that whatever they like poetry or not is irrelevant
• 3:47
  Speaker 2: to me.
• 3:48
  Speaker 1: Right. But the point is, like the things that we
• 3:51
  Speaker 1: put meaning in, they it's only us. The you know,
• 3:56
  Speaker 1: super massive black hole doesn't give a shit about a
• 3:59
  Speaker 1: great song.
• 4:00
  Speaker 2: And they talk about some super value, superculture, super things
• 4:04
  Speaker 2: superintelligence would like. And it's important that they are conscious
• 4:07
  Speaker 2: and experienced all that greatness in the universe.
• 4:11
  Speaker 1: But I would think that they would look at us
• 4:13
  Speaker 1: the same way we look at chimpanzees. We would say, yeah,
• 4:17
  Speaker 1: they're great, but don't give them guns. Yeah they're great,
• 4:19
  Speaker 1: but don't let them have airplanes, don't let them make
• 4:23
  Speaker 1: global geopolitical decisions.
• 4:28
  Speaker 2: So there are many reasons why they can decide that
• 4:32
  Speaker 2: we're dangerous. We may create competing AI, we may decide
• 4:37
  Speaker 2: we're want to shut them off. So for many reasons,
• 4:39
  Speaker 2: we would try to restrict our abilities. Restrict our capabilities
• 4:44
  Speaker 2: for sure.
• 4:45
  Speaker 1: Yeah, and there's no reason why they would not limit
• 4:49
  Speaker 1: our freedoms.
• 4:52
  Speaker 2: If there is something only a human can do. And
• 4:54
  Speaker 2: I don't think there is anything like that. But let's
• 4:57
  Speaker 2: say we are conscious, we have internal experience, and they
• 5:00
  Speaker 2: can never get it. I don't believe it. But let's
• 5:02
  Speaker 2: say it was true and for some reason they wanted
• 5:04
  Speaker 2: to have that capability. They would need us and give
• 5:08
  Speaker 2: us enough freedom to experience the universe, to collect those quality,
• 5:12
  Speaker 2: to kind of engage with what is fun about being
• 5:16
  Speaker 2: a living human being? What makes it meaningful?
• 5:19
  Speaker 1: Right? But that's such an egotistical perspective, right, that we're
• 5:23
  Speaker 1: so unique that even super intelligence would say, Wow, I
• 5:26
  Speaker 1: wish I was human. Humans have this unique quality of
• 5:30
  Speaker 1: confusion and creativity.
• 5:32
  Speaker 2: There is no value in it, mostly because we can't
• 5:34
  Speaker 2: even test for it. I have no idea if you
• 5:35
  Speaker 2: are actually conscious on that. So how valuable can it
• 5:38
  Speaker 2: be if I can't even detected? Only you know what
• 5:43
  Speaker 2: ice cream tastes like to you? Okay, that's great, sell it,
• 5:46
  Speaker 2: now make a product out of it? Right?
• 5:48
  Speaker 1: And there's obviously variables because there's things that people like
• 5:52
  Speaker 1: that I think are gross.
• 5:54
  Speaker 2: Absolutely, So really you can come up with some agent
• 5:56
  Speaker 2: which likes anything or finds anything fun.
• 6:00
  Speaker 1: Oh God, why are you freaking me out? Right away.
• 6:04
  Speaker 1: That's the problem. This podcast is eighteen minutes old, and
• 6:07
  Speaker 1: I'm like, we could just stop right now, a.
• 6:10
  Speaker 2: Couple hours at least, and then I'll.
• 6:14
  Speaker 1: I don't want to end. I have so many questions.
• 6:16
  Speaker 1: But it's just the problem is we got off to it.
• 6:20
  Speaker 1: We just cut to the chase right away, and the
• 6:23
  Speaker 1: chase seems to be it's something that must be confronted
• 6:27
  Speaker 1: because it is. It's right there. That's it. That's the
• 6:31
  Speaker 1: whole thing. And I've tried so hard to listen to
• 6:37
  Speaker 1: these people that don't think that it's a problem, and
• 6:39
  Speaker 1: listen to these people that think that it's going to
• 6:41
  Speaker 1: be a net positive for humanity, and oh God's good.
• 6:44
  Speaker 1: I feel better now. But it doesn't work. It doesn't resonate.
• 6:47
  Speaker 2: I wish they were right. Every time I have a
• 6:49
  Speaker 2: debate with someone like that, I'm like, please come up
• 6:52
  Speaker 2: with better arguments, prove me wrong. I don't want to
• 6:54
  Speaker 2: be right on this one. I want you to show
• 6:56
  Speaker 2: all the mistakes in my papers. I want you to
• 6:59
  Speaker 2: show me how to control superintelligence and give us utopia,
• 7:02
  Speaker 2: solve cancer, give us free stuff. That's great.
• 7:06
  Speaker 1: When you think about the future of the world and
• 7:10
  Speaker 1: you think about these incredible technology scaling upwards and exponentially
• 7:17
  Speaker 1: increasing in their capability. What do you see like? What
• 7:22
  Speaker 1: do you think is going to happen?
• 7:24
  Speaker 2: So there are many reasons to think they may cancel
• 7:28
  Speaker 2: us for whatever reasons. We started talking about some game
• 7:31
  Speaker 2: theoretical reasons for it. If we are successful at controlling them,
• 7:35
  Speaker 2: I can come up with some ways to provide sort
• 7:39
  Speaker 2: of partial solution to the value alignment problem. It's very
• 7:42
  Speaker 2: hard to value align eight billion people, all the animals,
• 7:46
  Speaker 2: you know, everyone, because we disagree, we like many different things.
• 7:49
  Speaker 2: So we have advanced virtual reality technology. We can technically
• 7:53
  Speaker 2: give every person their own virtual universe where you decide
• 7:56
  Speaker 2: what you want to be. You're a king, you're a slave,
• 7:58
  Speaker 2: whatever it is u wento, and you can share with others.
• 8:01
  Speaker 2: You can visit their universes. All we have to do
• 8:04
  Speaker 2: is figure out how to control the substrate, the superintelligence
• 8:07
  Speaker 2: running all those virtual universes. And if we manage to
• 8:10
  Speaker 2: do that at least part of the value alignment problem,
• 8:13
  Speaker 2: which is super difficult. How do you get different preferences
• 8:17
  Speaker 2: multi objective optimization? Essentially, how do you get different objectives
• 8:21
  Speaker 2: to all agree?
• 8:23
  Speaker 1: But when you think about how it plays out when
• 8:30
  Speaker 1: like if you're alone at night and you're worried, what
• 8:33
  Speaker 1: do you see? What do you see happening?
• 8:36
  Speaker 2: So there are multiple levels of risk. Immediate is what
• 8:39
  Speaker 2: we call EKEYGI risk, IIG risk will lose meaning you
• 8:42
  Speaker 2: lost your job, You're no longer the best interviewer in
• 8:45
  Speaker 2: the world, Like what's left? What are you going to do?
• 8:48
  Speaker 2: Maybe some people will find some other artificial things to do,
• 8:54
  Speaker 2: but for most people, their job is their definition who
• 8:57
  Speaker 2: they are, what makes a difference to them For quiet
• 9:00
  Speaker 2: a few people, especially in professional circles, so losing that
• 9:03
  Speaker 2: meaning will have terrible impact. In society. We always talk
• 9:07
  Speaker 2: about unconditional basic income. We never talk about unconditional basic
• 9:11
  Speaker 2: meaning what are you doing with your life if basic
• 9:15
  Speaker 2: needs are provided for you? Next level is existential risk.
• 9:19
  Speaker 2: The concern is it will kill everyone, but there is
• 9:22
  Speaker 2: also suffering risks. Well, whatever it is, and it's not
• 9:25
  Speaker 2: even killing us, it's keeping us around forever, and we
• 9:29
  Speaker 2: would rather be dead. It's so bad.
• 9:33
  Speaker 1: What do you see when you think of that?
• 9:37
  Speaker 2: It's hard to be specific about what it can do
• 9:41
  Speaker 2: and what specific ways of torture it can come up
• 9:46
  Speaker 2: with and why Again, if we're looking at worst case scenarios.
• 9:49
  Speaker 2: I found this set of papers about what happens then
• 9:55
  Speaker 2: young children have epileptic seizures, really bad ones, and that
• 10:00
  Speaker 2: sometimes helps is to remove half of your brain, just
• 10:04
  Speaker 2: cut it out. And there are two types of surgeries
• 10:07
  Speaker 2: for doing that. One is to remove it completely and
• 10:10
  Speaker 2: one is to kind of dissect connections leading to that
• 10:13
  Speaker 2: half and leave it inside. So it's like solitary confinement
• 10:17
  Speaker 2: with zero input output forever. And there are equivalents for
• 10:21
  Speaker 2: digital forms and things like that.
• 10:25
  Speaker 1: And you worry that AI would do that to the
• 10:28
  Speaker 1: human race. It is a pass essentially.
• 10:35
  Speaker 2: Well. Loss of control is a part of it. But
• 10:37
  Speaker 2: you can lose control and be quite happy. You can
• 10:40
  Speaker 2: be like an animal in a very cool zoo, enjoying yourself,
• 10:44
  Speaker 2: engaging in hedonistic pleasures, sex, food, whatever. You're not in control,
• 10:49
  Speaker 2: but you're safe. So there are separate problems. And then
• 10:52
  Speaker 2: there is for whatever reason, I don't know, if it's
• 10:54
  Speaker 2: malevolent payload from some psychopaths. Again, that would assume that
• 10:58
  Speaker 2: they could control a eye. I don't think they will,
• 11:01
  Speaker 2: but if they manage to do it, they can really
• 11:04
  Speaker 2: put any type of payload into it. So think about
• 11:07
  Speaker 2: all the doomsday called psychopaths, anyone providing their set of
• 11:12
  Speaker 2: goals into the system put on.
• 11:14
  Speaker 1: Those human characteristics. I mean, those are characteristics that I think,
• 11:19
  Speaker 1: if I had to guess, those exist because in the future,
• 11:25
  Speaker 1: there was some sort of a natural selection benefit to
• 11:28
  Speaker 1: being a psychopath in the days of tribal warfare, that
• 11:33
  Speaker 1: if you were the type of person that could sneak
• 11:36
  Speaker 1: into a tribe in the middle of the night and
• 11:38
  Speaker 1: slaughter innocent women and children, your genes would pass on.
• 11:44
  Speaker 1: There was a benefit to that.
• 11:46
  Speaker 2: Right, So if it's a human providing payload, that's what
• 11:48
  Speaker 2: would show up. If it's AI on its own deciding
• 11:51
  Speaker 2: what's going to happen, I cannot predict. I'm just looking
• 11:54
  Speaker 2: at worst case scenarios. There are also game theoretic reasons
• 11:58
  Speaker 2: where people talk about retro casale. See where if right now,
• 12:01
  Speaker 2: what is that word like trying to influence the past?
• 12:05
  Speaker 1: Say it again, say that retro causality, retro causology.
• 12:10
  Speaker 2: Causality causes. Oh okay, so think about like weird time
• 12:13
  Speaker 2: travel effects right now, if you're not helping to create superintelligence,
• 12:17
  Speaker 2: once it comes into existence, it will punish you really
• 12:20
  Speaker 2: hard for it and punishment needs to be so bad
• 12:22
  Speaker 2: that you start to help just to avoid that.
• 12:28
  Speaker 1: My thought about it was that it would just completely
• 12:34
  Speaker 1: render us benign, that it wouldn't be fearful of us
• 12:38
  Speaker 1: if we had no control, that it would just sort
• 12:41
  Speaker 1: of let us exist and it would be the dominant
• 12:44
  Speaker 1: force on the planet, and then it would stop. If
• 12:49
  Speaker 1: human beings have no control over you know, all the
• 12:55
  Speaker 1: different things that we have control over now, like international politics,
• 13:00
  Speaker 1: control over communication. If we have none of that anymore,
• 13:04
  Speaker 1: and we're reduced to a subsistence lifestyle, then we would
• 13:08
  Speaker 1: be no threat.
• 13:10
  Speaker 2: It is a possibility. I cannot say this will not
• 13:12
  Speaker 2: happen for sure, but look at our relationship with animals,
• 13:16
  Speaker 2: where we don't care about them so ants. If you
• 13:19
  Speaker 2: decide to build a house and there is an ant
• 13:21
  Speaker 2: colony and that property, you genocide them or you take
• 13:24
  Speaker 2: them out, not because you hate ants, but because you
• 13:27
  Speaker 2: just need that real estate. And it could be very similar. Again,
• 13:31
  Speaker 2: I cannot predict what it can do, but if it
• 13:33
  Speaker 2: needs to turn the planet into fuel, but it is
• 13:37
  Speaker 2: temperature of a planet, cool it down for servers, whatever
• 13:40
  Speaker 2: it needs to do, it wouldn't be concerned about your
• 13:42
  Speaker 2: well being.
• 13:42
  Speaker 1: It wouldn't be concerned about any life, right, because it
• 13:45
  Speaker 1: doesn't need biological life in order to function as long
• 13:48
  Speaker 1: as it has access to power, and assuming that it
• 13:53
  Speaker 1: is far more intelligent than us, there's abundant power in
• 13:57
  Speaker 1: the universe. There's abundant power. Just the the ability to
• 14:00
  Speaker 1: harness solar would be an infinite resource and it would
• 14:04
  Speaker 1: be completely free of being dependent upon any of the
• 14:10
  Speaker 1: things that we utilize.
• 14:11
  Speaker 2: And again we're kind of thinking what we would use
• 14:13
  Speaker 2: for power. If it's smarter than us, if it does
• 14:16
  Speaker 2: novel research and physics, it can come up with completely
• 14:18
  Speaker 2: novel ways of harnessing energy getting energy. So I have
• 14:22
  Speaker 2: no idea what side effects that would have for climate.
• 14:25
  Speaker 1: Right, right, Why would it care about biological life at all?
• 14:28
  Speaker 2: We don't know how to program it to care about us, and.
• 14:34
  Speaker 1: Even if we did, if it felt like that was
• 14:37
  Speaker 1: an issue, if that was a conflicting issue, it would
• 14:40
  Speaker 1: just change its programming.
• 14:42
  Speaker 2: So usually when we start training AI, we train it
• 14:46
  Speaker 2: on human data and it becomes really good, very quickly
• 14:49
  Speaker 2: becomes superhuman, and then the next level is usually zero knowledge,
• 14:53
  Speaker 2: where it goes all your human data is biased. Let
• 14:56
  Speaker 2: me figure it out from scratch, I'll do my own experiments,
• 14:58
  Speaker 2: i'll do some self play. I'll learn how to do
• 15:01
  Speaker 2: it better without you.