Latest / INSPIRE STORY / #090_ STORY INSPIRE: The AI Pioneer’s Warning: What They Don’t Want You to Know About Artificial Intelligence

Transcript

#090_ STORY INSPIRE: The AI Pioneer’s Warning: What They Don’t Want You to Know About Artificial Intelligence · INSPIRE STORY

• 0:00
  Speaker 1: They call you the godfather of AI. So what would
• 0:03
  Speaker 1: you be saying to people about their career prospects in
• 0:05
  Speaker 1: a world of superintelligence?
• 0:07
  Speaker 2: Train to be a plumber. Really? Yeah.
• 0:09
  Speaker 1: Okay, I'm going to become a plumber. Jeffrey Hinton is
• 0:13
  Speaker 1: the Nobel Prize winning pioneer whose groundbreaking work has shaped
• 0:16
  Speaker 1: AI and the future of humanity. Why do they call
• 0:20
  Speaker 1: you the godfather of AI? Because there weren't.
• 0:22
  Speaker 2: Many people who believed that we could model AI on
• 0:24
  Speaker 2: the brain so that it learned to do complicated things,
• 0:27
  Speaker 2: like recognize objects in images or even do reasoning. And
• 0:29
  Speaker 2: I pushed that approach for 50 years. And then Google
• 0:32
  Speaker 2: acquired that technology. And I worked there for 10 years
• 0:34
  Speaker 2: on something that's now used all the time in AI.
• 0:36
  Speaker 1: And then you left.
• 0:37
  Speaker 2: Yeah.
• 0:38
  Speaker 1: Why?
• 0:38
  Speaker 2: So that I could talk freely at a conference.
• 0:40
  Speaker 1: What did you want to talk about freely? How dangerous
• 0:43
  Speaker 1: AI could be.
• 0:45
  Speaker 2: I realized that these things will one day get smarter
• 0:48
  Speaker 2: than us. And we've never had to deal with that.
• 0:49
  Speaker 2: And if you want to know what life's like when
• 0:51
  Speaker 2: you're not the apex intelligence, ask a chicken. So there's
• 0:55
  Speaker 2: risks that come from people misusing AI, and then there's
• 0:58
  Speaker 2: risks from AI getting super smart and deciding it doesn't
• 1:00
  Speaker 2: need us.
• 1:01
  Speaker 1: Is that a real risk?
• 1:02
  Speaker 2: Yes, it is. But they're not going to stop it
• 1:03
  Speaker 2: because it's too good for too many things.
• 1:05
  Speaker 1: What about regulations?
• 1:06
  Speaker 2: They have some, but they're not designed to deal with
• 1:08
  Speaker 2: most of the threats. Like, the European regulations have a
• 1:11
  Speaker 2: clause that says none of these apply to military uses
• 1:13
  Speaker 2: of AI. Really? Yeah, it's crazy.
• 1:16
  Speaker 1: One of your students left OpenAI.
• 1:18
  Speaker 2: Yeah. He was probably the most important person behind the
• 1:20
  Speaker 2: development of the early versions of ChatGPT. And I think
• 1:23
  Speaker 2: he left because he had safety concerns. We should recognize
• 1:26
  Speaker 2: that this stuff is an existential threat. And we have
• 1:28
  Speaker 2: to face the possibility that unless we do something soon,
• 1:33
  Speaker 2: we're near the end.
• 1:34
  Speaker 1: So let's do the risks in what we end up
• 1:36
  Speaker 1: doing in such a world. This has always blown my
• 1:40
  Speaker 1: mind a little bit. 53% of you that listen to
• 1:43
  Speaker 1: this show regularly haven't yet subscribed to this show. So
• 1:46
  Speaker 1: could I ask you for a favor before we start?
• 1:48
  Speaker 1: If you like the show and you like what we
• 1:49
  Speaker 1: do here and you want to support us, the free,
• 1:51
  Speaker 1: simple way that you can do just that is by
• 1:53
  Speaker 1: hitting the subscribe button. And my commitment to you is
• 1:55
  Speaker 1: if you do that, then I'll do everything in my power,
• 1:58
  Speaker 1: me and my team, to make sure that this show
• 1:59
  Speaker 1: is better for you every single week. We'll listen to
• 2:02
  Speaker 1: your feedback, we'll find the guests that you want me
• 2:04
  Speaker 1: to speak to, and we'll continue to do what we do.
• 2:06
  Speaker 1: Thank you so much. Jeffrey Hinton. They call you the
• 2:14
  Speaker 1: godfather of AI.
• 2:16
  Speaker 2: Yes, they do.
• 2:17
  Speaker 1: Why do they call you that?
• 2:19
  Speaker 2: There weren't that many people who believed that we could
• 2:22
  Speaker 2: make neural networks work, artificial neural networks. So for a
• 2:26
  Speaker 2: long time in AI, from the 1950s onwards, there were
• 2:30
  Speaker 2: kind of two ideas about how to do AI. One
• 2:34
  Speaker 2: idea was that sort of core of human intelligence was reasoning.
• 2:38
  Speaker 2: And to do reasoning, you need to use some form
• 2:40
  Speaker 2: of logic. And so AI had to be based around logic.
• 2:45
  Speaker 2: And In your head, you must have something like symbolic
• 2:48
  Speaker 2: expressions that you manipulated with rules. And that's how intelligence worked.
• 2:53
  Speaker 2: And things like learning or reasoning by analogy, that'll come
• 2:56
  Speaker 2: later once we've figured out how basic reasoning works. There
• 2:59
  Speaker 2: was a different approach, which is to say, let's model
• 3:03
  Speaker 2: AI on the brain, because obviously the brain makes us intelligent.
• 3:08
  Speaker 2: So simulate a network of brain cells on a computer
• 3:12
  Speaker 2: and Try and figure out how you would learn strengths
• 3:14
  Speaker 2: of connections between brain cells so that it learned to
• 3:17
  Speaker 2: do complicated things, like recognize objects and images or recognize
• 3:21
  Speaker 2: speech or even do reasoning. I pushed that approach for
• 3:25
  Speaker 2: like 50 years. Because so few people believed in it,
• 3:30
  Speaker 2: there weren't many good universities that had groups that did that.
• 3:33
  Speaker 2: So if you did that, the best young students who
• 3:37
  Speaker 2: believed in that came and worked with you. So I
• 3:39
  Speaker 2: was very fortunate in getting a whole lot of really
• 3:42
  Speaker 2: good students.
• 3:44
  Speaker 1: Some of which have gone on to create and play
• 3:46
  Speaker 1: an instrumental role in creating platforms like OpenAI.
• 3:49
  Speaker 2: Yes, so Ilya Sutskova would be a nice example. A
• 3:53
  Speaker 2: whole bunch of them.
• 3:54
  Speaker 1: Why did you believe that modelling it off the brain
• 3:57
  Speaker 1: was a more effective approach?
• 3:59
  Speaker 2: It wasn't just me who believed it. Early on, von
• 4:02
  Speaker 2: Neumann believed it and Turing believed it. And if either
• 4:06
  Speaker 2: of those had lived, I think AI would have had
• 4:08
  Speaker 2: a very different history but they both died young.
• 4:12
  Speaker 1: You think AI would have been here sooner?
• 4:13
  Speaker 2: I think the neural net approach would have been accepted
• 4:18
  Speaker 2: much sooner if them had lived.
• 4:20
  Speaker 1: In this season of your life, what mission are you on?
• 4:25
  Speaker 2: My main mission now is to warn people how dangerous
• 4:29
  Speaker 2: AI could be.
• 4:30
  Speaker 1: Did you know that when you became the godfather of AI?
• 4:34
  Speaker 2: No, not really. I was quite slow to understand some
• 4:38
  Speaker 2: of the risks. Some of the risks were always very obvious,
• 4:40
  Speaker 2: like people would use AI to make autonomous lethal weapons.
• 4:44
  Speaker 2: That is, things that go around deciding by themselves who
• 4:47
  Speaker 2: to kill. Other risks, like the idea that they would
• 4:50
  Speaker 2: one day get smarter than us and maybe would become irrelevant,
• 4:55
  Speaker 2: I was slow to recognize that. Other people recognized it
• 4:58
  Speaker 2: 20 years ago. I only recognized it a few years
• 5:01
  Speaker 2: ago that that was a real risk that might be
• 5:04
  Speaker 2: coming quite soon.
• 5:05
  Speaker 1: How could you not have foreseen that if with everything
• 5:09
  Speaker 1: you know here about cracking the ability for these computers
• 5:12
  Speaker 1: to learn similar to how humans learn and just introducing
• 5:16
  Speaker 1: any rate of improvement?
• 5:18
  Speaker 2: It's a very good question. How could you not have
• 5:20
  Speaker 2: seen that? But remember, neural networks 20, 30 years ago
• 5:25
  Speaker 2: were very primitive in what they could do. They were
• 5:28
  Speaker 2: nowhere near as good as humans at things like vision
• 5:31
  Speaker 2: and language and speech recognition. The idea that you have
• 5:34
  Speaker 2: to now worry about it getting smarter than people, that
• 5:36
  Speaker 2: seems silly then.
• 5:38
  Speaker 1: When did that change?
• 5:39
  Speaker 2: It changed for the general population when ChatGPT came out.
• 5:44
  Speaker 2: It changed for me when I realized that the kinds
• 5:48
  Speaker 2: of digital intelligences we're making have something that makes them
• 5:52
  Speaker 2: far superior to the kind of biological intelligence we have.
• 5:55
  Speaker 2: If I want to share information with you, so I
• 5:58
  Speaker 2: go off and I learn something, and I'd like to
• 6:00
  Speaker 2: tell you what I learned. So I produce some sentences.
• 6:04
  Speaker 2: This is a rather simplistic model, but roughly right. Your
• 6:07
  Speaker 2: brain is trying to figure out, how can I change
• 6:08
  Speaker 2: the strengths of connections between neurons so I might have
• 6:10
  Speaker 2: put that word next? And so you'll do a lot
• 6:13
  Speaker 2: of learning when a very surprising word comes, and not
• 6:15
  Speaker 2: much learning when it's a very obvious word. If I
• 6:18
  Speaker 2: say fish and chips, you don't do much learning when
• 6:21
  Speaker 2: I say chips. But if I say fish and cucumber,
• 6:23
  Speaker 2: you do a lot more learning. You wonder, why did
• 6:25
  Speaker 2: I say cucumber? So that's roughly what's going on in
• 6:28
  Speaker 2: your brain.
• 6:29
  Speaker 1: I'm predicting what's coming next.
• 6:31
  Speaker 2: That's how we think it's working. Nobody really knows for
• 6:34
  Speaker 2: sure how the brain works. And nobody knows how it
• 6:36
  Speaker 2: gets the information about whether you should increase the strength
• 6:39
  Speaker 2: of a connection or decrease the strength of a connection.
• 6:42
  Speaker 2: That's the crucial thing. But what we do know now
• 6:45
  Speaker 2: from AI is that if you could get information about
• 6:50
  Speaker 2: whether to increase or decrease the connection strength so as
• 6:53
  Speaker 2: to do better at whatever task you're trying to do,
• 6:56
  Speaker 2: then we could learn incredible things because that's what we're
• 6:58
  Speaker 2: doing now with artificial neuron ads. It's just we don't
• 7:02
  Speaker 2: know for real brains how they get that signal about
• 7:04
  Speaker 2: whether to increase or decrease.
• 7:07
  Speaker 1: As we sit here today, what are the big concerns
• 7:08
  Speaker 1: you have around safety of AI? If we were to
• 7:12
  Speaker 1: list the top couple that are really front of mind
• 7:15
  Speaker 1: and that we should be thinking about.
• 7:17
  Speaker 2: Can I have more than a couple?
• 7:18
  Speaker 1: Go ahead. I'll write them all down and we'll go
• 7:20
  Speaker 1: through them.
• 7:21
  Speaker 2: Okay. First of all, I want to make a distinction
• 7:24
  Speaker 2: between two completely different kinds of risk. There's risks that
• 7:28
  Speaker 2: come from people misusing AI. And that's most of the
• 7:32
  Speaker 2: risks and all of the short-term risks. And then there's
• 7:37
  Speaker 2: risks that come from AI getting super smart and deciding
• 7:40
  Speaker 2: it doesn't need us.
• 7:41
  Speaker 1: Is that a real risk?
• 7:43
  Speaker 2: And I talk mainly about that second risk because lots
• 7:46
  Speaker 2: of people say, is that a real risk? And yes,
• 7:49
  Speaker 2: it is. Now, we don't know how much of a
• 7:52
  Speaker 2: risk it is. We've never been in that situation before.
• 7:54
  Speaker 2: We've never had to deal with things smarter than us.
• 7:57
  Speaker 2: So really the thing about that existential threat is is
• 8:01
  Speaker 2: that we have no idea how to deal with it.
• 8:04
  Speaker 2: We have no idea what it's going to look like.
• 8:06
  Speaker 2: And anybody who tells you they know just what's going
• 8:08
  Speaker 2: to happen and how to deal with it, they're talking nonsense.
• 8:11
  Speaker 2: So we don't know how to estimate the probabilities it'll
• 8:14
  Speaker 2: replace us. Some people say it's like less than 1%.
• 8:18
  Speaker 2: My friend Yann LeCun, who was a postdoc with me, thinks, no, no, no, no,
• 8:22
  Speaker 2: we're always going to be, we build these things, we're
• 8:25
  Speaker 2: always going to be in control. We'll build them to
• 8:27
  Speaker 2: be obedient. And Other people, like Yudkowsky, say, no, no, no,
• 8:34
  Speaker 2: these things are going to wipe us out for sure.
• 8:35
  Speaker 2: If anybody builds it, it's going to wipe us all out.
• 8:38
  Speaker 2: And he's confident of that. I think both of those
• 8:41
  Speaker 2: positions are extreme. It's very hard to estimate the probabilities
• 8:44
  Speaker 2: in between.
• 8:45
  Speaker 1: If you had to bet on who was right out
• 8:48
  Speaker 1: of your two friends?
• 8:51
  Speaker 2: I simply don't know. So if I had to bet,
• 8:54
  Speaker 2: I'd say the probability is in between. And I don't
• 8:56
  Speaker 2: know where to estimate it in between. I often say
• 8:59
  Speaker 2: 10% to 20% chance they'll wipe us out. But that's
• 9:02
  Speaker 2: just gut based on the idea that we're still making
• 9:06
  Speaker 2: them and we're pretty ingenious. And the hope is that
• 9:10
  Speaker 2: if enough smart people do enough research with enough resources,
• 9:14
  Speaker 2: we'll figure out a way to build them so they'll
• 9:16
  Speaker 2: never want to harm us.
• 9:19
  Speaker 1: Sometimes I think if we talk about that second path,
• 9:22
  Speaker 1: sometimes I think about nuclear bombs and the invention of,
• 9:25
  Speaker 1: the atomic bomb and how it compares, like how is
• 9:27
  Speaker 1: this different? Because the atomic bomb came along and I
• 9:29
  Speaker 1: imagine a lot of people at that time thought our
• 9:31
  Speaker 1: days are numbered.
• 9:33
  Speaker 2: Yes, I was there. We did. Yeah.
• 9:35
  Speaker 1: But, but, but what's, what, we're still here.
• 9:38
  Speaker 2: We're still here. Yes. So the atomic bomb was really
• 9:42
  Speaker 2: only good for one thing and it was very obvious
• 9:45
  Speaker 2: how it worked. Even if you hadn't had the pictures
• 9:47
  Speaker 2: of Hiroshima and Nagasaki, it was obvious that it was
• 9:51
  Speaker 2: a very big bomb that was very dangerous. With AI,
• 9:56
  Speaker 2: it's good for many, many things. It's going to be
• 10:00
  Speaker 2: magnificent in healthcare and education and more or less any
• 10:04
  Speaker 2: industry that needs to use its data is going to
• 10:07
  Speaker 2: be able to use it better with AI. So we're
• 10:10
  Speaker 2: not going to stop the development. People say, well, why
• 10:14
  Speaker 2: don't we just stop it now? We're not going to
• 10:17
  Speaker 2: stop it because it's too good for too many things. Also,
• 10:20
  Speaker 2: we're not going to stop it because it's good for
• 10:22
  Speaker 2: battle robots, and none of the countries that sell weapons
• 10:25
  Speaker 2: are going to want to stop it. Like the European regulations,
• 10:30
  Speaker 2: they have some regulations about AI, and it's good they
• 10:32
  Speaker 2: have some regulations, but they're not designed to deal with
• 10:35
  Speaker 2: most of the threats. And in particular, the European regulations
• 10:38
  Speaker 2: have a clause in them that say none of these
• 10:41
  Speaker 2: regulations apply to military uses of AI. So governments are
• 10:46
  Speaker 2: willing to regulate AI. companies and people, but they're not
• 10:50
  Speaker 2: willing to regulate themselves.
• 10:53
  Speaker 1: It seems pretty crazy to me that they, I go
• 10:56
  Speaker 1: back and forward, but if Europe has a regulation, but
• 10:58
  Speaker 1: the rest of the world doesn't, aren't.
• 11:00
  Speaker 2: We just putting ourselves at a competitive disadvantage?
• 11:02
  Speaker 3: Yeah.
• 11:03
  Speaker 1: And we're seeing this already. I don't think people realize
• 11:05
  Speaker 1: that when OpenAI release a new model or a new
• 11:07
  Speaker 1: piece of software in America, they can't release it to
• 11:11
  Speaker 1: Europe yet because of regulations here. So Sam Altman tweeted saying,
• 11:14
  Speaker 1: our new AI agent thing is available to everybody, but
• 11:17
  Speaker 1: it can't come to Europe yet because there's regulations.
• 11:20
  Speaker 2: Yes.
• 11:21
  Speaker 1: That gives us a productive disadvantage, productivity disadvantage.
• 11:25
  Speaker 2: What we need is, I mean, at this point in history,
• 11:28
  Speaker 2: when we're about to produce things more intelligent than ourselves,
• 11:31
  Speaker 2: what we really need is a kind of world government
• 11:35
  Speaker 2: that works run by intelligent, thoughtful people. And that's not
• 11:38
  Speaker 2: what we got.
• 11:40
  Speaker 1: So free for all.
• 11:43
  Speaker 2: Well, what we've got is sort of, We've got capitalism,
• 11:48
  Speaker 2: which is done very nicely by us. It's produced lots
• 11:51
  Speaker 2: of goods and services for us. But these big companies,
• 11:56
  Speaker 2: they're legally required to try and maximize profits. And that's
• 12:01
  Speaker 2: not what you want from the people developing this stuff.
• 12:05
  Speaker 1: So let's do the risks then. You talked about there's
• 12:07
  Speaker 1: human risks. So I've distinguished these two kinds of risks.
• 12:10
  Speaker 1: Let's talk about all the risks from bad human actors
• 12:13
  Speaker 1: using AI.
• 12:15
  Speaker 2: There's cyber attacks. So between 2023 and 2024, they increased
• 12:23
  Speaker 2: by about a factor of 12, 1200%. And that's probably
• 12:28
  Speaker 2: because these large language models make it much easier to
• 12:32
  Speaker 2: do phishing attacks.
• 12:33
  Speaker 1: And a phishing attack for anyone that doesn't know is?
• 12:36
  Speaker 2: It's they send you something saying, hi, I'm your friend, John,
• 12:40
  Speaker 2: and I'm stuck in El Salvador. Could you just wire
• 12:43
  Speaker 2: this money? That's one kind of attack. But the phishing
• 12:46
  Speaker 2: attacks are really trying to get your logon credentials.
• 12:49
  Speaker 1: And now with AI, they can clone my voice, my image.
• 12:51
  Speaker 2: They can do all that.
• 12:52
  Speaker 1: I'm struggling at the moment because there's a bunch of
• 12:54
  Speaker 1: AI scams on X and also Meta. And there's one
• 12:57
  Speaker 1: in particular on Meta, so Instagram, Facebook at the moment,
• 13:00
  Speaker 1: which is a paid advert where they've taken my voice
• 13:02
  Speaker 1: from the podcast. They've taken my mannerisms and they've made
• 13:05
  Speaker 1: a new video of me encouraging people to go and
• 13:08
  Speaker 1: take part in this crypto Ponzi scam or whatever. And
• 13:12
  Speaker 1: we've spent weeks and weeks and weeks and weeks and
• 13:14
  Speaker 1: then emailing Meta telling them, please take this down. They
• 13:16
  Speaker 1: take it down. Another one pops up. They take that
• 13:19
  Speaker 1: one down. Another one pops up. So it's like whack-a-mole.
• 13:21
  Speaker 1: It's very annoying. The heartbreaking part is you get the
• 13:23
  Speaker 1: messages from people that have fallen for the scam.
• 13:26
  Speaker 4: Right.
• 13:26
  Speaker 1: And they've lost 500 pounds or $ 500.
• 13:27
  Speaker 2: And they cross with you because you recommended it.
• 13:29
  Speaker 1: And I'm sad for them.
• 13:31
  Speaker 2: It's very annoying. Yeah. I have a smaller version of that,
• 13:34
  Speaker 2: which is some people now publish papers with me as
• 13:38
  Speaker 2: one of the authors. Mm-hmm. And it looks like it's
• 13:41
  Speaker 2: in order that they can get lots of citations to themselves.
• 13:46
  Speaker 1: So cyber attacks, a very real threat. There's been an
• 13:48
  Speaker 1: explosion of those.
• 13:49
  Speaker 2: And these already, obviously, AI is very patient. So they
• 13:53
  Speaker 2: can go through 100 million lines of code looking for
• 13:56
  Speaker 2: known ways of attacking them. That's easy to do. But
• 14:00
  Speaker 2: they're going to get more creative. And they may, some
• 14:03
  Speaker 2: people believe, and some people who know a lot believe
• 14:07
  Speaker 2: that maybe by 2030... They'll be creating new kinds of
• 14:12
  Speaker 2: cyber attacks, which no person ever thought of. So that's
• 14:17
  Speaker 2: very worrisome.
• 14:18
  Speaker 1: Because they can think for themselves and discover new ways
• 14:20
  Speaker 1: to attack.
• 14:21
  Speaker 2: They can draw new conclusions from much more data than
• 14:24
  Speaker 2: a person ever saw.
• 14:26
  Speaker 1: Is there anything you're doing to protect yourself from cyber
• 14:29
  Speaker 1: attacks at all?
• 14:30
  Speaker 2: Yes. It's one of the few places where I changed
• 14:34
  Speaker 2: what I do radically because I'm scared of cyber attacks.
• 14:37
  Speaker 2: Canadian banks are extremely safe. In 2008, no Canadian banks
• 14:42
  Speaker 2: came anywhere near going bust. So they're very safe banks
• 14:46
  Speaker 2: because they're well regulated, fairly well regulated. Nevertheless, I think
• 14:51
  Speaker 2: a cyber attack might be able to bring down a bank. Now,
• 14:54
  Speaker 2: if you have all my savings are in shares in banks,
• 14:58
  Speaker 2: held by banks. So if the bank gets attacked and
• 15:03
  Speaker 2: it holds your shares, they're still your shares. And so
• 15:07
  Speaker 2: I think you'd be okay unless... The attacker sells the
• 15:11
  Speaker 2: shares because the bank can sell the shares. If the
• 15:14
  Speaker 2: attacker sells your shares, I think you're screwed. I don't know.
• 15:18
  Speaker 2: I mean, maybe the bank would have to try and
• 15:21
  Speaker 2: reimburse you, but the bank's bust by now, right? So
• 15:24
  Speaker 2: I'm worried about a Canadian bank being taken down by
• 15:27
  Speaker 2: a cyber attack and the attacker selling shares that it holds.
• 15:32
  Speaker 2: So I spread my money, my children's money between three banks.
• 15:37
  Speaker 2: In the belief that if a cyber attack takes down
• 15:40
  Speaker 2: one Canadian bank, the other Canadian banks will very quickly
• 15:44
  Speaker 2: get very careful.
• 15:46
  Speaker 1: And do you have a phone that's not connected to
• 15:47
  Speaker 1: the internet? Do you have any, you know, I'm thinking
• 15:50
  Speaker 1: about storing data and stuff like that. Do you think
• 15:52
  Speaker 1: it's wise to consider having cold storage?
• 15:55
  Speaker 2: I have a little disk drive and I back up
• 15:58
  Speaker 2: my laptop on this hard drive. So I actually have
• 16:02
  Speaker 2: everything on my laptop on a hard drive. At least,
• 16:05
  Speaker 2: you know, if the whole internet went down, I had
• 16:07
  Speaker 2: the sense I still got it on my laptop and
• 16:09
  Speaker 2: I still got my information.
• 16:10
  Speaker 1: Okay.
• 16:12
  Speaker 2: Then the next thing is using AI to create nasty viruses.
• 16:18
  Speaker 1: Okay.
• 16:19
  Speaker 2: And the problem with that is that just requires one
• 16:24
  Speaker 2: crazy guy with a grudge. One guy who knows a
• 16:27
  Speaker 2: little bit of molecular biology, knows a lot about AI
• 16:30
  Speaker 2: and just wants to destroy the world. You can now
• 16:34
  Speaker 2: create new viruses relatively cheaply using AI. And you don't
• 16:40
  Speaker 2: have to be a very skilled molecular biologist to do it.
• 16:42
  Speaker 2: And that's very scary. So you could have a small cult,
• 16:45
  Speaker 2: for example. A small cult might be able to raise
• 16:49
  Speaker 2: a few million dollars. For a few million dollars, they
• 16:52
  Speaker 2: might be able to design a whole bunch of viruses.
• 16:54
  Speaker 1: Well, I'm thinking about some of our foreign adversaries. doing
• 16:57
  Speaker 1: government-funded programs. I mean, there was lots of talk around
• 16:59
  Speaker 1: COVID and the Wuhan laboratory and what they were doing
• 17:02
  Speaker 1: in gain-of-function research. But I'm wondering if in, you know,
• 17:05
  Speaker 1: a China or a Russia or an Iran or something,
• 17:08
  Speaker 1: the government could fund a program for a small group
• 17:11
  Speaker 1: of scientists to make a virus that they could, you know?
• 17:14
  Speaker 2: I think they could, yes. Now, they'd be worried about retaliation.
• 17:17
  Speaker 2: They'd be worried about other governments doing the same to them.
• 17:19
  Speaker 2: Hopefully that would help keep it under control. They might
• 17:22
  Speaker 2: also be worried about the virus spreading to their country. Okay.
• 17:26
  Speaker 2: Then there's corrupting elections. So if you wanted to use
• 17:32
  Speaker 2: AI to corrupt elections, a very effective thing is to
• 17:36
  Speaker 2: be able to do targeted political advertisements where you know
• 17:40
  Speaker 2: a lot about the person. So anybody who wanted to
• 17:46
  Speaker 2: use AI for corrupting elections would try and get as
• 17:49
  Speaker 2: much data as they could about everybody in the electorate.
• 17:54
  Speaker 2: With that in mind, it's a bit worrying what Musk
• 17:56
  Speaker 2: is doing at present in the States, going in and
• 17:59
  Speaker 2: insisting on getting access to all these things that were
• 18:01
  Speaker 2: very carefully siloed. The claim is it's to make things
• 18:05
  Speaker 2: more efficient, but it's exactly what you would want if
• 18:08
  Speaker 2: you intended to corrupt the next election.
• 18:10
  Speaker 1: How do you mean? Because you get all this data
• 18:12
  Speaker 1: on the people?
• 18:12
  Speaker 2: You get all this data on people. You know how
• 18:14
  Speaker 2: much they make, you know everything about them. Once you
• 18:17
  Speaker 2: know that, it's very easy to manipulate.
• 18:20
  Speaker 1: Because you can make an AI that... You.
• 18:23
  Speaker 2: Can send messages that they'll find very convincing, telling them
• 18:26
  Speaker 2: not to vote, for example. So I have no reason
• 18:32
  Speaker 2: other than common sense to think this, but I wouldn't
• 18:35
  Speaker 2: be surprised if part of the motivation of getting all
• 18:38
  Speaker 2: this data from American government sources is to corrupt elections.
• 18:43
  Speaker 2: Another part might be that it's very nice training data
• 18:46
  Speaker 2: for a big model.
• 18:48
  Speaker 1: But he would have to be taking that data from
• 18:50
  Speaker 1: the government and feeding it into his.
• 18:52
  Speaker 2: Yes. And what they've done is turned off lots of
• 18:55
  Speaker 2: the security controls, got rid of some of the organization
• 18:59
  Speaker 2: to protect against that.
• 19:02
  Speaker 1: So that's corrupting elections.
• 19:03
  Speaker 2: Okay. Then there's creating these two echo chambers by organizations
• 19:11
  Speaker 2: like YouTube and Facebook, showing people things that will make
• 19:17
  Speaker 2: them indignant. People love to be indignant.
• 19:20
  Speaker 1: Indignant as in angry? What does indignant mean?
• 19:23
  Speaker 2: Feeling I'm sort of angry but feeling righteous. Okay. So,
• 19:28
  Speaker 2: for example, if you were to show me something that said,
• 19:32
  Speaker 2: Trump did this crazy thing. Here's a video of Trump
• 19:35
  Speaker 2: doing this completely crazy thing. I would immediately click on it. Yeah.
• 19:40
  Speaker 1: Okay, so putting us in echo chambers and dividing us.
• 19:43
  Speaker 2: Yes, and that's... The policy that YouTube and Facebook and
• 19:48
  Speaker 2: others use for deciding what to show you next is
• 19:53
  Speaker 2: causing that. If they had a policy of showing you
• 19:57
  Speaker 2: balanced things, they wouldn't get so many clicks and they
• 20:00
  Speaker 2: wouldn't be able to sell so many advertisements. And so
• 20:03
  Speaker 2: it's basically the profit motive is saying, show them whatever
• 20:06
  Speaker 2: will make them click. And what will make them click is...
• 20:10
  Speaker 2: things that are more and more extreme.
• 20:12
  Speaker 1: And that confirmed my existing bias.
• 20:14
  Speaker 2: That confirmed my existing bias. So you're getting your biases
• 20:16
  Speaker 2: confirmed all the time.
• 20:18
  Speaker 1: Further and further and further and further, which means you're
• 20:20
  Speaker 1: driving away.
• 20:21
  Speaker 2: Which is now there's in the States, there's two communities
• 20:23
  Speaker 2: that don't hardly talk to each other.
• 20:25
  Speaker 1: I'm not sure people realize that this is actually happening
• 20:27
  Speaker 1: every time they open an app. But if you go
• 20:28
  Speaker 1: on a TikTok or a YouTube or one of these
• 20:30
  Speaker 1: big social networks, the algorithm, as you said, is designed
• 20:34
  Speaker 1: to show you more of the things that you had
• 20:37
  Speaker 1: interest in last time, So if you just play that
• 20:39
  Speaker 1: out over 10 years, it's going to drive you further
• 20:41
  Speaker 1: and further and further into whatever ideology or belief you
• 20:44
  Speaker 1: have and further away from nuance and common sense and parity,
• 20:49
  Speaker 1: which is a pretty remarkable thing. People don't know it's happening.
• 20:53
  Speaker 1: They just open their phones and experience something and think
• 20:56
  Speaker 1: this is the news or the experience everyone else is having.
• 21:00
  Speaker 2: Right. So basically, if you have a newspaper and everybody
• 21:04
  Speaker 2: gets the same newspaper, you get to see all sorts
• 21:07
  Speaker 2: of things you weren't looking for. And you get a
• 21:09
  Speaker 2: sense that if it's in the newspaper, it's an important
• 21:11
  Speaker 2: thing or significant thing. But if you have your own newsfeed,
• 21:15
  Speaker 2: my newsfeed on my iPhone, three quarters of the stories
• 21:19
  Speaker 2: are about AI. And I find it very hard to
• 21:22
  Speaker 2: know if the whole world's talking about AI all the time,
• 21:25
  Speaker 2: or if it's just my newsfeed.
• 21:28
  Speaker 1: Okay, so driving me into my echo chambers, which is
• 21:32
  Speaker 1: going to continue to divide us further and further, I'm
• 21:34
  Speaker 1: actually noticing that the algorithms are becoming even more What's
• 21:38
  Speaker 1: the word? Tailored. And people might go, oh, that's great.
• 21:42
  Speaker 1: But what it means is they're becoming even more personalised,
• 21:44
  Speaker 1: which means that my reality is becoming even further from
• 21:48
  Speaker 1: your reality.
• 21:49
  Speaker 2: Yeah, it's crazy. We don't have a shared reality anymore.
• 21:53
  Speaker 2: I share reality with other people who watch the BBC
• 21:56
  Speaker 2: and other BBC News and other people who read The
• 21:59
  Speaker 2: Guardian and other people who read The New York Times.
• 22:02
  Speaker 2: I have almost no shared reality with people who watch
• 22:05
  Speaker 2: Fox News.
• 22:08
  Speaker 1: It's pretty worrisome.
• 22:11
  Speaker 2: Yeah. Behind all this is the idea that these companies
• 22:14
  Speaker 2: just want to make profit and they'll do whatever it
• 22:17
  Speaker 2: takes to make more profit.
• 22:18
  Speaker 1: Because they have to.
• 22:20
  Speaker 2: They're legally obliged to do that.
• 22:23
  Speaker 1: So we almost can't blame the company, can we?
• 22:26
  Speaker 2: Well, capitalism has done very well for us. It's produced
• 22:30
  Speaker 2: lots of goodies. But you need to have it very
• 22:32
  Speaker 2: well regulated. So what you really want... is to have
• 22:37
  Speaker 2: rules so that when some company is trying to make
• 22:40
  Speaker 2: as much profit as possible, in order to make that profit,
• 22:44
  Speaker 2: they have to do things that are good for people
• 22:46
  Speaker 2: in general, not things that are bad for people in general.
• 22:49
  Speaker 2: So once you get to a situation where in order
• 22:51
  Speaker 2: to make more profit, the company starts doing things that
• 22:54
  Speaker 2: are very bad for society, like showing you things that
• 22:57
  Speaker 2: are more and more extreme, that's what regulations are for.
• 23:01
  Speaker 2: So you need regulations with capitalism. Now, companies will always say,
• 23:06
  Speaker 2: regulations get in the way, make us less efficient. And
• 23:09
  Speaker 2: that's true. The whole point of regulations is to stop
• 23:12
  Speaker 2: them doing things to make profit that hurt society. And
• 23:16
  Speaker 2: we need strong regulation.
• 23:17
  Speaker 1: Who's going to decide whether it hurts society or not? Because,
• 23:20
  Speaker 1: you know.
• 23:21
  Speaker 2: That's the job of politicians. Unfortunately, if the politicians are
• 23:25
  Speaker 2: owned by the companies, that's not so good.
• 23:27
  Speaker 1: And also the politicians might not understand the technology. You've
• 23:29
  Speaker 1: probably seen the Senate hearings where they wheel out, you know,
• 23:31
  Speaker 1: Mark Zuckerberg and these big tech CEOs. And it is
• 23:34
  Speaker 1: quite embarrassing because they're asking the wrong questions.
• 23:37
  Speaker 2: Well, I've seen the video of the U.S. Education Secretary
• 23:41
  Speaker 2: talking about how they're going to get AI in the classrooms,
• 23:45
  Speaker 2: except she thought it was called A1. She's actually there
• 23:49
  Speaker 2: saying we're going to have all the kids interacting with A1.
• 23:53
  Speaker 3: There is a school system that's going to start making
• 23:56
  Speaker 3: sure that first graders or even pre-Ks have A1 teaching
• 24:01
  Speaker 3: every year starting that far down in the grades. And
• 24:04
  Speaker 3: that's just a wonderful thing.
• 24:11
  Speaker 1: And these are the people that.
• 24:12
  Speaker 2: These are the people in charge.
• 24:14
  Speaker 1: Ultimately, the tech companies are in charge because they were smart.
• 24:17
  Speaker 2: The tech companies in the States now, at least a
• 24:21
  Speaker 2: few weeks ago when I was there, they were running
• 24:25
  Speaker 2: an advertisement about how it was very important not to
• 24:28
  Speaker 2: regulate AI because it would hurt us in the competition
• 24:30
  Speaker 2: with China.
• 24:31
  Speaker 1: Yeah. And that's a plausible argument, no?
• 24:34
  Speaker 2: Yes, it will. But you have to decide. Do you
• 24:38
  Speaker 2: want to compete with China? By doing things that will
• 24:42
  Speaker 2: do a lot of harm to your society. And you
• 24:46
  Speaker 2: probably don't.
• 24:49
  Speaker 1: I guess they would say that it's not just China.
• 24:52
  Speaker 1: It's Denmark and Australia and Canada.
• 24:54
  Speaker 2: And the UK.
• 24:55
  Speaker 1: They're not so worried about. And Germany. But if they
• 24:57
  Speaker 1: kneecap themselves with regulation, if they slow themselves down, then
• 25:01
  Speaker 1: the founders, the entrepreneurs, the investors are going to go invest.
• 25:03
  Speaker 2: I think calling it kneecapping is taking a particular point
• 25:06
  Speaker 2: of view. It's taking the point of view that regulations
• 25:09
  Speaker 2: are sort of very harmful. What you need to do
• 25:11
  Speaker 2: is just constrain the big companies so that in order
• 25:14
  Speaker 2: to make profit, they have to do things that are
• 25:17
  Speaker 2: socially useful. Like Google Search is a great example. That
• 25:21
  Speaker 2: didn't need regulation because it just made information available to people.
• 25:25
  Speaker 2: It was great. But then if you take YouTube, which
• 25:28
  Speaker 2: starts showing you adverts and showing you more and more
• 25:32
  Speaker 2: extreme things, that needs regulation.
• 25:35
  Speaker 1: But we don't have the people to regulate it. As
• 25:37
  Speaker 1: we've identified.
• 25:38
  Speaker 2: I think people know pretty well that particular problem of
• 25:42
  Speaker 2: showing you more and more extreme things. That's a well-known
• 25:44
  Speaker 2: problem that the politicians understand. They just need to get
• 25:49
  Speaker 2: on and regulate it.
• 25:50
  Speaker 1: So that was the next point, which was that the
• 25:52
  Speaker 1: algorithms are going to drive us further into our echo chambers.
• 25:55
  Speaker 2: Right.
• 25:56
  Speaker 1: What's next?
• 25:57
  Speaker 2: Lethal autonomous weapons.
• 26:00
  Speaker 1: Lethal autonomous weapons.
• 26:03
  Speaker 2: That means things that can kill you and make their
• 26:06
  Speaker 2: own decision about whether to kill you.
• 26:08
  Speaker 1: Which is the great dream, I guess, of the military
• 26:11
  Speaker 1: industrial complex, being able to create such weapons.
• 26:14
  Speaker 2: So the worst thing about them is big, powerful countries
• 26:19
  Speaker 2: always have the ability to invade smaller, poorer countries. They're
• 26:24
  Speaker 2: just more powerful. But if you do that using actual soldiers,
• 26:30
  Speaker 2: you get bodies coming back in bags and the relatives
• 26:34
  Speaker 2: of the soldiers that were killed don't like it. So
• 26:37
  Speaker 2: you get something out of Vietnam. In the end, there's
• 26:39
  Speaker 2: a lot of protest at home. If instead of bodies
• 26:43
  Speaker 2: coming back in bags, it was dead robots, there'd be
• 26:47
  Speaker 2: much less protest and the military industrial complex would like
• 26:51
  Speaker 2: it much more because robots are expensive. And suppose you
• 26:55
  Speaker 2: had something that could get killed and was expensive to replace.
• 26:59
  Speaker 2: That would be just great. Big countries can invade small
• 27:03
  Speaker 2: countries much more easily because they don't have their soldiers
• 27:06
  Speaker 2: being killed.
• 27:07
  Speaker 1: And the risk here is... that these robots will malfunction
• 27:13
  Speaker 1: or they'll just be more.
• 27:14
  Speaker 2: No, no. Even if the robots do exactly what the
• 27:17
  Speaker 2: people who built the robots want them to do, the
• 27:19
  Speaker 2: risk is that it's going to make big countries invade.
• 27:22
  Speaker 1: Small countries more often. More often because they can. Yeah,
• 27:24
  Speaker 1: and it's not a nice thing to do. So it
• 27:25
  Speaker 1: brings down the friction of war.
• 27:27
  Speaker 2: It brings down the cost of doing an invasion.
• 27:30
  Speaker 1: And these machines will be smarter at warfare as well.
• 27:34
  Speaker 2: Well, even when the machines aren't smarter. So the lethal
• 27:37
  Speaker 2: autonomous weapons, they can make them now. And I think
• 27:42
  Speaker 2: all the big defense firms are busy making them. Even
• 27:44
  Speaker 2: if they're not smarter than people, they're still very nasty,
• 27:47
  Speaker 2: scary things.
• 27:48
  Speaker 1: Because I'm thinking that, you know, they could show just
• 27:50
  Speaker 1: a picture, go get this guy.
• 27:53
  Speaker 2: Yeah.
• 27:53
  Speaker 1: And go take out anyone he's been texting and this
• 27:57
  Speaker 1: little wasp.
• 27:58
  Speaker 2: So two days ago, I was visiting a friend of
• 28:00
  Speaker 2: mine in Sussex who had a drone that cost less
• 28:03
  Speaker 2: than 200 pounds. And the drone went up. It took
• 28:08
  Speaker 2: a good look at me. And then it could follow
• 28:11
  Speaker 2: me through the woods. And it was very spooky having
• 28:15
  Speaker 2: this drone. It was about two meters behind me. It
• 28:17
  Speaker 2: was looking at me. If I moved over there, it
• 28:19
  Speaker 2: moved over there. It could just track me for 200 pounds.
• 28:24
  Speaker 2: But it was already quite spooky.
• 28:26
  Speaker 1: Yeah, I imagine there's, as you say, a race going
• 28:28
  Speaker 1: on as we speak to who can build the most
• 28:30
  Speaker 1: complex autonomous weapons. There is a risk I often hear
• 28:35
  Speaker 1: that some of these things will combine and the cyber
• 28:38
  Speaker 1: attack will release weapons Sure.
• 28:42
  Speaker 2: You can get combinatorially many risks by combining these other risks.
• 28:48
  Speaker 2: So I mean, for example, you could get a super
• 28:51
  Speaker 2: intelligent AI that decides to get rid of people. And
• 28:55
  Speaker 2: the obvious way to do that is just to make
• 28:56
  Speaker 2: one of these nasty viruses. If you made a virus
• 28:59
  Speaker 2: that was very contagious, very lethal, and very slow, everybody
• 29:06
  Speaker 2: would have it before they realized what was happening. I mean,
• 29:09
  Speaker 2: I think if a superintelligence wanted to get rid of us,
• 29:12
  Speaker 2: it would probably go for something biological like that that
• 29:14
  Speaker 2: wouldn't affect it.
• 29:15
  Speaker 1: Do you think it could just very quickly turn us
• 29:17
  Speaker 1: against each other? For example, it could send a warning
• 29:20
  Speaker 1: on the nuclear systems in America that there's a nuclear
• 29:23
  Speaker 1: bomb coming from Russia or vice versa, and one retaliates.
• 29:26
  Speaker 2: Yeah, I mean, my basic view is there's so many
• 29:30
  Speaker 2: ways in which a superintelligence could get rid of us.
• 29:32
  Speaker 2: It's not worth speculating about.
• 29:35
  Speaker 1: What is to stop What you have to do.
• 29:38
  Speaker 2: Is prevent it ever wanting to. That's what we should
• 29:41
  Speaker 2: be doing research on. There's no way we're going to
• 29:43
  Speaker 2: prevent it from... It's smarter than us, right? There's no
• 29:46
  Speaker 2: way we're going to prevent it getting rid of us
• 29:48
  Speaker 2: if it wants to. We're not used to thinking about
• 29:51
  Speaker 2: things smarter than us. If you want to know what
• 29:54
  Speaker 2: life's like when you're not the apex intelligence, ask a chicken.
• 30:03
  Speaker 1: Yeah, I was thinking about my dog Pablo, my French bulldog,
• 30:05
  Speaker 1: this morning as I left home. He has no idea
• 30:08
  Speaker 1: where I'm going. He has no idea what I do.
• 30:10
  Speaker 4: Right.
• 30:10
  Speaker 1: I can't even talk to him.
• 30:12
  Speaker 2: Yeah. And the intelligence gap will be like that.
• 30:15
  Speaker 1: So you're telling me that if I'm Pablo, my French bulldog,
• 30:18
  Speaker 1: I need to figure out a way to make my
• 30:21
  Speaker 1: owner not wipe me out.
• 30:24
  Speaker 2: Yeah. So we have one example of that, which is
• 30:27
  Speaker 2: mothers and babies. Evolution put a lot of work into that.
• 30:31
  Speaker 2: Mothers are smarter than babies, but babies are in control.
• 30:34
  Speaker 2: And they're in control because the mother just can't bear
• 30:37
  Speaker 2: lots of hormones and things, but the mother just can't
• 30:40
  Speaker 2: bear the sound of the baby crying. Not all mothers.
• 30:43
  Speaker 2: Not all mothers. And then the baby's not in control
• 30:46
  Speaker 2: and then bad things happen. We somehow need to figure
• 30:50
  Speaker 2: out how to make them not want to take over.
• 30:53
  Speaker 2: The analogy I often use is forget about intelligence, just
• 30:56
  Speaker 2: think about physical strength. Suppose you have a nice little
• 30:59
  Speaker 2: tiger cub. It's sort of a bit bigger than a cat.
• 31:02
  Speaker 2: It's really cute. It's very cuddly, very interesting to watch.
• 31:07
  Speaker 2: Except that you better be sure that when it grows up,
• 31:09
  Speaker 2: it never wants to kill you. Because if it ever
• 31:11
  Speaker 2: wanted to kill you, you'd be dead in a few seconds.
• 31:15
  Speaker 1: And you're saying the AI we have now is the
• 31:16
  Speaker 1: tiger cub?
• 31:17
  Speaker 2: Yep.
• 31:18
  Speaker 1: And it's growing up?
• 31:19
  Speaker 4: Yep.
• 31:21
  Speaker 1: So we need to train it as it's, when it's
• 31:23
  Speaker 1: a baby.
• 31:23
  Speaker 2: Well, now a tiger has lots of innate stuff built in,
• 31:25
  Speaker 2: so you know when it grows up. It's not a
• 31:28
  Speaker 2: safe thing to have around.
• 31:29
  Speaker 1: But lions, people that have lions as pets.
• 31:32
  Speaker 2: Yes.
• 31:32
  Speaker 1: Sometimes the lion is affectionate to its creator, but not
• 31:34
  Speaker 1: to others.
• 31:35
  Speaker 2: Yes. And we don't know whether these AIs, we simply
• 31:41
  Speaker 2: don't know whether we can make them not want to
• 31:43
  Speaker 2: take over and not want to hurt us.
• 31:45
  Speaker 1: Do you think we can? Do you think it's possible
• 31:46
  Speaker 1: to train superintelligence? I don't think it's clear that we can.
• 31:49
  Speaker 1: So I think it might be hopeless. But I also
• 31:53
  Speaker 1: think we might.
• 31:55
  Speaker 2: Be able to. And it'd be sort of crazy if
• 31:58
  Speaker 2: people went extinct because we couldn't be bothered to try.
• 32:01
  Speaker 1: If that's even a possibility, how do you feel about
• 32:04
  Speaker 1: your life's work? Because you were.
• 32:06
  Speaker 2: Yeah. It sort of takes the edge off it, doesn't it?
• 32:10
  Speaker 2: I mean, the idea is going to be wonderful in
• 32:12
  Speaker 2: healthcare and wonderful in education and wonderful. I mean, it's
• 32:16
  Speaker 2: going to make call centres much more efficient. Though one
• 32:18
  Speaker 2: worries a bit about what the people who are doing
• 32:20
  Speaker 2: that job now do. It makes me sad. I don't
• 32:24
  Speaker 2: feel particularly guilty about developing AI like 40 years ago
• 32:29
  Speaker 2: because at that time we had no idea that this
• 32:32
  Speaker 2: stuff was going to happen this fast. We thought we
• 32:35
  Speaker 2: had plenty of time to worry about things like that.
• 32:37
  Speaker 2: When you can't get the AI to do much and
• 32:40
  Speaker 2: you want to get it to do a little bit more,
• 32:41
  Speaker 2: you don't worry about this stupid little thing is going
• 32:44
  Speaker 2: to take over from people. You just want it to
• 32:46
  Speaker 2: be able to do a little bit more of the
• 32:48
  Speaker 2: things people can do. It's not like I knowingly did
• 32:52
  Speaker 2: something thinking, this might wipe us all out, but I'm
• 32:55
  Speaker 2: going to do it anyway. But it is a bit
• 32:58
  Speaker 2: sad that it's not just going to be something for good.
• 33:03
  Speaker 2: So I feel I have a duty now to talk
• 33:05
  Speaker 2: about the risks.
• 33:07
  Speaker 1: And if you could play it forward and you could
• 33:08
  Speaker 1: go forward 30, 50 years and you found out that
• 33:10
  Speaker 1: it led to the extinction of humanity, and if that
• 33:13
  Speaker 1: does end up being the outcome?
• 33:21
  Speaker 2: Well, if you played it forward and... It led to
• 33:23
  Speaker 2: the extinction of humanity. I would use that to tell people,
• 33:28
  Speaker 2: to tell their governments that we really have to work
• 33:31
  Speaker 2: on how we're going to keep this stuff under control.
• 33:34
  Speaker 2: I think we need people to tell governments that governments
• 33:36
  Speaker 2: have to force the companies to use their resources to
• 33:39
  Speaker 2: work on safety. And they're not doing much of that
• 33:43
  Speaker 2: because you don't make profits that way.
• 33:45
  Speaker 1: One of your students we talked about earlier, Ilya?
• 33:49
  Speaker 2: Yep.
• 33:50
  Speaker 1: Ilya left OpenAI.
• 33:51
  Speaker 2: Yep.
• 33:53
  Speaker 1: And there was lots of conversation around the fact that
• 33:55
  Speaker 1: he left because he had safety concerns.
• 33:57
  Speaker 4: Yes.
• 33:58
  Speaker 1: And he's gone on to set up an AI safety company.
• 34:02
  Speaker 2: Yes.
• 34:04
  Speaker 1: Why do you think he left?
• 34:06
  Speaker 2: I think he left because he had safety concerns.
• 34:08
  Speaker 1: Really?
• 34:08
  Speaker 2: I still have lunch with him from time to time. Oh, okay.
• 34:12
  Speaker 2: His parents live in Toronto. When he comes to Toronto,
• 34:14
  Speaker 2: we have lunch together. He doesn't talk to me about
• 34:17
  Speaker 2: what went on at OpenAI, so I have no inside
• 34:19
  Speaker 2: information about that. But I know Ilya very well. And
• 34:22
  Speaker 2: he is genuinely concerned with safety. So I think that's
• 34:25
  Speaker 2: why he left. Because he was one of the top people.
• 34:27
  Speaker 1: I mean, he was.
• 34:28
  Speaker 2: He was probably the most important person behind the development
• 34:31
  Speaker 2: of ChatGPT. The early versions like GPT-2, he was very
• 34:36
  Speaker 2: important in the development of that.
• 34:38
  Speaker 1: You know him personally, so you know his character.
• 34:41
  Speaker 2: Yes. He has a good moral compass. He's not like
• 34:44
  Speaker 2: someone like Musk who has no moral compass.
• 34:47
  Speaker 1: Does Sam Altman have a good moral compass?
• 34:50
  Speaker 2: We'll see. I don't know Sam, so I don't want
• 34:56
  Speaker 2: to comment on that.
• 34:57
  Speaker 1: But from what you've seen, are you concerned about the
• 35:00
  Speaker 1: actions that they've taken? Because if you know Ilya, and
• 35:03
  Speaker 1: Ilya's a good guy and he's left.
• 35:06
  Speaker 2: That would give you some insight, yes. It would give
• 35:09
  Speaker 2: you some reason to believe that there's a problem there.
• 35:12
  Speaker 2: And if you look at Sam's statements some years ago,
• 35:16
  Speaker 2: he sort of happily said in one interview, and this
• 35:21
  Speaker 2: stuff will probably kill us all, That's not exactly what
• 35:23
  Speaker 2: he said, but that's what it amounted to. Now he's
• 35:25
  Speaker 2: saying you don't need to worry too much about it.
• 35:28
  Speaker 2: And I suspect that's not driven by seeking after the truth.
• 35:34
  Speaker 2: That's driven by seeking after money.
• 35:37
  Speaker 1: Is it money or is it power?
• 35:39
  Speaker 2: Yeah, I shouldn't have said money. It's some combination of those, yes.
• 35:43
  Speaker 1: Okay, I guess money is a proxy for power. But
• 35:46
  Speaker 1: I've got a friend who's a billionaire and he is
• 35:49
  Speaker 1: in those circles. And when I went to his house
• 35:52
  Speaker 1: and had lunch with him one day, he knows lots
• 35:54
  Speaker 1: of people in AI, building the biggest AI companies in
• 35:56
  Speaker 1: the world. And he gave me a cautionary warning across
• 35:59
  Speaker 1: his kitchen table in London, where he gave me an
• 36:03
  Speaker 1: insight into the private conversations these people have. Not the
• 36:05
  Speaker 1: media interviews they do, where they talk about safety and
• 36:08
  Speaker 1: all these things, but actually what some of these individuals
• 36:10
  Speaker 1: think is going to happen.
• 36:12
  Speaker 2: And what do they think is going to happen?
• 36:14
  Speaker 1: It's not what they say publicly. person who I shouldn't name,
• 36:19
  Speaker 1: who is the, who's leading one of the biggest AI
• 36:21
  Speaker 1: companies in the world. He told me that he knows
• 36:23
  Speaker 1: this person very well. And he privately thinks that we're
• 36:26
  Speaker 1: heading towards this kind of dystopian world where we have
• 36:29
  Speaker 1: just huge amounts of free time. We don't work anymore.
• 36:32
  Speaker 1: And this person doesn't really give a fuck about the
• 36:34
  Speaker 1: harm that it's going to have on the world. And
• 36:36
  Speaker 1: this person who I'm referring to is building one of
• 36:38
  Speaker 1: the biggest AI companies in the world. And I then
• 36:40
  Speaker 1: watch this person's interviews online.
• 36:41
  Speaker 2: Trying to figure out which of three people it is.
• 36:43
  Speaker 1: Yeah, well, it's one of those three people.
• 36:45
  Speaker 2: Okay.
• 36:45
  Speaker 1: And I watch this person's interviews online and I reflect
• 36:47
  Speaker 1: on the conversation that my billionaire friend had with me
• 36:50
  Speaker 1: who knows him. And I go, fucking hell, this guy's
• 36:52
  Speaker 1: lying publicly. Like he's not telling the truth to the world.
• 36:56
  Speaker 1: And that's haunted me a little bit. It's part of
• 36:57
  Speaker 1: the reason I have so many conversations around AR on
• 36:59
  Speaker 1: this podcast because I'm like, I don't know if they're,
• 37:02
  Speaker 1: I think some of them are a little bit sadistic
• 37:05
  Speaker 1: about power. I think they like the idea that they
• 37:09
  Speaker 1: will change the world, that they will be the one
• 37:12
  Speaker 1: that fundamentally shifts the world.
• 37:15
  Speaker 2: I think Musk is clearly like that, right?
• 37:19
  Speaker 1: He's such a complex character that I don't really know
• 37:22
  Speaker 1: how to place Musk.
• 37:23
  Speaker 2: He's done some really good things, like pushing electric cars.
• 37:28
  Speaker 2: That was a really good thing to do. Some of
• 37:30
  Speaker 2: the things he said about self-driving were a bit exaggerated,
• 37:33
  Speaker 2: but that was a really useful thing he did. giving
• 37:37
  Speaker 2: the Ukrainians communication during the war with Russia. That was
• 37:42
  Speaker 2: a really good thing he did. There's a bunch of
• 37:44
  Speaker 2: things like that. But he's also done some very bad things.
• 37:49
  Speaker 1: So coming back to this point of the possibility of
• 37:55
  Speaker 1: destruction and the motives of these big companies, are you
• 38:02
  Speaker 1: at all hopeful that anything can be done To slow
• 38:04
  Speaker 1: down the pace and acceleration of AI.
• 38:07
  Speaker 2: Okay, there's two issues. One is, can you slow it down?
• 38:10
  Speaker 1: Yeah.
• 38:11
  Speaker 2: And the other is, can you make it so it
• 38:13
  Speaker 2: will be safe in the end? It won't wipe us
• 38:15
  Speaker 2: all out. I don't believe we're going to slow it down.
• 38:20
  Speaker 4: Yeah.
• 38:20
  Speaker 2: And the reason I don't believe we're going to slow
• 38:21
  Speaker 2: it down is because there's competition between countries and competition
• 38:25
  Speaker 2: between companies within a country. And all of that is
• 38:28
  Speaker 2: making it go faster and faster. And if the US
• 38:31
  Speaker 2: slowed it down, China wouldn't slow it down.
• 38:34
  Speaker 1: Does Ilya think it's possible to make AI safe?
• 38:40
  Speaker 2: I think he does. He won't tell me what his
• 38:42
  Speaker 2: secret source is. I'm not sure how many people know
• 38:46
  Speaker 2: what his secret source is. I think a lot of
• 38:47
  Speaker 2: the investors don't know what his secret source is, but
• 38:50
  Speaker 2: they've given him billions of dollars anyway because they have
• 38:52
  Speaker 2: so much faith in Ilya, which isn't foolish. I mean,
• 38:56
  Speaker 2: he was very important in AlexNet, which got object recognition
• 39:00
  Speaker 2: working well. He was the main... the main force behind
• 39:04
  Speaker 2: the things like GPT-2, which then led to ChatGPT. So
• 39:11
  Speaker 2: I think having a lot of faith in Ilya is
• 39:12
  Speaker 2: a very reasonable decision.
• 39:14
  Speaker 1: There's something quite haunting about the guy that made and
• 39:17
  Speaker 1: was the main force behind GPT-2, which led Rise to
• 39:20
  Speaker 1: this whole revolution, left the company because of safety reasons.
• 39:25
  Speaker 1: He knows something that I don't know about what might
• 39:28
  Speaker 1: happen next.
• 39:29
  Speaker 2: Well, the company had, now I don't know the precise details,
• 39:34
  Speaker 2: but I'm fairly sure the company had indicated that it
• 39:37
  Speaker 2: would use a significant fraction of its resources of the
• 39:41
  Speaker 2: compute time for doing safety research. And then it reduced
• 39:45
  Speaker 2: that fraction. I think that's one of the things that happened.
• 39:48
  Speaker 1: Yeah, that was reported publicly.
• 39:49
  Speaker 2: Yes.
• 39:49
  Speaker 1: Yeah. We've gotten to the autonomous weapons part of the
• 39:54
  Speaker 1: risk framework.
• 39:56
  Speaker 2: Right. So the next one is joblessness. Yeah. In the past,
• 40:00
  Speaker 2: new technologies have come in which didn't lead to joblessness.
• 40:04
  Speaker 2: New jobs were created. So the classic example people use
• 40:08
  Speaker 2: is automatic teller machines. When automatic teller machines came in,
• 40:13
  Speaker 2: A lot of bank tellers didn't lose their jobs. They
• 40:15
  Speaker 2: just got to do more interesting things. But here, I
• 40:19
  Speaker 2: think this is more like when they got machines in
• 40:22
  Speaker 2: the Industrial Revolution. And you can't have a job digging
• 40:27
  Speaker 2: ditches now because a machine can dig ditches much better
• 40:30
  Speaker 2: than you can. And I think for mundane intellectual labor,
• 40:35
  Speaker 2: AI is just going to replace everybody. Now, it may
• 40:40
  Speaker 2: well be in the form of You have fewer people
• 40:44
  Speaker 2: using AI assistants. So it's a combination of a person
• 40:47
  Speaker 2: and an AI assistant are now doing the work that
• 40:50
  Speaker 2: 10 people could do previously.
• 40:52
  Speaker 1: People say that it will create new jobs, though, so
• 40:54
  Speaker 1: we'll be fine.
• 40:56
  Speaker 2: Yes, and that's been the case for other technologies. But
• 40:59
  Speaker 2: this is a very different kind of technology. If it
• 41:01
  Speaker 2: can do all mundane human intellectual labor, then what new
• 41:06
  Speaker 2: jobs is it going to create? You'd have to be
• 41:09
  Speaker 2: very skilled to have a job that it couldn't just do.
• 41:12
  Speaker 2: So I don't think they're right. I think you can
• 41:15
  Speaker 2: try and generalize from other technologies that come in, like
• 41:18
  Speaker 2: computers or automatic telemachines. But I think this is different.
• 41:23
  Speaker 1: People use this phrase, they say, AI won't take your job.
• 41:25
  Speaker 1: A human using AI will take your job.
• 41:27
  Speaker 2: Yes, I think that's true. But for many jobs, that'll
• 41:31
  Speaker 2: mean you need far fewer people. My niece answers letters
• 41:35
  Speaker 2: of complaint to a health service. It used to take
• 41:38
  Speaker 2: her 25 minutes. She'd read the complaint and she'd think
• 41:41
  Speaker 2: how to reply and she'd write a letter. Now she
• 41:45
  Speaker 2: just scans it into a chatbot and it writes the letter.
• 41:51
  Speaker 2: She just checks the letter. Occasionally she tells it to
• 41:54
  Speaker 2: revise it in some ways. The whole process takes her
• 41:57
  Speaker 2: five minutes. That means she can answer five times as
• 42:00
  Speaker 2: many letters. And that means they need five times fewer
• 42:05
  Speaker 2: of her so she can do the job that five
• 42:07
  Speaker 2: of her used to do. that will mean they need
• 42:12
  Speaker 2: less people. In other jobs, like in healthcare, they're much
• 42:16
  Speaker 2: more elastic. So if you could make doctors five times
• 42:20
  Speaker 2: as efficient, we could all have five times as much
• 42:23
  Speaker 2: healthcare for the same price. And that would be great.
• 42:26
  Speaker 2: There's almost no limit to how much healthcare people can absorb.
• 42:30
  Speaker 2: They always want more healthcare if there's no cost to it.
• 42:34
  Speaker 2: There are jobs where you can make a person with
• 42:36
  Speaker 2: an AI assistant much more efficient and you won't lead
• 42:39
  Speaker 2: to less people, because you'll just have much more of
• 42:43
  Speaker 2: that being done. But most jobs, I think, are not
• 42:46
  Speaker 2: like that.
• 42:47
  Speaker 1: Am I right in thinking the sort of industrial revolution
• 42:49
  Speaker 1: played a role in replacing muscles?
• 42:53
  Speaker 2: Yes, exactly.
• 42:54
  Speaker 1: And this revolution in AI replaces intelligence, the brain.
• 42:57
  Speaker 2: Yeah. So mundane intellectual labour is like having strong muscles,
• 43:02
  Speaker 2: and it's not worth much anymore.
• 43:05
  Speaker 1: So muscles have been replaced. Now intelligence is being replaced.
• 43:08
  Speaker 2: Yeah.
• 43:09
  Speaker 1: So... what remains?
• 43:11
  Speaker 2: Maybe for a while, some kinds of creativity, but the
• 43:14
  Speaker 2: whole idea of super intelligence is nothing remains.
• 43:18
  Speaker 1: These things will get to be better than us at everything.
• 43:20
  Speaker 1: So what do we end up doing in such a world?
• 43:22
  Speaker 2: Well, if they work for us, we end up getting
• 43:27
  Speaker 2: lots of goods and services for not much effort. Okay.
• 43:31
  Speaker 1: But that sounds tempting and nice, but I don't know,
• 43:34
  Speaker 1: there's a cautionary tale in creating more and more ease
• 43:37
  Speaker 1: for humans and it going Yes.
• 43:41
  Speaker 2: And we need to figure out if we can make
• 43:44
  Speaker 2: it go well. So the nice scenario is, imagine a
• 43:47
  Speaker 2: company with a CEO who is very dumb, probably the
• 43:52
  Speaker 2: son of the former CEO, and he has an executive
• 43:56
  Speaker 2: assistant who's very smart. And he says, I think we
• 44:02
  Speaker 2: should do this. And the executive assistant makes it all work.
• 44:05
  Speaker 2: The CEO feels great. He doesn't understand that he's not
• 44:08
  Speaker 2: really in control. And in some sense, he is in control.
• 44:12
  Speaker 2: He suggests what the company should do. She just makes
• 44:15
  Speaker 2: it all work. Everything's great. That's the good scenario.
• 44:19
  Speaker 1: And the bad scenario?
• 44:20
  Speaker 2: The bad scenario. She thinks, why do we need him? Yeah.
• 44:26
  Speaker 1: I mean, in a world where we have superintelligence, which
• 44:29
  Speaker 1: you don't believe is that far away, Yeah.
• 44:32
  Speaker 2: I think it might not be that far away. It's
• 44:33
  Speaker 2: very hard to predict, but I think we might get
• 44:36
  Speaker 2: it in like 20 years or even less.
• 44:38
  Speaker 1: I made the biggest investment I've ever made in a
• 44:41
  Speaker 1: company because of my girlfriend. I came home one night
• 44:44
  Speaker 1: and my lovely girlfriend was up at 1am in the morning,
• 44:48
  Speaker 1: pulling her hair out as she tried to piece together
• 44:50
  Speaker 1: her own online store for her business. And in that moment,
• 44:54
  Speaker 1: I remembered an email I'd had from a guy called John,
• 44:57
  Speaker 1: the founder of Stan's Store. our new sponsor, and a
• 45:01
  Speaker 1: company I've invested incredibly heavily in. And Stan still helps
• 45:04
  Speaker 1: creators to sell digital products, courses, coaching, and memberships all
• 45:07
  Speaker 1: through a simple, customizable link in bio system. And it
• 45:11
  Speaker 1: handles everything. Payments, bookings, emails, community engagement, and even links
• 45:15
  Speaker 1: with Shopify. And I believe in it so much that
• 45:18
  Speaker 1: I'm going to launch a Stan Challenge. And as part
• 45:22
  Speaker 1: of this challenge, I'm going to give away $ 100, 000 to
• 45:25
  Speaker 1: one of you. If you want to take part in
• 45:27
  Speaker 1: this challenge, if you want to monetize the knowledge that
• 45:29
  Speaker 1: you have, visit stephenbartlett.stan.store to sign up. And you'll also
• 45:34
  Speaker 1: get an extended 30-day free trial of Stan's store if
• 45:38
  Speaker 1: you use that link. Your next move could quite frankly
• 45:41
  Speaker 1: change everything. Because I talked about ketosis on this podcast
• 45:44
  Speaker 1: and ketones, a brand called Ketone IQ sent me their
• 45:47
  Speaker 1: little product here. And it was on my desk when
• 45:49
  Speaker 1: I got to the office. I picked it up, it
• 45:51
  Speaker 1: sat on my desk for a couple of weeks, then
• 45:52
  Speaker 1: one day I tried it. And honestly, I have not
• 45:57
  Speaker 1: looked back ever since. I now have this everywhere I go.
• 46:01
  Speaker 1: When I travel all around the world, it's in my
• 46:02
  Speaker 1: hotel room, my team will put it there. Before I
• 46:04
  Speaker 1: did the podcast recording today that I've just finished, I
• 46:07
  Speaker 1: had a shot of Ketone IQ. And as is always
• 46:10
  Speaker 1: the case, when I fall in love with a product,
• 46:11
  Speaker 1: I called the CEO and asked if I could invest
• 46:13
  Speaker 1: a couple of million quid into their company. So I'm
• 46:16
  Speaker 1: now an investor in the company as well as them
• 46:18
  Speaker 1: being a brand sponsor. I find it so easy to
• 46:20
  Speaker 1: drop into deep focused work when I've had one of these.
• 46:23
  Speaker 1: I would love you to try one and see the
• 46:25
  Speaker 1: impact it has on you, your focus, your productivity and
• 46:28
  Speaker 1: your endurance. So if you want to try it today,
• 46:29
  Speaker 1: visit ketone.com slash Stephen for 30% off your subscription. Plus
• 46:33
  Speaker 1: you'll receive a free gift with your second shipment. That's ketone.com
• 46:38
  Speaker 1: slash Stephen. I'm excited for you.
• 46:41
  Speaker 2: I am.
• 46:42
  Speaker 1: So what's the difference between what we have now and
• 46:43
  Speaker 1: super intelligence? Because it seems to be really intelligent to
• 46:46
  Speaker 1: me when I use like, ChatGPT 3.0 or Gemini?
• 46:50
  Speaker 2: Okay, so it's already, AI is already better than us
• 46:53
  Speaker 2: at a lot of things. In particular areas like chess,
• 46:57
  Speaker 2: for example, AI is so much better than us that
• 47:00
  Speaker 2: people will never beat those things again. Maybe the occasional win,
• 47:04
  Speaker 2: but basically they'll never be comfortable again. Obviously the same
• 47:08
  Speaker 2: in Go. In terms of the amount of knowledge they have,
• 47:12
  Speaker 2: something like GPT-4 knows thousands of times more than you do.
• 47:16
  Speaker 2: There's a few areas in which your knowledge is better
• 47:19
  Speaker 2: than it's. And in almost all areas, it just knows
• 47:22
  Speaker 2: more than you do.
• 47:24
  Speaker 1: What areas am I better than it?
• 47:27
  Speaker 2: Probably in interviewing CEOs. You're probably better at that. You've
• 47:33
  Speaker 2: got a lot of experience at it. You're a good interviewer.
• 47:36
  Speaker 2: You know a lot about it. If you got GPT-4
• 47:40
  Speaker 2: to interview a CEO, probably do a worse job. Okay.
• 47:46
  Speaker 1: I'm trying to think if I agree with that statement. GPT-4,
• 47:49
  Speaker 1: I think, for sure. Yeah. But I guess you could
• 47:52
  Speaker 1: train one.
• 47:52
  Speaker 2: But it may not be long before.
• 47:53
  Speaker 1: Yeah, I guess you could train one on this, how
• 47:56
  Speaker 1: I ask questions and what I do.
• 47:57
  Speaker 2: Sure. And if you took a general purpose sort of
• 48:01
  Speaker 2: foundation model and then you trained it up on not
• 48:05
  Speaker 2: just you, but every interview you could find doing interviews
• 48:09
  Speaker 2: like this, but especially you, you'd probably get to be
• 48:12
  Speaker 2: quite good at doing your job, but probably not as
• 48:14
  Speaker 2: good as you for a while.
• 48:17
  Speaker 1: Okay. So there's a few areas left and then super
• 48:19
  Speaker 1: intelligence becomes when it's better than us at all things.
• 48:23
  Speaker 2: When it's much smarter than you and almost all things
• 48:26
  Speaker 2: is better than you. Yeah.
• 48:27
  Speaker 1: And you say that this might be a decade away
• 48:29
  Speaker 1: or so?
• 48:29
  Speaker 2: Yeah, it might be. It might be even closer. Some
• 48:33
  Speaker 2: people think it's even closer. It might well be much further.
• 48:36
  Speaker 2: It might be 50 years away. That's still a possibility.
• 48:40
  Speaker 2: It might be that somehow training on human data limits
• 48:43
  Speaker 2: you to not be much smarter than humans. My guess
• 48:46
  Speaker 2: is between 10 and 20 years we'll have superintelligence.
• 48:50
  Speaker 1: On this point of joblessness, it's something that I've been
• 48:52
  Speaker 1: thinking a lot about in particular because I started messing
• 48:54
  Speaker 1: around with AI agents. And we released an episode on
• 48:56
  Speaker 1: the podcast actually this morning where we had a debate
• 48:59
  Speaker 1: about AI agents with a CEO of a big AI
• 49:02
  Speaker 1: agent company and a few other people. And it was
• 49:04
  Speaker 1: the first moment where I had, no, it was another
• 49:07
  Speaker 1: moment where I had a eureka moment about what the
• 49:10
  Speaker 1: future might look like when I was able in the
• 49:12
  Speaker 1: interview to tell this agent to order all of us drinks.
• 49:15
  Speaker 1: And then five minutes later in the interview, you see
• 49:17
  Speaker 1: the guy show up with the drinks and I didn't
• 49:19
  Speaker 1: touch anything. I just told it to order us drinks
• 49:21
  Speaker 1: to the studio.
• 49:22
  Speaker 2: And you didn't know about who you normally got your
• 49:24
  Speaker 2: drinks from it. Figure that out from the web.
• 49:26
  Speaker 1: Yeah, figured it out. Cause it went on Uber Eats.
• 49:28
  Speaker 1: It has my, my, my data, I guess. And it,
• 49:31
  Speaker 1: we put it on the screen in real time. So
• 49:32
  Speaker 1: everyone at home could see the agent going through the internet,
• 49:35
  Speaker 1: picking the drinks, adding a tip for the driver. putting
• 49:38
  Speaker 1: my address in, putting my credit card details in. And
• 49:40
  Speaker 1: then the next thing you see is the drinks show up.
• 49:42
  Speaker 1: So that was one moment. And then the other moment
• 49:44
  Speaker 1: was when I used a tool called Replit and I
• 49:49
  Speaker 1: built software by just telling the agent what I wanted.
• 49:51
  Speaker 2: Yes. It's amazing, right?
• 49:53
  Speaker 1: It's amazing and terrifying at the same time.
• 49:56
  Speaker 2: Yes.
• 49:57
  Speaker 1: Because.
• 49:57
  Speaker 2: And if you can build software like that, right? Yeah.
• 50:00
  Speaker 2: Remember that the AI, when it's training, is using code.
• 50:05
  Speaker 2: And if it can modify its own code... then it
• 50:08
  Speaker 2: gets quite scary, right?
• 50:10
  Speaker 1: Because it can modify itself.
• 50:10
  Speaker 2: It can change itself in a way we can't change ourselves.
• 50:14
  Speaker 2: We can't change our innate endowment, right? There's nothing about
• 50:18
  Speaker 2: itself that it couldn't change.
• 50:21
  Speaker 1: On this point of joblessness, you have kids.
• 50:23
  Speaker 2: I do.
• 50:24
  Speaker 1: And they have kids? No, they don't have kids. No
• 50:26
  Speaker 1: grandkids yet. What would you be saying to people about
• 50:29
  Speaker 1: their career prospects in a world of superintelligence? What should
• 50:32
  Speaker 1: we be thinking about?
• 50:34
  Speaker 2: In the meantime, I'd say it's going to be a
• 50:36
  Speaker 2: long time before it's as good at physical manipulation as us.
• 50:40
  Speaker 1: Okay.
• 50:41
  Speaker 2: And so a good bet would be to be a plumber.
• 50:47
  Speaker 1: Until the humanoid robots show up. In such a world
• 50:50
  Speaker 1: where there is mass joblessness, which is not something that
• 50:53
  Speaker 1: you just predict, but this is something that Sam Altman
• 50:55
  Speaker 1: at OpenAI, I've heard him predict, and many of the CEOs,
• 50:58
  Speaker 1: Elon Musk, I watched an interview, which I'll play on
• 51:00
  Speaker 1: screen of him being asked this question. And it's very
• 51:03
  Speaker 1: rare that you see Elon Musk silent for 12 seconds
• 51:05
  Speaker 1: or whatever it was.
• 51:05
  Speaker 4: Right.
• 51:07
  Speaker 1: He basically says something about he actually is living in
• 51:10
  Speaker 1: suspended disbelief. He's basically just not thinking about it.
• 51:13
  Speaker 2: When you think about advising your children on a career
• 51:16
  Speaker 2: with so much that is changing, what do you tell
• 51:19
  Speaker 2: them is going to be of value?
• 51:33
  Speaker 4: Well, that is a tough question to answer. I would
• 51:38
  Speaker 4: just say, you know, to sort of follow their heart
• 51:40
  Speaker 4: in terms of what they find interesting to do or
• 51:43
  Speaker 4: fulfilling to do. I mean, if I think about it
• 51:45
  Speaker 4: too hard, it can be dispiriting and demotivating. Because, I mean,
• 51:55
  Speaker 4: I go through, I've put a lot of blood, sweat,
• 51:59
  Speaker 4: and tears into building the companies. And then I'm like, wait,
• 52:04
  Speaker 4: should I be doing this?
• 52:05
  Speaker 2: Because.
• 52:06
  Speaker 4: if I'm sacrificing time with friends and family that I
• 52:09
  Speaker 4: would prefer to do, but then ultimately the AI can
• 52:13
  Speaker 4: do all these things.
• 52:14
  Speaker 2: Does that make sense? I don't know.
• 52:17
  Speaker 4: Um, to some extent I have to have deliberate suspension
• 52:20
  Speaker 4: of disbelief in order to be, to remain motivated. Um,
• 52:25
  Speaker 4: so I, I guess I would say just, you know,
• 52:31
  Speaker 4: work on things that you find interesting, fulfilling, and, um,
• 52:34
  Speaker 4: and that contribute some good to the rest of society.
• 52:37
  Speaker 2: Yeah, a lot of these threats, it's very hard to,
• 52:41
  Speaker 2: intellectually you can see the threat, but it's very hard
• 52:45
  Speaker 2: to come to terms with it emotionally. I haven't come
• 52:48
  Speaker 2: to terms with it emotionally yet.
• 52:50
  Speaker 1: What do you mean by that?
• 52:53
  Speaker 2: I haven't come to terms with what the development of
• 52:57
  Speaker 2: superintelligence could do to my children's future. I'm okay, I'm 77.
• 53:04
  Speaker 2: I'm going to be out of here soon. But for
• 53:07
  Speaker 2: my children and my younger friends, my nephews and nieces
• 53:13
  Speaker 2: and their children, I just don't like to think about
• 53:19
  Speaker 2: what could happen.
• 53:23
  Speaker 1: Why?
• 53:25
  Speaker 2: Because it could be awful.
• 53:30
  Speaker 1: In what way?
• 53:32
  Speaker 2: Well, if I ever decided to take over, I mean,
• 53:35
  Speaker 2: it would need people for a while to run the
• 53:37
  Speaker 2: power stations until it designed better analog machines to run
• 53:42
  Speaker 2: the power stations. There's so many ways it could get
• 53:46
  Speaker 2: rid of people, all of which would, of course, be
• 53:48
  Speaker 2: very nasty.
• 53:50
  Speaker 1: Is that part of the reason you do what you
• 53:51
  Speaker 1: do now? Yeah.
• 53:53
  Speaker 2: I mean, I think we should be making a huge
• 53:55
  Speaker 2: effort right now to try and figure out if we
• 53:59
  Speaker 2: can develop it safely.
• 54:00
  Speaker 1: Are you concerned about the midterm impact potentially on your
• 54:03
  Speaker 1: nephews and your kids in terms of their jobs as well?
• 54:06
  Speaker 2: Yeah, I'm concerned about all that.
• 54:08
  Speaker 1: Are there any particular industries that you think are most
• 54:10
  Speaker 1: at risk? People talk about the creative industries a lot
• 54:12
  Speaker 1: and it's sort of knowledge work. They talk about lawyers
• 54:16
  Speaker 1: and accountants and stuff like that.
• 54:17
  Speaker 2: Yeah, so that's why I mentioned plumbers. I think plumbers
• 54:19
  Speaker 2: are less at risk.
• 54:20
  Speaker 1: Okay, I'm going to become a plumber.
• 54:21
  Speaker 2: Someone like a legal assistant, a paralegal, they're not going
• 54:27
  Speaker 2: to be needed for very long.
• 54:29
  Speaker 1: And is there a wealth inequality issue here that will
• 54:32
  Speaker 1: rise from this?
• 54:32
  Speaker 2: Yeah, I think in a society which shared out things fairly,
• 54:37
  Speaker 2: if you get a big increase in productivity, everybody should
• 54:41
  Speaker 2: be better off. But if you can replace lots of
• 54:45
  Speaker 2: people by AIs, then the people who get replaced will
• 54:50
  Speaker 2: be worse off. And the company that supplies the AIs
• 54:55
  Speaker 2: will be much better off. and the company that uses
• 55:00
  Speaker 2: the AIs. So it's going to increase the gap between
• 55:03
  Speaker 2: rich and poor. And we know that if you look
• 55:06
  Speaker 2: at that gap between rich and poor, that basically tells
• 55:09
  Speaker 2: you how nice a society is. If you have a
• 55:11
  Speaker 2: big gap, you get very nasty societies in which people
• 55:14
  Speaker 2: live in walled communities and put other people in mass jails.
• 55:20
  Speaker 2: It's not good to increase the gap between rich and poor.
• 55:22
  Speaker 1: The International Monetary Fund has expressed profound concerns that generative
• 55:26
  Speaker 1: AI could cause massive labor disruptions and rising inequality, and
• 55:30
  Speaker 1: has called for policies that prevent this from happening. I
• 55:33
  Speaker 1: read that in the Business Insider.
• 55:35
  Speaker 2: Have they given any idea what the policies should look like?
• 55:38
  Speaker 1: No.
• 55:39
  Speaker 2: Yeah, that's the problem. I mean, if AI can make
• 55:41
  Speaker 2: everything much more efficient and get rid of people for
• 55:43
  Speaker 2: most jobs, or have a person assisted by AI doing many,
• 55:48
  Speaker 2: many people's work, it's not obvious what to do about it.
• 55:53
  Speaker 1: Universal basic income? Give everybody money?
• 55:56
  Speaker 2: Yeah, I think that's a good start. And it stops
• 56:01
  Speaker 2: people starving. But for a lot of people, their dignity
• 56:05
  Speaker 2: is tied up with their job. I mean, who you
• 56:06
  Speaker 2: think you are is tied up with you doing this job, right? Yeah.
• 56:11
  Speaker 2: And if we said, we'll give you the same money
• 56:13
  Speaker 2: just to sit around, that would impact your dignity.
• 56:18
  Speaker 1: You said something earlier about it surpassing or being superior
• 56:22
  Speaker 1: to human intelligence. A lot of people, I think, like
• 56:24
  Speaker 1: to believe that AI is on a computer and it's
• 56:27
  Speaker 1: something you can just turn off if you don't like it.
• 56:30
  Speaker 2: Well, let me tell you why I think it's superior.
• 56:32
  Speaker 1: Okay.
• 56:34
  Speaker 2: It's digital. And because it's digital, you can simulate a
• 56:39
  Speaker 2: neural network on one piece of hardware. Yeah. And you
• 56:42
  Speaker 2: can simulate exactly the same neural network on a different
• 56:45
  Speaker 2: piece of hardware. So you can have clones of the
• 56:48
  Speaker 2: same intelligence. Now, you could get this one to go
• 56:52
  Speaker 2: off and look at one bit of the internet and
• 56:54
  Speaker 2: this other one to look at different bit of the internet.
• 56:57
  Speaker 2: And while they're looking at these different bits of the internet,
• 57:00
  Speaker 2: they can be syncing with each other. So they keep
• 57:02
  Speaker 2: their weights the same, the connection strings the same, weights
• 57:05
  Speaker 2: of connection strings. So this one might look at something
• 57:08
  Speaker 2: on the internet and say, oh, I'd like to increase
• 57:10
  Speaker 2: this strength, this connection a bit. And it can convey
• 57:14
  Speaker 2: that information to this one. So it can increase the
• 57:16
  Speaker 2: strength of that connection a bit based on this one's experience.
• 57:19
  Speaker 1: And when you say the strength of the connection, you're
• 57:21
  Speaker 1: talking about learning.
• 57:22
  Speaker 2: That's learning, yes. Learning consists of saying, instead of this
• 57:26
  Speaker 2: one giving 2.4 votes for whether that one should turn on,
• 57:29
  Speaker 2: we'll have this one give 2.5 votes for whether this
• 57:31
  Speaker 2: one should turn on. That would be a little bit
• 57:34
  Speaker 2: of learning. So these two different copies of the same
• 57:37
  Speaker 2: neural net are getting different experiences. They're looking at different data,
• 57:42
  Speaker 2: but they're sharing what they've learned by averaging their weights together.
• 57:47
  Speaker 2: And they can do that averaging at like a You
• 57:49
  Speaker 2: can average a trillion weights. When you and I transfer information,
• 57:54
  Speaker 2: we're limited to the amount of information in a sentence.
• 57:56
  Speaker 2: And the amount of information in a sentence is maybe
• 57:58
  Speaker 2: 100 bits. It's very little information. We're lucky if we're
• 58:02
  Speaker 2: transferring like 10 bits a second. These things are transferring
• 58:06
  Speaker 2: trillions of bits a second. So they're billions of times
• 58:09
  Speaker 2: better than us at sharing information. And that's because they're digital.
• 58:14
  Speaker 2: And you can have two bits of hardware using the
• 58:16
  Speaker 2: connection strengths in exactly the same way. We're analog and
• 58:19
  Speaker 2: you can't do that. Your brain's different from my brain.
• 58:22
  Speaker 2: And if I could see the connection strengths between all
• 58:25
  Speaker 2: your neurons, it wouldn't do me any good because my
• 58:27
  Speaker 2: neurons work slightly differently and they're connected up slightly differently.
• 58:31
  Speaker 2: So when you die, all your knowledge dies with you.
• 58:35
  Speaker 2: When these things die, suppose you take these two digital
• 58:38
  Speaker 2: intelligences that are clones of each other and you destroy
• 58:41
  Speaker 2: the hardware they run on. As long as you've stored
• 58:44
  Speaker 2: the connection strength somewhere, you can just build new hardware
• 58:48
  Speaker 2: that executes the same instructions, so it'll know how to
• 58:51
  Speaker 2: use those connection strengths, and you've recreated that intelligence. So
• 58:55
  Speaker 2: they're immortal. We've actually solved the problem of immortality, but
• 59:00
  Speaker 2: it's only for digital things.
• 59:02
  Speaker 1: So it knows, it will essentially know everything that humans
• 59:06
  Speaker 1: know but more, because it will learn new things.
• 59:09
  Speaker 2: It will learn new things. It will also see all
• 59:11
  Speaker 2: sorts of analogies that people probably never saw. So, for example,
• 59:17
  Speaker 2: at the point when GPT-4 couldn't look on the web,
• 59:20
  Speaker 2: I asked it, why is a compost heap like an
• 59:23
  Speaker 2: atom bomb? Off you go.
• 59:27
  Speaker 1: I have no idea. Exactly.
• 59:28
  Speaker 2: Exactly. Excellent. That's exactly what most people would say. It said, well,
• 59:32
  Speaker 2: the time scales are very different and the energy scales
• 59:35
  Speaker 2: are very different. But then it went on to talk
• 59:38
  Speaker 2: about how a compost heap, as it gets hotter, generates
• 59:40
  Speaker 2: heat faster. And an atom bomb, as it produces more neutrons,
• 59:45
  Speaker 2: generates neutrons faster. And so they're both chain reactions, but
• 59:50
  Speaker 2: at very different time and energy scales. And I believe
• 59:53
  Speaker 2: GPT-4 had seen that during its training. It had understood
• 59:57
  Speaker 2: the analogy between a compost heap and an atom bomb.
• 59:59
  Speaker 2: And the reason I believe that is, if you've only
• 1:00:02
  Speaker 2: got a trillion connections, remember you have 100 trillion, and
• 1:00:06
  Speaker 2: you need to have thousands of times more knowledge than
• 1:00:08
  Speaker 2: a person, you need to compress information into those connections.
• 1:00:13
  Speaker 2: And to compress information, you need to see analogies between
• 1:00:16
  Speaker 2: different things. In other words, it needs to see all
• 1:00:19
  Speaker 2: the things that are chain reactions and understand the basic
• 1:00:22
  Speaker 2: idea of a chain reaction and code that, and then
• 1:00:24
  Speaker 2: code the ways in which they're different. And that's just
• 1:00:26
  Speaker 2: a more efficient way of coding things than coding each
• 1:00:29
  Speaker 2: of them separately. So it's seen many, many analogies, probably
• 1:00:34
  Speaker 2: many analogies that people have never seen. That's why I
• 1:00:37
  Speaker 2: also think that people say these things will never be creative.
• 1:00:40
  Speaker 2: They're going to be much more creative than us because
• 1:00:42
  Speaker 2: they're going to see all sorts of analogies we never saw.
• 1:00:45
  Speaker 2: And a lot of creativity is about seeing strange analogies.
• 1:00:49
  Speaker 1: People are somewhat romantic about the specialness of what it
• 1:00:51
  Speaker 1: is to be human. And you hear lots of people saying,
• 1:00:53
  Speaker 1: it's very, very different. It's a computer. We are, you know,
• 1:00:56
  Speaker 1: we're conscious. We are creatives. We have these sort of innate,
• 1:01:01
  Speaker 1: unique abilities that the computers will never have. What do
• 1:01:04
  Speaker 1: you say to those people?
• 1:01:05
  Speaker 2: I'd argue a bit with the innate. So the first
• 1:01:12
  Speaker 2: thing I say is we have a long history of
• 1:01:14
  Speaker 2: believing people were special. And we should have learned by now.
• 1:01:18
  Speaker 2: We thought we were at the center of the universe.
• 1:01:20
  Speaker 2: We thought we were made in the image of God.
• 1:01:23
  Speaker 2: White people thought they were very special. We just tend
• 1:01:27
  Speaker 2: to want to think we're special. My belief is that
• 1:01:31
  Speaker 2: more or less everyone has a completely wrong model of
• 1:01:35
  Speaker 2: what the mind is. Let's suppose I drink a lot
• 1:01:37
  Speaker 2: or I drop some acid and not recommend it. And
• 1:01:42
  Speaker 2: I say to you, I have the subjective experience of
• 1:01:46
  Speaker 2: little pink elephants floating in front of me. Most people
• 1:01:51
  Speaker 2: interpret that as there's some kind of inner theater called
• 1:01:55
  Speaker 2: the mind, and only I can see what's in my mind.
• 1:02:01
  Speaker 2: And in this inner theater, there's little pink elephants floating around.
• 1:02:06
  Speaker 2: So in other words, what's happened is my perceptual system's
• 1:02:08
  Speaker 2: gone wrong, and I'm trying to indicate to you how
• 1:02:12
  Speaker 2: it's gone wrong and what it's trying to tell me.
• 1:02:15
  Speaker 2: And the way I do that is by telling you
• 1:02:17
  Speaker 2: what would have to be out there in the real
• 1:02:19
  Speaker 2: world for it to be telling the truth. And so
• 1:02:25
  Speaker 2: these little pink elephants, they're not in some inner theater.
• 1:02:29
  Speaker 2: These little pink elephants are hypothetical things in the real world.
• 1:02:33
  Speaker 2: And that's my way of telling you how my perceptual
• 1:02:36
  Speaker 2: system's telling me fibs. So now let's do that with
• 1:02:39
  Speaker 2: a chatbot. Yeah. Because I believe that current multimodal chatbots
• 1:02:43
  Speaker 2: have subjective experiences. And very few people believe that. But
• 1:02:48
  Speaker 2: I'll try and make you believe it. So suppose I
• 1:02:50
  Speaker 2: have a multimodal chatbot. It's got a robot arm so
• 1:02:53
  Speaker 2: it can point, and it's got a camera so it
• 1:02:55
  Speaker 2: can see things. And I put an object in front
• 1:02:58
  Speaker 2: of it, and I say, point at the object. It
• 1:03:01
  Speaker 2: goes like this. No problem. Then I put a prism
• 1:03:04
  Speaker 2: in front of its lens. And so then I put
• 1:03:06
  Speaker 2: an object in front of it, and I say, point
• 1:03:09
  Speaker 2: at the object, and it goes there. And I say, no,
• 1:03:13
  Speaker 2: that's not where the object is. The object's actually straight
• 1:03:15
  Speaker 2: in front of you, but I put a prism in
• 1:03:17
  Speaker 2: front of your lens. And the chatbot says, Oh, I see.
• 1:03:21
  Speaker 2: The prism bent the light rays. So the object's actually there,
• 1:03:25
  Speaker 2: but I had the subjective experience that it was there. Now,
• 1:03:29
  Speaker 2: if the chatbot says that, it's using the word subjective
• 1:03:32
  Speaker 2: experience exactly the way people use them. It's an alternative
• 1:03:35
  Speaker 2: view of what's going on. They're hypothetical states of the world,
• 1:03:40
  Speaker 2: which if they were true, would mean my perceptual system
• 1:03:42
  Speaker 2: wasn't lying. And that's the best way I can tell
• 1:03:44
  Speaker 2: you what my perceptual system's doing when it's lying to me. Now,
• 1:03:49
  Speaker 2: We need to go further to deal with sentience and
• 1:03:50
  Speaker 2: consciousness and feelings and emotions. But I think in the end,
• 1:03:53
  Speaker 2: they're all going to be dealt with in a similar way.
• 1:03:55
  Speaker 2: There's no reason machines can't have them all. But people
• 1:03:58
  Speaker 2: say machines can't have feelings. And people are curiously confident
• 1:04:03
  Speaker 2: about that. I have no idea why. Suppose I make
• 1:04:06
  Speaker 2: a battle robot. And it's a little battle robot. And
• 1:04:09
  Speaker 2: it sees a big battle robot that's much more powerful
• 1:04:12
  Speaker 2: than it. It will be really useful if it got scared. Now,
• 1:04:19
  Speaker 2: when I get scared, various physiological things happen that we
• 1:04:23
  Speaker 2: don't need to go into, and those won't happen with
• 1:04:25
  Speaker 2: the robot. But all the cognitive things, like I better
• 1:04:29
  Speaker 2: get the hell out of here, and I better sort
• 1:04:31
  Speaker 2: of change my way of thinking so I focus and
• 1:04:35
  Speaker 2: focus and focus and don't get distracted, all of that
• 1:04:38
  Speaker 2: will happen with robots too. People will build in things
• 1:04:42
  Speaker 2: so that they, when the circumstances such they should get
• 1:04:45
  Speaker 2: the hell out of there, they get scared and run away.
• 1:04:48
  Speaker 2: They'll have emotions then. They won't have the physiological aspects,
• 1:04:52
  Speaker 2: but they will have all the cognitive aspects. And I
• 1:04:55
  Speaker 2: think it would be odd to say they're just simulating emotions. No,
• 1:04:58
  Speaker 2: they're really having those emotions. The little robot got scared
• 1:05:00
  Speaker 2: and ran away.
• 1:05:02
  Speaker 1: It's not running away because of adrenaline. It's running away
• 1:05:04
  Speaker 1: because of a sequence of sort of neurological, in its
• 1:05:08
  Speaker 1: neural net, processes happened.
• 1:05:10
  Speaker 2: Which have the equivalent effect to adrenaline. And it's not
• 1:05:14
  Speaker 2: just adrenaline, right? There's a lot of cognitive stuff goes
• 1:05:16
  Speaker 2: on when you get scared. Yeah.
• 1:05:18
  Speaker 1: So do you think that there is conscious AI? And
• 1:05:23
  Speaker 1: when I say conscious, I mean that represents the same
• 1:05:26
  Speaker 1: properties of consciousness that a human has.
• 1:05:29
  Speaker 2: There's two issues here. There's a sort of empirical one
• 1:05:31
  Speaker 2: and a philosophical one. I don't think there's anything in
• 1:05:34
  Speaker 2: principle that stops machines from being conscious. I'll give you
• 1:05:39
  Speaker 2: a little demonstration of that before we carry on. Suppose
• 1:05:41
  Speaker 2: I take your brain and I take one brain cell
• 1:05:44
  Speaker 2: in your brain And I replace it by, it's a
• 1:05:47
  Speaker 2: bit Black Mirror-like, I replace it by a little piece
• 1:05:50
  Speaker 2: of nanotechnology that's just the same size that behaves in
• 1:05:55
  Speaker 2: exactly the same way when it gets pings from other neurons.
• 1:05:57
  Speaker 2: It sends out pings just as the brain cell would have.
• 1:06:00
  Speaker 2: So the other neurons don't know anything's changed. Okay, I've
• 1:06:04
  Speaker 2: just replaced one of your brain cells with this little
• 1:06:06
  Speaker 2: piece of nanotechnology. Would you still be conscious? Yeah. Now
• 1:06:11
  Speaker 2: you can see where this argument's going. Yeah, so if
• 1:06:13
  Speaker 2: you replaced all of them, As I replace them all,
• 1:06:16
  Speaker 2: at what point do you stop being conscious?
• 1:06:19
  Speaker 1: Well, people think of consciousness as this like ethereal thing
• 1:06:22
  Speaker 1: that exists maybe beyond the brain cells.
• 1:06:25
  Speaker 2: Yeah, well, people have a lot of crazy ideas. People
• 1:06:30
  Speaker 2: don't know what consciousness is and they often don't know
• 1:06:32
  Speaker 2: what they mean by it. And then they fall back
• 1:06:35
  Speaker 2: on saying, well, I know it because I've got it
• 1:06:37
  Speaker 2: and I can see that I've got it. And they
• 1:06:39
  Speaker 2: fall back on this theater model of the mind, which
• 1:06:42
  Speaker 2: I think is nonsense.
• 1:06:44
  Speaker 1: What do you think of consciousness as if you had
• 1:06:45
  Speaker 1: to try and define it? Because I think of it
• 1:06:47
  Speaker 1: as just like the awareness of myself. I don't know.
• 1:06:50
  Speaker 2: I think it's a term we'll stop using. Suppose you
• 1:06:53
  Speaker 2: want to understand how a car works. Well, you know,
• 1:06:56
  Speaker 2: some cars have a lot of oomph and other cars
• 1:06:58
  Speaker 2: have a lot less oomph. Like an Aston Martin's got
• 1:07:01
  Speaker 2: lots of oomph and a little Toyota Corolla doesn't have
• 1:07:05
  Speaker 2: much oomph. But oomph isn't a very good concept for
• 1:07:09
  Speaker 2: understanding cars. If you want to understand cars, you need
• 1:07:12
  Speaker 2: to understand about electric engines or petrol engines and how
• 1:07:15
  Speaker 2: they work. And it gives rise to oomph. But oomph
• 1:07:19
  Speaker 2: isn't a very useful explanatory concept. It's a kind of
• 1:07:21
  Speaker 2: essence of a car. It's the essence of an Aston Martin.
• 1:07:24
  Speaker 2: But it doesn't explain much. I think consciousness is like that.
• 1:07:28
  Speaker 2: And I think we'll stop using that term. But I
• 1:07:30
  Speaker 2: don't think there's anything, any reason why a machine shouldn't
• 1:07:34
  Speaker 2: have it. If your view of consciousness is that it
• 1:07:37
  Speaker 2: intrinsically involves self-awareness, then the machine's got to have self-awareness.
• 1:07:42
  Speaker 2: It's got to have cognition about its own cognition and stuff.
• 1:07:45
  Speaker 2: But I'm a materialist through and through, and I don't
• 1:07:51
  Speaker 2: think there's any reason why a machine shouldn't have consciousness.
• 1:07:54
  Speaker 1: Do you think they do then have the same consciousness
• 1:07:57
  Speaker 1: that we think of ourselves as being uniquely given as
• 1:08:02
  Speaker 1: a gift when we're born?
• 1:08:03
  Speaker 2: I'm ambivalent about that at present. So I don't think
• 1:08:09
  Speaker 2: there's this hard line. I think as soon as you
• 1:08:10
  Speaker 2: have a machine that has some self-awareness, it's got some consciousness.
• 1:08:17
  Speaker 2: I think it's an emergent property of a complex system.
• 1:08:21
  Speaker 2: It's not a sort of essence that's throughout the universe.
• 1:08:25
  Speaker 2: You make this really complicated system that's complicated enough to
• 1:08:28
  Speaker 2: have a model of itself, and it does perception. And
• 1:08:32
  Speaker 2: I think then you're beginning to get conscious machine. So
• 1:08:37
  Speaker 2: I don't think there's any sharp distinction between what we've
• 1:08:39
  Speaker 2: got now and conscious machines. I don't think one day
• 1:08:42
  Speaker 2: we're going to wake up and say, hey, if you
• 1:08:44
  Speaker 2: put this special chemical in, it becomes conscious. It's not
• 1:08:47
  Speaker 2: going to be like that.
• 1:08:49
  Speaker 1: I think we all wonder if these computers are thinking
• 1:08:51
  Speaker 1: like we are on their own when we're not there.
• 1:08:55
  Speaker 1: And if they're experiencing emotions, if they're contending with... I
• 1:08:58
  Speaker 1: think we think about things like love and things that
• 1:09:01
  Speaker 1: feel unique to biological species. Are they sat there thinking?
• 1:09:07
  Speaker 1: Do they have concerns? I think they really are thinking.
• 1:09:10
  Speaker 1: And I think as soon as you make AI agents,
• 1:09:12
  Speaker 1: they will have concerns. If you wanted to make an
• 1:09:15
  Speaker 1: effective AI agent, suppose you... Let's take a call center.
• 1:09:19
  Speaker 1: In a call center, you have people at present. They
• 1:09:22
  Speaker 1: have all sorts of emotions and feelings which are kind
• 1:09:25
  Speaker 1: of useful. So suppose I call up.
• 1:09:28
  Speaker 2: The call center and I'm actually lonely and I don't
• 1:09:32
  Speaker 2: actually want to know the answer to why my computer
• 1:09:35
  Speaker 2: isn't working. I just want somebody to talk to. After
• 1:09:39
  Speaker 2: a while, the person in the call center will either
• 1:09:43
  Speaker 2: get bored or get annoyed with me and will terminate it. Well,
• 1:09:48
  Speaker 2: you replace them by an AI agent. The AI agent
• 1:09:51
  Speaker 2: needs to have the same kind of responses. If someone's
• 1:09:53
  Speaker 2: just called up because they just want to talk to
• 1:09:55
  Speaker 2: the AI agent and were happy to talk for the
• 1:09:58
  Speaker 2: whole day to the AI agent, that's not good for business.
• 1:10:01
  Speaker 2: And you want an AI agent that either gets bored
• 1:10:03
  Speaker 2: or gets irritated and says, I'm sorry, but I don't
• 1:10:06
  Speaker 2: have time for this. Once it does that, I think
• 1:10:09
  Speaker 2: it's got emotions. Now, like I say, emotions have two
• 1:10:14
  Speaker 2: aspects to them. There's the cognitive aspect. and the behavioral aspect.
• 1:10:18
  Speaker 2: And then there's a physiological aspect. And those go together
• 1:10:21
  Speaker 2: with us. And if the AI agent gets embarrassed, he
• 1:10:26
  Speaker 2: won't go red.
• 1:10:27
  Speaker 1: Yeah.
• 1:10:27
  Speaker 2: So there's no physiological. And his skin won't start sweating.
• 1:10:30
  Speaker 3: Yeah.
• 1:10:31
  Speaker 2: But it might have all the same behavior. And in
• 1:10:32
  Speaker 2: that case, I'd say, yeah, it's having emotion. It's got
• 1:10:34
  Speaker 2: an emotion.
• 1:10:35
  Speaker 1: So it's going to have the same sort of cognitive thought,
• 1:10:38
  Speaker 1: and then it's going to act upon that cognitive thought.
• 1:10:40
  Speaker 2: In the same way, but without the physiological responses.
• 1:10:43
  Speaker 1: And does that matter that it doesn't go red in
• 1:10:45
  Speaker 1: the face and it's just a different, I mean, that's
• 1:10:47
  Speaker 1: a response to the.
• 1:10:48
  Speaker 2: It makes it somewhat different from us. Yeah. For some things,
• 1:10:52
  Speaker 2: the physiological aspects are very important, like love. They're a
• 1:10:55
  Speaker 2: long way from having love the same way we do.
• 1:10:58
  Speaker 2: But I don't see why they shouldn't have emotions. So
• 1:11:02
  Speaker 2: I think what's happened is people have a model of
• 1:11:05
  Speaker 2: how the mind works and what feelings are and what
• 1:11:08
  Speaker 2: emotions are, and their model is just wrong.
• 1:11:12
  Speaker 1: What brought you to Google? You worked at Google for
• 1:11:16
  Speaker 1: about a decade, right?
• 1:11:17
  Speaker 3: Yeah.
• 1:11:18
  Speaker 1: What brought you there?
• 1:11:19
  Speaker 2: I have a son who has learning difficulties. And in
• 1:11:24
  Speaker 2: order to be sure he would never be out on
• 1:11:27
  Speaker 2: the street, I needed to get several million dollars. And
• 1:11:32
  Speaker 2: I wasn't going to get that as an academic. I tried.
• 1:11:36
  Speaker 2: So I taught a Coursera course in the hope that
• 1:11:38
  Speaker 2: I'd make lots of money that way, but there was
• 1:11:40
  Speaker 2: no money in that. So I figured out, well, the
• 1:11:43
  Speaker 2: only way to get millions of dollars is to sell
• 1:11:48
  Speaker 2: myself to a big company. And so when I was 65,
• 1:11:54
  Speaker 2: fortunately for me, I had two brilliant students who produced
• 1:11:58
  Speaker 2: something called AlexNet, which was neural net that was very
• 1:12:01
  Speaker 2: good at recognizing objects and images. And so Ilya and
• 1:12:07
  Speaker 2: Alex and I set up a little company and auctioned it.
• 1:12:12
  Speaker 2: And we actually set up an auction where we had
• 1:12:14
  Speaker 2: a number of big companies bidding for us.
• 1:12:18
  Speaker 1: And that company was called AlexNet?
• 1:12:21
  Speaker 2: No, the network that recognized objects was called AlexNet. The
• 1:12:25
  Speaker 2: company was called DNN Research, Deep Neural Network Research.
• 1:12:29
  Speaker 1: And it was doing things like this. I'll put this
• 1:12:30
  Speaker 1: graph up on the screen.
• 1:12:31
  Speaker 2: Yeah, that's AlexNet.
• 1:12:33
  Speaker 1: This picture shows eight images and AlexNet's ability, which is
• 1:12:38
  Speaker 1: your company's ability, to spot what was in those images.
• 1:12:41
  Speaker 4: Yeah.
• 1:12:42
  Speaker 2: So it could tell the difference between various kinds of mushroom.
• 1:12:46
  Speaker 2: And about 12% of ImageNet is dogs. And to be
• 1:12:50
  Speaker 2: good at ImageNet, you have to tell the difference between
• 1:12:52
  Speaker 2: very similar kinds of dog. And it would got to
• 1:12:55
  Speaker 2: be very good at that.
• 1:12:57
  Speaker 1: And your company AlexNet won several awards, I believe, for
• 1:13:01
  Speaker 1: its ability to outperform its competitors. And so Google ultimately
• 1:13:05
  Speaker 1: ended up acquiring your technology. Google acquired that technology and
• 1:13:10
  Speaker 1: some other technology. And you went to work at Google
• 1:13:13
  Speaker 1: at age, what, 66?
• 1:13:15
  Speaker 2: I went at age 65 to work at Google. 65.
• 1:13:19
  Speaker 1: And you left at age 76?
• 1:13:20
  Speaker 3: 75.
• 1:13:21
  Speaker 2: 75, okay. I worked there for more or less exactly
• 1:13:23
  Speaker 2: 10 years.
• 1:13:24
  Speaker 1: And what were you doing there?
• 1:13:26
  Speaker 2: Okay, they were very nice to me. They said pretty much,
• 1:13:29
  Speaker 2: you can do what you like. I worked on something
• 1:13:32
  Speaker 2: called distillation that did really work well. And that's now
• 1:13:36
  Speaker 2: used all the time. In AI?
• 1:13:37
  Speaker 1: In AI.
• 1:13:38
  Speaker 2: In AI, and distillation is a way of taking what
• 1:13:40
  Speaker 2: a big model knows, a big neural net knows, and
• 1:13:44
  Speaker 2: getting that knowledge into a small neural net. Then at
• 1:13:46
  Speaker 2: the end, I got very interested in analog computation and
• 1:13:50
  Speaker 2: whether it would be possible to get these big language
• 1:13:52
  Speaker 2: models running in analog hardware so they used much less energy.
• 1:13:57
  Speaker 2: And it was when I was doing that work that
• 1:14:00
  Speaker 2: I began to really realize how much better digital is
• 1:14:03
  Speaker 2: for sharing information.
• 1:14:05
  Speaker 1: Was there a eureka moment?
• 1:14:08
  Speaker 2: There was a Eureka month or two, and it was
• 1:14:11
  Speaker 2: a sort of coupling of ChatGPT coming out, although Google
• 1:14:15
  Speaker 2: had very similar things a year earlier. And I'd seen those,
• 1:14:19
  Speaker 2: and that had a big effect on me. The closest
• 1:14:22
  Speaker 2: I had to a Eureka moment was when a Google
• 1:14:26
  Speaker 2: system called Palm was able to say why a joke
• 1:14:29
  Speaker 2: was funny. And I'd always thought of that as a
• 1:14:32
  Speaker 2: kind of landmark. If it can say why a joke's funny,
• 1:14:35
  Speaker 2: it really does understand. And it could say why a
• 1:14:37
  Speaker 2: joke was funny. And that coupled with realizing why digital
• 1:14:44
  Speaker 2: is so much better than analog for sharing information suddenly
• 1:14:47
  Speaker 2: made me very interested in AI safety and that these
• 1:14:52
  Speaker 2: things were going to get a lot smarter than us.
• 1:14:55
  Speaker 1: Why did you leave Google?
• 1:14:57
  Speaker 2: The main reason I left Google was because I was
• 1:15:00
  Speaker 2: 75 and I wanted to retire. I've done a very
• 1:15:03
  Speaker 2: bad job of that. The precise timing of when I
• 1:15:06
  Speaker 2: left Google was so that I could talk freely at
• 1:15:09
  Speaker 2: a conference at MIT. But I left because I'm old
• 1:15:14
  Speaker 2: and I was finding it harder to program. I was
• 1:15:16
  Speaker 2: making many more mistakes when I programmed, which is very annoying.
• 1:15:19
  Speaker 1: You wanted to talk freely at a conference at MIT?
• 1:15:22
  Speaker 2: Yes, organized by MIT Tech Review.
• 1:15:25
  Speaker 1: What did you want to talk about freely?
• 1:15:26
  Speaker 2: AI safety.
• 1:15:28
  Speaker 1: And you couldn't do that while you were at Google?
• 1:15:30
  Speaker 2: Well, I could have done it while I was at Google.
• 1:15:32
  Speaker 2: And Google encouraged me to stay and work on AI
• 1:15:33
  Speaker 2: safety and said I could do whatever I liked on
• 1:15:36
  Speaker 2: AI safety. You kind of censor yourself. If you work
• 1:15:39
  Speaker 2: for a big company, you don't feel right saying things
• 1:15:43
  Speaker 2: that'll damage the big company. Even if you could get
• 1:15:46
  Speaker 2: away with it, it just feels wrong to me. I
• 1:15:49
  Speaker 2: didn't leave because I was crossed with anything Google was doing.
• 1:15:51
  Speaker 2: I think Google actually behaved very responsibly. When they had
• 1:15:54
  Speaker 2: these big chatbots, they didn't release them, possibly because they
• 1:15:58
  Speaker 2: were worried about their reputation. They had a very good
• 1:16:01
  Speaker 2: reputation and they didn't want to damage it. So OpenAI
• 1:16:05
  Speaker 2: didn't have a reputation and so they could afford to
• 1:16:07
  Speaker 2: take the gamble.
• 1:16:08
  Speaker 1: I mean, there's also big conversation happening around how it
• 1:16:11
  Speaker 1: will cannibalize their core business and search there is now
• 1:16:14
  Speaker 1: yes yeah yeah and it's the older innovators dilemmas to
• 1:16:18
  Speaker 1: some degree i guess exactly yes good ending with bad
• 1:16:21
  Speaker 1: skin i've had it and i'm sure many of you
• 1:16:23
  Speaker 1: listening have had it too or maybe you have it
• 1:16:26
  Speaker 1: right now i know how draining it can be especially
• 1:16:29
  Speaker 1: if you're in a job where you're presenting often like
• 1:16:32
  Speaker 1: i am so let me tell you about something that's
• 1:16:34
  Speaker 1: helped both my partner and me and my sister which
• 1:16:36
  Speaker 1: is red light therapy i only got into this a
• 1:16:39
  Speaker 1: couple of years ago but I wish I'd known a
• 1:16:41
  Speaker 1: little bit sooner. I've been using our show sponsors Bond
• 1:16:44
  Speaker 1: Charge's infrared sauna blanket for a while now, but I
• 1:16:47
  Speaker 1: just got hold of their red light therapy mask as well.
• 1:16:50
  Speaker 1: Red light has been proven to have so many benefits
• 1:16:52
  Speaker 1: for the body. Like any area of your skin that's exposed,
• 1:16:55
  Speaker 1: we'll see a reduction in scarring, wrinkles, and even blemishes.
• 1:16:58
  Speaker 1: It also helps with complexion. It boosts collagen and it
• 1:17:01
  Speaker 1: does that by targeting the upper layers of your skin.
• 1:17:04
  Speaker 1: And Bond Charge ships worldwide with easy returns, and a
• 1:17:07
  Speaker 1: year-long warranty on all of their products. So if you'd
• 1:17:10
  Speaker 1: like to try it yourself, head over to bondcharge.com slash
• 1:17:12
  Speaker 1: diary and use code diary for 25% off any product site-wide.
• 1:17:18
  Speaker 1: Just make sure you order through this link, bondcharge.com slash
• 1:17:21
  Speaker 1: diary with code diary. Make sure you keep what I'm
• 1:17:25
  Speaker 1: about to say to yourself. I'm inviting 10,000 of you
• 1:17:28
  Speaker 1: to come even deeper into the diary of a CEO.
• 1:17:30
  Speaker 1: Welcome to my inner circle. This is a brand new
• 1:17:34
  Speaker 1: private community that I'm launching to the world. We have
• 1:17:36
  Speaker 1: so many incredible things that happen that you are never shown.
• 1:17:40
  Speaker 1: We have the briefs that are on my iPad when
• 1:17:42
  Speaker 1: I'm recording the conversation. We have clips we've never released.
• 1:17:45
  Speaker 1: We have behind the scenes conversations with the guests and
• 1:17:47
  Speaker 1: also the episodes that we've never, ever released. And so
• 1:17:51
  Speaker 1: much more. In the circle, you'll have direct access to me.
• 1:17:55
  Speaker 1: You can tell us what you want this show to be,
• 1:17:57
  Speaker 1: who you want us to interview, and the types of
• 1:17:59
  Speaker 1: conversations you would love us to have. But remember, for now,
• 1:18:02
  Speaker 1: we're only inviting the first 10,000 people that join before
• 1:18:05
  Speaker 1: it closes. So if you want to join our private
• 1:18:07
  Speaker 1: closed community, head to the link in the description below
• 1:18:10
  Speaker 1: or go to doaccircle.com. I will speak to you then.
• 1:18:16
  Speaker 1: I'm continually shocked by the types of individuals that listen
• 1:18:18
  Speaker 1: to this conversation. because they come up to me sometimes.
• 1:18:21
  Speaker 1: So I hear from politicians, I hear from some rural people,
• 1:18:23
  Speaker 1: I hear from entrepreneurs all over the world, whether they
• 1:18:26
  Speaker 1: are the entrepreneurs building some of the biggest companies in
• 1:18:27
  Speaker 1: the world or they're early stage startups. For those people
• 1:18:32
  Speaker 1: that are listening to this conversation now, that are in
• 1:18:35
  Speaker 1: positions of power and influence, world leaders, let's say, what's
• 1:18:40
  Speaker 1: your message to them?
• 1:18:42
  Speaker 2: I'd say what you need is highly regulated capitalism. That's
• 1:18:45
  Speaker 2: what seems to work best.
• 1:18:47
  Speaker 1: And what would you say to the average person? Doesn't
• 1:18:50
  Speaker 1: work in the industry, somewhat concerned about the future, doesn't
• 1:18:54
  Speaker 1: know if they're helpless or not. What should they be
• 1:18:57
  Speaker 1: doing in their own lives?
• 1:18:59
  Speaker 2: My feeling is there's not much they can do. This
• 1:19:02
  Speaker 2: isn't going to be decided by, just as climate change
• 1:19:06
  Speaker 2: isn't going to be decided by people separating out the
• 1:19:09
  Speaker 2: plastic bags from the compostables, that's not going to have
• 1:19:13
  Speaker 2: much effect. It's going to be decided by whether the
• 1:19:16
  Speaker 2: lobbyists for the big energy companies can be kept under control.
• 1:19:20
  Speaker 2: I don't think there's much people can do to accept for,
• 1:19:26
  Speaker 2: try and pressure their governments to force the big companies
• 1:19:31
  Speaker 2: to work on AI safety. That they can do.
• 1:19:36
  Speaker 1: You've lived a fascinating, fascinating, winding life. I think one
• 1:19:40
  Speaker 1: of the things most people don't know about you is
• 1:19:42
  Speaker 1: that your family has a big history of being involved
• 1:19:47
  Speaker 1: in tremendous things. You have a family tree, which is
• 1:19:50
  Speaker 1: one of the most impressive that I've ever seen or
• 1:19:52
  Speaker 1: read about. Your great, great grandfather, George Ball, founded the
• 1:19:58
  Speaker 1: Boolean algebra logic, which is one of the foundational principles
• 1:20:01
  Speaker 1: of modern computer science. You have your great, great grandmother,
• 1:20:05
  Speaker 1: Mary Everest Ball, who was a mathematician and educator who
• 1:20:09
  Speaker 1: made huge leaps forward in mathematics from what I was
• 1:20:12
  Speaker 1: able to ascertain. I mean, the list goes on and
• 1:20:15
  Speaker 1: on and on. I mean, your great-great-uncle, George Everest, is
• 1:20:19
  Speaker 1: what Mount Everest is named after. Is that correct?
• 1:20:24
  Speaker 2: I think he's my great-great-great-uncle. His niece married George Bull.
• 1:20:33
  Speaker 2: So Mary Bull was Mary Everest Bull. She was the
• 1:20:37
  Speaker 2: niece of Everest.
• 1:20:39
  Speaker 1: And your first cousin once removed, Joan Hinton, was involved
• 1:20:42
  Speaker 1: in a nuclear physicist who worked on the Manhattan Project,
• 1:20:46
  Speaker 1: which is the World War II development of the first
• 1:20:48
  Speaker 1: nuclear bomb.
• 1:20:49
  Speaker 2: Yeah, she was one of the two female physicists at
• 1:20:52
  Speaker 2: Las Almas. And then, after they dropped the bomb, she
• 1:20:56
  Speaker 2: moved to China.
• 1:20:58
  Speaker 1: Why?
• 1:20:59
  Speaker 2: She was very cross with them dropping the bomb. And
• 1:21:02
  Speaker 2: her family had a lot of links with China. Her mother...
• 1:21:06
  Speaker 2: was friends with Chairman Mao. Quite weird.
• 1:21:13
  Speaker 1: When you look back at your life, Jeffrey, with the
• 1:21:17
  Speaker 1: hindsight you have now and the retrospective clarity, what might
• 1:21:23
  Speaker 1: you have done differently if you were advising me?
• 1:21:26
  Speaker 2: I guess I have two pieces of advice. One is,
• 1:21:31
  Speaker 2: if you have an intuition that people are doing things
• 1:21:35
  Speaker 2: wrong and there's a better way to do things. Don't
• 1:21:37
  Speaker 2: give up on that intuition just because people say it's silly.
• 1:21:41
  Speaker 2: Don't give up on the intuition until you've figured out
• 1:21:44
  Speaker 2: why it's wrong. Figure it out for yourself why that
• 1:21:46
  Speaker 2: intuition isn't correct. And usually it's wrong if it disagrees
• 1:21:52
  Speaker 2: with everybody else and you'll eventually figure out why it's wrong.
• 1:21:56
  Speaker 2: But just occasionally you'll have an intuition that's actually right
• 1:22:00
  Speaker 2: and everybody else is wrong. And I lucked out that
• 1:22:03
  Speaker 2: way Early on, I thought neural nets are definitely the
• 1:22:06
  Speaker 2: way to go to make AI. And almost everybody said
• 1:22:11
  Speaker 2: that was crazy. And I stuck with it because I couldn't,
• 1:22:14
  Speaker 2: it just seemed to me it was obviously right. Now,
• 1:22:18
  Speaker 2: the idea that you should stick with your intuitions isn't
• 1:22:21
  Speaker 2: going to work if you have bad intuitions. But if
• 1:22:24
  Speaker 2: you have bad intuitions, you're never going to do anything anyway,
• 1:22:26
  Speaker 2: so you might as well stick with them.
• 1:22:30
  Speaker 1: And in your own career journey, is there anything you
• 1:22:32
  Speaker 1: look back on and say, with the hindsight I have now,
• 1:22:35
  Speaker 1: I should have taken a different approach at that juncture?
• 1:22:39
  Speaker 2: I wish I spent more time with my wife. And
• 1:22:47
  Speaker 2: with my children when they were little. I was kind
• 1:22:51
  Speaker 2: of obsessed with work.
• 1:22:55
  Speaker 1: Your wife passed away?
• 1:22:56
  Speaker 2: Yeah.
• 1:22:57
  Speaker 1: From ovarian cancer?
• 1:22:59
  Speaker 2: No, that was another wife. I had two wives to
• 1:23:03
  Speaker 2: have cancer. Oh, really? Sorry. The first one died of
• 1:23:06
  Speaker 2: ovarian cancer and the second one died of pancreatic cancer.
• 1:23:08
  Speaker 1: And you wish you'd spent more time with her?
• 1:23:10
  Speaker 2: With the second wife, yeah, who was a wonderful person.
• 1:23:15
  Speaker 1: Why do you say that in your 70s? What is
• 1:23:17
  Speaker 1: it that you've figured out that I might not know yet?
• 1:23:21
  Speaker 2: Oh, just because she's gone and I can't spend more
• 1:23:23
  Speaker 2: time with her now.
• 1:23:26
  Speaker 1: But you didn't know that at the time?
• 1:23:30
  Speaker 2: At the time, you think, I mean, it was likely
• 1:23:34
  Speaker 2: I would die before her just because she was a
• 1:23:36
  Speaker 2: woman and I was a man. I didn't, I just
• 1:23:41
  Speaker 2: didn't spend enough time when I could.
• 1:23:44
  Speaker 1: I think I inquire there because I think there's many
• 1:23:46
  Speaker 1: of us that are so consumed with what we're doing
• 1:23:48
  Speaker 1: professionally that we kind of assume immortality with our partners
• 1:23:52
  Speaker 1: because they've always been there.
• 1:23:53
  Speaker 2: Yeah. She was very supportive of me spending a lot
• 1:23:56
  Speaker 2: of time working, but.
• 1:23:59
  Speaker 1: And why do you say your children as well? What's the,
• 1:24:01
  Speaker 1: what's the.
• 1:24:02
  Speaker 2: Well, I didn't spend enough time with them when they
• 1:24:03
  Speaker 2: were little.
• 1:24:05
  Speaker 1: And you regret that now?
• 1:24:07
  Speaker 2: Yeah.
• 1:24:13
  Speaker 1: If you had a closing message for my listeners about
• 1:24:16
  Speaker 1: AI and AI safety, what would that be, Geoffrey?
• 1:24:20
  Speaker 2: There's still a chance that we can figure out how
• 1:24:23
  Speaker 2: to develop AI that won't want to take over from us.
• 1:24:27
  Speaker 2: And because there's a chance... We should put enormous resources
• 1:24:30
  Speaker 2: into trying to figure that out. Because if we don't,
• 1:24:32
  Speaker 2: it's going to take over.
• 1:24:34
  Speaker 1: And are you hopeful?
• 1:24:36
  Speaker 2: I just don't know. I'm agnostic.
• 1:24:40
  Speaker 1: You must get in bed at night. And when you're
• 1:24:43
  Speaker 1: thinking to yourself about probabilities of outcomes, there must be
• 1:24:47
  Speaker 1: a bias in one direction. Because there certainly is for me.
• 1:24:50
  Speaker 1: I mean, imagine everyone listening now has a internal prediction
• 1:24:55
  Speaker 1: that they might not say out loud, but of how
• 1:24:57
  Speaker 1: they think it's going to play out.
• 1:24:59
  Speaker 2: I really don't know. I genuinely don't know. I think
• 1:25:02
  Speaker 2: it's incredibly uncertain. When I'm feeling slightly depressed, I think
• 1:25:07
  Speaker 2: people are toast. The air is going to take over.
• 1:25:10
  Speaker 2: When I'm feeling cheerful, I think we'll figure out a way.
• 1:25:13
  Speaker 1: Maybe one of the facets of being a human is
• 1:25:16
  Speaker 1: because we've always been here, like we were saying about
• 1:25:19
  Speaker 1: our loved ones and our relationships, we assume casually that
• 1:25:23
  Speaker 1: we will always be here and we'll always figure everything out.
• 1:25:25
  Speaker 1: But there's a beginning and an end to everything, as
• 1:25:28
  Speaker 1: we saw from the dinosaurs.
• 1:25:29
  Speaker 2: I mean, Yeah. And we have to face the possibility
• 1:25:34
  Speaker 2: that unless we do something soon, we're near the end.
• 1:25:42
  Speaker 1: We have a closing tradition on this podcast where the
• 1:25:44
  Speaker 1: last guest leaves a question in their diary. And the
• 1:25:47
  Speaker 1: question that they've left for you is, with everything that
• 1:25:55
  Speaker 1: you see ahead of us, What is the biggest threat
• 1:25:59
  Speaker 1: you see to human happiness?
• 1:26:04
  Speaker 2: I think the joblessness is a fairly urgent short-term threat
• 1:26:08
  Speaker 2: to human happiness. I think if you make lots and
• 1:26:11
  Speaker 2: lots of people unemployed, even if they get universal basic income,
• 1:26:17
  Speaker 2: they're not going to be happy.
• 1:26:19
  Speaker 1: Because they need purpose.
• 1:26:21
  Speaker 2: Because they need purpose, yes.
• 1:26:23
  Speaker 1: And struggle.
• 1:26:23
  Speaker 2: They need to feel they're contributing something. They're useful.
• 1:26:27
  Speaker 1: And do you think that outcome, that there's going to
• 1:26:29
  Speaker 1: be huge job displacement, is more probable than not?
• 1:26:32
  Speaker 2: Yes, I do. That one I think is definitely more
• 1:26:36
  Speaker 2: probable than not. If I worked in a call centre,
• 1:26:39
  Speaker 2: I'd be terrified.
• 1:26:41
  Speaker 1: And what's the timeframe for that in terms of mass
• 1:26:43
  Speaker 1: job displacement?
• 1:26:43
  Speaker 2: I think it's beginning to happen already. I read an
• 1:26:46
  Speaker 2: article in The Atlantic recently that said it's already getting
• 1:26:50
  Speaker 2: hard for university graduates to get jobs and Part of
• 1:26:55
  Speaker 2: that may be that people are already using AI for
• 1:26:57
  Speaker 2: the jobs they would have got.
• 1:27:00
  Speaker 1: I spoke to the CEO of a major company that
• 1:27:02
  Speaker 1: everyone will know of, lots of people use. And he
• 1:27:05
  Speaker 1: said to me in DMs that they used to have
• 1:27:08
  Speaker 1: just over 7,000 employees. He said by last year, they
• 1:27:11
  Speaker 1: were down to, I think, 5,000. He said right now
• 1:27:14
  Speaker 1: they have 3,600. And he said by the end of summer,
• 1:27:16
  Speaker 1: because of AI agents, they'll be down to 3,000.
• 1:27:19
  Speaker 2: So it's happening already.
• 1:27:21
  Speaker 1: Yes. He's halved his workforce because AI agents can now
• 1:27:24
  Speaker 1: handle 80% of the customer service inquiries and other things.
• 1:27:29
  Speaker 1: So it's happening already. So urgent action is needed.
• 1:27:33
  Speaker 2: Yep.
• 1:27:33
  Speaker 1: I don't know what that urgent action is.
• 1:27:36
  Speaker 2: That's a tricky one because that depends very much on
• 1:27:38
  Speaker 2: the political system. And political systems are all going in
• 1:27:42
  Speaker 2: the wrong direction at present.
• 1:27:44
  Speaker 1: And what do we need to do? Save up money? Like,
• 1:27:46
  Speaker 1: do we save money? Do we move to another part
• 1:27:47
  Speaker 1: of the world?
• 1:27:49
  Speaker 2: I don't know.
• 1:27:50
  Speaker 1: What would you tell your kids to do? They said, Dad, look,
• 1:27:53
  Speaker 1: there's going to be loads of job displacement.
• 1:27:56
  Speaker 2: Because I worked for Google for 10 years, they have
• 1:27:58
  Speaker 2: enough money.
• 1:27:59
  Speaker 1: Okay. So they're not typical. What if they didn't have money?
• 1:28:04
  Speaker 2: Train to be a plumber. Really? Yeah.
• 1:28:11
  Speaker 1: Geoffrey, thank you so much. You're the first Nobel Prize
• 1:28:13
  Speaker 1: winner that I've ever had a conversation with, I think,
• 1:28:17
  Speaker 1: in my life. So that's a tremendous honor. And you
• 1:28:21
  Speaker 1: received that award for a lifetime of exceptional work and
• 1:28:23
  Speaker 1: pushing the world forward in so many profound ways that
• 1:28:25
  Speaker 1: will lead to great and that have led to great
• 1:28:28
  Speaker 1: advancements and things that matter so much to us. And
• 1:28:30
  Speaker 1: now you've turned this season in your life to shining
• 1:28:33
  Speaker 1: a light on some of your own work, but also
• 1:28:36
  Speaker 1: on the the broader risks of ai and how um
• 1:28:40
  Speaker 1: and how it might impact us adversely and there's very
• 1:28:42
  Speaker 1: few people that have worked inside the machine of a
• 1:28:46
  Speaker 1: google or a big tech company that have contributed to
• 1:28:48
  Speaker 1: the field of ai that are now at the very
• 1:28:51
  Speaker 1: forefront of warning us against the very thing that they
• 1:28:53
  Speaker 1: worked upon.
• 1:28:55
  Speaker 2: There are actually a surprising number.
• 1:28:57
  Speaker 1: Of us now they're not as uh And they're actually
• 1:29:02
  Speaker 1: quite hard to get to have these kinds of conversations
• 1:29:04
  Speaker 1: because many of them are still in that industry. So,
• 1:29:07
  Speaker 1: you know, someone who tries to contact these people often
• 1:29:09
  Speaker 1: and invites them to have conversations, they often are a
• 1:29:11
  Speaker 1: little bit hesitant to speak openly. So they speak privately,
• 1:29:15
  Speaker 1: but they're less willing to openly because maybe they still
• 1:29:17
  Speaker 1: have some sort of incentives at play.
• 1:29:20
  Speaker 2: I have an advantage over them, which is I'm older,
• 1:29:23
  Speaker 2: so I'm unemployed, so I can say what I have.
• 1:29:25
  Speaker 1: Well, there you go. So thank you for doing what
• 1:29:27
  Speaker 1: you do. It's a real honour. And please do continue
• 1:29:29
  Speaker 1: to do it. Thank you. Thank you so much.
• 1:29:35
  Speaker 2: People think I'm joking when I say that, but I'm not.
• 1:29:37
  Speaker 1: The plumbing fish.
• 1:29:38
  Speaker 2: Yeah. And plumbers are pretty well paid. Thank you.