Latest / Faster, Please! — The Podcast / 🤖🧠My chat (+transcript) with Google DeepMind's Séb Krier on AGI and public policy
Transcript
- 0:06SPEAKER_01: Welcome to Faster Please, the podcast.
- 0:09SPEAKER_01: I'm your host, Jim Pethokoukis.
- 0:12SPEAKER_01: Several times a month,
- 0:13SPEAKER_01: I'll feature a lively conversation with a fascinating and provocative guest about
- 0:18SPEAKER_01: how to make the world a better place through scientific discovery,
- 0:21SPEAKER_01: technological innovation,
- 0:23SPEAKER_01: and economic growth.
- 0:24SPEAKER_01: You're also going to want to check out my Faster Please newsletter.
- 0:27SPEAKER_01: You're on Substack throughout the week for fresh essays,
- 0:30SPEAKER_01: Q&As,
- 0:31SPEAKER_01: and stories from around the internet and around the world.
- 0:35SPEAKER_01: In a world of artificial general intelligence,
- 0:38SPEAKER_01: machines will be able to match or even exceed human cognitive abilities.
- 0:45SPEAKER_01: AGI might still be science fiction,
- 0:47SPEAKER_01: but Seb Krier sees this technology as not only possible,
- 0:50SPEAKER_01: but inevitable.
- 0:52SPEAKER_01: Today on Faster Please,
- 0:53SPEAKER_01: the podcast,
- 0:55SPEAKER_01: I chat with Krier about how our public policy should facilitate AGI's arrival and flourishing.
- 1:01SPEAKER_01: Krier is an AI policy expert, advisor, and attorney.
- 1:04SPEAKER_01: He currently works in policy development and strategy at Google DeepMind.
- 1:09SPEAKER_01: He previously served as head of regulation for the UK government's Office for
- 1:13SPEAKER_01: Artificial Intelligence and was a senior tech policy researcher at Stanford Cyber
- 1:18SPEAKER_01: Policy Center.
- 1:19SPEAKER_01: Seb, welcome to the podcast.
- 1:21SPEAKER_01: Thank you.
- 1:22SPEAKER_01: Great to be here.
- 1:23SPEAKER_01: Let's start with a bit of context that may influence the rest of the conversation.
- 1:31SPEAKER_01: So what is the vision or image of the future regarding...
- 1:40SPEAKER_01: AI, you can find it as machine learning or generative AI.
- 1:45SPEAKER_01: Like that excites you,
- 1:47SPEAKER_01: that makes it like,
- 1:49SPEAKER_01: that gets you going in the day that you feel like you're part of something important.
- 1:54SPEAKER_01: What is sort of that vision?
- 1:57SPEAKER_00: Right,
- 1:57SPEAKER_00: so I think it's a great question,
- 1:59SPEAKER_00: but so in my mind,
- 2:00SPEAKER_00: I think,
- 2:01SPEAKER_00: you know,
- 2:01SPEAKER_00: when AI has been going on for quite a long time,
- 2:04SPEAKER_00: but I think the aim has always been,
- 2:06SPEAKER_00: you know,
- 2:07SPEAKER_00: artificial general intelligence.
- 2:09SPEAKER_00: And in a sense, I think of this as a huge deal.
- 2:12SPEAKER_00: And the vision I have for the future is being able to have a very,
- 2:18SPEAKER_00: very large supply of cognitive resources that you can allocate to quite a wide
- 2:23SPEAKER_00: range of different problems,
- 2:24SPEAKER_00: whether that's
- 2:25SPEAKER_00: energy, health care, governance.
- 2:29SPEAKER_00: There's many, many ways in which this technology can be applied as a general purpose technology.
- 2:34SPEAKER_00: And so I guess my vision is seeing that being used to solve quite a wide range of
- 2:39SPEAKER_00: problems that humans have had for decades,
- 2:43SPEAKER_00: centuries,
- 2:43SPEAKER_00: millennia.
- 2:44SPEAKER_00: And I think you could go in so many different directions with that,
- 2:47SPEAKER_00: whether it's curing diseases or optimizing energy grids and more.
- 2:52SPEAKER_00: But I think broadly, that's, I guess, the way I think about it.
- 2:56SPEAKER_00: So the objective, I guess, in a sense, is safe AGI.
- 3:00SPEAKER_00: And I think from that, I think you can go even further.
- 3:02SPEAKER_00: And I think in many ways, this can be hugely beneficial to science, R&D, and humanity as a whole.
- 3:10SPEAKER_00: Of course, that also comes with ways in which this could be misused or accidents and so on.
- 3:14SPEAKER_00: And so, you know, huge emphasis on the safe development of AGI.
- 3:17SPEAKER_01: Sure.
- 3:19SPEAKER_01: So you're viewing it as...
- 3:22SPEAKER_01: way of as a tool as a way to apply intelligence across a variety of fields a
- 3:30SPEAKER_01: variety of problems to solve those problems and of course the the word in there uh
- 3:38SPEAKER_01: doing a lot of lifting is safely given the discussion of the past 18 months about
- 3:43SPEAKER_01: that word safely is
- 3:47SPEAKER_01: One would I think someone someone who maybe only pays passing attention to this
- 3:52SPEAKER_01: issue might think that it's almost impossible to do it safely without jeopardizing
- 4:00SPEAKER_01: all those upside benefits.
- 4:02SPEAKER_01: But you're confident that those two things can ultimately be in harmony.
- 4:06SPEAKER_00: Yeah, absolutely.
- 4:07SPEAKER_00: Otherwise, I wouldn't be.
- 4:09SPEAKER_00: working on an AGI policy.
- 4:11SPEAKER_00: I'm very confident this can be done well.
- 4:15SPEAKER_00: I think it also depends what we mean by safety and what kind of safety we have in mind.
- 4:19SPEAKER_00: Any technology will have costs and trade-offs.
- 4:22SPEAKER_00: Of course, the upside here is enormous, and in my mind, very much outweighs potential downsides.
- 4:28SPEAKER_00: However,
- 4:29SPEAKER_00: I think for certain risks,
- 4:31SPEAKER_00: things like potentially catastrophic risk and so on,
- 4:33SPEAKER_00: there is an argument in
- 4:35SPEAKER_00: treading some careful,
- 4:36SPEAKER_00: careful path,
- 4:37SPEAKER_00: and making sure you know,
- 4:38SPEAKER_00: this is done kind of scientifically with scientific method in mind,
- 4:41SPEAKER_00: I guess,
- 4:42SPEAKER_00: and in doing that,
- 4:42SPEAKER_00: well,
- 4:43SPEAKER_00: but I don't think there's fundamentally a necessary tension anything.
- 4:47SPEAKER_00: In fact, what many people sometimes underestimate is how AI itself as a technology will be helpful in
- 4:56SPEAKER_00: mitigating a lot of the risks we're kind of foreseeing and thinking about.
- 4:59SPEAKER_00: So,
- 4:59SPEAKER_00: you know,
- 5:00SPEAKER_00: there's obviously ways in which kind of AI can be used for cyber offense,
- 5:03SPEAKER_00: but many ways in which you can also use that for defense,
- 5:05SPEAKER_00: for example.
- 5:07SPEAKER_00: So,
- 5:07SPEAKER_00: yeah,
- 5:08SPEAKER_00: I definitely,
- 5:08SPEAKER_00: you know,
- 5:09SPEAKER_00: I'm kind of cautiously optimistic about how this can be developed and used in the
- 5:13SPEAKER_00: long run.
- 5:15SPEAKER_01: Since these large language models and chatbots were sort of rolled out to public
- 5:22SPEAKER_01: awareness in late 2022,
- 5:24SPEAKER_01: I think I may have added an extra 20,
- 5:25SPEAKER_01: 2022,
- 5:26SPEAKER_01: has the sort of safety regulatory debate continued?
- 5:33SPEAKER_01: changed in any way it seems to me that there was a lot of talk early on about sort
- 5:37SPEAKER_01: of these existential risks now the risks now i think i seem to be hearing sort of
- 5:42SPEAKER_01: less about that and more about issues about maybe it's disinformation or bias from
- 5:48SPEAKER_01: your perspective has that debate changed and has it changed for the better or worse
- 5:53SPEAKER_00: Yeah,
- 5:54SPEAKER_00: I think it has evolved quite a lot over the past,
- 5:56SPEAKER_00: you know,
- 5:56SPEAKER_00: I've been working in AI policy since 2017,
- 5:58SPEAKER_00: and there's been kind of different phases.
- 6:00SPEAKER_00: And,
- 6:01SPEAKER_00: you know,
- 6:01SPEAKER_00: at first,
- 6:02SPEAKER_00: a lot of skepticism around kind of AI even being kind of useful or,
- 6:05SPEAKER_00: you know,
- 6:05SPEAKER_00: hype and so on,
- 6:06SPEAKER_00: and then kind of seeing more and more of these kind of what these general models
- 6:09SPEAKER_00: could do.
- 6:10SPEAKER_00: And I think,
- 6:12SPEAKER_00: you know,
- 6:12SPEAKER_00: initially,
- 6:12SPEAKER_00: a lot of the concerns were around things like bias and discrimination and errors.
- 6:16SPEAKER_00: So even things like, you know, early on, facial recognition technologies were very kind of
- 6:21SPEAKER_00: problematic in many ways,
- 6:22SPEAKER_00: not in just ways in which they were applied,
- 6:23SPEAKER_00: but they were prone to a lot of errors and biases that could become unfair,
- 6:27SPEAKER_00: whereas they're much better now,
- 6:28SPEAKER_00: and therefore the concern now is more on misuse than it accidentally misidentifying someone,
- 6:32SPEAKER_00: I would say.
- 6:33SPEAKER_00: So I think in that sense, these things have changed.
- 6:35SPEAKER_00: And then a lot of the discourse around existential risk and so on,
- 6:39SPEAKER_00: there was a bit of a peak a bit last year,
- 6:41SPEAKER_00: and then this switched a bit towards more catastrophic risks and misuse.
- 6:46SPEAKER_00: I think...
- 6:48SPEAKER_00: There's a few different things.
- 6:48SPEAKER_00: So broadly, I think it's good that these risks are taken seriously.
- 6:51SPEAKER_00: So in some sense, I'm happy that these
- 6:53SPEAKER_00: have taken more space in a way.
- 6:55SPEAKER_00: But I think there's also been a lot of alarmism and unnecessary kind of doomerism
- 7:01SPEAKER_00: of conquering worlds a little bit too early.
- 7:02SPEAKER_00: I think what happens is that sometimes people also conflate a capability of a
- 7:07SPEAKER_00: system and how that fits within a wider risk or threat model or something.
- 7:13SPEAKER_00: And the latter is often kind of underdefined.
- 7:15SPEAKER_00: And there's a tendency of people to often kind of see the worst in technology,
- 7:18SPEAKER_00: I think particularly in certain regions of the world.
- 7:21SPEAKER_00: And so I think sometimes a lot of it has been a little bit kind of exaggerated, overhyped.
- 7:24SPEAKER_00: But having said that, I think it's very good.
- 7:26SPEAKER_00: There's lots of research going on on the many ways in which this could potentially
- 7:30SPEAKER_00: be harmful to the research side,
- 7:32SPEAKER_00: the evaluation side.
- 7:33SPEAKER_00: There's a lot of great work.
- 7:35SPEAKER_00: We've published some papers on socio-technical evaluations, dangerous capabilities and so on.
- 7:40SPEAKER_00: And all of that is great.
- 7:42SPEAKER_00: But I think there has also been
- 7:44SPEAKER_00: some more polarized parts,
- 7:46SPEAKER_00: kind of calling for excessive measures,
- 7:48SPEAKER_00: whether regulatory or pausing AI and so on,
- 7:51SPEAKER_00: that I think have been a little bit too trigger-happy.
- 7:55SPEAKER_00: So I'm less happy about these bits, but yeah, there's been a lot of good as well.
- 8:01SPEAKER_01: And much debate about policy has been about the right sort of policy to prevent bad
- 8:08SPEAKER_01: things from happening.
- 8:09SPEAKER_01: How should we think about policy that maximizes the odds of good things happening?
- 8:16SPEAKER_01: What should policymakers do to help promote
- 8:19SPEAKER_01: AI to reshape science, to help promote AI diffusing as efficiently as possible throughout an economy?
- 8:27SPEAKER_01: Like sort of like how do we optimize the upside through policy rather than just
- 8:31SPEAKER_01: focusing on making sure the bad things don't happen?
- 8:34SPEAKER_00: I think the very first thing is not having, you know,
- 8:37SPEAKER_00: rushed regulation.
- 8:39SPEAKER_00: I'm not personally a huge fan of the precautionary principle.
- 8:43SPEAKER_00: And I think that very often, regulations can cause a lot of harm downstream.
- 8:49SPEAKER_00: And they're very sticky, hard to remove.
- 8:52SPEAKER_00: The other thing that you can do beyond avoiding bad policy
- 8:56SPEAKER_00: is,
- 8:56SPEAKER_00: you know,
- 8:57SPEAKER_00: I think a lot of the levers to making sure that the development goes well aren't
- 9:00SPEAKER_00: necessarily all directly AI related.
- 9:02SPEAKER_00: So it'll be things like immigration,
- 9:04SPEAKER_00: you know,
- 9:05SPEAKER_00: having kind of attracting a lot of talent,
- 9:07SPEAKER_00: for example,
- 9:08SPEAKER_00: I think will be very important.
- 9:10SPEAKER_00: So immigration is a big one.
- 9:11SPEAKER_00: Power and energy.
- 9:13SPEAKER_00: You want there to be a lot more kind of, you know, I'm a big fan of nuclear.
- 9:15SPEAKER_00: So I think that kind of thing is also very helpful in terms of the expected needs
- 9:20SPEAKER_00: for AI development in the future.
- 9:21SPEAKER_00: And then there are certain things governments could potentially do with some narrow
- 9:24SPEAKER_00: domains like advanced market commitments,
- 9:26SPEAKER_00: for example,
- 9:27SPEAKER_00: although,
- 9:27SPEAKER_00: you know,
- 9:28SPEAKER_00: that's not a panacea.
- 9:29SPEAKER_00: Commitments to do what?
- 9:31SPEAKER_00: Oh,
- 9:31SPEAKER_00: advanced market commitments,
- 9:32SPEAKER_00: like,
- 9:32SPEAKER_00: you know,
- 9:33SPEAKER_00: pull mechanisms,
- 9:34SPEAKER_00: I guess,
- 9:34SPEAKER_00: to like create a market for a particular solution.
- 9:36SPEAKER_00: So like,
- 9:37SPEAKER_00: you know,
- 9:37SPEAKER_00: operational warp speeds,
- 9:38SPEAKER_00: but you could have an AI,
- 9:40SPEAKER_00: you know,
- 9:40SPEAKER_00: equivalent for certain applications.
- 9:41SPEAKER_00: But of course,
- 9:42SPEAKER_00: like that,
- 9:42SPEAKER_00: you know,
- 9:43SPEAKER_00: there's a lot of parameters in doing that well,
- 9:44SPEAKER_00: and I wouldn't want a large industrial policy type kind of approach to AI.
- 9:49SPEAKER_00: But I think generally it's around kind of ensuring that all the enablers,
- 9:51SPEAKER_00: all the kind of different ingredients and factors of a rich research and
- 9:55SPEAKER_00: development ecosystem continue to thrive.
- 9:58SPEAKER_00: And so,
- 9:58SPEAKER_00: you know,
- 9:58SPEAKER_00: I think to a large extent,
- 10:00SPEAKER_00: avoiding bad regulation and ensuring that a lot of things like energy immigration
- 10:03SPEAKER_00: and so on go well is already a huge part of the battle.
- 10:06SPEAKER_01: How serious of a potential bottleneck is the energy issue?
- 10:10SPEAKER_01: It seems to me like it's a serious issue that's coming fast,
- 10:15SPEAKER_01: but the solutions seem like they'll take more time.
- 10:19SPEAKER_01: And I'm worried about the mismatch between sort of the problem and finding a solution to the problem.
- 10:27SPEAKER_00: Yeah, I think the, you know, I suspect that over the coming years, you know,
- 10:32SPEAKER_00: we will see more and more of these AI systems being increasingly useful,
- 10:36SPEAKER_00: capable,
- 10:37SPEAKER_00: and then integrated into kind of economic systems.
- 10:40SPEAKER_00: And I think as you start seeing these benefits more and more,
- 10:42SPEAKER_00: I think it'll be easier to make the case for why you need to solve some of these
- 10:46SPEAKER_00: kind of policy issues a bit faster.
- 10:49SPEAKER_00: And I also think these solutions aren't that difficult ultimately.
- 10:53SPEAKER_00: So I think there's a...
- 10:55SPEAKER_00: There's a lot that can be done around nuclear,
- 10:58SPEAKER_00: wind and solar and so on,
- 11:00SPEAKER_00: and many kind of processes and regulatory processes that could be kind of
- 11:03SPEAKER_00: simplified and accelerated and improved to kind of avoid the vitocracy system we're
- 11:09SPEAKER_00: in at the moment.
- 11:10SPEAKER_00: So I don't think the solutions are that difficult.
- 11:13SPEAKER_00: I think mustering the political will might be right now,
- 11:16SPEAKER_00: but I expect that to be less of a challenge in the coming years with AI showing
- 11:23SPEAKER_00: more and more promise,
- 11:23SPEAKER_00: I think.
- 11:24SPEAKER_01: Yeah,
- 11:24SPEAKER_01: I mean,
- 11:25SPEAKER_01: speaking of vitocracy,
- 11:27SPEAKER_01: whatever the exact substance of the regulation,
- 11:29SPEAKER_01: I am concerned,
- 11:30SPEAKER_01: at least in the United States,
- 11:32SPEAKER_01: that we have 50 states and perhaps even more bodies,
- 11:36SPEAKER_01: if you look at like cities,
- 11:38SPEAKER_01: who all have a lot of ideas about AI regulation.
- 11:41SPEAKER_01: And I'm extremely concerned that that sort of fractured policy landscape will create a bottleneck.
- 11:48SPEAKER_01: Can we get to where we need to go if that's the regulatory environment we are
- 11:53SPEAKER_01: looking at,
- 11:54SPEAKER_01: at least in the United States?
- 11:56SPEAKER_01: And does ultimately there need to be a federal regulation
- 12:01SPEAKER_01: I think the technical word is preemption of all those efforts.
- 12:03SPEAKER_01: So there is a federal approach and there aren't a federal approach plus a 50 state
- 12:08SPEAKER_01: approach plus 175 city approach to regulation.
- 12:12SPEAKER_01: Because if it's going to be what I just described,
- 12:14SPEAKER_01: that seems like a very difficult environment to deal with.
- 12:17SPEAKER_00: I'm not wildly optimistic around a patchwork of different state-level regulatory systems.
- 12:24SPEAKER_00: I think that will come with various externalities.
- 12:27SPEAKER_00: You'll have distortionary effects.
- 12:30SPEAKER_00: And it will be very difficult environment, I think, from a commercial perspective to operate in smoothly.
- 12:35SPEAKER_00: I think I'm a lot more
- 12:36SPEAKER_00: open to something at a federal level at some point,
- 12:40SPEAKER_00: rather than kind of a big patchwork of city level or state level regulation.
- 12:43SPEAKER_00: Now, it depends on exactly also what we're talking about.
- 12:45SPEAKER_00: There might be specific,
- 12:47SPEAKER_00: you know,
- 12:48SPEAKER_00: domain and context and application specific regulation that might make sense,
- 12:51SPEAKER_00: you know,
- 12:51SPEAKER_00: in some state and not another.
- 12:53SPEAKER_00: But in general,
- 12:54SPEAKER_00: you know,
- 12:55SPEAKER_00: from a first principles level,
- 12:56SPEAKER_00: at least I think that that would probably not be desirable.
- 12:59SPEAKER_01: The second regulatory concern, and maybe this is dissipating as policymakers
- 13:05SPEAKER_01: I think learn more,
- 13:06SPEAKER_01: especially the federal level,
- 13:07SPEAKER_01: maybe learn more about AI is at least initially,
- 13:11SPEAKER_01: it seems to me that whatever,
- 13:14SPEAKER_01: whatever your policy idea was for social media or to about content moderation or
- 13:21SPEAKER_01: what have you,
- 13:22SPEAKER_01: you just kind of took that policy framework and applied it to AI because like,
- 13:28SPEAKER_01: that was what you had.
- 13:29SPEAKER_01: Hey, you pull that baby right off the shelf.
- 13:31SPEAKER_01: Are we still seeing that or are people beginning to think like this is its own
- 13:36SPEAKER_01: thing and my ideas for social media,
- 13:41SPEAKER_01: that may be fine for social media,
- 13:43SPEAKER_01: but I need to think differently about AI.
- 13:46SPEAKER_01: Obviously, the technology is different.
- 13:47SPEAKER_01: Also, I think both the risks and potential rewards are very different.
- 13:52SPEAKER_00: Yeah, totally.
- 13:53SPEAKER_00: I think that has been an issue.
- 13:54SPEAKER_00: Now, I wouldn't say that's the case for everyone.
- 13:56SPEAKER_00: There's been some groups and some institutions doing some very careful work that
- 14:01SPEAKER_00: really kind of thinks about AI and AGI and so on in kind of careful,
- 14:06SPEAKER_00: more calibrated ways.
- 14:06SPEAKER_00: But I also have seen quite a lot of reports where you could have easily imagined
- 14:10SPEAKER_00: the same text being about social media platforms or some kind of other policy issue
- 14:15SPEAKER_00: or blockchain or something just being kind of repurposed for AI.
- 14:21SPEAKER_00: And there's a lot of stuff out there that's just,
- 14:22SPEAKER_00: you know,
- 14:22SPEAKER_00: very high level and kind of,
- 14:23SPEAKER_00: you know,
- 14:24SPEAKER_00: it's hard to kind of disagree with at a high level or something,
- 14:26SPEAKER_00: but it's far harder to kind of apply and look at from an operational or practical perspective.
- 14:33SPEAKER_00: So I've been seeing quite a lot of that.
- 14:35SPEAKER_00: However, I think over time, you know, the field is maturing more and more.
- 14:38SPEAKER_00: And,
- 14:40SPEAKER_00: you know,
- 14:40SPEAKER_00: you're seeing kind of better thinking around,
- 14:43SPEAKER_00: you know,
- 14:44SPEAKER_00: AI,
- 14:44SPEAKER_00: what it really is,
- 14:46SPEAKER_00: what,
- 14:46SPEAKER_00: you know,
- 14:46SPEAKER_00: what the model,
- 14:47SPEAKER_00: you know,
- 14:47SPEAKER_00: what's appropriate at the model level versus at the application level.
- 14:51SPEAKER_00: and the existing landscape of laws and regulations and how these might apply as well,
- 14:55SPEAKER_00: which is often that's something that's forgotten,
- 14:57SPEAKER_00: where you have lots of academics coming in and just trying to re-regulate
- 15:04SPEAKER_00: everything from first principles.
- 15:05SPEAKER_00: And then you're like, well, there's tort law, and there's this and that over there.
- 15:07SPEAKER_00: You've got to do your gap analysis first before coming up with all this stuff.
- 15:11SPEAKER_00: But I think we're seeing kind of, yeah,
- 15:14SPEAKER_00: the kind of field of AI governance and policy maturing in that space, and I expect it to continue.
- 15:19SPEAKER_00: But I still,
- 15:20SPEAKER_00: of course,
- 15:20SPEAKER_00: see a lot of bad heuristics and poor thinking here,
- 15:24SPEAKER_00: and particularly an underestimation of the benefits of AI and AGI.
- 15:29SPEAKER_00: I think there's a tendency to always think of the worst for everything,
- 15:32SPEAKER_00: and it's necessary you need to do that too,
- 15:33SPEAKER_00: but few are really internalizing how significant AGI would be for growth,
- 15:39SPEAKER_00: for welfare,
- 15:40SPEAKER_00: and for solving a lot of the issues that we've been talking about in the first place.
- 15:44SPEAKER_01: Is AGI, is it an if issue or is it a when issue?
- 15:50SPEAKER_01: And if it's a when, when?
- 15:53SPEAKER_01: And I say this with the caveat that predictions are difficult, especially about the future.
- 15:59SPEAKER_00: Yeah, absolutely.
- 16:01SPEAKER_00: So in my mind, it's definitely a when question.
- 16:03SPEAKER_00: I see no real strong reason why it would be an if and that being completely impossible.
- 16:10SPEAKER_00: And there's been many,
- 16:10SPEAKER_00: many,
- 16:10SPEAKER_00: many examples over the last 10 years of people saying,
- 16:13SPEAKER_00: well,
- 16:14SPEAKER_00: this is not possible with neural networks.
- 16:16SPEAKER_00: And then like 10 minutes later, it is proven to be possible.
- 16:19SPEAKER_00: as a recurring theme.
- 16:20SPEAKER_00: And I mean,
- 16:21SPEAKER_00: that may not be sufficient to think that AGI is feasible and possible,
- 16:24SPEAKER_00: but I'm pretty confident for a variety of reasons.
- 16:28SPEAKER_00: And so by AGI,
- 16:30SPEAKER_00: by the way,
- 16:30SPEAKER_00: I think there's an excellent paper by Morris and others on levels of AGI and
- 16:37SPEAKER_00: operationalizing progress on the path of AGI.
- 16:39SPEAKER_00: And I think it's a very good paper.
- 16:40SPEAKER_00: And that goes back to one point I made earlier in that at some point,
- 16:45SPEAKER_00: you know,
- 16:45SPEAKER_00: you'll have systems that will be capable of quite a lot of things.
- 16:48SPEAKER_00: And probably, you know, I could do probably anything that your average human can do.
- 16:54SPEAKER_00: at least virtually remotely to start with, and eventually to the physical world.
- 16:58SPEAKER_00: But I think they'll be capable in that sense.
- 17:01SPEAKER_00: Now,
- 17:01SPEAKER_00: there's a difference between these systems being capable in kind of in vitro lab
- 17:05SPEAKER_00: settings or something,
- 17:06SPEAKER_00: and then being actually deployed and used in industrial processes,
- 17:10SPEAKER_00: commercial applications,
- 17:12SPEAKER_00: in ways that is productive,
- 17:13SPEAKER_00: adds value,
- 17:14SPEAKER_00: creates profits,
- 17:14SPEAKER_00: and so on.
- 17:15SPEAKER_00: And I think there's a bit of a gap here.
- 17:16SPEAKER_00: So I don't think we'll have a day where we'll wake up and say, oh, you know, that's it.
- 17:20SPEAKER_00: Today we have AGI.
- 17:21SPEAKER_00: I think it'll be more of a kind of blurry spectrum.
- 17:23SPEAKER_00: But gradually, I think it'll be harder and harder for people to deny that we have kind of reached AGI.
- 17:29SPEAKER_00: And as this stuff gets kind of integrated into production systems,
- 17:32SPEAKER_00: I think the effects on growth in the economy will speak for themselves.
- 17:36SPEAKER_00: And as to the when exactly,
- 17:37SPEAKER_00: I would think that at least the capabilities,
- 17:40SPEAKER_00: I would expect that in the next five years,
- 17:43SPEAKER_00: you could easily see a point where people make a very confident claim that,
- 17:48SPEAKER_00: yeah,
- 17:49SPEAKER_00: we've got systems now that are AGI level.
- 17:51SPEAKER_00: They're generally capable,
- 17:52SPEAKER_00: and they are pretty competent,
- 17:55SPEAKER_00: or even expert level,
- 17:57SPEAKER_00: at least 90th percentile of skilled adults.
- 18:00SPEAKER_00: And then the challenge will be then,
- 18:01SPEAKER_00: you know,
- 18:02SPEAKER_00: operationalizing that and integrating that into a lot of systems.
- 18:05SPEAKER_00: But yeah, in my mind, it's definitely not an if.
- 18:08SPEAKER_00: And I would say the next five to 10 years is the kind of relevant period I have in mind, at least.
- 18:13SPEAKER_00: It could be longer.
- 18:14SPEAKER_00: And I think,
- 18:14SPEAKER_00: you know,
- 18:15SPEAKER_00: tech community has a tendency to sometimes over index,
- 18:18SPEAKER_00: particularly on the kind of R&D side.
- 18:19SPEAKER_01: Do you have any thoughts or maybe you don't have any thoughts about the notion that as
- 18:26SPEAKER_01: as perhaps AGI seems closer and maybe the geopolitical race intensifies,
- 18:32SPEAKER_01: that this becomes more of a national security issue and the government takes a
- 18:35SPEAKER_01: greater role.
- 18:36SPEAKER_01: And maybe the government makes itself a not so silent partner with tech companies.
- 18:42SPEAKER_01: And it really becomes almost like a Manhattan Project kind of deal to get there first and
- 18:49SPEAKER_01: And Leopold Aschenbrenner wrote this very long, very long paper.
- 18:53SPEAKER_01: Is that an issue that you have any thoughts on?
- 18:56SPEAKER_01: Is it something that you discuss or does it seem kind of or does it seem just science fictional to you?
- 19:01SPEAKER_00: Yeah, I do a lot of thinking on that.
- 19:04SPEAKER_00: And I've read up for Leopold's report, and I think there's a lot of good things in there.
- 19:08SPEAKER_00: I don't necessarily agree with everything.
- 19:10SPEAKER_00: I think things like security is really critical.
- 19:13SPEAKER_00: I think thinking about things like alignment and so on is important.
- 19:17SPEAKER_00: And I think that one thing I really agree with Leopold's report,
- 19:21SPEAKER_00: and I'm glad he emphasized,
- 19:22SPEAKER_00: was the need to secure and cement against liberal democracy.
- 19:27SPEAKER_00: The free world must prevail kind of thing.
- 19:28SPEAKER_00: I think that is indeed true, and people
- 19:30SPEAKER_00: tend to underestimate the implication on that front.
- 19:34SPEAKER_00: Now, what that looks like, what that means and requires in practice is not fully clear to me yet.
- 19:40SPEAKER_00: I think people talk about a Manhattan Project.
- 19:42SPEAKER_00: There's many other potential blueprints or ways to think about that.
- 19:45SPEAKER_00: They could be just normal procurement partnerships.
- 19:48SPEAKER_00: There could be different models for this.
- 19:50SPEAKER_00: At some point,
- 19:51SPEAKER_00: something like that could be defensible,
- 19:54SPEAKER_00: but it's very hard to predict that in advance,
- 19:56SPEAKER_00: given particularly
- 19:58SPEAKER_00: well, how hard it is to predict anything with AI to start with.
- 20:01SPEAKER_00: And secondly,
- 20:02SPEAKER_00: there's loads of kind of trade-offs with all these different options and some might
- 20:06SPEAKER_00: be a lot better than others.
- 20:07SPEAKER_00: So I think certainly more work might be needed there.
- 20:09SPEAKER_00: But in principle, I think, you know, the idea doesn't seem completely crazy or science fiction to me.
- 20:15SPEAKER_01: You recently posted on X that you were baffled at how many people don't use large,
- 20:21SPEAKER_01: these language models or chatbots daily.
- 20:25SPEAKER_01: I think a lot of people, they just, they don't know what they would use them for.
- 20:30SPEAKER_01: And do you have any recommendations for ways that people who are,
- 20:34SPEAKER_01: you know,
- 20:34SPEAKER_01: not in your line,
- 20:35SPEAKER_01: who are not coders,
- 20:37SPEAKER_01: that people can use them?
- 20:39SPEAKER_01: I mean,
- 20:39SPEAKER_01: do you use them in ways that are applicable to just,
- 20:42SPEAKER_01: you know,
- 20:42SPEAKER_01: the way regular people might use them?
- 20:45SPEAKER_00: Yeah, I think so.
- 20:45SPEAKER_00: I think under the post, I gave a few examples of how I use it.
- 20:48SPEAKER_00: Now, admittedly, most of these wouldn't be something that, you know, anyone would do.
- 20:52SPEAKER_00: But,
- 20:53SPEAKER_00: you know,
- 20:53SPEAKER_00: I had this actually,
- 20:54SPEAKER_00: I thought about this last weekend when I was seeing my parents and I was trying to
- 20:58SPEAKER_00: get them to,
- 20:58SPEAKER_00: A,
- 20:59SPEAKER_00: understand what,
- 20:59SPEAKER_00: you know,
- 20:59SPEAKER_00: what Claude or Gemini is and how to think about it.
- 21:03SPEAKER_00: You know, what kind of questions are worth asking and what kind of questions are not worth asking.
- 21:07SPEAKER_00: And it's very hard to come up with a very crisp kind of way of, you know,
- 21:13SPEAKER_00: sharing these intuitions.
- 21:15SPEAKER_00: And I think the first piece of advice I'd give is probably to just kind of,
- 21:18SPEAKER_00: you know,
- 21:19SPEAKER_00: take one of these models and have a very long conversation with it about some sort
- 21:22SPEAKER_00: of topic,
- 21:23SPEAKER_00: you know,
- 21:24SPEAKER_00: like try to poke holes,
- 21:25SPEAKER_00: try to kind of like contradict,
- 21:26SPEAKER_00: try to,
- 21:27SPEAKER_00: and I think that starts giving you maybe a bit,
- 21:29SPEAKER_00: you know,
- 21:29SPEAKER_00: a few better intuitions about what this can do,
- 21:32SPEAKER_00: as opposed to just treating it as some sort of,
- 21:34SPEAKER_00: you know,
- 21:35SPEAKER_00: question and answer Oracle type search engine,
- 21:37SPEAKER_00: which I think is not the right use case.
- 21:39SPEAKER_01: That is probably the most unsatisfying way to look at it and just treat it as,
- 21:44SPEAKER_01: you know,
- 21:44SPEAKER_01: a better Google search engine.
- 21:46SPEAKER_01: I mean,
- 21:48SPEAKER_01: yeah,
- 21:48SPEAKER_01: really that sort of conversational,
- 21:50SPEAKER_01: you know,
- 21:51SPEAKER_01: curious aspect rather than saying like,
- 21:54SPEAKER_01: right.
- 21:55SPEAKER_01: Find me a link.
- 21:55SPEAKER_01: Find me a link isn't a great use.
- 21:59SPEAKER_00: Exactly.
- 22:00SPEAKER_00: And people will often do that.
- 22:01SPEAKER_00: We'll do a thing,
- 22:02SPEAKER_00: we'll get some incorrect answer,
- 22:05SPEAKER_00: a hallucination or whatever,
- 22:06SPEAKER_00: and then we'll say,
- 22:06SPEAKER_00: oh,
- 22:07SPEAKER_00: these things are not good,
- 22:07SPEAKER_00: they're not accurate,
- 22:08SPEAKER_00: and I'll stop using it.
- 22:09SPEAKER_00: And to me, that is just crazy.
- 22:11SPEAKER_00: It's very fundamentally incurious.
- 22:13SPEAKER_00: And I think there's ways of using them and thinking them that is very useful.
- 22:17SPEAKER_00: So what have I done recently?
- 22:18SPEAKER_00: I'm trying to think of an example.
- 22:21SPEAKER_00: I had some papers that I couldn't understand very well.
- 22:23SPEAKER_00: And I would just ask it for better analogies,
- 22:25SPEAKER_00: explanations,
- 22:26SPEAKER_00: try to dig into certain concepts and ideas and just play around with them until the
- 22:30SPEAKER_00: insights and intuitions were easier for me to internalize and understand.
- 22:33SPEAKER_00: And I think you can do that at different levels and regular people also want to understand things.
- 22:38SPEAKER_00: So I think that might be potentially an example.
- 22:40SPEAKER_00: But the very first thing I would do is simply kind of long,
- 22:43SPEAKER_00: protracted conversations to really get a sense of how far the model can really go.
- 22:50SPEAKER_00: And then as you do that,
- 22:52SPEAKER_00: you'll find things that are a bit more creative than can you please rewrite this
- 22:55SPEAKER_00: email for me?
- 22:56SPEAKER_00: Can you find typos or can you fill in my tax report or something?
- 23:01SPEAKER_00: You know,
- 23:02SPEAKER_00: I think one way a friend used it,
- 23:03SPEAKER_00: and of course,
- 23:03SPEAKER_00: like,
- 23:04SPEAKER_00: you know,
- 23:04SPEAKER_00: there's limitations to that,
- 23:05SPEAKER_00: you know,
- 23:06SPEAKER_00: get a lawyer and everything,
- 23:06SPEAKER_00: but like he had a legal contract that someone said to him and he couldn't afford a
- 23:09SPEAKER_00: lawyer straight away.
- 23:10SPEAKER_00: And he just said, well, can you help me find potential issues and errors in here?
- 23:14SPEAKER_00: Here's like why I'm in this kind of contract.
- 23:17SPEAKER_00: Here's what I'm concerned with.
- 23:18SPEAKER_00: And, you know, it's the first starting point, you know, it can be useful, give you interesting insights.
- 23:22SPEAKER_00: Doesn't mean it replaces a lawyer straight away,
- 23:24SPEAKER_00: but this is one potential kind of interesting way that,
- 23:26SPEAKER_00: you know,
- 23:26SPEAKER_00: everyday people could use.
- 23:32Thank you.