Latest / Faster, Please! — The Podcast / 🤖 AI risks and rewards: My chat (+transcript) with AI researcher Miles Brundage
Transcript
- 0:06SPEAKER_01: Welcome to Faster Please, the podcast.
- 0:09SPEAKER_01: I'm your host, Jim Pethikoukis.
- 0:12SPEAKER_01: Several times a month,
- 0:13I'll feature a lively conversation with a fascinating and
- 0:16provocative guest about
- 0:18SPEAKER_01: how to make the world a better place through scientific discovery,
- 0:21SPEAKER_01: technological innovation,
- 0:23SPEAKER_01: and economic growth.
- 0:24SPEAKER_01: You're also going to want to check out my Faster Please newsletter.
- 0:27SPEAKER_01: You're on Substack throughout the week for fresh essays,
- 0:30SPEAKER_01: Q&As,
- 0:31SPEAKER_01: and stories from around the internet and around the world.
- 0:36Artificial intelligence may prove to be one of the most
- 0:39transformative technologies in history.
- 0:42SPEAKER_01: But like any tool,
- 0:43SPEAKER_01: its immense power for good comes with a unique array of risks,
- 0:47SPEAKER_01: both large and small.
- 0:49SPEAKER_01: Today on Faster Please,
- 0:50SPEAKER_01: the podcast,
- 0:51I chat with Miles Brundage about extracting the most out of
- 0:55AI's potential while
- 0:57SPEAKER_01: mitigating harms.
- 0:58We discuss the evolving expectations for AI development and how
- 1:02to reconcile with
- 1:03SPEAKER_01: the technology's most daunting challenges.
- 1:06SPEAKER_01: Brundage is an AI policy researcher.
- 1:09He's a non-resident fellow at the Institute for Progress and
- 1:12formerly held a number
- 1:13SPEAKER_01: of senior roles at OpenAI.
- 1:15SPEAKER_01: He's also the author of his own Substack.
- 1:17SPEAKER_01: Earlier this year,
- 1:19I was moderating a discussion between an economist here at AEI
- 1:25and a CEO of a
- 1:27SPEAKER_01: leading AI company.
- 1:29And when I asked each of them how AI might impact our lives,
- 1:36our economists said,
- 1:37well,
- 1:38SPEAKER_01: I could imagine,
- 1:40SPEAKER_01: for instance,
- 1:40a doctor's productivity increasing because AI could accurately
- 1:45and deeply translate
- 1:47and transcribe an appointment with a patient in a way that's
- 1:49far better than what's
- 1:50SPEAKER_01: currently available.
- 1:51SPEAKER_01: So that was his scenario.
- 1:53SPEAKER_01: And then I asked the same question of the AI company CEO,
- 1:57SPEAKER_01: who said,
- 1:58SPEAKER_01: by contrast,
- 1:59SPEAKER_01: well,
- 1:59SPEAKER_01: I think within a decade,
- 2:00SPEAKER_01: all human death will be optional.
- 2:03SPEAKER_01: thanks to AI-driven medical advances.
- 2:05SPEAKER_01: On that rather broad spectrum,
- 2:07SPEAKER_01: more efficient doctor appointments and immortality,
- 2:12SPEAKER_01: how do you see the potential of this technology?
- 2:15SPEAKER_00: Yeah, it's a good question.
- 2:18SPEAKER_00: I don't think those are necessarily mutually exclusive.
- 2:20I think in general,
- 2:21AI can both augment productivity and substitute for human
- 2:26labor.
- 2:27And the ratio of those things is kind of hard to predict and
- 2:30might be very policy
- 2:31SPEAKER_00: dependent and social norm dependent.
- 2:34But what I will say is that,
- 2:36in general,
- 2:36it seems to me like the pace of progress is very fast.
- 2:39And so both augmentation and substitution seem to be,
- 2:43you know,
- 2:44picking up steam.
- 2:45And it's kind of interesting watching the debate between AI
- 2:50researchers and economists.
- 2:51SPEAKER_00: And, you know, I have a...
- 2:53colleague who has said that you know the ai researchers
- 2:56sometimes underestimate the
- 2:57practical challenges in deployment at scale and conversely the
- 3:02economists sometimes
- 3:03underestimate how just how quickly the the technology is
- 3:06advancing and so you know
- 3:07i think there there's maybe some happy uh happy middle to be
- 3:11found or perhaps you
- 3:12know one of the the more extreme perspectives is true but you
- 3:16know personally
- 3:18SPEAKER_00: You know,
- 3:18SPEAKER_00: I'm not an economist,
- 3:19SPEAKER_00: I can't really speak to all of the details of,
- 3:23SPEAKER_00: you know,
- 3:23SPEAKER_00: substitution and augmentation and all the policy variables here.
- 3:28But what I will say is that at least the technical potential
- 3:31for very significant
- 3:33SPEAKER_00: amounts of augmentation of human labor,
- 3:36SPEAKER_00: as well as substitution for human labor seem pretty likely on,
- 3:40SPEAKER_00: you know,
- 3:41SPEAKER_00: even,
- 3:41SPEAKER_00: you know,
- 3:42SPEAKER_00: well less than 10 years,
- 3:43SPEAKER_00: but certainly within 10 years,
- 3:44SPEAKER_00: things will change a lot.
- 3:46SPEAKER_01: It seems to me that the vibe has shifted a bit.
- 3:49When I talk to people from the Bay Area and I kind of give them
- 3:52the Washington or
- 3:53SPEAKER_01: Wall Street economists view,
- 3:56SPEAKER_01: To them,
- 3:56SPEAKER_01: I sound unbelievably gloomy and cautious,
- 3:59SPEAKER_01: but it seems the vibe has shifted,
- 4:01SPEAKER_01: at least recently,
- 4:03to where a lot of people think major advancements like
- 4:07superintelligence are
- 4:09SPEAKER_01: further out than they previously thought.
- 4:11SPEAKER_01: Like we should be viewing AI as an important technology,
- 4:14SPEAKER_01: but more like what we've seen before with the internet and PC.
- 4:18SPEAKER_00: Yeah, it's hard for me to comment.
- 4:20It seems to me like there are multiple vibe shifts happening at
- 4:26different cadences
- 4:27SPEAKER_00: and in different directions.
- 4:28SPEAKER_00: It seems like several years ago,
- 4:31there was more of a consensus that what people today would call
- 4:34AGI was decades
- 4:36SPEAKER_00: away or more.
- 4:38And it does seem like that kind of timeframe has shifted closer
- 4:41to the present.
- 4:42SPEAKER_00: And so there's still debate between the
- 4:45SPEAKER_00: you know,
- 4:45SPEAKER_00: I'd say the kind of next few years crowd versus the,
- 4:48SPEAKER_00: you know,
- 4:49SPEAKER_00: more like 10 years crowd.
- 4:50But that is a much narrower range than we saw several years ago
- 4:55when when when
- 4:55there was a wider range of expert opinions and people who used
- 5:00to be seen as
- 5:01SPEAKER_00: you know,
- 5:03SPEAKER_00: on one end of the spectrum,
- 5:05SPEAKER_00: like,
- 5:05SPEAKER_00: for example,
- 5:05SPEAKER_00: Gary Marcus and Francois Cholet,
- 5:08SPEAKER_00: you know,
- 5:09SPEAKER_00: who were seen as kind of the skeptics of AI progress,
- 5:11SPEAKER_00: even they now are saying,
- 5:13SPEAKER_00: oh,
- 5:13SPEAKER_00: it's like maybe,
- 5:14SPEAKER_00: you know,
- 5:14SPEAKER_00: 10 years or so,
- 5:15SPEAKER_00: maybe five years for very high levels of capability.
- 5:18SPEAKER_00: And so I think there's been some compression in that respect.
- 5:22SPEAKER_00: So that's one thing that's going on.
- 5:24SPEAKER_00: There's also a way in which people are starting to
- 5:27think less abstractly and more concretely about the
- 5:30applications of AI and seeing
- 5:32it less as this kind of mysterious thing that might happen
- 5:36suddenly and thinking of
- 5:37SPEAKER_00: it more as incremental,
- 5:39more as something that requires some work to apply in various
- 5:44parts of the economy,
- 5:45SPEAKER_00: that there's some friction associated with.
- 5:48SPEAKER_00: I think both of the...
- 5:49SPEAKER_00: these aren't inconsistent.
- 5:50SPEAKER_00: They're just kind of different vibe shifts that are happening.
- 5:55SPEAKER_00: So getting back to the question of,
- 5:56SPEAKER_00: you know,
- 5:56SPEAKER_00: is this just a normal technology,
- 5:58SPEAKER_00: et cetera,
- 5:58SPEAKER_00: I would say that at the very least,
- 5:59it does seem faster in some respects than some other
- 6:04technological changes that
- 6:05SPEAKER_00: we've seen.
- 6:06SPEAKER_00: So I think...
- 6:08chat GPT's adoption going from zero to double digit percentages
- 6:14of use across many
- 6:15SPEAKER_00: professions in the US and in a matter of high number of months,
- 6:20SPEAKER_00: low number of years is quite stark.
- 6:22SPEAKER_01: Would you be surprised if five years from now,
- 6:26SPEAKER_01: we viewed AI as something much more important than just another sort of
- 6:31SPEAKER_01: incremental technological advance,
- 6:34something far more transformative than technologies that have
- 6:38come before.
- 6:39No,
- 6:40I wouldn't be surprised by that at all,
- 6:41if I understand your question correctly.
- 6:43My kind of baseline expectation is that it will be seen as one
- 6:48of the most
- 6:48SPEAKER_00: important technologies ever,
- 6:49SPEAKER_00: if not the most important technology ever.
- 6:52And I'm not sure that there's a standard consensus on how to
- 6:56rate the internet
- 6:57SPEAKER_00: versus electricity,
- 6:58SPEAKER_00: et cetera.
- 6:58But it does seem to me like it's of the same caliber of
- 7:03electricity in the sense of turning
- 7:06essentially converting one kind of energy into various kinds of
- 7:10useful economic work.
- 7:12And similarly,
- 7:12AI is converting various types of electricity into cognitive
- 7:18work.
- 7:18SPEAKER_00: And I think that's a huge deal.
- 7:21However you want to define society or the aspect of society
- 7:24that you focus on,
- 7:25SPEAKER_01: government,
- 7:25SPEAKER_01: businesses,
- 7:26SPEAKER_01: individuals,
- 7:27are we collectively doing what we need to do to fully exploit
- 7:33the upsides of this
- 7:35SPEAKER_01: technology over the next half decade to decade,
- 7:38SPEAKER_01: as well as minimizing potential downsides?
- 7:42SPEAKER_00: I think we are not.
- 7:44And this is something that I sometimes find frustrating about
- 7:48the way that the
- 7:48SPEAKER_00: debate plays out is that there's sometimes this zero-sum mentality of,
- 7:53SPEAKER_00: you know,
- 7:54SPEAKER_00: doomers versus boomers or,
- 7:56SPEAKER_00: you know,
- 7:56SPEAKER_00: there's a term that Karen Howe uses.
- 7:58SPEAKER_00: And,
- 8:00SPEAKER_00: you know,
- 8:00this idea that there's this inherent tension between mitigating
- 8:04the risks and
- 8:05SPEAKER_00: maximizing the benefits.
- 8:06SPEAKER_00: And there are some tensions, but I don't think that we are
- 8:10SPEAKER_00: on the Pareto frontier, so to speak, of those issues.
- 8:15SPEAKER_00: So like right now,
- 8:16I think there's a lot of value being left on the table in terms
- 8:20of fairly low cost
- 8:21SPEAKER_00: risk mitigations.
- 8:22SPEAKER_00: There's also a lot of value being left on the table in terms of
- 8:26uh finding new ways to exploit the the upsides and kind of
- 8:30accelerate particularly
- 8:32beneficial applications so i'll give just one example on the on
- 8:35you know because i
- 8:36write a lot about the risk but uh i also am very interested in
- 8:40maximizing the
- 8:40SPEAKER_00: upside so i'll just give one example
- 8:43um protecting critical infrastructure and improving the cyber
- 8:46security of uh
- 8:48SPEAKER_00: various uh parts of critical infrastructure in the U.S.
- 8:51so hospitals for example get you know rent get attacked with
- 8:55ransomware all the
- 8:56time and this this causes real harm to patients because uh you
- 9:00know machines get
- 9:01SPEAKER_00: bricked essentially and and you know the and
- 9:04SPEAKER_00: they have,
- 9:04SPEAKER_00: you know,
- 9:05SPEAKER_00: like one or two people on the IT team,
- 9:08SPEAKER_00: and they're kind of overwhelmed by these,
- 9:11SPEAKER_00: you know,
- 9:11SPEAKER_00: not even always that sophisticated,
- 9:12SPEAKER_00: but perhaps more sophisticated attackers.
- 9:16SPEAKER_00: And that's a huge problem.
- 9:18It matters for national security,
- 9:21in addition to,
- 9:22you know,
- 9:22patients' lives.
- 9:24And it matters for national security in the sense that this is
- 9:27something that China
- 9:28SPEAKER_00: and Russia and others could hold at risk in a
- 9:31SPEAKER_00: In the context of a war,
- 9:32SPEAKER_00: they could kind of threaten this critical infrastructure as part of a,
- 9:36SPEAKER_00: you know,
- 9:38SPEAKER_00: a bargaining strategy.
- 9:40And I don't think that there's that much interest in kind of
- 9:43helping hospitals have
- 9:45a better kind of automated cybersecurity engineer helper among
- 9:50the big tech
- 9:51SPEAKER_00: companies.
- 9:51SPEAKER_00: because there just aren't that many hospital administrators.
- 9:54SPEAKER_00: It's kind of a market failure if you think it,
- 9:56SPEAKER_00: you know,
- 9:57SPEAKER_00: I'm not sure if it would mean the technical definition of market failure,
- 9:59SPEAKER_00: but it's at least a national security failure in that this is a very,
- 10:02SPEAKER_00: you know,
- 10:02SPEAKER_00: that's a kind of fragmented market.
- 10:04SPEAKER_00: There's, you know, a water plant here, a hospital administrator there.
- 10:09And so I recently put out a report with the Institute for
- 10:12Progress arguing that
- 10:13philanthropists and government could kind of put some
- 10:16additional kind of gasoline
- 10:19in the tank of cybersecurity by kind of incentivizing
- 10:22innovation that specifically
- 10:24SPEAKER_00: helps these under-resourced defenders more so than the usual customers of
- 10:28SPEAKER_00: cybersecurity companies like Fortune 500 companies.
- 10:32I'm confident that companies and entrepreneurs will figure out
- 10:35how to extract value
- 10:36SPEAKER_01: from AI and create new products and new services,
- 10:41SPEAKER_01: barring any sort of regulatory slowdowns.
- 10:44But since you mentioned low hanging fruit,
- 10:46what are some examples of that?
- 10:48SPEAKER_00: Yeah,
- 10:49so I would say that transparency is one of the areas where a
- 10:54lot of AI policy
- 10:55SPEAKER_00: experts seem to be in pretty strong agreement.
- 10:58SPEAKER_00: And,
- 10:59SPEAKER_00: you know,
- 10:59SPEAKER_00: obviously,
- 11:00there's there is still some debate and disagreement about the
- 11:03details of what
- 11:04SPEAKER_00: should be required.
- 11:05SPEAKER_00: But just to give you some illustration.
- 11:08SPEAKER_00: it is typical for the leading AI companies,
- 11:12SPEAKER_00: sometimes called frontier AI companies,
- 11:14to put out some kind of documentation about the safety steps
- 11:18that they've taken.
- 11:19SPEAKER_00: It's typical for them to say,
- 11:21SPEAKER_00: here's our safety strategy,
- 11:23SPEAKER_00: and here's some evidence that we're following this strategy.
- 11:26This includes things like assessing whether their systems can
- 11:30be used for cyber attacks and
- 11:31SPEAKER_00: Adam Chapnick, Ph.D.
- 11:32SPEAKER_00: :
- 11:32Assessing whether they could be used to create biological
- 11:34weapons or assessing the
- 11:36extent to which they make up facts and and kind of make make
- 11:40mistakes that that but
- 11:42state them very confidently in a way that could pose risks to
- 11:45users of the
- 11:46SPEAKER_00: technology that tends to be totally voluntary and.
- 11:51There started to be some momentum as a result of various
- 11:55voluntary commitments that
- 11:56SPEAKER_00: were made in recent years.
- 11:57But as the technology gets more high stakes and there's more
- 12:02kind of cutthroat
- 12:03competition and there's maybe more lawsuits where companies
- 12:06might be tempted to
- 12:08SPEAKER_00: kind of retreat a bit in terms of the information that they share.
- 12:12i think that could you know things could kind of backslide and
- 12:15at the very least
- 12:16not advance as far as i would like from the perspective of
- 12:19making sure that there's
- 12:20sharing of lessons learned from one company to another as well
- 12:24as making sure that
- 12:25the uh that the that investors and users of the technology can
- 12:29make informed
- 12:29decisions about okay do i purchase do i purchase the services
- 12:33of openei or google
- 12:35SPEAKER_00: or
- 12:35SPEAKER_00: Or anthropic and kind of make making these informed decisions,
- 12:39making informed capital investment seems to require
- 12:41transparency to some degree.
- 12:43And so this is something that is kind of actively being debated
- 12:47in a few contexts.
- 12:48For example,
- 12:48in California,
- 12:49there's a bill that has that and a few other things called SB
- 12:5353.
- 12:54SPEAKER_00: But in general,
- 12:55SPEAKER_00: you know,
- 12:55we're at a bit of a fork in the road in terms of both how
- 12:59certain regulations will
- 13:00SPEAKER_00: be implemented,
- 13:01SPEAKER_00: such as the EU.
- 13:03SPEAKER_00: Is it going to become,
- 13:05SPEAKER_00: you know,
- 13:05SPEAKER_00: actually an adaptive kind of nimble approach to risk mitigation?
- 13:10SPEAKER_00: Or is it going to become a compliance checklist that,
- 13:13SPEAKER_00: you know,
- 13:13SPEAKER_00: just kind of makes big four accounting firms richer?
- 13:16SPEAKER_00: So there are implementation questions.
- 13:17Then there are just,
- 13:18you know,
- 13:19does the law pass or not kind of questions here.
- 13:23SPEAKER_01: Sort of in my probably overly simplistic way of looking at it,
- 13:26SPEAKER_01: I kind of think of like two buckets and you have issues like,
- 13:30SPEAKER_01: you know,
- 13:32SPEAKER_01: is there are these things biased?
- 13:34SPEAKER_01: Are they giving misinformation?
- 13:36Are they interacting with young people in a way that's bad for
- 13:39their mental health?
- 13:40SPEAKER_01: And I feel like.
- 13:41Like we have a lot of rules and we have a huge legal system for
- 13:44liability that can
- 13:46SPEAKER_01: probably handle those.
- 13:48Then in the other bucket are these kinds of what may for the
- 13:51moment be science
- 13:52SPEAKER_01: fictional kinds of existential risks,
- 13:54SPEAKER_01: whether it's
- 13:56machines taking over or just being able to give humans the
- 13:59ability to do very bad
- 14:01things in a way we couldn't before and within that and within
- 14:05that second bucket i
- 14:06SPEAKER_01: think sort of needs to be flexible like right now i'm pretty happy with
- 14:12voluntary standards and market discipline and maybe the
- 14:16government creating some benchmarks.
- 14:19But I can imagine the technology advancing to where the
- 14:24voluntary aspect seems less
- 14:26viable and there might need to be actual mandates about
- 14:29transparency or testing or
- 14:32SPEAKER_01: red teaming or whatever you want to call it.
- 14:35SPEAKER_00: Yeah,
- 14:35SPEAKER_00: I mean,
- 14:35I think that's a reasonable distinction in the sense that there
- 14:39are there are risks
- 14:40SPEAKER_00: at different scales.
- 14:42There are some that are kind of these large scale catastrophic
- 14:44risks and they might
- 14:45SPEAKER_00: have lower like lower likelihood,
- 14:47SPEAKER_00: but higher magnitude of impact.
- 14:49SPEAKER_00: And then there are things that are,
- 14:50SPEAKER_00: I would say,
- 14:52SPEAKER_00: literally happening millions of times a day,
- 14:54SPEAKER_00: like chat GPT making up,
- 14:56SPEAKER_00: you know,
- 14:56SPEAKER_00: making up.
- 14:57SPEAKER_00: citations to articles that don't exist or,
- 15:00SPEAKER_00: or,
- 15:01SPEAKER_00: you know,
- 15:01SPEAKER_00: Claude,
- 15:02SPEAKER_00: you know,
- 15:02SPEAKER_00: kind of saying that it fixed your code,
- 15:05SPEAKER_00: but actually didn't fix the code.
- 15:06SPEAKER_00: And, you know, the user is kind of too lazy to notice and so forth.
- 15:09SPEAKER_00: So, so there, so there are these different kinds of risks.
- 15:13SPEAKER_00: I personally,
- 15:14don't make a super strong distinction between them in terms of
- 15:19like different time
- 15:19SPEAKER_00: horizons,
- 15:20SPEAKER_00: precisely because I think things are going so quickly.
- 15:22SPEAKER_00: I think science fiction is becoming science fact very,
- 15:25SPEAKER_00: you know,
- 15:26SPEAKER_00: much sooner than many people expected.
- 15:28SPEAKER_00: But in any case,
- 15:29SPEAKER_00: I think that similar logic around like,
- 15:32SPEAKER_00: let's make sure that there's transparency,
- 15:35SPEAKER_00: even if we don't know exactly what the right
- 15:37SPEAKER_00: you know,
- 15:38SPEAKER_00: risk thresholds are,
- 15:39and we want to allow a fair degree of kind of flexibility and
- 15:43kind of what measures
- 15:45SPEAKER_00: companies take,
- 15:45SPEAKER_00: it seems good that they share what they're doing and kind of,
- 15:49SPEAKER_00: in my opinion,
- 15:50SPEAKER_00: ideally go another step further and allow,
- 15:52SPEAKER_00: you know,
- 15:53third parties to audit their practices and kind of make sure
- 15:56that if they say,
- 15:58SPEAKER_00: well,
- 15:58SPEAKER_00: we did a rigorous test for hallucination or something like that,
- 16:01SPEAKER_00: that that's actually true.
- 16:03And so that's kind of what I would like to see for both what
- 16:06you might call the
- 16:06SPEAKER_00: mundane and the more science fiction risks.
- 16:10SPEAKER_00: But,
- 16:10SPEAKER_00: you know,
- 16:10SPEAKER_00: again,
- 16:11SPEAKER_00: I think it's,
- 16:12SPEAKER_00: you know,
- 16:12it's kind of hard to say how things will play out and different
- 16:16people have,
- 16:16SPEAKER_00: you know,
- 16:17SPEAKER_00: different perspectives on these things.
- 16:19SPEAKER_00: I happen to be on the more aggressive end of the spectrum.
- 16:22I am worried about the spread of the sort of apocalyptic high
- 16:25risk AI narrative
- 16:27SPEAKER_01: that we heard so much about when ChatGPT first rolled out.
- 16:31SPEAKER_01: I mean,
- 16:32SPEAKER_01: that seems to have quieted somewhat,
- 16:33but I worry about it ramping up again and stifling innovation
- 16:37in an attempt to
- 16:38SPEAKER_01: reduce risk.
- 16:40SPEAKER_00: Yeah, no, I mean, these are very fair concerns.
- 16:42SPEAKER_00: And I will say that,
- 16:43SPEAKER_00: like,
- 16:43SPEAKER_00: there are lots of bills and,
- 16:46SPEAKER_00: you know,
- 16:46SPEAKER_00: and laws out there that have,
- 16:47SPEAKER_00: in fact,
- 16:48SPEAKER_00: slowed down innovation in certain contexts,
- 16:50SPEAKER_00: like something,
- 16:51SPEAKER_00: you know,
- 16:51SPEAKER_00: the EU,
- 16:51SPEAKER_00: I think,
- 16:51SPEAKER_00: has gone too far in some areas around,
- 16:55SPEAKER_00: you know,
- 16:55SPEAKER_00: social media platforms.
- 16:57I do think some of the at least some of the state bills that
- 17:00have been floated are,
- 17:02SPEAKER_00: you know,
- 17:02would kind of lead to just a lot of red tape and burdensome
- 17:07burdens to small
- 17:08SPEAKER_00: businesses.
- 17:09SPEAKER_00: I personally think this is avoidable.
- 17:11SPEAKER_00: I mean, there are going to be mistakes.
- 17:13I don't want to be misleading about how high quality
- 17:19policymakers' understanding of
- 17:21SPEAKER_00: some of these issues are.
- 17:22SPEAKER_00: There will be mistakes,
- 17:23SPEAKER_00: even in cases where,
- 17:25SPEAKER_00: for example,
- 17:25SPEAKER_00: in California,
- 17:26SPEAKER_00: there was a kind of blue ribbon commission of AI experts kind of...
- 17:31SPEAKER_00: producing a report over several months,
- 17:33and then that directly informing legislation and a lot of
- 17:36industry back and forth
- 17:37SPEAKER_00: and negotiation over the details.
- 17:39SPEAKER_00: And,
- 17:39SPEAKER_00: you know,
- 17:39I would say that's probably the high watermark SB 53 of kind of
- 17:44like fairly
- 17:45SPEAKER_00: stakeholder slash expert informed legislation,
- 17:49SPEAKER_00: even there,
- 17:49SPEAKER_00: I'm sure there will be some,
- 17:51SPEAKER_00: some things that,
- 17:52SPEAKER_00: you know,
- 17:52SPEAKER_00: we look back on and say,
- 17:54SPEAKER_00: you know,
- 17:54SPEAKER_00: it's not ideal,
- 17:55SPEAKER_00: but
- 17:55SPEAKER_00: In my opinion,
- 17:56SPEAKER_00: it's better to do something,
- 17:58SPEAKER_00: you know,
- 17:58SPEAKER_00: something that is as informed as we can do,
- 18:01because it does seem like there are these kind of market
- 18:03failures and incentive
- 18:05SPEAKER_00: problems that are going to arise if we do nothing,
- 18:08such as kind of companies retrenching and holding back
- 18:11information that makes it
- 18:12SPEAKER_00: hard for the field as a whole to
- 18:14SPEAKER_00: to kind of tackle these issues.
- 18:16SPEAKER_00: And I'll just make one more point,
- 18:18which is kind of adapting to the compliance capability of
- 18:22different companies.
- 18:23SPEAKER_00: Like how rich are they?
- 18:24SPEAKER_00: How expensive are the models they're training,
- 18:28I think is a key factor in the legislation that I tend to be
- 18:33more sympathetic to.
- 18:34SPEAKER_00: So just to make a contrast,
- 18:36SPEAKER_00: there was a building in Colorado that was kind of just one size fits all
- 18:40SPEAKER_00: you know, regulate all the kind of algorithms.
- 18:44SPEAKER_00: And that, I think, is very burdensome to small businesses.
- 18:47SPEAKER_00: I think something like SB53,
- 18:50SPEAKER_00: where it says,
- 18:51SPEAKER_00: okay,
- 18:52SPEAKER_00: if you can afford to train an AI system for $100 million,
- 18:56SPEAKER_00: you can probably afford to put out,
- 18:59SPEAKER_00: you know,
- 19:00SPEAKER_00: a dozen pages about your safety and security practices.
- 19:04SPEAKER_01: Yeah.
- 19:06SPEAKER_01: Hopefully Grok did not create this tweet of yours.
- 19:14SPEAKER_01: If it did, well, there we go.
- 19:15SPEAKER_01: You won't have to answer it.
- 19:16SPEAKER_01: I just want to understand what you meant by it.
- 19:20SPEAKER_01: A lot of AI safety people really,
- 19:22really want to find evidence that we have a lot of time to AGI
- 19:25or for AGI.
- 19:26SPEAKER_01: What does that mean?
- 19:27What I was trying to get at is that and I guess this is not
- 19:31necessarily just AI
- 19:31SPEAKER_00: safety people,
- 19:32SPEAKER_00: but I sometimes kind of try to poke at people in my,
- 19:35SPEAKER_00: you know,
- 19:36SPEAKER_00: social network who,
- 19:37SPEAKER_00: you know,
- 19:38SPEAKER_00: I'm often,
- 19:38SPEAKER_00: you know,
- 19:39SPEAKER_00: you know,
- 19:40on the same side of but also try to be a friendly critic to and
- 19:43that includes
- 19:43SPEAKER_00: people who are working on AI safety.
- 19:45SPEAKER_00: I think there's a common tendency to kind of
- 19:48grasp at what I would consider straws when reading papers and
- 19:53interpreting product
- 19:56SPEAKER_00: launches in a way that kind of suggests like,
- 19:59SPEAKER_00: well,
- 19:59SPEAKER_00: we've hit a wall,
- 20:00SPEAKER_00: AI is slowing down,
- 20:02SPEAKER_00: you know,
- 20:03SPEAKER_00: this was a flop,
- 20:04SPEAKER_00: like,
- 20:04SPEAKER_00: who cares?
- 20:06SPEAKER_00: And I think part of the,
- 20:08SPEAKER_00: you know,
- 20:08SPEAKER_00: I'm doing my kind of,
- 20:10SPEAKER_00: you know,
- 20:10SPEAKER_00: maybe uncharitable psychoanalysis.
- 20:12What I was getting at is that I think one reason why some
- 20:15people might be tempted
- 20:16SPEAKER_00: to do that is that it makes things seem easier and less scary if,
- 20:21SPEAKER_00: you know,
- 20:21SPEAKER_00: if...
- 20:22SPEAKER_00: well,
- 20:22we don't have to worry about really powerful AI enabled cyber
- 20:26attacks for another
- 20:26SPEAKER_00: five years or,
- 20:27SPEAKER_00: you know,
- 20:28SPEAKER_00: bio,
- 20:29SPEAKER_00: you know,
- 20:29SPEAKER_00: biological weapons for another two years or whatever.
- 20:32SPEAKER_00: Maybe, maybe not.
- 20:34SPEAKER_00: And I think that it,
- 20:35SPEAKER_00: you know,
- 20:35SPEAKER_00: and this,
- 20:36SPEAKER_00: I think the specific example that sparked that was GPT-5,
- 20:39SPEAKER_00: where there were a lot of people who,
- 20:41SPEAKER_00: in my opinion,
- 20:42were kind of reading the tea leaves in a particular way and
- 20:47kind of missing
- 20:47SPEAKER_00: important parts of the context.
- 20:49SPEAKER_00: Like, for example, that GPT-5 was not a
- 20:51not a particularly you know it wasn't a much larger or more uh
- 20:56expensive to train
- 20:57model than uh than gpd4 which may be surprising by the name and
- 21:02i think opening i
- 21:03did kind of screw up the naming and gave people the wrong
- 21:06impression but from my
- 21:07perspective you know there was nothing particularly surprising
- 21:10but to some people
- 21:11SPEAKER_00: it was kind of a
- 21:12a flop that they kind of you know wanted to say well things are
- 21:15slowing down but in
- 21:16my opinion you know the the if you look at more objective
- 21:20measures of progress like
- 21:22scores on math and coding and you know the reduction in the
- 21:27rate of hallucinations
- 21:28and solving chemistry and and biology problems and and uh you
- 21:33know designing new
- 21:34SPEAKER_00: chips and so forth there's there's quite rapid progress happening still
- 21:39I'm not sure if you're familiar with some of the work being
- 21:41done by former Google
- 21:43SPEAKER_01: CEO Eric Schmidt,
- 21:45SPEAKER_01: who's been doing a lot of work on national security and AI and his work.
- 21:53SPEAKER_01: It doesn't use the word AGI,
- 21:54SPEAKER_01: but it talks about AI,
- 21:57SPEAKER_01: certainly smart enough to be able to have certain capabilities,
- 22:00SPEAKER_01: which our national security establishment should be aware of,
- 22:04SPEAKER_01: should be planning.
- 22:06SPEAKER_01: And those capabilities,
- 22:07SPEAKER_01: I think to most people would seem sort of science fictional,
- 22:10SPEAKER_01: being able to launch technology.
- 22:13SPEAKER_01: incredibly sophisticated cyber attacks or be,
- 22:16SPEAKER_01: or be able to,
- 22:17SPEAKER_01: you know,
- 22:17SPEAKER_01: improve itself or be able to create some other sorts of capabilities.
- 22:22SPEAKER_01: And from that,
- 22:22SPEAKER_01: I'm like,
- 22:23SPEAKER_01: whether or not you think that's possible to me,
- 22:25SPEAKER_01: like the odds of that being possible are not zero.
- 22:28SPEAKER_01: And if they're not,
- 22:28SPEAKER_01: they're not zero,
- 22:30like some bit of like the bandwidth of the Pentagon should be
- 22:34thinking about that.
- 22:36SPEAKER_01: Is that sensible?
- 22:37SPEAKER_00: Yeah, it's totally sensible.
- 22:38SPEAKER_00: I mean, yeah, I'm not going to argue with you there.
- 22:42SPEAKER_00: And in fact,
- 22:43SPEAKER_00: I've done some collaboration with the RAND Corporation,
- 22:46SPEAKER_00: which has a pretty heavy investment in,
- 22:48SPEAKER_00: you know,
- 22:49SPEAKER_00: what they call the geopolitics of AGI and kind of studying,
- 22:53SPEAKER_00: you know,
- 22:53SPEAKER_00: what are the scenarios,
- 22:55SPEAKER_00: including AI and AGI being used to produce,
- 22:59SPEAKER_00: you know,
- 22:59SPEAKER_00: quote unquote,
- 23:00SPEAKER_00: wonder weapons and super weapons of some kind.
- 23:03SPEAKER_00: Basically, I think this is super important.
- 23:05SPEAKER_00: In fact,
- 23:05SPEAKER_00: I have a paper coming out,
- 23:07SPEAKER_00: you know,
- 23:08SPEAKER_00: that was in collaboration with some folks there pretty soon.
- 23:10SPEAKER_00: And, you know, I won't spoil all the details.
- 23:15SPEAKER_00: But if you search kind of Miles Brundage,
- 23:17SPEAKER_00: US,
- 23:17SPEAKER_00: China,
- 23:18SPEAKER_00: you know,
- 23:18SPEAKER_00: you'll kind of see some things that I've discussed there.
- 23:21SPEAKER_00: And,
- 23:21SPEAKER_00: you know,
- 23:21SPEAKER_00: basically,
- 23:22SPEAKER_00: my perspective is,
- 23:23SPEAKER_00: you know,
- 23:23we need to strike a balance between competing vigorously on the
- 23:29commercial side
- 23:30SPEAKER_00: with countries like China and Russia on AI.
- 23:32SPEAKER_00: I mean, more...
- 23:33more so china russia is less of a kind of you know uh less of a
- 23:37threat on the
- 23:37commercial side at least um and also making sure that we're
- 23:40fielding national
- 23:42security applications of ai in a responsible way um but also
- 23:46recognizing that there
- 23:47are these ways in which things could spiral out of control in a
- 23:51scenario with
- 23:52SPEAKER_00: totally unbridled competition and and i i want to avoid
- 23:55a scenario like you know the cuban missile crisis or uh other
- 23:59other kinds of things
- 24:00you know or ways in which that could have been much worse than
- 24:03the actual uh cuban
- 24:04missile crisis um uh happening as a result of ai and agi if you
- 24:10think that again if
- 24:12SPEAKER_01: the odds are not zero
- 24:14SPEAKER_01: that a technology which is fast evolving,
- 24:17SPEAKER_01: that we have no previous really experience with,
- 24:19SPEAKER_01: because it's fast evolving,
- 24:22could create the kinds of doomsday scenarios that there's new
- 24:27books out about,
- 24:28SPEAKER_01: people are talking about.
- 24:30SPEAKER_01: And so if you think,
- 24:31SPEAKER_01: okay,
- 24:32SPEAKER_01: not a 0% chance that could happen,
- 24:35SPEAKER_01: but it is kind of a 0% chance that we're going to stop AI,
- 24:39SPEAKER_01: smash the GPUs,
- 24:41SPEAKER_01: as someone who cares about policy,
- 24:45Are you just hoping for the best or are the kinds of things
- 24:48we've already talked
- 24:49SPEAKER_01: about,
- 24:49SPEAKER_01: you know,
- 24:50SPEAKER_01: transparency,
- 24:51SPEAKER_01: testing,
- 24:51SPEAKER_01: maybe that testing becoming mandatory at some point?
- 24:55SPEAKER_01: Like, is that enough?
- 24:57SPEAKER_00: It's hard to say what's enough.
- 24:59SPEAKER_00: And I agree that,
- 25:00SPEAKER_00: you know,
- 25:01SPEAKER_00: I don't know if I give it zero,
- 25:02SPEAKER_00: you know,
- 25:03SPEAKER_00: maybe if there's some like major pandemic caused by AI,
- 25:06SPEAKER_00: and then,
- 25:07SPEAKER_00: you know,
- 25:07SPEAKER_00: Xi Jinping and Trump,
- 25:09SPEAKER_00: you know,
- 25:09SPEAKER_00: get together and say,
- 25:09SPEAKER_00: okay,
- 25:10SPEAKER_00: like,
- 25:10SPEAKER_00: this is this is getting out of control,
- 25:11SPEAKER_00: maybe,
- 25:12SPEAKER_00: maybe things could change.
- 25:13SPEAKER_00: But yeah,
- 25:13SPEAKER_00: it does seem like continued investment and,
- 25:16SPEAKER_00: and large scale deployment of AI is the most likely scenario.
- 25:20um generally the way that i see this playing out is that
- 25:24they're kind of three
- 25:25pillars of a solution there's kind of some degree of safety and
- 25:28security standards
- 25:30um you know maybe we won't agree on everything but we should at
- 25:33least be able to
- 25:33agree that you know you don't want to lose control of your ai
- 25:36system you don't want
- 25:37it to get stolen uh you know you don't want a uh you know 10
- 25:41billion dollar ai
- 25:42system to be stolen you know with with a by a you know 10
- 25:46million dollar you know
- 25:47SPEAKER_00: scale hacking effort and so
- 25:49I think there are sensible standards you can come up with
- 25:51around safety and security.
- 25:53SPEAKER_00: I think you can have evidence produced or required that,
- 25:57SPEAKER_00: you know,
- 25:58SPEAKER_00: companies are following these things.
- 26:00SPEAKER_00: And so that includes transparency.
- 26:02SPEAKER_00: It also includes,
- 26:03SPEAKER_00: I would say,
- 26:05SPEAKER_00: third party auditing,
- 26:06where there's kind of third parties like checking the claims
- 26:08and making sure that
- 26:10SPEAKER_00: these standards are being followed.
- 26:11SPEAKER_00: And then you need some incentives to actually, you know,
- 26:14SPEAKER_00: participate in this regime and follow it.
- 26:16SPEAKER_00: And I think the incentives part is tricky,
- 26:18SPEAKER_00: particularly at an international scale,
- 26:20SPEAKER_00: like what incentive does China have to,
- 26:23SPEAKER_00: to play ball other than kind of,
- 26:24SPEAKER_00: you know,
- 26:25SPEAKER_00: they don't,
- 26:25SPEAKER_00: you know,
- 26:26SPEAKER_00: other than like,
- 26:27SPEAKER_00: obviously,
- 26:27SPEAKER_00: they don't want to kind of,
- 26:28SPEAKER_00: you know,
- 26:28SPEAKER_00: have their AI,
- 26:29SPEAKER_00: you know,
- 26:30SPEAKER_00: kill them or,
- 26:31SPEAKER_00: you know,
- 26:31SPEAKER_00: overthrow their government or whatever.
- 26:32SPEAKER_00: So like, you know, where exactly are the interests aligned and or not?
- 26:38SPEAKER_00: And,
- 26:38you know are is there some kind of system of you know export
- 26:41control policies or or
- 26:44sanctions or something that would kind of drive compliance or
- 26:47do you kind of uh you
- 26:49know is there is there some other approach and i think that
- 26:51that's the tricky part
- 26:52but to me that's those are kind of the rough outlines of a
- 26:56solution maybe that's
- 26:57enough but i think you know right now it it's it's not even
- 27:01really clear what the
- 27:02SPEAKER_00: rough rules of the road are
- 27:04SPEAKER_00: who's playing by the rules and,
- 27:06SPEAKER_00: and,
- 27:06SPEAKER_00: you know,
- 27:07SPEAKER_00: we're kind of relying a lot on,
- 27:09SPEAKER_00: on kind of,
- 27:09SPEAKER_00: you know,
- 27:10SPEAKER_00: goodwill and voluntary reporting.
- 27:11SPEAKER_00: So I think we can do better, but is that enough?
- 27:13SPEAKER_00: That's harder to say.
- 27:15Did your experience at open AI make you more or make you more
- 27:20optimistic or worried
- 27:21SPEAKER_01: that when we look back 10 years from now,
- 27:24SPEAKER_01: that AI will have overall on that made the world world a better place.
- 27:32SPEAKER_00: I'm sorry to not give you a simpler answer here.
- 27:37SPEAKER_00: And maybe I should sit on this one and come up with a kind of clearer,
- 27:42SPEAKER_00: more optimistic,
- 27:42SPEAKER_00: more pessimistic answer.
- 27:43SPEAKER_00: But I'll give you kind of two updates in different directions.
- 27:47SPEAKER_00: And I think they're not totally inconsistent.
- 27:50I would say that I have gotten more optimistic about the
- 27:54solvability of the problem
- 27:56SPEAKER_00: in the following sense.
- 27:57SPEAKER_00: I think that things were very fuzzy five, 10 years ago.
- 28:03SPEAKER_00: And when I joined OpenAI almost seven years ago now,
- 28:09there was a lot of concern that it could kind of come about
- 28:14suddenly that one day
- 28:15SPEAKER_00: you don't have AI,
- 28:16SPEAKER_00: the next day you have
- 28:17AGI and then on the third day you have artificial super
- 28:20intelligence and so forth
- 28:22and we don't live between the fourth day yeah and exactly and
- 28:25so it seems more
- 28:26gradual to me now and I think that is a good thing it also
- 28:31means that you know this
- 28:33is where I differ from some of the the kind of uh you know more
- 28:37extreme voices in
- 28:37SPEAKER_00: terms of shutting it all down is it seems to me like
- 28:41SPEAKER_00: there is at least some room for learning from experience,
- 28:45SPEAKER_00: iterating,
- 28:46SPEAKER_00: kind of taking the lessons from,
- 28:48SPEAKER_00: you know,
- 28:48GPT-5 and translating them into GPT-6 rather than it being
- 28:52something that we have
- 28:53SPEAKER_00: to get,
- 28:53SPEAKER_00: you know,
- 28:54SPEAKER_00: 100% right on the first shot and there being no room for error.
- 28:58SPEAKER_00: So in that sense, I'm more optimistic.
- 29:01um i would say in another respect i'm maybe more pessimistic in
- 29:06that i i'm seeing
- 29:08value being left on the table like i it seems to me like as i
- 29:11said we're not on the
- 29:12pareto frontier it seems like they're pretty straightforward um
- 29:17you know things
- 29:17that could be done you know at a for a very small fraction of
- 29:20say you know the u.s
- 29:22SPEAKER_00: uh you know federal budget or um
- 29:25SPEAKER_00: you know,
- 29:25SPEAKER_00: very small fraction of,
- 29:26SPEAKER_00: you know,
- 29:26SPEAKER_00: billionaires,
- 29:27SPEAKER_00: like personal philanthropy or whatever,
- 29:28SPEAKER_00: that,
- 29:29SPEAKER_00: in my opinion,
- 29:29SPEAKER_00: would dramatically reduce the likelihood of,
- 29:32SPEAKER_00: you know,
- 29:32SPEAKER_00: an AI enabled,
- 29:34SPEAKER_00: you know,
- 29:35SPEAKER_00: pandemic or various other issues and would dramatically increase the
- 29:40SPEAKER_00: the benefits of AI.
- 29:41SPEAKER_00: And so it's,
- 29:42SPEAKER_00: you know,
- 29:43SPEAKER_00: it's been a bit sad to kind of continuously see those opportunities being
- 29:47SPEAKER_00: neglected.
- 29:48SPEAKER_00: I hope that as AI becomes more of a salient,
- 29:52SPEAKER_00: you know,
- 29:52SPEAKER_00: salient issue to more people and people start to appreciate,
- 29:55SPEAKER_00: okay,
- 29:55SPEAKER_00: this is a real thing.
- 29:57SPEAKER_00: You know,
- 29:57SPEAKER_00: the benefits are real,
- 29:58SPEAKER_00: the risks are real,
- 29:59that there will be more of a kind of efficient policy market
- 30:02and people take those
- 30:03SPEAKER_00: opportunities.
- 30:04SPEAKER_00: But right now, it seems pretty inefficient to me.
- 30:07SPEAKER_00: And so that's kind of that's where my pessimism comes from.
- 30:10SPEAKER_00: It's not that it's unsolvable.
- 30:12SPEAKER_00: It's just,
- 30:12SPEAKER_00: OK,
- 30:13SPEAKER_00: from a kind of political economy and kind of public choice perspective,
- 30:16SPEAKER_00: like are the policymakers going to make the right decisions?