Latest / Statecraft / How to Give Money + Should You Quit Your Job?
Transcript
- 0:00[music]
- 0:05SPEAKER_00: Hi, I'm Santi Ruiz and you're listening to Statecraft.
- 0:08SPEAKER_00: Today, we've got an interesting episode, an unorthodox one on a couple axes.
- 0:13SPEAKER_00: For one, our guest today is Caleb Watney, who is formerly...
- 0:19SPEAKER_00: Well, he is the co-founder of IFP, the institution that Statecraft runs under.
- 0:24SPEAKER_00: Caleb, welcome aboard.
- 0:26SPEAKER_00: Great to be here.
- 0:26SPEAKER_00: Thanks, Santi.
- 0:27SPEAKER_00: Caleb is no longer my boss.
- 0:30SPEAKER_00: As of this summer,
- 0:30SPEAKER_00: he became managing director of public policy at Coefficient Given,
- 0:34SPEAKER_00: where he runs its new US AI policy team,
- 0:37SPEAKER_00: oversees its abundance and growth fund, and handles government relations.
- 0:41SPEAKER_00: The other interesting thing about this episode is I've got a co-pilot, a co-host, Leah Sargent.
- 0:47SPEAKER_00: Will you introduce yourself and welcome aboard?
- 0:50SPEAKER_01: I'm Leah Sargent.
- 0:51SPEAKER_01: I'm Santi's successor as the editorial director at IFP and generally someone
- 0:57SPEAKER_01: who likes examining interesting people.
- 1:00SPEAKER_01: So glad to join you on Statecraft.
- 1:02SPEAKER_00: I'm very glad to have you.
- 1:04SPEAKER_00: Leah, I would say,
- 1:06SPEAKER_00: has been one of the most dedicated supporters of Statecraft from very early on.
- 1:10SPEAKER_00: So it's very funny that we're now co-hosting.
- 1:13SPEAKER_02: You have two long-time listeners, first-time callers.
- 1:16SPEAKER_00: That's right.
- 1:17SPEAKER_00: As is probably obvious from these intros,
- 1:19SPEAKER_00: we all have conflicts of interest here on this episode.
- 1:22SPEAKER_00: We're going to talk about philanthropy,
- 1:24SPEAKER_00: the state of AI policy in DC,
- 1:26SPEAKER_00: the state of policy entrepreneurship generally,
- 1:29SPEAKER_00: like what are the levers to change and shape and influence
- 1:33SPEAKER_00: and make policy better right now.
- 1:35SPEAKER_00: But I just want to put it all out there.
- 1:36SPEAKER_00: Caleb used to be my boss.
- 1:38SPEAKER_00: Leah is my successor at IFP as editorial director.
- 1:42SPEAKER_00: And I really like both Caleb and Leah.
- 1:44SPEAKER_00: So all those caveats aside, let's get into it.
- 1:47SPEAKER_00: Caleb, when you joined Coefficient Giving,
- 1:50SPEAKER_00: you wrote a great essay I really enjoyed called A Long Sequence
- 1:54SPEAKER_00: of Small Correct Decisions.
- 1:56SPEAKER_00: What are you talking about?
- 1:58SPEAKER_02: John Van Neumann, savant, man of many different things named after him.
- 2:02SPEAKER_02: I actually encourage readers to go to the Wikipedia page for things named
- 2:05SPEAKER_02: after John Van Neumann.
- 2:06SPEAKER_02: It's much, much longer than you might expect.
- 2:08SPEAKER_02: In addition to being a polymath, he was also a great writer.
- 2:12SPEAKER_02: He wrote this essay in He was writing about nuclear weapons and climate engineering.
- 2:16SPEAKER_02: Humanity, in some sense, has had a slowly increasing ability to cause both harm
- 2:28SPEAKER_02: And good.
- 2:29SPEAKER_02: But, you know,
- 2:29SPEAKER_02: there's only so much damage when you're fighting with sticks and stones and
- 2:32SPEAKER_02: then eventually you get swords and then you get dynamite and
- 2:35SPEAKER_02: then now we have nuclear weapons.
- 2:36SPEAKER_02: And his basic thesis is that the situation is very unstable and that
- 2:41SPEAKER_02: the most we can kind of hope for given that.
- 2:43SPEAKER_02: Is a sort of like institutional solution.
- 2:46SPEAKER_02: It's going to be really hard to articulate ahead of time what the correct answer
- 2:50SPEAKER_02: to nuclear stability is going to be.
- 2:53SPEAKER_02: And all we have are sort of the I think I really like the way he answers this.
- 2:56SPEAKER_02: He says, can we produce?
- 2:58SPEAKER_02: And elsewhere in the essay,
- 2:59SPEAKER_02: he says that the only safeguards he could sort of endorse
- 3:01SPEAKER_02: through this framing were...
- 3:23SPEAKER_02: Day-to-day,
- 3:24SPEAKER_02: or perhaps year-to-year,
- 3:25SPEAKER_02: opportunistic measures, a long sequence of small correct decisions.
- 3:29SPEAKER_02: And so that's what the essay is sort of riffing on and where it comes from.
- 3:33SPEAKER_02: Our ability to sort of really reckon with our age of AI is unlikely
- 3:37SPEAKER_02: to be one big master plan that we sort of theorize from first principles.
- 3:41SPEAKER_02: And instead,
- 3:41SPEAKER_02: the best we can hope to do is build institutions that can make this small series
- 3:45SPEAKER_02: of correct decisions and help us kind of like muddle through this period together.
- 3:49SPEAKER_01: And I am curious just to orient us as we get started.
- 3:52SPEAKER_01: His examples were nuclear power and nuclear weapons and climate change.
- 3:57SPEAKER_01: For you, Caleb,
- 3:58SPEAKER_01: do you think right now for each of those two domains,
- 4:00SPEAKER_01: we're still in kind of the sequence he described where we need this slow series
- 4:06SPEAKER_01: of correct decisions to keep navigating those technologies?
- 4:09SPEAKER_01: Or have we passed out of that phase for either or both of those technologies?
- 4:13SPEAKER_01: Yeah.
- 4:14SPEAKER_02: One thing I like about the framing is that it sort of moves from a sort of static,
- 4:18SPEAKER_02: like, did you survive, yes or no, to more of like a dynamic process.
- 4:22SPEAKER_02: And I think that is like something that's very important.
- 4:24SPEAKER_02: I think our own understanding of kind of the risk vectors across all
- 4:27SPEAKER_02: of these different areas and many more in the future is Is not something
- 4:30SPEAKER_02: that we're ever going to have a full complete picture of.
- 4:32SPEAKER_02: What the correct intervention is or even what's the sign
- 4:34SPEAKER_02: of any particular intervention kind of like flip quite rapidly.
- 4:37SPEAKER_02: And so whether you have institutions that are able to kind of like notice
- 4:41SPEAKER_02: in real time how things are doing, make an adjustment.
- 4:44SPEAKER_02: Those plans kind of over time and then have public legitimacy to be able to act on
- 4:48SPEAKER_02: the basis of kind of the new evidence they generate is the meta principle
- 4:52SPEAKER_02: that you need to sort of optimize for.
- 4:53SPEAKER_02: And so we definitely are in a less perilous situation,
- 4:56SPEAKER_02: I think, with hopefully both of those than I think we were before.
- 4:59SPEAKER_02: But that's in part because we took concerted action and effort
- 5:02SPEAKER_02: and we set up new institutions that were able to help us kind of like navigate that.
- 5:06SPEAKER_02: So it's not to say that those risks have gone away, but we,
- 5:08SPEAKER_02: I think,
- 5:08SPEAKER_02: are in a more stable equilibrium than we were before because we kind
- 5:11SPEAKER_02: of have these new institutions.
- 5:12SPEAKER_01: And for most of this episode,
- 5:13SPEAKER_01: we're going to be focusing on AI as one of the domains that needs that long sequence
- 5:17SPEAKER_01: of small correct decisions.
- 5:18SPEAKER_01: But I have two social technologies where I'm curious if they fall into the same bucket for you.
- 5:23SPEAKER_01: And the social technologies are short form video and contraception.
- 5:28SPEAKER_01: Do either of these represent a technological advance that requires a long sequence
- 5:32SPEAKER_01: of small correct decisions to navigate?
- 5:35SPEAKER_01: Or are we capable of grappling with them without that kind of full court press?
- 5:40SPEAKER_02: Good question.
- 5:41SPEAKER_02: I mean, I almost want to back up and say that I think this is just an attribute
- 5:44SPEAKER_02: of technology in general.
- 5:46SPEAKER_02: One thing I've always liked about our name,
- 5:48SPEAKER_02: the Institute for Progress,
- 5:49SPEAKER_02: is that in our heads,
- 5:51SPEAKER_02: progress has never been like a one-dimensional thing that goes faster or slower.
- 5:55SPEAKER_02: It is multifaceted.
- 5:56SPEAKER_02: The technology tree is like one way you can envision this.
- 5:59SPEAKER_02: And I'd actually really recommend our kind of mutual colleague,
- 6:01SPEAKER_02: Tau Berga,
- 6:02SPEAKER_02: had a really great essay a couple of months ago called Do Not Surrender to
- 6:06SPEAKER_02: the Technology Tree.
- 6:07SPEAKER_02: Kind of all about this theme and kind of wrestling with these themes
- 6:10SPEAKER_02: of technological determinism.
- 6:11SPEAKER_02: To what extent is it kind of preordained that we are going to come up
- 6:14SPEAKER_02: with certain sorts of technologies or not?
- 6:16SPEAKER_02: And wrestling with the fact that there is some kind of,
- 6:18SPEAKER_02: I think, weak version of techno determinism that does seem kind of true.
- 6:22SPEAKER_02: If a technology is sufficiently obvious and possible to combine,
- 6:25SPEAKER_02: and in the case of a lot of these things,
- 6:27SPEAKER_02: it sort of is,
- 6:28SPEAKER_02: like once you have the basic scientific kind of preconditions,
- 6:30SPEAKER_02: it seems inevitable that someone somewhere is going to do some version of it.
- 6:34SPEAKER_02: So it's very difficult to,
- 6:35SPEAKER_02: I think,
- 6:35SPEAKER_02: build a world that is premised on keeping the genie in the bottle,
- 6:38SPEAKER_02: so to speak, for any sort of technology.
- 6:41SPEAKER_02: But on the flip side,
- 6:41SPEAKER_02: there's a huge amount of path dependence in the ordering in
- 6:45SPEAKER_02: which these technologies come,
- 6:46SPEAKER_02: in the business models that they get embedded in,
- 6:49SPEAKER_02: in the cultural values that are shaping them geographically.
- 6:55SPEAKER_02: I strongly endorse is that you can neither hope to just contain technology in
- 7:01SPEAKER_02: a neat box and say,
- 7:02SPEAKER_02: we're never opening up that jar,
- 7:04SPEAKER_02: but to also say that we should not consider ourselves helpless.
- 7:06SPEAKER_02: We have a huge amount of agency as a society, as an individual.
- 7:10SPEAKER_02: And so zooming back in then for things like short form video or lots
- 7:12SPEAKER_02: of other things,
- 7:13SPEAKER_02: I think it would be a mistake to say that they either exist as a technology
- 7:17SPEAKER_02: that should be shaped or should not be shaped.
- 7:19SPEAKER_02: All technologies at all times are being shaped by laws, cultural wars.
- 7:38SPEAKER_02: Investments, taxation, whatnot.
- 7:40SPEAKER_02: Obviously,
- 7:41SPEAKER_02: depending on the overall scope of the technology and what we think
- 7:44SPEAKER_02: the downsides are,
- 7:45SPEAKER_02: that merits stronger or weaker forms of, I think, intervention and shaping.
- 7:49SPEAKER_02: But I don't think any single class of technology is immune from it.
- 7:53SPEAKER_00: One interesting thing is that von Neumann,
- 7:56SPEAKER_00: who's writing this essay and making this – I mean,
- 7:59SPEAKER_00: it's not exactly an incrementalist take.
- 8:02SPEAKER_00: Some of the things that one had to do in the nuclear age were quite non-incremental, right?
- 8:07SPEAKER_00: But it's a real evolution for him,
- 8:09SPEAKER_00: widely thought of as one of the most intelligent men ever lived because
- 8:12SPEAKER_00: a few years prior, he'd been advocating for a nuclear first strike on the Russians.
- 8:18SPEAKER_00: Which I would not classify as a potentially small correct decision.
- 8:22SPEAKER_00: It would have been a very large catastrophic decision.
- 8:26SPEAKER_00: Agreed, agreed.
- 8:27SPEAKER_00: So even the very wise among us can change and learn.
- 8:31SPEAKER_00: I'm curious to root this in the work you're doing.
- 8:35SPEAKER_00: So you're a philanthropist now.
- 8:38SPEAKER_00: That's now the hat you wear.
- 8:39SPEAKER_00: As many listeners of Statecraft will be familiar,
- 8:42SPEAKER_00: there's a lot of discussion about the role of philanthropy today.
- 8:46SPEAKER_00: Because there's a huge amount of capital that is expected to come online
- 8:50SPEAKER_00: with various companies potentially IPO-ing, including SpaceX, which already IPO-ed.
- 8:55SPEAKER_00: And people don't really talk about where that capital is going.
- 8:57SPEAKER_00: I have a lot of questions there.
- 8:58SPEAKER_00: But a side note, there's been a lot of discussion.
- 9:01SPEAKER_00: And I'll link some of these good essays in the show notes
- 9:03SPEAKER_00: about how should new philanthropists think or how should people working on big,
- 9:08SPEAKER_00: ambitious projects think about capital coming to the space.
- 9:11SPEAKER_00: So Caleb,
- 9:12SPEAKER_00: given this focus on small,
- 9:13SPEAKER_00: correct decisions, How are you thinking about capital allocation?
- 9:38SPEAKER_02: And that is simultaneously a really exciting thing.
- 9:42SPEAKER_02: If you think about the history of the American project,
- 9:45SPEAKER_02: it has been shaped many times over by not just the state,
- 9:48SPEAKER_02: but also by large waves of philanthropic capital,
- 9:51SPEAKER_02: whether it was the Rockefellers or the Carnegies or the Gates Foundation
- 9:55SPEAKER_02: or any number of other actors.
- 9:56SPEAKER_02: And
- 9:59SPEAKER_02: On the flip side, each of those forms of capital has their own set of like biases.
- 10:06SPEAKER_02: And I think we'll end up focusing on certain kinds of problems and certain kinds
- 10:09SPEAKER_02: of solutions that if like left to their own devices are probably going
- 10:12SPEAKER_02: to be unhealthy.
- 10:13SPEAKER_02: I mean, Santis,
- 10:14SPEAKER_02: you know,
- 10:14SPEAKER_02: my And in kind of previous IFP hat,
- 10:15SPEAKER_02: one of the big areas that I did a lot of work on was meta science.
- 10:19SPEAKER_02: And I think there's actually a lot of analogies to the way that we think
- 10:21SPEAKER_02: through like a healthy portfolio of funding that exists on the scientific layer and
- 10:26SPEAKER_02: then how we as a society want to have kind of like high level capital
- 10:29SPEAKER_02: And I think one of the insights on the science funding side is that any form
- 10:35SPEAKER_02: of like scientific production is going to have its own set of like weak points,
- 10:38SPEAKER_02: blind spots, biases,
- 10:39SPEAKER_02: and they're going to ignore certain kinds of solutions or be sort of predisposed
- 10:42SPEAKER_02: towards certain kind of methods or methods.
- 10:44SPEAKER_02: Thank you.
- 11:06SPEAKER_02: And so I think that same principle actually applies to philanthropy.
- 11:09SPEAKER_02: And this is one of the things I'm most excited about with philanthropy is
- 11:12SPEAKER_02: that we can tackle different kinds of problems that the public sector can't.
- 11:16SPEAKER_02: On the flip side, philanthropy has its own set of biases and weak points.
- 11:19SPEAKER_02: And it would be bad if obviously All capital allocation were left to the philanthropic sector.
- 11:23SPEAKER_02: That would be its own kind of mini hell.
- 11:25SPEAKER_02: But I think as a part of a larger ecosystem that includes the private sector,
- 11:30SPEAKER_02: that includes businesses,
- 11:31SPEAKER_02: philanthropy has a really meaningful role to play in trying to take crazy shots
- 11:35SPEAKER_02: on goal or bets.
- 11:36SPEAKER_02: And certainly within AI,
- 11:37SPEAKER_02: I think philanthropy has been an absolutely massive shaper of the conversation.
- 11:41SPEAKER_02: I mean, coefficient giving,
- 11:43SPEAKER_02: which was previously open philanthropy,
- 11:44SPEAKER_02: I think first started taking these issues really seriously in 2014, 2015.
- 11:48SPEAKER_02: This was almost a decade before COVID.
- 11:51SPEAKER_02: You know, chat GPT and a lot of this like most recent wave of large language models.
- 11:56SPEAKER_02: And at the time, the idea of like taking AI seriously was like really weird.
- 11:59SPEAKER_02: And I think the kinds of people that were willing to take this seriously
- 12:02SPEAKER_02: or think there could be large scale societal harms
- 12:04SPEAKER_02: that are worth taking proactive action.
- 12:06SPEAKER_02: Yeah, it was not at all a mainstream idea.
- 12:09SPEAKER_02: And I'm really glad that there were philanthropic voices out there,
- 12:12SPEAKER_02: not just coefficient giving,
- 12:13SPEAKER_02: but lots of other people as well,
- 12:15SPEAKER_02: who were willing to fund early investment in research
- 12:17SPEAKER_02: in areas like mechanistic interpretability, in areas like benchmarking.
- 12:21SPEAKER_02: Benchmarking and evaluation.
- 12:38SPEAKER_01: You guys are trying to help spin up a lot of new organizations
- 12:42SPEAKER_01: through Project Tailwind,
- 12:43SPEAKER_01: where you're listing organizations you'd like someone to found.
- 12:46SPEAKER_01: And I'm kind of curious,
- 12:47SPEAKER_01: as you advise people who are making grants,
- 12:49SPEAKER_01: as you encourage people to start these,
- 12:50SPEAKER_01: I think your optimal failure rate for these projects is not zero,
- 12:54SPEAKER_01: which is different than a lot of philanthropies that are looking to fund winners.
- 12:58SPEAKER_01: How do you talk to someone who's allocating funds or a founder where you want them
- 13:01SPEAKER_01: to quit their normal job and do something you're like,
- 13:03SPEAKER_01: I think this might have a 10% chance of working?
- 13:06SPEAKER_01: And I want you to put your IPO money or put your life and your livelihoods
- 13:11SPEAKER_01: or something I think probably won't work.
- 13:14SPEAKER_01: But is EV positive as part of my overall portfolio?
- 13:17SPEAKER_02: This is not a novel problem that philanthropy is trying to solve here.
- 13:21SPEAKER_02: In some sense, the entire ethos of Silicon Valley is this.
- 13:24SPEAKER_02: Most startup founders have a rational exuberance.
- 13:28SPEAKER_01: But definitely not most philanthropic foundations.
- 13:31SPEAKER_02: Correct, correct.
- 13:32SPEAKER_02: And so in some sense,
- 13:33SPEAKER_02: I think motivating people by the shape of the problem and the impact
- 13:37SPEAKER_02: that you can have in society is a form of compensation that people get
- 13:42SPEAKER_02: in some sense.
- 13:42SPEAKER_02: I think lots of people find themselves wanting to leave their mark on the world.
- 13:46SPEAKER_02: And so tackling some of the biggest,
- 13:47SPEAKER_02: most important problems in society, I think, motivates a lot of people.
- 13:50SPEAKER_02: And so trying to help people navigate their way through that ecosystem and make sure
- 13:54SPEAKER_02: that they're orienting towards problems
- 13:56SPEAKER_02: that actually are really high stakes is itself,
- 13:59SPEAKER_02: I think, just like a thing that a lot of people intrinsically find motivating.
- 14:02SPEAKER_02: I don't find it that hard,
- 14:03SPEAKER_02: actually,
- 14:03SPEAKER_02: to motivate people to work on problems that only have a 10% chance of working,
- 14:07SPEAKER_02: so long as they are motivated that the underlying problem is really the right one
- 14:11SPEAKER_02: to be working on.
- 14:11SPEAKER_01: Well, I'm kind of curious,
- 14:12SPEAKER_01: are the founders then folks who are slightly delusional where they're like,
- 14:15SPEAKER_01: most of them have a 10% chance, but I've got the good one.
- 14:18SPEAKER_01: I'm the 80% chance.
- 14:19SPEAKER_01: And you, Caleb, know it's 10, but you're working with them.
- 14:24SPEAKER_01: Or are they kind of in that hold swarm mentality where they say,
- 14:27SPEAKER_01: As a side note,
- 14:32SPEAKER_01: I don't
- 14:35SPEAKER_00: like that we're now referring to normal human altruism as
- 14:39SPEAKER_00: a whole swarm mentality.
- 14:40SPEAKER_00: I think we want to retain that one.
- 14:41SPEAKER_00: I don't think this is just normal human altruism.
- 14:43SPEAKER_01: For me, it's always been timeless decision theory
- 14:46SPEAKER_02: moving me here.
- 14:47SPEAKER_02: Expected value is a very old framework.
- 14:51SPEAKER_02: I think at IFP, this was like a thing we ran against a lot.
- 14:54SPEAKER_02: Like in some sense,
- 14:54SPEAKER_02: the whole IFP thesis is that policy is really hard to change and you should expect
- 14:59SPEAKER_02: that most of the time you're going to totally fail.
- 15:01SPEAKER_02: But you only need a couple of wins to kind of pay for the overall portfolio.
- 15:05SPEAKER_02: And I think we've like seen that now across like IFP's five years
- 15:08SPEAKER_02: that we have some like pretty concrete, tangible wins, which are exciting.
- 15:10SPEAKER_02: And I think have like paid for the portfolio in some philanthropic kind
- 15:14SPEAKER_02: of like social value sense.
- 15:15SPEAKER_02: But it's good to maintain,
- 15:17SPEAKER_02: I think,
- 15:18SPEAKER_02: a community and positive vibes that allows people
- 15:20SPEAKER_02: to both be roughly correctly anchored on what's the underlying probability
- 15:24SPEAKER_02: of any given solution working,
- 15:26SPEAKER_02: but also gives you social supports and status In some sense,
- 15:31SPEAKER_02: for running after hard problems in an admirable way.
- 15:34SPEAKER_02: And I don't think that always works, but I think it's a correct thing.
- 15:37SPEAKER_02: I think the other thing is that you can also get too lost in the sauce of trying
- 15:40SPEAKER_02: to weigh down, oh, what is the exact probability of success?
- 15:43SPEAKER_02: In some sense,
- 15:44SPEAKER_02: if you're running after a really big problem and you think you have
- 15:46SPEAKER_02: a plausible chance of doing it,
- 15:48SPEAKER_02: it doesn't actually matter sometimes whether it's a 30% chance or a 10% chance or
- 15:52SPEAKER_02: a 50% chance.
- 15:53SPEAKER_02: If it's a sufficiently big and important problem,
- 15:55SPEAKER_02: usually the thing is kind of worth running after.
- 15:57SPEAKER_01: I'm really curious for Tailwind or for other projects like how you give people
- 16:01SPEAKER_01: the social support to do ambitious things that are likely to fail.
- 16:04SPEAKER_01: I seldom see people praised for shutting down
- 16:06SPEAKER_01: a project early once it's clear it's not panning out.
- 16:09SPEAKER_01: The piece that stands out to me is why we shut down in Asterisk Magazine
- 16:13SPEAKER_01: where people talk about why they closed their NGO and it wasn't because
- 16:16SPEAKER_01: the money dried up.
- 16:17SPEAKER_01: They weren't convinced of the impact.
- 16:19SPEAKER_01: Do you at CG or should you at CG I mean, it's funny.
- 16:37SPEAKER_02: I think that being a failed founder makes you quite qualified to trade again
- 16:41SPEAKER_02: on some other thing.
- 16:42SPEAKER_02: And the fact that you had the wits about you to realize the thing you were running
- 16:47SPEAKER_02: after was no longer the correct move is in fact a thing that's celebrated in
- 16:51SPEAKER_02: that community and is a marker of status in some sense.
- 16:54SPEAKER_02: And so I don't know that we're quite there on the philanthropic level,
- 16:58SPEAKER_02: but I have seen organizations or people Wrestle very seriously with I had a theory
- 17:04SPEAKER_02: of the case that turned out to not be true or facts about the world have changed in
- 17:08SPEAKER_02: the last two years that no longer make the thing that I'm running after kind of
- 17:11SPEAKER_02: the like highest leverage opportunity for me anymore.
- 17:14SPEAKER_02: And so I want to like really seriously wrestle with and think about
- 17:16SPEAKER_02: if there are other things across this like larger funded ecosystem
- 17:19SPEAKER_02: that would be useful for me to do.
- 17:21SPEAKER_02: And I again,
- 17:21SPEAKER_02: I think it's like,
- 17:22SPEAKER_02: not quite as a liquid a market,
- 17:23SPEAKER_02: you might say as as VC,
- 17:25SPEAKER_02: but I think there's like something approximating that
- 17:27SPEAKER_00: that seems
- 17:27SPEAKER_02: useful.
- 17:28SPEAKER_00: Here's a kind of like a broad question for you.
- 17:29SPEAKER_00: Nan Ranselhoff's essay,
- 17:31SPEAKER_00: which you mentioned a moment ago, says, okay,
- 17:33SPEAKER_00: there's something like $30 or $40 billion potentially coming online
- 17:36SPEAKER_00: a year starting next year.
- 17:39SPEAKER_00: That would, you know, roughly fund 5,000 IFPs.
- 17:42SPEAKER_00: She has a couple different examples,
- 17:43SPEAKER_00: you know, of like how you should conceptualize that number one thing, 5,000 IFPs.
- 17:48SPEAKER_00: Seems clear to me that 5,000 of IFP would be way too many.
- 17:51SPEAKER_00: I don't know what that would even mean to have that many new think tanks,
- 17:55SPEAKER_00: possibly even 1,000 or 500 or 200.
- 17:58SPEAKER_00: That's the wrong concept.
- 18:00SPEAKER_00: So where should that money go?
- 18:02SPEAKER_00: You're deploying some more than nominal chunk of that because CG is a very large organization.
- 18:08SPEAKER_00: You can't say, hey,
- 18:09SPEAKER_00: everybody,
- 18:09SPEAKER_00: you should go start a C3 and do policy research and development in Washington, DC.
- 18:15SPEAKER_00: Where does the money go?
- 18:15SPEAKER_00: It's an insane amount of money.
- 18:17SPEAKER_02: One thing to note is just that different areas have,
- 18:20SPEAKER_02: I guess,
- 18:20SPEAKER_02: to use the econ term,
- 18:21SPEAKER_02: very different elasticities in terms of how elastic are they to additional supply.
- 18:28SPEAKER_02: And I think some areas like global health Are extremely elastic.
- 18:31SPEAKER_02: And it's like quite obvious if you get an additional $10 billion to spend on kind
- 18:35SPEAKER_02: of new vaccines or kind of like new clinical trial design, new,
- 18:40SPEAKER_02: you know,
- 18:40SPEAKER_02: humanitarian interventions all across the world,
- 18:41SPEAKER_02: there is just like such a enormous amount.
- 18:45SPEAKER_02: I think a policy is one of those.
- 18:46SPEAKER_02: Because fundamentally, the thing you are sort of like supporting with fundraising.
- 18:59SPEAKER_02: Philanthropic dollars are just like people's time.
- 19:02SPEAKER_02: That makes it a lot less scalable than I think other interventions.
- 19:05SPEAKER_02: And so when you're buying things in the world,
- 19:07SPEAKER_02: be it vaccines or compute or whatever,
- 19:10SPEAKER_02: like physical goods,
- 19:11SPEAKER_02: I think it's just like inherently a much more scalable kind of solution.
- 19:15SPEAKER_02: We fund a lot of great organizations across the ecosystem,
- 19:18SPEAKER_02: like IFP, like RAND, like Horizon or something.
- 19:21SPEAKER_02: But you cannot just arbitrarily scale up and say, okay, I like IFP.
- 19:25SPEAKER_02: I want you guys to be a thousand times bigger.
- 19:27SPEAKER_02: They just would never work and there's not enough people.
- 19:30SPEAKER_02: And so policy,
- 19:30SPEAKER_02: it's very,
- 19:30SPEAKER_02: very important to have some baseline level of support to make sure that sort of like
- 19:33SPEAKER_02: the most thoughtful
- 19:44SPEAKER_02: And I find that now a big part of my job is actually trying to pull new deal flow
- 19:47SPEAKER_02: into the world.
- 20:07SPEAKER_02: Yeah.
- 20:08SPEAKER_02: Yeah.
- 20:25SPEAKER_02: Sometimes it is talking to an organization that's already doing some strain of work
- 20:29SPEAKER_02: that seems really useful,
- 20:30SPEAKER_02: but actually
- 20:30SPEAKER_02: with one slight tweak could very obviously fill this huge policymaking gap
- 20:35SPEAKER_02: where there's enormous demand from policymakers for financial models
- 20:38SPEAKER_02: or legal analysis or whatever.
- 20:40SPEAKER_02: And so helping basically do matchmaking in some big aggregate 30,000 foot view
- 20:45SPEAKER_02: between Who are the most talented and promising people across the ecosystem?
- 20:48SPEAKER_02: What are the most important policy problems to be working on?
- 20:51SPEAKER_02: Where's the biggest demand?
- 20:52SPEAKER_02: And what are the institutions that can kind of make that all happen?
- 20:55SPEAKER_02: And unfortunately, labor markets are just like really not all that efficient.
- 20:59SPEAKER_02: And that means that there's actually a lot of alpha in trying to do like better matchmaking.
- 21:04SPEAKER_01: Yeah, I'm actually quite curious about how you approach this part of your job
- 21:06SPEAKER_01: because this has been really highlighted to me by other folks
- 21:09SPEAKER_01: when I was interviewing about you in preparation for this interview,
- 21:13SPEAKER_01: that Caleb is going around telling people to quit their jobs and take new jobs,
- 21:17SPEAKER_01: requisitioning labor, etc.
- 21:19SPEAKER_00: Can confirm.
- 21:20SPEAKER_01: I don't know at what point people are going to bang you from their offices
- 21:24SPEAKER_01: because you're like the angel of death coming through and taking people out.
- 21:28SPEAKER_01: But I'm kind of curious, why is a Caleb necessary to do this?
- 21:32SPEAKER_01: Why is the labor market for talent so inefficient?
- 21:35SPEAKER_01: Why aren't people finding their ways to these better opportunities naturally such
- 21:38SPEAKER_01: that you've got to go out and kind of one by one talk to them and say,
- 21:41SPEAKER_01: have you thought about quitting your job and taking this job instead?
- 21:44SPEAKER_02: Yeah.
- 21:45SPEAKER_02: There's maybe a couple of ways to think about this.
- 21:48SPEAKER_02: One is a sort of more general Tyler Cowen ask.
- 21:53SPEAKER_02: There are just enormous returns to raising the ambitions of others.
- 21:57SPEAKER_02: I was an early emergent venture grantee, Alec and I, for Yeah, that's right.
- 22:15SPEAKER_02: You can go and do things in the world.
- 22:17SPEAKER_02: And it's like so easy to kind of get stuck in the day to day tranche
- 22:21SPEAKER_02: of like whatever it is that you're working on to kind of like miss the forest for
- 22:24SPEAKER_02: the trees.
- 22:25SPEAKER_02: I think I'm not gonna be able to quote it verbatim,
- 22:27SPEAKER_02: but some really great like quotes from like Steve Jobs about,
- 22:28SPEAKER_02: you know,
- 22:29SPEAKER_02: you should look out in the world and realize that all these structures were made
- 22:32SPEAKER_02: by people that are like, No more ambitious, no smarter than you.
- 22:35SPEAKER_02: And like you can go out there and you can change them.
- 22:37SPEAKER_02: And I think that is just like a deeply true baseline about the world.
- 22:40SPEAKER_02: And then two,
- 22:40SPEAKER_02: I would say this is even more true in the nonprofit space,
- 22:44SPEAKER_02: which is just like not an efficient market.
- 22:46SPEAKER_02: Again, this was like a big part of our theory of change with AIF.
- 22:48SPEAKER_02: And so I think that's a really good thing to do.
- 23:03SPEAKER_02: And so I think like,
- 23:18SPEAKER_02: It is unusually important for funders to have sort of an inside view of what is good
- 23:24SPEAKER_02: to do in the world and who is doing good work and what is like a useful theory
- 23:28SPEAKER_02: of change for doing that and then try to help like on the margin,
- 23:31SPEAKER_02: like push and nudge and people.
- 23:33SPEAKER_02: I do want to say that policy is an area that benefits unusually from having
- 23:49SPEAKER_02: A similar skill set that has done direct work versus doing capital allocation in that space.
- 23:54SPEAKER_02: I think in some spaces,
- 23:55SPEAKER_02: like international development,
- 23:56SPEAKER_02: actually these are two extremely different skill sets.
- 23:58SPEAKER_02: If I'm trying to allocate vaccine funding or something,
- 24:02SPEAKER_02: I do not have to be an expert in supply chain cold storage logistics.
- 24:07SPEAKER_02: I don't have to second guess the route around the Horn of Africa
- 24:10SPEAKER_02: for delivering malaria bed nets or something because
- 24:13SPEAKER_02: the outcomes are all very verifiable.
- 24:15SPEAKER_02: But I think in policy research and analysis and advocacy, it's much softer.
- 24:21SPEAKER_02: It's more art than science.
- 24:22SPEAKER_02: It's much harder to know what things are true in the world.
- 24:25SPEAKER_02: And so I think it benefits an unusual amount from having done the thing yourself
- 24:29SPEAKER_02: before doing capital allocation.
- 24:30SPEAKER_01: Because this is hard to scale, Caleb, I want to save you some time.
- 24:33SPEAKER_01: There are at least 15 people listening to this podcast right now
- 24:37SPEAKER_01: who should quit their job and find a more impactful one.
- 24:40SPEAKER_01: And to save you time tracking them down,
- 24:42SPEAKER_01: I'd like you to enumerate the criteria of them and of their current job
- 24:46SPEAKER_01: that should lead them to realize a visit from Caleb Watney is in their future,
- 24:50SPEAKER_01: that they might want to make the change now so that you can move faster.
- 24:53SPEAKER_00: Before you answer that, Caleb, I'm going to say the real number is much larger than 15.
- 24:57SPEAKER_00: 15 is a very low bar in terms of number of listeners of this podcast who should get
- 25:03SPEAKER_00: the talk from Caleb and maybe change their job.
- 25:05SPEAKER_01: But let's try and realistically knock off 15 and leave a comment in the show notes
- 25:09SPEAKER_01: if it reaches you.
- 25:10SPEAKER_01: Caleb, how should these people recognize themselves without your help personally?
- 25:14SPEAKER_02: What an opportunity.
- 25:15SPEAKER_02: I guess it is hard to give very broad-based and But if you have in any way kind of
- 25:21SPEAKER_02: an interest in policy, here's how I sort of think about it.
- 25:38SPEAKER_02: First is,
- 25:39SPEAKER_02: I think things are going to get quite crazy quite quickly,
- 25:42SPEAKER_02: especially with regard to AI.
- 25:44SPEAKER_02: And that means that a lot of theories of change that rely on things happening 15,
- 25:48SPEAKER_02: 20 years down the road are just going to very quickly find themselves kind
- 25:52SPEAKER_02: of no longer...
- 25:53SPEAKER_02: Yeah, that's right.
- 26:08SPEAKER_02: And I think in this current environment,
- 26:10SPEAKER_02: the discount rate is just unusually high beyond our typical 2% or 3%.
- 26:15SPEAKER_02: I don't have a number, maybe something more like 20%.
- 26:18SPEAKER_02: And that's mostly because things are happening so quickly that
- 26:21SPEAKER_02: the world could completely change before your eyes.
- 26:23SPEAKER_02: And the correct thing to do could be radically different.
- 26:25SPEAKER_02: And so somewhat shorter timelines, I think is like warranted in this current environment.
- 26:30SPEAKER_02: Another subset of that is that there's really only a couple of governments
- 26:32SPEAKER_02: that matter in the short term.
- 26:34SPEAKER_02: And I think it's like the US and China.
- 26:36SPEAKER_02: And so I have a lot of friends that are doing great work New York is great too.
- 27:08SPEAKER_02: It is not a center of power for the most important forces that are affecting
- 27:13SPEAKER_02: the American economy over the next five years, Santi.
- 27:16SPEAKER_00: Agree to disagree.
- 27:17SPEAKER_02: So move to the U.S.,
- 27:19SPEAKER_02: move to SF or D.C.,
- 27:20SPEAKER_02: and then I think think seriously about the issues that you're working on.
- 27:24SPEAKER_02: One particular thing that feels very top of mind right now to me and kind
- 27:27SPEAKER_02: of like almost on fire in like why aren't more people working on this is
- 27:31SPEAKER_02: if you assume that Congress is going to want to act,
- 27:34SPEAKER_02: do something on AI governance, what should it actually do?
- 27:37SPEAKER_02: Despite this being an extremely obvious question,
- 27:40SPEAKER_02: I don't know that like the people that have been thinking the longest or the deepest
- 27:44SPEAKER_02: about AI policy actually feel like we have a kind of robust answer to that.
- 27:47SPEAKER_02: Partially that's again because the strategic situation is changing so quickly.
- 27:51SPEAKER_02: I think there are some areas where we have some certainty,
- 27:54SPEAKER_02: like there's been a lot of momentum around this idea
- 27:56SPEAKER_02: of embedded third party auditors.
- 27:58SPEAKER_02: Wouldn't it be nice if there were kind of like some entity that has visibility into
- 28:01SPEAKER_02: the lab and then can I think there's some sort of vague directions that we feel.
- 28:18SPEAKER_02: So AI security,
- 28:20SPEAKER_02: we know that kind of like who has access to model weights
- 28:22SPEAKER_02: and how do those model weights get distributed around and
- 28:25SPEAKER_02: under what security conditions are they held?
- 28:27SPEAKER_02: Seems extremely important, especially for kind of like US-China competition.
- 28:30SPEAKER_02: But like, what does that mean practically?
- 28:32SPEAKER_02: Do we want to move most frontier training runs to like
- 28:35SPEAKER_02: a security level four data centers to use the like RAND terminology?
- 28:39SPEAKER_02: Seems plausible.
- 28:39SPEAKER_02: But like, again, I think the details here are really kind of like underbaked.
- 28:44SPEAKER_02: What sorts of new governmental institutions do we need to create?
- 28:47SPEAKER_02: Again, think about this.
- 28:49SPEAKER_02: One example I cited in my essay and
- 28:50SPEAKER_02: that other people have mentioned too is something like NTSB,
- 28:54SPEAKER_02: National Transportation Safety Board.
- 28:56SPEAKER_02: But for AI,
- 28:57SPEAKER_02: this is an independent group that doesn't actually have any regulatory authority
- 29:00SPEAKER_02: but can just go in and investigate and find out literally what happened and create,
- 29:05SPEAKER_02: again,
- 29:05SPEAKER_02: public awareness about what we could change about the safety systems
- 29:09SPEAKER_02: the security mechanisms to prevent that from happening in the future.
- 29:12SPEAKER_02: Right now,
- 29:12SPEAKER_02: this role is being informally served by various third-party groups like METER,
- 29:17SPEAKER_02: and I'm very glad that they exist,
- 29:18SPEAKER_02: but it'd be nice to have a more institutionalized version of this.
- 29:21SPEAKER_02: There's a hundred other questions.
- 29:22SPEAKER_02: What do we want to do about building in more resilience to the labor market?
- 29:26SPEAKER_02: What do we want to do about congressional empowerment vis-a-vis the executive branch?
- 29:30SPEAKER_02: How do we want to think about concentration of power more generally?
- 29:33SPEAKER_02: How does the nature of Power change between corporations and the state
- 29:38SPEAKER_02: as you have one of
- 29:39SPEAKER_02: the most powerful technologies ever being developed almost entirely within
- 29:42SPEAKER_02: the auspice of a private company.
- 29:43SPEAKER_02: There's just an extremely long list of questions
- 29:46SPEAKER_02: that I think we don't actually have good answers to, and that feels quite worrying.
- 29:50SPEAKER_02: What do we actually want to do here?
- 29:51SPEAKER_02: And I think we're far away from that point currently.
- 30:09SPEAKER_00: Can I add or propose one more criteria for listeners here?
- 30:12SPEAKER_00: This might be kind of banal or maybe skipping over it,
- 30:14SPEAKER_00: but if your institution doesn't do anything,
- 30:17SPEAKER_00: you should consider going somewhere else.
- 30:18SPEAKER_00: I just want to make sure that that's explicit.
- 30:21SPEAKER_02: To finish the hierarchy,
- 30:22SPEAKER_02: I guess there's sort of consider whether you should change countries to go to
- 30:25SPEAKER_02: the US.
- 30:25SPEAKER_02: If you're in the US, consider whether you should go to like DC or SF.
- 30:28SPEAKER_02: If you're in DC or SF and you're working at an institution that doesn't do anything,
- 30:31SPEAKER_02: you should consider going to an institution or starting an institution
- 30:33SPEAKER_02: that does do something.
- 30:34SPEAKER_02: And if you're in an institution that kind of does something,
- 30:36SPEAKER_02: but is oriented towards like maybe not the most important or pressing problems,
- 30:39SPEAKER_02: you should consider pivoting towards the most important and pressing problems.
- 30:42SPEAKER_02: And I think that that rough hierarchy will actually get you pretty far.
- 30:45SPEAKER_00: I want to use that as a jumping off point here.
- 30:47SPEAKER_00: We had Greg Berman on several months back to talk about problems with
- 30:51SPEAKER_00: the traditional philanthropy apparatus as it exists.
- 30:55SPEAKER_00: I was a little bit more bearish.
- 30:56SPEAKER_00: He was a little more bullish,
- 30:58SPEAKER_00: but he'd written a book about here's some kind That's one of the key problems
- 31:01SPEAKER_00: that traditional philanthropists who've been in the game a while need to think
- 31:04SPEAKER_00: about.
- 31:04SPEAKER_00: A lot of those were about the particular failures of philanthropy.
- 31:09SPEAKER_00: If philanthropy has strengths and weaknesses that we talked about, I think you're right.
- 31:13SPEAKER_00: Philanthropists like venture capitalists are often herd animals.
- 31:17SPEAKER_00: They're faddish.
- 31:19SPEAKER_00: They're maybe more consensus-driven than domains that are inherently adversarial.
- 31:25SPEAKER_00: Traditional philanthropies also face these days a ton of internal pressure
- 31:29SPEAKER_00: from employees to employees,
- 31:30SPEAKER_00: and the leaders of the orgs are often way out of step culturally,
- 31:34SPEAKER_00: what they want politically, philosophically.
- 31:36SPEAKER_00: I think in some ways,
- 31:38SPEAKER_00: what Nan Ransomhoff would call the third wave of philanthropy can avoid some
- 31:42SPEAKER_00: of these problems.
- 31:43SPEAKER_00: I think there's not a huge gap between the rank and file,
- 31:46SPEAKER_00: which in a traditional org might be really committed
- 31:49SPEAKER_00: to environmental justice versus leadership.
- 31:51SPEAKER_00: There's some of these problems that may not be the case.
- 31:53SPEAKER_00: I'm curious,
- 31:54SPEAKER_00: if you were going to give the bear case or the worries that you have for this wave
- 31:58SPEAKER_00: of philanthropy, what are the failure modes that you want to avoid?
- 32:03SPEAKER_00: What would some of those be?
- 32:04SPEAKER_02: Interesting question.
- 32:05SPEAKER_02: I have one big failure mode that we have certainly seen struck previous waves
- 32:09SPEAKER_02: of philanthropy that I'm worried about with this one.
- 32:12SPEAKER_02: And this is especially relevant to the previous discussion
- 32:14SPEAKER_02: about discount rates is how quickly are you actually able to deploy capital?
- 32:18SPEAKER_02: And historically,
- 32:19SPEAKER_02: there's been a long tradition of,
- 32:21SPEAKER_02: you know,
- 32:21SPEAKER_02: wealthy industrialists or whatever,
- 32:23SPEAKER_02: making lots of money and then deciding,
- 32:24SPEAKER_02: you know what,
- 32:25SPEAKER_02: I'm going to set up a foundation when I die and then kind of we'll deploy the money
- 32:28SPEAKER_02: then.
- 32:29SPEAKER_02: And that means both that like There are tons of really great opportunities
- 32:33SPEAKER_02: that could have used that funding much, much earlier.
- 32:36SPEAKER_02: In the same way that money can compound in a savings account,
- 32:39SPEAKER_02: impact can compound in the real world, institutions that you set up and support.
- 32:42SPEAKER_02: I'm so glad that IFB was set up five years ago rather than today.
- 32:46SPEAKER_02: It just gives us such a better starting ground.
- 32:47SPEAKER_02: We've been able to have tangible impact and our future impact is much bigger because
- 32:51SPEAKER_02: of that five-year head start.
- 32:52SPEAKER_02: And so how quickly can you actually get some of these new things off the ground?
- 32:56SPEAKER_02: Are we deciding to wait till people die?
- 32:58SPEAKER_02: And then also there's kind of like,
- 33:00SPEAKER_02: hopefully the principle has some useful expertise to bring to bear in terms
- 33:03SPEAKER_02: of deciding and orienting which problems are important.
- 33:07SPEAKER_02: That is also useful to take advantage of.
- 33:09SPEAKER_02: But again,
- 33:10SPEAKER_02: if you're kind of like waiting too long or waiting
- 33:12SPEAKER_02: until you can really do this thing kind of full time,
- 33:14SPEAKER_02: I think that's like a major failure mode to try to avoid.
- 33:17SPEAKER_02: I'm also worried,
- 33:18SPEAKER_02: I think this particular wave,
- 33:19SPEAKER_02: one interesting angle is going to be like how to navigate around conflicts
- 33:23SPEAKER_02: of interest,
- 33:24SPEAKER_02: where I think a lot of this new money is coming from sort of like the AI sector.
- 33:28SPEAKER_02: But then understandably,
- 33:30SPEAKER_02: A lot of the problems that they think are the most important that they want
- 33:32SPEAKER_02: to help contribute to are within AI governance or AI evaluation or whatnot.
- 33:38SPEAKER_02: And so how can you take like large scale funding that is obviously good that we want
- 33:43SPEAKER_02: to deploy,
- 33:44SPEAKER_02: but kind of like find I mean, again,
- 33:50SPEAKER_02: this is sort of why it's great in some sense that some of the problems that are
- 33:57SPEAKER_02: the most elastic to additional supply are also just completely orthogonal to AI.
- 34:00SPEAKER_02: I think there's a huge amount of great work to be done in global health
- 34:03SPEAKER_02: and well-being,
- 34:04SPEAKER_02: I think in biosecurity,
- 34:06SPEAKER_02: in tons of other fields that this money should hopefully just pour into.
- 34:10SPEAKER_02: But then within AI governance,
- 34:11SPEAKER_02: I think we just have to be really thoughtful about how we're setting these things up
- 34:14SPEAKER_02: and making sure that there is independent decision making that does not have
- 34:17SPEAKER_02: a vested interest with the places that are then giving the money.
- 34:20SPEAKER_02: Because the entire point is that AI governance needs to be separate from the people in the labs.
- 34:24SPEAKER_00: There's a kind of related question here about whether you should prioritize,
- 34:30SPEAKER_00: quote unquote,
- 34:31SPEAKER_00: secret Congress ideas or things that are kind of firmly in
- 34:35SPEAKER_00: the public's Overton window, right?
- 34:37SPEAKER_00: So IFP...
- 34:40SPEAKER_00: In the main, focused on issues that were not incredibly high salience.
- 34:43SPEAKER_00: That was part of the original thesis was issues that are important, tractable, and neglected.
- 34:50SPEAKER_00: To coin a phrase, which no one has ever said before.
- 34:53SPEAKER_00: We're going to focus on those because A,
- 34:55SPEAKER_00: we'll be more effective as a small team working on these neglected issues
- 34:58SPEAKER_00: and jumping in as the 20,000th organization speaking up on,
- 35:02SPEAKER_00: I don't know, cost of living.
- 35:07SPEAKER_00: Those problems are more amenable to just like backroom dealing or like influence
- 35:11SPEAKER_00: in this or that way.
- 35:12SPEAKER_00: AI plausibly was one of those domains and now obviously no longer is.
- 35:17SPEAKER_00: In fact, might Congress act?
- 35:19SPEAKER_00: Maybe it would be because it's become a massive salient issue,
- 35:23SPEAKER_00: increasing in salience faster than any other political issue.
- 35:25SPEAKER_00: If you're a philanthropist and you're like,
- 35:27SPEAKER_00: I want to affect policy,
- 35:29SPEAKER_00: I want to help American policy be better on the topic of AI,
- 35:33SPEAKER_00: can you really do the secret Congress game anymore?
- 35:35SPEAKER_00: Should you just be doing mailers in New Hampshire and Iowa?
- 35:39SPEAKER_00: How should you change your theory of impact?
- 35:43SPEAKER_00: When this stuff has been coming, seeing it so quickly.
- 35:44SPEAKER_02: I think there's a couple of angles here.
- 35:46SPEAKER_02: One is, I mean,
- 35:46SPEAKER_02: as I was sort of articulating earlier,
- 35:48SPEAKER_02: I think there are some areas that are sort of less political and more policy
- 35:53SPEAKER_02: that still just need an enormous amount of work.
- 35:55SPEAKER_02: All this like policy development that I think has like yet to happen
- 35:58SPEAKER_02: or just like needs kind of like a really urgent strike team on.
- 36:01SPEAKER_02: Again,
- 36:01SPEAKER_02: my hope is that is coming from a set of actors that are just like pretty pluralistic
- 36:05SPEAKER_02: and are sort of like inherently connected to a lot of
- 36:07SPEAKER_02: the like pre-existing American political traditions such that you're sort
- 36:10SPEAKER_02: of actually creating a thing that can kind of move.
- 36:13SPEAKER_02: One thing that's interesting about some of these crisis moments,
- 36:17SPEAKER_02: if we expect that might be the policymaking environment
- 36:19SPEAKER_02: that Congress feels like it's under,
- 36:21SPEAKER_02: is that it also sometimes is a luxury of being slightly more bipartisan than
- 36:25SPEAKER_02: the day-to-day grind of a big tax bill or a big health bill.
- 36:29SPEAKER_02: I mean, a lot of the COVID response packages passed with flying colors across party lines.
- 36:34SPEAKER_02: I wrote in my essay about thinking about policy as a kind of punctuated equilibrium,
- 36:39SPEAKER_02: where especially on the congressional side,
- 36:40SPEAKER_02: you go through long periods where nothing happens and then everything kind
- 36:43SPEAKER_02: of happens all at once.
- 36:45SPEAKER_02: And usually that's like when an issue has become so salient that
- 36:47SPEAKER_02: the public demands Congress has to do something about it right now.
- 36:50SPEAKER_02: And it sort of like overwhelms the day-to-day responsibilities
- 36:53SPEAKER_02: of whatever latest partisan infighting thing that we're doing.
- 36:56SPEAKER_02: And maybe to like briefly paint a picture
- 36:58SPEAKER_02: because I think this could actually happen much sooner
- 37:00SPEAKER_02: than maybe people are expecting.
- 37:01SPEAKER_02: I think it's like quite plausible that we have a window in spring,
- 37:05SPEAKER_02: summer of like 2027 that is in some sense driven by a lot
- 37:09SPEAKER_02: of crazy cyber capabilities that I think could be very near on the horizon.
- 37:13SPEAKER_02: Typically,
- 37:13SPEAKER_02: the trend is that like open source models will end up recreating
- 37:16SPEAKER_02: the frontier capabilities between four to six months after we have seen them on
- 37:20SPEAKER_02: the frontier.
- 37:21SPEAKER_02: And as we look at mythos level cyber capabilities,
- 37:25SPEAKER_02: which sort of became broadly available to the public in like roughly And
- 37:28SPEAKER_02: so my expectation
- 37:43SPEAKER_02: As a technical prediction,
- 37:44SPEAKER_02: we will start to see really crazy cyber capabilities kind of across the winter and
- 37:48SPEAKER_02: then early spring next year.
- 37:50SPEAKER_02: And I think these will be like quite bad.
- 37:53SPEAKER_02: And I think our institutions are in no way prepared for this.
- 37:56SPEAKER_02: There were some kind of like attempts to kind of prepare society.
- 37:59SPEAKER_02: I mean, we at IFP wrote a lot of,
- 38:01SPEAKER_02: I think,
- 38:01SPEAKER_02: really great pieces through the launch sequence about kind
- 38:03SPEAKER_02: of what would it look like to really do like an all hands on deck,
- 38:05SPEAKER_02: like let's kind of prepare for the incoming cyber craziness.
- 38:08SPEAKER_02: And unfortunately, I don't think our society has taken us up on those.
- 38:11SPEAKER_02: And so it might look like things like,
- 38:13SPEAKER_02: you know,
- 38:14SPEAKER_02: a 100x increase in the number of like very sophisticated grandparent scam attacks.
- 38:18SPEAKER_02: And like everyone personally knows someone in their family who's been affected by this.
- 38:21SPEAKER_02: Or the FAA has to randomly cancel flights,
- 38:24SPEAKER_02: you know,
- 38:24SPEAKER_02: a dozen times over the course of a month
- 38:26SPEAKER_02: because they don't actually have full confidence in their communications networks.
- 38:29SPEAKER_02: Or local hospital systems are just under a constant barrage
- 38:32SPEAKER_02: of very sophisticated ransomware attacks or any number of other things.
- 38:35SPEAKER_02: And again, I think these are bad and I don't want them to happen.
- 38:37SPEAKER_02: But I think to some extent they are inevitable unless we sort
- 38:39SPEAKER_02: of like really batten down our hatches and kind of get our things together from
- 38:42SPEAKER_02: a policy perspective for cyber resilience.
- 38:44SPEAKER_01: For those cyber risks,
- 38:45SPEAKER_01: I'm kind of curious how you would allocate your probability mass to the thing
- 38:48SPEAKER_01: that makes people wake up to the risk is non-state fraudster actors, right?
- 38:54SPEAKER_01: Where it is pig butchering grandparents just for money using these tools,
- 38:59SPEAKER_01: non-state political or terrorist actors, state actors, the AIs themselves.
- 39:05SPEAKER_01: So across those four buckets,
- 39:07SPEAKER_01: like you're anticipating large cyber disruption,
- 39:10SPEAKER_01: which do you think is most likely to be the driver of that?
- 39:13SPEAKER_02: There may be some combination of all.
- 39:14SPEAKER_02: I feel like a lot of the political or non-state actor stuff is kind of hard
- 39:18SPEAKER_02: to predict because it depends a lot on what the geopolitical situation is,
- 39:21SPEAKER_02: obviously.
- 39:22SPEAKER_01: That's why I'm just asking you to allocate probability mass.
- 39:25SPEAKER_01: You don't have to tell me what's going on in the world,
- 39:27SPEAKER_01: just where you're putting the probability mass.
- 39:28SPEAKER_02: I think I probably put the most probability masks,
- 39:31SPEAKER_02: at least in terms of the day-to-day,
- 39:33SPEAKER_02: just most number of incidents being driven for normal financial interests.
- 39:37SPEAKER_02: There's already a very large cybercrime operation.
- 39:41SPEAKER_02: One analogy I heard of this recently is,
- 39:43SPEAKER_02: imagine there's just another Russia that exists out in the world in some sense
- 39:47SPEAKER_02: in cyberspace that is just passively doing a bunch of cybercrime,
- 39:51SPEAKER_02: and that is maybe the best way to conceptualize what is going to happen here,
- 39:54SPEAKER_02: and different attacks will be at different levels of sophistication.
- 39:57SPEAKER_02: One area
- 39:58SPEAKER_02: where I would put surprisingly high probability is maybe just like autonomous AI
- 40:02SPEAKER_02: stuff that is not actually even intended by anyone.
- 40:05SPEAKER_02: Our kind of mutual friend Dean Ball wrote a really interesting post a couple
- 40:08SPEAKER_02: of months ago about the idea of like self-sovereign AI.
- 40:11SPEAKER_02: The fact that it seems like it will be trivially easy in the near future to set up
- 40:15SPEAKER_02: a Bitcoin wallet and buy some dark compute that's like not tracked anywhere and kind
- 40:19SPEAKER_02: of just like set an agent free and sort of say, you know, do whatever you want.
- 40:23SPEAKER_02: And there's a long history of people doing crazy things like this.
- 40:37SPEAKER_02: I want to see what can happen when you set up these systems.
- 40:40SPEAKER_02: And I think like some amount of stuff from that may also kind of cause really crazy cyber harms.
- 40:45SPEAKER_02: But it's hard to, I think, reason about the exact probabilities.
- 40:49SPEAKER_02: Mostly, I just feel like I expect cyber to become very salient.
- 40:52SPEAKER_02: And so I think people trying to anticipate or prepare for what is going
- 40:55SPEAKER_02: to be policy demand early next year,
- 40:57SPEAKER_02: I would say,
- 40:57SPEAKER_02: should be thinking hard about what sorts of cyber incident response do we want?
- 41:00SPEAKER_02: How do we batten down our hatches?
- 41:01SPEAKER_02: And then how do we bridge from that to kind of like the broader question
- 41:04SPEAKER_02: of frontier AI governance?
- 41:05SPEAKER_00: I want to spend a little more time on this because one thing that, Caleb,
- 41:09SPEAKER_00: you and I have reflected on over the past couple years is
- 41:12SPEAKER_00: that you get these moments,
- 41:13SPEAKER_00: whether you call it the Overton window suddenly opens up or
- 41:16SPEAKER_00: the train is suddenly leaving the station.
- 41:18SPEAKER_00: Do you have something put on it?
- 41:20SPEAKER_00: There's a lot of work you can do to develop good policy ideas abstractly before these moments.
- 41:25SPEAKER_00: But A, other people have their bad ideas that they're trying to get on the train.
- 41:29SPEAKER_00: You know, as it's leaving the station, a lot of those, they've also been working for a while.
- 41:33SPEAKER_00: Or there's all kinds of other actors who are generating ideas.
- 41:36SPEAKER_00: Often the thing that policymakers pick up in a moment of crisis is not
- 41:39SPEAKER_00: a very good idea, even if you had the good idea lying around ready to hand.
- 41:43SPEAKER_00: Or they haven't learned lessons in the way that you'd imagine they should learn lessons.
- 41:47SPEAKER_00: So post-COVID,
- 41:49SPEAKER_00: I think one reflection that many people had,
- 41:51SPEAKER_00: yourself included,
- 41:52SPEAKER_00: was like, Man, our political establishment did not learn the lessons of COVID.
- 41:56SPEAKER_00: Everyone kind of memory hold it.
- 41:58SPEAKER_00: It's like we're not doing all the pandemic preparedness stuff
- 42:00SPEAKER_00: that you would have wanted to do in 2019.
- 42:03SPEAKER_00: Also, on the other hand,
- 42:04SPEAKER_00: parts of our health expert establishment really did not – And
- 42:09SPEAKER_00: so you lost trust there.
- 42:10SPEAKER_00: And so in some ways, are we better prepared having experienced COVID for the next bio risk?
- 42:16SPEAKER_00: Maybe yes, maybe no.
- 42:16SPEAKER_00: But just planning ahead,
- 42:19SPEAKER_00: putting together your package for that next moment is necessary,
- 42:22SPEAKER_00: but maybe not sufficient.
- 42:23SPEAKER_00: So what color would you put?
- 42:24SPEAKER_00: How would you advise people?
- 42:25SPEAKER_00: It's not just write a really good research paper so that when the moment comes,
- 42:29SPEAKER_00: you can wave it around and say, I wrote a really good paper about this.
- 42:32SPEAKER_00: That clearly might be important, but it's not going to be the sum total of work.
- 42:36SPEAKER_00: If you're funding or advising people who want to influence policy,
- 42:41SPEAKER_00: what beyond come up with good ideas is necessary right now?
- 42:45SPEAKER_02: I think there's kind of like cultivating a diverse set of skill sets and expertise
- 42:50SPEAKER_02: because sometimes it's unpredictable what sorts of things are going
- 42:53SPEAKER_02: to end up being really useful.
- 42:54SPEAKER_02: I think this COVID example is interesting where if you think about what was
- 42:58SPEAKER_02: the most successful part of the policy response coming out of COVID,
- 43:01SPEAKER_02: it was in large part the kind of like Design and implementation
- 43:05SPEAKER_02: of advanced smart commitments for the COVID-19 vaccines.
- 43:09SPEAKER_02: And this is an area where it was very fortunate that we actually had a kind
- 43:12SPEAKER_02: of like small,
- 43:14SPEAKER_02: you know,
- 43:14SPEAKER_02: a bundle of,
- 43:15SPEAKER_02: I guess, like really talented policy economists who had been thinking Yeah.
- 43:34SPEAKER_02: And I think if we'd not have those sort of policy investments or that kind
- 43:42SPEAKER_02: of like living tacit knowledge expertise on how
- 43:45SPEAKER_02: to use advanced market amendments kind of lying around,
- 43:47SPEAKER_02: it seems very plausible to me that we're not going to have that.
- 43:48SPEAKER_02: That our COVID response would have been dramatically worse than it even was because
- 43:52SPEAKER_02: that was like, you know, one of the few kind of golden areas.
- 43:54SPEAKER_02: And maybe to some extent,
- 43:55SPEAKER_02: kind of our failure on some of the lack of COVID follow through was in part the fact
- 44:00SPEAKER_02: that we didn't actually have fully baked plans on the COVID.
- 44:04SPEAKER_02: How do we actually want to do good disease surveillance?
- 44:07SPEAKER_02: We sort of set up ad hoc these sorts of wastewater surveillance systems
- 44:12SPEAKER_02: where we were able to see where COVID was kind of like floating around
- 44:16SPEAKER_02: in sewer systems afterwards and sort of get us some sense of incidence.
- 44:19SPEAKER_02: But a lot of those systems didn't continue.
- 44:21SPEAKER_02: I didn't certainly see kind of like really robust,
- 44:24SPEAKER_02: well-developed plans for like how you could operationalize that on a tactical level.
- 44:27SPEAKER_02: There wasn't coalition building with like local municipal cities to try to like
- 44:33SPEAKER_02: I think there were some thoughtful people working on this,
- 44:38SPEAKER_02: but it was just like so small and kind of a little bit too late
- 44:42SPEAKER_02: to really take advantage of the momentum.
- 44:44SPEAKER_02: And if someone I think had been building out,
- 44:46SPEAKER_02: here's I think that would have dramatically increased the odds that
- 44:48SPEAKER_02: that could have been kind of like a key component of what the U.S. COVID response
- 45:02SPEAKER_02: And so, of course,
- 45:05SPEAKER_02: a lot of the responsibility here is also on legislatures who have been kind
- 45:08SPEAKER_02: of like letting their own policy development apparatus kind of like atrophy.
- 45:12SPEAKER_02: And I think this is why sort
- 45:14SPEAKER_02: of generic broad based interventions like let's beef up congressional capacity
- 45:17SPEAKER_02: and give them more staff,
- 45:19SPEAKER_02: more resources,
- 45:20SPEAKER_02: more ability to really,
- 45:21SPEAKER_02: really wrestle with these crises,
- 45:23SPEAKER_02: both in particular moments and kind of beforehand seems so important.
- 45:27SPEAKER_02: But I think there's also some ways that the outside ecosystem can supplement that.
- 45:30SPEAKER_00: Let me push you on this, though,
- 45:31SPEAKER_00: because I think I take your point that there's a lot of value
- 45:34SPEAKER_00: to having good plans ahead of time.
- 45:37SPEAKER_00: And obviously, plans are fallible, but planning is essential.
- 45:40SPEAKER_00: Functional institutions like the US military does a lot of planning,
- 45:42SPEAKER_00: and maybe you don't end up using the plan, but you built that muscle.
- 45:46SPEAKER_00: So I take that point.
- 45:47SPEAKER_00: But I guess it would be a failure of this philanthropic moment.
- 45:51SPEAKER_00: If what you seeded was a bunch of traditional looking think tanks who thought of it
- 45:55SPEAKER_00: as their job to produce good ideas and write them up and create good memos
- 45:59SPEAKER_00: and white papers.
- 46:01SPEAKER_00: And part of the reason that I think IFP has been successful is that
- 46:04SPEAKER_00: that has not felt like the sum total,
- 46:06SPEAKER_00: has not been explicitly the vision of the important work.
- 46:08SPEAKER_00: We're going to prepare good ideas so that when a critical moment comes,
- 46:11SPEAKER_00: a legislator will reach over behind them and be like, what's a good idea in the box?
- 46:16SPEAKER_00: There's actually a lot of socializing you have to do.
- 46:17SPEAKER_00: There's a lot of like What would the implementation look like?
- 46:20SPEAKER_00: Can we actually do that for you?
- 46:22SPEAKER_00: Can we grease the skids in this or that way?
- 46:25SPEAKER_00: Talk a little bit more about that because I'm unconvinced that we need
- 46:27SPEAKER_00: to develop better ideas ahead of a crisis is an especially big component of
- 46:32SPEAKER_00: the whole package that needs to be done.
- 46:34SPEAKER_02: This is a useful push.
- 46:35SPEAKER_02: And I think Congress is one of many institutions that's going to have
- 46:38SPEAKER_02: to respond well during any particular crisis.
- 46:40SPEAKER_02: But I guess maybe going back to COVID,
- 46:42SPEAKER_02: I think we've all kind of acknowledged that like the FDA and
- 46:45SPEAKER_02: the CDC were institutions that were supposed to be set up to kind
- 46:48SPEAKER_02: of like give us almost like Live response capacity in a crisis moment and just kind
- 46:52SPEAKER_02: of completely failed on the job.
- 46:54SPEAKER_02: I think the general diagnosis I've seen from I think the previous CDC director was
- 46:58SPEAKER_02: that like,
- 46:58SPEAKER_02: over time,
- 46:59SPEAKER_02: the CDC kind of atrophied their institutional like live readiness muscle
- 47:04SPEAKER_02: and they I think post-World War II,
- 47:05SPEAKER_02: and kind of like really beginning,
- 47:07SPEAKER_02: they succeeded so much that they actually like mostly eliminated malaria,
- 47:09SPEAKER_02: but then after
- 47:19SPEAKER_02: After that success,
- 47:21SPEAKER_02: and they were very much at that time kind of like
- 47:22SPEAKER_02: a wartime logistics style organization.
- 47:25SPEAKER_02: And the kinds of people that they brought into
- 47:26SPEAKER_02: the organization were like logistics supply chain experts who really knew how
- 47:30SPEAKER_02: to like manufacture countermeasures and kind of deploy them in the field and kind
- 47:34SPEAKER_02: of like ring fence diseases and try to push them back.
- 47:36SPEAKER_02: And then after we succeeded at that, it didn't seem like there was a next thing to do.
- 47:40SPEAKER_02: And so the kinds of people who ended up getting hired by the CDC were like,
- 47:43SPEAKER_02: And so that is, I guess, a version of having unwise institutions.
- 47:58SPEAKER_02: Substitutions are maybe uncapable in this case,
- 48:01SPEAKER_02: is that when you have a response function,
- 48:04SPEAKER_02: you have to find some way to keep practicing it in the world and instantiate
- 48:07SPEAKER_02: and sort of keep alive a body of knowledge that has tacit knowledge.
- 48:10SPEAKER_02: And so maybe this is where a lot of the think tanks or policy apparatus are round.
- 48:17SPEAKER_02: I mean,
- 48:22SPEAKER_02: when I think about IFP's previous successes,
- 48:25SPEAKER_02: a lot of them have been more kind of like agency focused,
- 48:28SPEAKER_02: kind of trying to help the NIH or the NSF really like think about their kind
- 48:32SPEAKER_02: of like science funding mechanism.
- 48:34SPEAKER_02: And it was not just like throwing ideas over the fence at Congress and expecting them to act.
- 48:38SPEAKER_02: And so I think that's like another component of
- 48:39SPEAKER_02: a good response here is like both you have to think about the new institutions and
- 48:42SPEAKER_02: then actually have societal mechanisms around them to make sure that they kind
- 48:46SPEAKER_02: of like stay ready for primetime.
- 48:48SPEAKER_00: One other piece here, important intervention that comes to mind.
- 48:52SPEAKER_00: And I think we all we're all in violent agreement here that implementation really matters.
- 48:56SPEAKER_00: Listeners are all clapping and cheering.
- 48:57SPEAKER_02: They're all standing up on their chairs and saying,
- 48:59SPEAKER_02: whoo, Statecraft is talking about implementation and state capacity again.
- 49:03SPEAKER_00: But I'm thinking of folks at IFP who came from the CHIPS program office,
- 49:06SPEAKER_00: CHIPS Act being the kind of legislative push to spur an investment boom
- 49:11SPEAKER_00: in semiconductors on American soil.
- 49:13SPEAKER_00: And the CHIPS team,
- 49:14SPEAKER_00: they've written quite a lot about they had some kind of great carve-outs from
- 49:18SPEAKER_00: the existing paper and bureaucracy that you usually go through that enabled them
- 49:22SPEAKER_00: to move quite quickly.
- 49:23SPEAKER_00: And then they had other cases where they did not get the kind of full extent of
- 49:27SPEAKER_00: the carve out that they wanted or that would have been useful
- 49:29SPEAKER_00: unless they spent hundreds of thousands or tens of thousands
- 49:32SPEAKER_00: of man hours filing forms in triplicate.
- 49:35SPEAKER_00: Basically, they thought that had a real drag on their performance.
- 49:39SPEAKER_00: To bring everything back to my favorite pet topic,
- 49:41SPEAKER_00: State Capacity,
- 49:42SPEAKER_00: there's plausibly an argument
- 49:43SPEAKER_00: that you should be not just prepping good policy ideas,
- 49:46SPEAKER_00: but like Doing work on, okay, how might you want to set this agency up?
- 49:50SPEAKER_00: Or how might you want to carve out this institution from a given set of bureaucratic restraints?
- 49:55SPEAKER_00: Or how might you want to loosen those restraints in general on the assumption
- 49:58SPEAKER_00: that anything that we do in the next couple years will really benefit from speed?
- 50:01SPEAKER_02: I think that's really important.
- 50:02SPEAKER_02: I guess I almost want to push this even farther and say that I,
- 50:05SPEAKER_02: of course, love the classic state capacity.
- 50:07SPEAKER_02: Let's do things.
- 50:08SPEAKER_02: Let's think about the legal mechanisms.
- 50:09SPEAKER_02: But I still don't want to underrate how potentially crazy or insane this moment
- 50:14SPEAKER_02: with AI could feel and that actually a lot of our normal processes for dealing
- 50:18SPEAKER_02: with institutional failure and adaptation and kind of like,
- 50:20SPEAKER_02: oh, we messed up that procurement contract.
- 50:23SPEAKER_02: We'll definitely get it right this time.
- 50:25SPEAKER_02: That kind of works in a normal cycle and this trial and error
- 50:28SPEAKER_02: and iteration process has served us well across a lot of mechanisms,
- 50:32SPEAKER_02: especially when you have the chance to get it right.
- 50:35SPEAKER_02: One of the things that I'm most concerned about from AI Yeah.
- 50:53SPEAKER_02: Yeah.
- 51:08SPEAKER_02: And I kind of feel like from an institutional level,
- 51:09SPEAKER_02: we're going to need to do things like that.
- 51:23SPEAKER_02: And I think we need like 10 other ideas like that where we're able
- 51:24SPEAKER_02: to do like really rapid iteration experimentation and testing
- 51:27SPEAKER_02: for new institutional forms outside of the confines of government,
- 51:29SPEAKER_02: but like with the government.
- 51:43SPEAKER_01: And what does that look like?
- 51:44SPEAKER_01: That's a good example of the government successfully eating an organization
- 51:48SPEAKER_01: that spun up independently with the nimbleness that wouldn't have happened
- 51:52SPEAKER_01: if it had been incubated in government but was successfully integrated later.
- 51:56SPEAKER_02: It's a good question.
- 51:57SPEAKER_02: I don't know that I actually have like super concrete examples here for like the pod.
- 52:03SPEAKER_02: I mean, I feel like there's got to be some like post-World War II where,
- 52:06SPEAKER_02: you know,
- 52:06SPEAKER_02: there was like the National Defense Research Board that was kind of like
- 52:10SPEAKER_02: a quasi-governmental thing that was basically borrowing a bunch of kind
- 52:13SPEAKER_02: of like scientific expertise from like universities and then sort of de facto formed
- 52:18SPEAKER_02: that into like the NSF and the NIH.
- 52:21SPEAKER_01: What do we gain or lose if Meter gets eaten by Casey tomorrow?
- 52:24SPEAKER_02: I mean, if Meter gets eaten by Casey tomorrow, it could potentially be quite good.
- 52:29SPEAKER_02: But I think it depends a lot on how robust.
- 52:32SPEAKER_02: I mean,
- 52:32SPEAKER_02: I think Meter would be the first ones to tell you right now that they are not at
- 52:35SPEAKER_02: the size or scale.
- 52:38SPEAKER_02: What they are doing right now in terms of embedded audits is very far away from
- 52:42SPEAKER_02: a science and it's still kind of more like an art they are trying to go in and sort
- 52:46SPEAKER_02: of make an all things considered sense of like,
- 52:48SPEAKER_02: which areas here seem high risk or low risk or which things are Are we most worried
- 52:51SPEAKER_02: about kind of like being off track or ways in which kind of AIs could escape
- 52:54SPEAKER_02: the different kinds of like defense mechanisms that we've set up?
- 52:57SPEAKER_02: But usually when things get like instantiated within the government,
- 53:00SPEAKER_02: we want there to be something more like a process or a procedure.
- 53:03SPEAKER_02: We have a checklist of things that we're going to like look for.
- 53:04SPEAKER_02: And I think there's some areas of AI that we can do that for things like,
- 53:08SPEAKER_02: you know, are you below some threshold of risk for like CBRN risk?
- 53:11SPEAKER_02: It seems like the kind of thing you could theoretically set up kind of like
- 53:14SPEAKER_02: a whole apparatus to do.
- 53:15SPEAKER_02: And I think there's a lot of really thoughtful people at KC and other places
- 53:18SPEAKER_02: that are trying to do basically versions of this.
- 53:20SPEAKER_02: But I think this is like one of the tricky parts is that our understanding,
- 53:23SPEAKER_02: our like scientific understanding of AI is actually still at a premature phase.
- 53:26SPEAKER_02: And yet we are now starting to reach kind of like quite dangerous levels of capability.
- 53:30SPEAKER_02: And so I would be really excited about a world where like Casey was empowered
- 53:34SPEAKER_02: to still do this kind of like scientific exploration at the same time
- 53:37SPEAKER_02: that they have kind of the importure of the federal government and kind of
- 53:40SPEAKER_02: that stamp of legitimacy.
- 53:42SPEAKER_02: And yeah,
- 53:42SPEAKER_02: hopefully a lot of the like preexisting capacity around evaluation can be a big part
- 53:46SPEAKER_02: of informing that.
- 53:47SPEAKER_02: But I think one of my concerns right now, yeah,
- 53:49SPEAKER_02: if Casey were to get eaten,
- 53:50SPEAKER_02: is that you would lose the kind of more scientific,
- 53:54SPEAKER_02: exploratory,
- 53:55SPEAKER_02: all things considered view, and it would become more like formulaic checklist.
- 53:58SPEAKER_00: I will say this is one of the reasons I think,
- 54:01SPEAKER_00: maybe this is different ways of talking about the same thing,
- 54:03SPEAKER_00: but even if you think that you want a big portfolio of bets,
- 54:07SPEAKER_00: that you want to beef up Casey and you want to beef up external evaluators
- 54:11SPEAKER_00: and you want to beef up the British ACs,
- 54:14SPEAKER_00: You still want to maximize the value of your bets,
- 54:17SPEAKER_00: which might mean Casey needs to be able to pay people much more,
- 54:20SPEAKER_00: that it needs to be able to hire much faster than it needs.
- 54:23SPEAKER_00: All kinds of things might fall under a traditional state capacity rubric,
- 54:26SPEAKER_00: even if you also think we need to fund a bunch of private or nonprofit
- 54:30SPEAKER_00: or other novel kinds of entities.
- 54:33SPEAKER_02: Totally.
- 54:33SPEAKER_02: This is not to say that the traditional state capacity agenda has nothing to say here.
- 54:37SPEAKER_02: In fact, I think it is deeply, deeply relevant.
- 54:38SPEAKER_02: Also, just that it is, I think, not by itself going to be enough to solve the problem.
- 54:42SPEAKER_00: Agreed.
- 54:43SPEAKER_00: With some of the time we have left,
- 54:45SPEAKER_00: I want to talk a little bit about the orientation you had in this essay,
- 54:49SPEAKER_00: Caleb, was largely around wise institutions.
- 54:52SPEAKER_00: You use the language of the virtues to talk about what are the kinds of institutions
- 54:56SPEAKER_00: that we might need.
- 54:58SPEAKER_00: One version of that is institutions need an information feedback loop.
- 55:03SPEAKER_00: They need to be able to learn in the outside world.
- 55:05SPEAKER_00: When things change in the real world, they need to adjust accordingly.
- 55:08SPEAKER_00: And I think that's a very obvious feature of good institutions that they don't put
- 55:14SPEAKER_00: on their horse blinders about things that happen.
- 55:17SPEAKER_00: And we can all think of institutions that are unable to do that.
- 55:19SPEAKER_00: Beyond feedback loops,
- 55:22SPEAKER_00: like getting better information,
- 55:24SPEAKER_00: making decisions as a result of that information,
- 55:27SPEAKER_00: what are other qualifications that maybe inside or outside
- 55:30SPEAKER_00: the government you think right now it's really important
- 55:33SPEAKER_00: to cultivate those institutional virtues?
- 55:35SPEAKER_02: Yeah.
- 55:36SPEAKER_02: Maybe a couple of examples that come to mind.
- 55:38SPEAKER_02: Some partially motivated by the COVID discussion that we were having.
- 55:40SPEAKER_02: But when I think about what were the failures of, say,
- 55:43SPEAKER_02: the CDC or our public health institutions,
- 55:46SPEAKER_02: I think one of them was they didn't actually understand the role
- 55:49SPEAKER_02: that they were supposed to play.
- 55:50SPEAKER_02: And I think they overstepped their bounds in terms of what kind of authority
- 55:54SPEAKER_02: that they were trying to claim.
- 55:55SPEAKER_02: And that was a big part of the backlash was basically these were institutions
- 55:58SPEAKER_02: that were largely scientific,
- 56:01SPEAKER_02: technical bodies that were trying
- 56:02SPEAKER_02: to make all things considered political determinations that were,
- 56:05SPEAKER_02: I think, outside of their scope.
- 56:08SPEAKER_02: It's one thing to say that we think risk level from an epidemiological perspective
- 56:13SPEAKER_02: of making policy decision X is Y.
- 56:15SPEAKER_02: It's very different to say you have to do this or something and
- 56:18SPEAKER_02: that this is what it takes to be a good person or a person of good upstanding
- 56:23SPEAKER_02: in our political system.
- 56:24SPEAKER_02: And so knowing what our political determinations that you as an institution are able
- 56:28SPEAKER_02: or have a legitimacy to make and what are ones that you do not is actually like
- 56:32SPEAKER_02: a key function of wisdom,
- 56:33SPEAKER_02: like knowing what role you are supposed to play in the ecosystem.
- 56:35SPEAKER_02: And I think especially in AI,
- 56:36SPEAKER_02: this is something that you could see come up to speed and kind of knowing about
- 56:40SPEAKER_02: that fundamental tradeoff is, I think, one key thing.
- 56:43SPEAKER_02: Another is kind of knowing how to fail gracefully when possible.
- 56:47SPEAKER_02: Again, I think we should expect that
- 56:48SPEAKER_02: if we are actually having ambitious institutions attempting to do lots
- 56:51SPEAKER_02: of different things that have never been done before or have been done
- 56:54SPEAKER_02: in very small settings or outside of government, we should expect that they are not.
- 56:59SPEAKER_02: But there are ways of failing spectacularly and there are ways of failing gracefully
- 57:03SPEAKER_02: and trying to think through like what are robust solutions?
- 57:06SPEAKER_02: What are going to be ones that we sort of like are glad we took in most to all worlds?
- 57:11SPEAKER_02: And how do we sort of like game that out or think about that ahead of time?
- 57:15SPEAKER_02: I think there's avoiding this kind of executing a script mentality
- 57:19SPEAKER_02: that government agencies very frequently fall into.
- 57:21SPEAKER_02: I think we want to cultivate live players that in some sense are able
- 57:25SPEAKER_02: to constantly reassess the field and reevaluate what is the good thing to do.
- 57:28SPEAKER_02: And then maybe finally, there's a form of having democratic legitimacy.
- 57:33SPEAKER_02: I think if we want to have institutions that are actually able
- 57:36SPEAKER_02: to make meaningful decisions for us as society,
- 57:40SPEAKER_02: In key moments of crisis and on a fast basis point,
- 57:44SPEAKER_02: like to do that well requires some sort of buy-in from the set of actors
- 57:49SPEAKER_02: that you were claiming to represent.
- 57:51SPEAKER_02: I think that both is like inputs and outputs.
- 57:53SPEAKER_02: It both means like, are you representing?
- 57:55SPEAKER_02: Presenting perspectives from a very pluralistic diverse set of kind
- 57:58SPEAKER_02: of coalitional actors such that when you make determinations
- 58:01SPEAKER_02: or decisions you are not just speaking to like one very narrow technical community
- 58:05SPEAKER_02: but like actually have buy-in across a pretty large set of society and for a lot of
- 58:09SPEAKER_02: the kinds of governance mechanisms that And I think that can be really easy to miss
- 58:12SPEAKER_02: and is kind of really important.
- 58:28SPEAKER_01: Kind of a theme of what we've been talking about this whole time is how
- 58:30SPEAKER_01: to form people in institutions for a high volatility, short timelines environment.
- 58:36SPEAKER_01: And I've worked places where we thought about where do we find people
- 58:39SPEAKER_01: who are well adapted to the zone we need to work in?
- 58:43SPEAKER_01: And when we were hiring ops people,
- 58:45SPEAKER_01: we found we were unusually likely to find good ops people
- 58:47SPEAKER_01: who weren't already ops people by looking at stage managers.
- 58:51SPEAKER_01: People who had handled...
- 58:52SPEAKER_01: Chaos in the physical world and non-negotiable deadlines well.
- 58:56SPEAKER_01: When you look for people who aren't currently working in these fields or don't think
- 59:01SPEAKER_01: of themselves as defined by working in a high volatility policy future,
- 59:05SPEAKER_01: where is the unexpected place you're likely to find that they exist
- 59:09SPEAKER_01: where they're being well-formed for these challenges?
- 59:12SPEAKER_02: Yeah, it's interesting.
- 59:13SPEAKER_02: I think there are some fields that by their nature are kind of like faster moving
- 59:19SPEAKER_02: or rely on kind of like Both more delegation and iteration.
- 59:24SPEAKER_02: I think the national security community and
- 59:26SPEAKER_02: the intelligence community are interesting places for this.
- 59:28SPEAKER_02: They are also well used to dealing with issues of national security.
- 59:34SPEAKER_02: They have something more like a security mindset where you imagine all
- 59:37SPEAKER_02: the different ways something could fail and try to proactively plan for that rather
- 59:41SPEAKER_02: than just thinking about the two most common ones and saying,
- 59:43SPEAKER_02: okay, we've probably got it handled.
- 59:45SPEAKER_02: We've got it good.
- 59:45SPEAKER_02: So cultivating the security mindset seems good.
- 59:48SPEAKER_02: I think people have like pivoted across different fields before are kind
- 59:51SPEAKER_02: of unusually able to kind of like pivot to this one.
- 59:54SPEAKER_02: I mean, it's interesting, again,
- 59:56SPEAKER_02: not to keep going back to COVID,
- 59:57SPEAKER_02: but it was interesting to think about the kinds of characters and institutions
- 1:00:01SPEAKER_02: that were able to actually like not that many.
- 1:00:03SPEAKER_02: So many people in society had been thinking about biodefense prior to COVID.
- 1:00:07SPEAKER_02: And yet,
- 1:00:08SPEAKER_02: actually,
- 1:00:09SPEAKER_02: one way that Alec,
- 1:00:10SPEAKER_02: my co-founder at IFP,
- 1:00:11SPEAKER_02: and I first really worked together on a project was
- 1:00:13SPEAKER_02: the Mercatus Center did this really interesting call for novel policy solutions.
- 1:00:18SPEAKER_02: So the vibe was like, hey, we're probably going to need to really think outside the box.
- 1:00:22SPEAKER_02: We do not have a well-developed set of policy actions here.
- 1:00:25SPEAKER_02: And Alec and I ended up working together on a policy brief that was,
- 1:00:28SPEAKER_02: we didn't have the language for at the time,
- 1:00:29SPEAKER_02: but effectively like a Operation Warp Speed, but for mask production.
- 1:00:33SPEAKER_02: And really trying to kind of accelerate that dramatically and kind
- 1:00:36SPEAKER_02: of give manufacturing demand certainty to sort
- 1:00:38SPEAKER_02: of incentivize more upfront investment.
- 1:00:41SPEAKER_02: I think it was like both good for Alec and I,
- 1:00:42SPEAKER_02: even though it was entirely outside of our wheelhouse.
- 1:00:45SPEAKER_02: But the fact that like Mercatus as an institution kind of like saw this as part
- 1:00:48SPEAKER_02: of their role of like putting out the call for proposals and really trying to send
- 1:00:51SPEAKER_02: the demand signal that like this is where we needed more effort was I think like
- 1:00:53SPEAKER_02: a big credit to them as an institution.
- 1:00:55SPEAKER_02: Yeah, I'm sure there are others I can think of as well,
- 1:00:57SPEAKER_02: but those are the first ones that come to mind.
- 1:00:58SPEAKER_01: And just from my personal curiosity,
- 1:01:00SPEAKER_01: you know,
- 1:01:01SPEAKER_01: how do the virtues of being Someone working in a high volatility environment as
- 1:01:06SPEAKER_01: a philanthropist,
- 1:01:07SPEAKER_01: working with a wide variety of recalcitrant stakeholders,
- 1:01:10SPEAKER_01: and being a parent to young kids.
- 1:01:12SPEAKER_01: How do those virtues cross over?
- 1:01:14SPEAKER_01: Which ones are helpful in both domains?
- 1:01:16SPEAKER_02: Emotional stability, I think, is very important.
- 1:01:19SPEAKER_02: Sort of a kind of compartmentalization, I think, is also useful.
- 1:01:23SPEAKER_02: I find it very helpful when you're working on AI policy.
- 1:01:25SPEAKER_02: I think you're oftentimes thinking about these extremely heady, uncertain things.
- 1:01:29SPEAKER_02: But if they happen in some sense,
- 1:01:31SPEAKER_02: in the way some people are predicting, they have existential stakes.
- 1:01:34SPEAKER_02: And it can be very easy to kind of let that become all-consuming and demanding.
- 1:01:38SPEAKER_02: And then I find it very helpful and balancing to go home and Have my two and
- 1:01:41SPEAKER_02: a half year old say like, daddy, I want to play like catch.
- 1:01:44SPEAKER_02: And just like it's a nice circuit breaker.
- 1:01:46SPEAKER_02: And it kind of both connects you and it roots you with like, why does this all matter?
- 1:01:49SPEAKER_02: What are we all doing this for?
- 1:01:50SPEAKER_02: And also it's kind of like a healthy way of just like forcing some kind
- 1:01:53SPEAKER_02: of like emotional balance.
- 1:01:54SPEAKER_02: And so I personally have found that very useful.
- 1:01:55SPEAKER_00: The classical answer to your question, we have which virtues carry over is all of them.
- 1:01:59SPEAKER_00: That's why they're the virtues.
- 1:02:02SPEAKER_00: If there were domain specific, they would not in fact be virtues.
- 1:02:04SPEAKER_00: Let me ask one more kind of concrete question,
- 1:02:07SPEAKER_00: which is about existing legacy institutions that don't work very well.
- 1:02:10SPEAKER_00: So there's,
- 1:02:11SPEAKER_00: I think,
- 1:02:11SPEAKER_00: a real need,
- 1:02:12SPEAKER_00: especially for philanthropists,
- 1:02:14SPEAKER_00: to think hard about what are the new kinds of institutions we might need
- 1:02:17SPEAKER_00: or what are these kind of big creative bets that we can take
- 1:02:20SPEAKER_00: a la Operation Warp Speed that are worth taking and they're robust.
- 1:02:25SPEAKER_00: But we're also just as a result of our history,
- 1:02:28SPEAKER_00: given a bunch of institutions like Congress,
- 1:02:31SPEAKER_00: which just everybody knows they don't work very well, or they kind of lumber along.
- 1:02:35SPEAKER_00: They're hard to fix.
- 1:02:36SPEAKER_00: Many have tried, few have succeeded.
- 1:02:39SPEAKER_00: And nevertheless, they're going to be Very important in these kinds of fast moving moments.
- 1:02:45SPEAKER_00: If you're somebody who's coming to this set of problems of frontier AI policy
- 1:02:50SPEAKER_00: and you want to deploy capital,
- 1:02:53SPEAKER_00: do you have any argument that they should try and make Congress work better?
- 1:02:56SPEAKER_00: I think the natural impulse of many from the West Coast is going to be like, Screw that stuff.
- 1:03:01SPEAKER_00: If I'm going to do federal stuff, it's going to be like an Elon style.
- 1:03:03SPEAKER_00: We want to go in, work on the executive or agency side.
- 1:03:07SPEAKER_00: IFP spent a lot of time working on the agency side because Congress is less tractable.
- 1:03:12SPEAKER_00: How do you weight your effort?
- 1:03:15SPEAKER_00: Should somebody be really trying to double the amount of money
- 1:03:18SPEAKER_00: that Congress sets aside for itself in the next Congress, for instance?
- 1:03:23SPEAKER_00: How high leverage is that?
- 1:03:25SPEAKER_02: Yeah.
- 1:03:26SPEAKER_02: I think like a vacuum, it seems quite good.
- 1:03:28SPEAKER_02: I mean, you do run into again,
- 1:03:29SPEAKER_02: like a timelines issue here of like,
- 1:03:30SPEAKER_02: if you're running on an appropriations process and you're trying to get Congress
- 1:03:33SPEAKER_02: to double, you know, it's like resourcing.
- 1:03:35SPEAKER_02: A year or two from now,
- 1:03:36SPEAKER_02: and then by the time that kicks in and you start hiring people
- 1:03:39SPEAKER_02: and building IT systems, you're now looking five, seven years down the line.
- 1:03:42SPEAKER_02: Again, I think we should do that.
- 1:03:44SPEAKER_02: That seems like a pretty robustly good solution.
- 1:03:46SPEAKER_02: Really hard to imagine how that goes wrong, but it may also not be enough by itself.
- 1:03:51SPEAKER_02: For a long time,
- 1:03:51SPEAKER_02: I think people in a variety of different policy communities have realized
- 1:03:54SPEAKER_02: because Congress is slow,
- 1:03:56SPEAKER_02: slow acting,
- 1:03:56SPEAKER_02: there's a huge amount of leverage or alpha and focusing on things
- 1:04:00SPEAKER_02: the executive branch can do.
- 1:04:01SPEAKER_02: You know, famously, I have a pen and I have a phone.
- 1:04:05SPEAKER_02: But Congress, I think, is going to act on AI.
- 1:04:09SPEAKER_02: And I think it would then be a mistake,
- 1:04:10SPEAKER_02: given Congress is going to act,
- 1:04:12SPEAKER_02: to not allocate pretty substantial efforts to improve how those responses are going
- 1:04:17SPEAKER_02: to go.
- 1:04:17SPEAKER_02: There's tons of things the executive branch cannot do by themselves and
- 1:04:20SPEAKER_02: that Congress is the ultimate owner for.
- 1:04:22SPEAKER_02: But I think there's sort of like,
- 1:04:24SPEAKER_02: you know, the technical folks sometimes like to say, update all the way.
- 1:04:27SPEAKER_02: If you notice that you are constantly kind of every three months kind
- 1:04:31SPEAKER_02: of making your AI timelines a little bit shorter,
- 1:04:32SPEAKER_02: a little bit shorter,
- 1:04:33SPEAKER_02: a little bit shorter,
- 1:04:34SPEAKER_02: anticipate where they are going to go and just make that update now.
- 1:04:37SPEAKER_02: And there's a similar kind of thing on political will that people have sort
- 1:04:40SPEAKER_02: of like underrated where...
- 1:04:42SPEAKER_02: You know,
- 1:04:42SPEAKER_02: if you keep noticing, oh, wow,
- 1:04:43SPEAKER_02: there's like this endogenous social response,
- 1:04:45SPEAKER_02: it seems like the American Congress,
- 1:04:47SPEAKER_02: the American government,
- 1:04:47SPEAKER_02: the Arab people are going to want to have a meaningful say in how this is going
- 1:04:50SPEAKER_02: to go, which they absolutely should, obviously.
- 1:04:52SPEAKER_02: But I think this is maybe the thing that my Silicon Valley friends have been like
- 1:04:55SPEAKER_02: the most wrong on is sort of like,
- 1:04:57SPEAKER_02: And so even if you are not interested in the government,
- 1:05:02SPEAKER_02: the government is interested in you.
- 1:05:04SPEAKER_02: And so even if you think the response is by default going to be bad,
- 1:05:07SPEAKER_02: taking what might be by default like a D minus response and making it a B minus,
- 1:05:11SPEAKER_02: there's like an enormous amount of like leverage in that.
- 1:05:13SPEAKER_02: And so, you know, don't let the enemy be the perfect or the good.
- 1:05:15SPEAKER_02: They're going to act,
- 1:05:16SPEAKER_02: it seems like really worthwhile and in high leverage to try to like make that
- 1:05:19SPEAKER_02: as good as we can.
- 1:05:20SPEAKER_02: There's also just like a lot of like problems here that I think are kind
- 1:05:23SPEAKER_02: of really hard to imagine solving without the government,
- 1:05:26SPEAKER_02: you know, collective action problems all the way down.
- 1:05:29SPEAKER_02: And that's like kind of the classical role of the government is trying
- 1:05:31SPEAKER_02: to like help us as a society navigate to these collective action problems.
- 1:05:34SPEAKER_00: Caleb, as we close,
- 1:05:36SPEAKER_00: I want you to give your charge one more time in brief to the listeners who need
- 1:05:41SPEAKER_00: to quit their jobs.
- 1:05:43SPEAKER_02: You...
- 1:05:44SPEAKER_02: You only have so much time in your life in general.
- 1:06:03SPEAKER_02: I think this is like a period of intense leverage over not just your life,
- 1:06:07SPEAKER_02: but over kind of like the next century of kind of human civilization and society.
- 1:06:12SPEAKER_02: And I think if you are in any way the sidelines of kind of wondering
- 1:06:15SPEAKER_02: whether you can contribute to this,
- 1:06:17SPEAKER_02: wondering whether you You can have a positive impact.
- 1:06:43SPEAKER_02: Really important work at basically every layer of the stack,
- 1:06:45SPEAKER_02: on the technical level,
- 1:06:46SPEAKER_02: on the governance layer,
- 1:06:47SPEAKER_02: on the organizing level,
- 1:06:48SPEAKER_02: on research production,
- 1:06:49SPEAKER_02: on empirical results, on financial modeling, on any number of things.
- 1:06:53SPEAKER_02: And so I think now is the time to get in the game.
- 1:06:55SPEAKER_00: That's a perfect note to end on.
- 1:06:57SPEAKER_00: Leah Sargent and Caleb Watney, thank you for joining Statecraft.
- 1:07:01SPEAKER_00: Great to be here.
- 1:07:02SPEAKER_01: Very fun.
- 1:07:03[music]