Latest / Sharp China Podcast / Sharp China: The Limited Potential for a Pacing Deal; MSS on PRC AI Risks; Distillation and Data Security; The Party's Approach to Social Media

Transcript

Sharp China: The Limited Potential for a Pacing Deal; MSS on PRC AI Risks; Distillation and Data Security; The Party's Approach to Social Media · Sharp China Podcast

• 0:04
  SPEAKER_02: Hello and welcome to Sharp China.
• 0:06
  SPEAKER_02: I'm Andrew Sharp and you are listening to a free preview of today's episode.
• 0:12
  SPEAKER_01: Hello and welcome back to another episode of Sharp China.
• 0:15
  SPEAKER_01: I'm Andrew Sharp and on the other line, Bill Bishop.
• 0:20
  SPEAKER_01: Bill, how you doing?
• 0:22
  SPEAKER_00: I'm doing well, Andrew.
• 0:23
  SPEAKER_00: It's good to see you.
• 0:23
  SPEAKER_00: And hi, everybody.
• 0:24
  SPEAKER_00: Apologies for last week.
• 0:25
  SPEAKER_00: We had a scheduling glitch.
• 0:27
  SPEAKER_00: That's right.
• 0:27
  SPEAKER_01: It is good to be back here.
• 0:30
  SPEAKER_01: And it is AI Day on Sharp China.
• 0:32
  SPEAKER_01: Do you have your models nearby ready for this conversation?
• 0:36
  SPEAKER_00: I'm actually, this is the agent talking.
• 0:38
  SPEAKER_01: Great.
• 0:39
  SPEAKER_01: Okay, good.
• 0:40
  SPEAKER_00: I'm talking to Astra for the next 30 or 40 minutes.
• 0:42
  SPEAKER_00: No, no.
• 0:43
  SPEAKER_00: I ride it through Seed Dance.
• 0:44
  SPEAKER_00: This is a ByteDance avatar.
• 0:46
  SPEAKER_01: Oh, perfect.
• 0:47
  SPEAKER_00: I look much better.
• 0:48
  SPEAKER_00: I look younger.
• 0:48
  SPEAKER_00: It's all good.
• 0:50
  SPEAKER_01: You do look good.
• 0:51
  SPEAKER_00: Just don't mention the name of a certain leader because otherwise I'll just sort of freeze.
• 0:57
  SPEAKER_01: Right.
• 0:58
  SPEAKER_01: Okay.
• 0:58
  SPEAKER_01: Good to know.
• 1:00
  SPEAKER_01: Well, AI is at the top of the world's agenda right now.
• 1:04
  SPEAKER_01: And the PRC is central to the conversation, central to the conversation in the United States.
• 1:11
  SPEAKER_01: So we can talk about a couple different dimensions of AI pacing and the PRC.
• 1:18
  SPEAKER_01: I will start with the potential for pausing or pacing tweet from Bernie Sanders on Sunday.
• 1:24
  SPEAKER_01: He says,
• 1:39
  SPEAKER_01: To establish a pause on advanced AI development and a ban on superintelligence.
• 1:45
  SPEAKER_01: In order to prevent a nuclear war,
• 1:47
  SPEAKER_01: Reagan and Gorbachev negotiated a nuclear arms agreement in the 1980s.
• 1:52
  SPEAKER_01: In order to prevent AI from acting independently of human control,
• 1:57
  SPEAKER_01: Trump and Xi must do the same.
• 2:00
  SPEAKER_01: So what do you think of the potential for an AI global treaty on slowing the pace
• 2:07
  SPEAKER_01: of AI progress?
• 2:09
  SPEAKER_01: Because Bernie is pushing toward that outcome.
• 2:11
  SPEAKER_01: I've seen more centrist Dems like John Ossoff pushing for the same thing.
• 2:16
  SPEAKER_01: Kamala Harris came out lobbying for the same sort of outcome.
• 2:21
  SPEAKER_01: I've read you say in a few different places that you are skeptical.
• 2:24
  SPEAKER_01: How do you think about the possibilities there?
• 2:26
  SPEAKER_00: I mean, I think it's close or at zero at this point.
• 2:30
  SPEAKER_00: One, you know,
• 2:31
  SPEAKER_00: Bernie also has a bill where he wants to jail researchers for 20 years
• 2:35
  SPEAKER_00: who are working on certain aspects of artificial intelligence.
• 2:39
  SPEAKER_00: So, you know, again,
• 2:40
  SPEAKER_00: I know he's an important politician in the Democratic Party,
• 2:42
  SPEAKER_00: but I don't think we should be looking to him for ideas about how
• 2:46
  SPEAKER_00: to solve this mess.
• 2:47
  SPEAKER_01: I agree.
• 2:48
  SPEAKER_01: Yes.
• 2:49
  SPEAKER_00: Sorry, we're probably going to get some hate mail on that one.
• 2:51
  SPEAKER_01: I don't think we will, honestly, as far as our audience is concerned.
• 2:54
  SPEAKER_00: We don't have a lot of Bernie bros in the audience.
• 2:56
  SPEAKER_01: I can't imagine that we do.
• 2:59
  SPEAKER_01: But I mentioned the tweet only to say that lots of other people also want to go that direction.
• 3:05
  SPEAKER_00: Right.
• 3:06
  SPEAKER_00: And you can see why, okay,
• 3:07
  SPEAKER_00: if you're just looking at it from the outside and saying this is what needs
• 3:10
  SPEAKER_00: to happen, you can sort of say, okay, this should be what happens.
• 3:14
  SPEAKER_00: This makes sense.
• 3:15
  SPEAKER_00: You know,
• 3:15
  SPEAKER_00: although again,
• 3:16
  SPEAKER_00: I think you,
• 3:16
  SPEAKER_00: you know,
• 3:18
  SPEAKER_00: Talk about whether or not,
• 3:19
  SPEAKER_00: you know, this sort of P-Doom, 10%, we're all going to die.
• 3:23
  SPEAKER_00: Is it legit?
• 3:24
  SPEAKER_00: Is it hysteria?
• 3:25
  SPEAKER_00: Is it some mixture of it?
• 3:26
  SPEAKER_00: Is it completely being blown up by the media?
• 3:29
  SPEAKER_00: I mean, I certainly think it's...
• 3:32
  SPEAKER_00: Obviously a risk, but it's maybe a bit overdone.
• 3:35
  SPEAKER_00: I know Ben has written about it.
• 3:36
  SPEAKER_00: You've written about it too.
• 3:37
  SPEAKER_00: It seems like a little skeptical.
• 3:40
  SPEAKER_00: But the reality is China has a say and China has no interest in pacing.
• 3:44
  SPEAKER_00: And the idea that China is going to sit down and suddenly say,
• 3:47
  SPEAKER_00: oh yeah,
• 3:47
  SPEAKER_00: we're going to agree to pacing even though we're behind and you're crippling us
• 3:50
  SPEAKER_00: with all these export controls.
• 3:52
  SPEAKER_00: It's fantasy land.
• 3:54
  SPEAKER_00: The party and Xi Jinping have made it clear they see AI development as...
• 4:00
  SPEAKER_00: Effectively existential too.
• 4:02
  SPEAKER_01: Yeah, right.
• 4:04
  SPEAKER_01: Well, and they don't trust the United States.
• 4:06
  SPEAKER_01: I mean, that's part of the issue here is if they were to agree to pacing,
• 4:12
  SPEAKER_01: I would imagine there would be some real skepticism that the US would comply
• 4:17
  SPEAKER_01: with any treaty.
• 4:18
  SPEAKER_00: Well, there'd be skepticism either side would comply.
• 4:20
  SPEAKER_00: And this idea that, oh, well, Reagan and Gorbachev did it.
• 4:24
  SPEAKER_00: Mm-hmm.
• 4:47
  SPEAKER_01: Government to government channels in the event of catastrophe.
• 4:51
  SPEAKER_01: That would be a good thing.
• 4:52
  SPEAKER_00: There are a whole bunch of things to talk about.
• 4:55
  SPEAKER_00: Nuclear command and control, cyber,
• 4:58
  SPEAKER_00: some level of cyber risk,
• 5:01
  SPEAKER_00: financial market risk,
• 5:03
  SPEAKER_00: things around working on creating mirror life and other biology forms
• 5:09
  SPEAKER_00: or other organisms.
• 5:10
  SPEAKER_00: There's all sorts of stuff that can be And scams.
• 5:14
  SPEAKER_00: But the idea that somehow we're going to jump straight to some overarching treaty
• 5:18
  SPEAKER_00: that freezes development now is just not the reality of where China is and where
• 5:25
  SPEAKER_00: the Xi and the Communist Party and he wants the country to go.
• 5:30
  SPEAKER_01: Are you surprised by how mainstream the conversation has become?
• 5:33
  SPEAKER_01: Because like House Democrats are also pushing for some sort of global AI treaty.
• 5:39
  SPEAKER_01: And this is going to be the top of the Democrat agenda if they take Congress in the midterms.
• 5:44
  SPEAKER_00: It's clearly polls well, right?
• 5:46
  SPEAKER_00: You've got the data center backlash and now you've got the AI freak out.
• 5:49
  SPEAKER_00: And so I think they see an opportunity,
• 5:52
  SPEAKER_00: you know,
• 5:53
  SPEAKER_00: and obviously the Trump administration,
• 5:55
  SPEAKER_00: you know,
• 5:55
  SPEAKER_00: we saw yesterday Trump's multiple tweets,
• 5:57
  SPEAKER_00: his, you know, dialing in to, you know, talk to Jensen Huang at the All In podcast.
• 6:02
  SPEAKER_00: Yeah.
• 6:03
  SPEAKER_00: I mean, good for the folks at the All-In to get that stage.
• 6:07
  SPEAKER_00: I assume that was David Sachs or maybe it was Jensen Huang directly.
• 6:10
  SPEAKER_00: I mean, because he obviously has a good relationship with the president.
• 6:13
  SPEAKER_01: It was a pretty compelling social video.
• 6:16
  SPEAKER_01: So good job on all sides executing that.
• 6:20
  SPEAKER_00: Made the ticket.
• 6:21
  SPEAKER_00: I don't know what they charged,
• 6:22
  SPEAKER_00: but it made that ticket worthwhile, I think, for a lot of people.
• 6:25
  SPEAKER_00: It certainly was good for the All-In podcast conference.
• 6:27
  SPEAKER_00: But again,
• 6:28
  SPEAKER_00: back to the point being, though,
• 6:29
  SPEAKER_00: that The government,
• 6:32
  SPEAKER_00: the Republican Party especially,
• 6:33
  SPEAKER_00: it's so far on the other side of sort of where the Democrats are.
• 6:37
  SPEAKER_00: And he's still the president for two and a half plus, two and a half more years, right?
• 6:42
  SPEAKER_01: Sure.
• 6:43
  SPEAKER_01: And they can frame themselves.
• 6:51
  SPEAKER_01: I'm sure they will make political hay out of that over the next two months
• 6:56
  SPEAKER_01: and hammer on this issue.
• 6:58
  SPEAKER_01: I ask the question because every now and then I'll come across people
• 7:02
  SPEAKER_01: who are pushing for something like a global AI treaty with China.
• 7:06
  SPEAKER_01: And it is an understandable goal, to be clear.
• 7:09
  SPEAKER_01: It's a natural, rational reaction to the concerns that have been expressed.
• 7:14
  SPEAKER_01: And so I sometimes find myself wondering,
• 7:29
  SPEAKER_01: What world are you living in and have you been paying attention to China for
• 7:33
  SPEAKER_01: the last 10 or 15 years if you think this is remotely realistic?
• 7:38
  SPEAKER_01: Has the PRC cooperated with the U.S. on nuclear arms control even over the last several years?
• 7:44
  SPEAKER_00: No, they are not because their view is we're way behind
• 7:48
  SPEAKER_00: and – I'm vastly oversimplifying, but we're way behind.
• 7:52
  SPEAKER_01: And these are defensive capabilities.
• 7:54
  SPEAKER_00: Well, no,
• 7:56
  SPEAKER_00: not necessarily,
• 7:57
  SPEAKER_00: but they want to build out their arsenal to the point where they basically can have
• 8:03
  SPEAKER_00: the ability to basically say to the U.S.,
• 8:07
  SPEAKER_00: look, we're going to do things and you can't intervene because...
• 8:11
  SPEAKER_00: Hey, we have a full nuclear triad, which is- Right.
• 8:14
  SPEAKER_00: Right?
• 8:15
  SPEAKER_00: And that was one of the points of their nuclear,
• 8:17
  SPEAKER_00: the ICBM test from the submarine- Two months ago.
• 8:23
  SPEAKER_00: Sure, yeah.
• 8:24
  SPEAKER_00: To show that they actually had the underwater, the submarine leg of their nuclear triad.
• 8:28
  SPEAKER_00: The point being though,
• 8:29
  SPEAKER_00: the Chinese view is, again,
• 8:30
  SPEAKER_00: oversimplifying is,
• 8:32
  SPEAKER_00: of course you want to lock us in because you're ahead,
• 8:35
  SPEAKER_00: but we're behind and we're going to catch up.
• 8:37
  SPEAKER_00: And I think a similar dynamic applies in AI.
• 8:40
  SPEAKER_00: And at the same time, the Chinese are two things.
• 8:44
  SPEAKER_00: One is that any discussions around,
• 8:47
  SPEAKER_00: any dialogues around AI, they always are going to bring up all the export controls.
• 8:52
  SPEAKER_00: And so why would they agree to some overarching treaty that would The closed source,
• 8:56
  SPEAKER_00: sure, yeah.
• 8:56
  SPEAKER_00: Right.
• 9:23
  SPEAKER_00: So for them to suddenly just say,
• 9:24
  SPEAKER_00: oh, no, we're just going to sign on this great treaty of the US and kumbaya.
• 9:29
  SPEAKER_00: It isn't,
• 9:30
  SPEAKER_00: I don't think,
• 9:31
  SPEAKER_00: at all reflecting what the Chinese themselves are doing or saying internally
• 9:35
  SPEAKER_00: and sometimes externally.
• 9:37
  SPEAKER_00: And so it is divorced from the sort of facts on the ground.
• 9:40
  SPEAKER_00: But again,
• 9:41
  SPEAKER_00: that has never stopped politicians in D.C.
• 9:44
  SPEAKER_00: And I will say on the other side,
• 9:46
  SPEAKER_00: on the Republican side,
• 9:46
  SPEAKER_00: what I was trying to say earlier about President Trump is part of the challenge,
• 9:50
  SPEAKER_00: too,
• 9:50
  SPEAKER_00: is that,
• 9:50
  SPEAKER_00: you know,
• 9:51
  SPEAKER_00: the Trump administration,
• 9:52
  SPEAKER_00: they're clearly,
• 9:52
  SPEAKER_00: you know, all, you know, this sort of dumerism, it's a hoax, et cetera, et cetera.
• 9:57
  SPEAKER_00: They're also completely incoherent on their AI policy.
• 10:01
  SPEAKER_00: And so for the Chinese,
• 10:03
  SPEAKER_00: how do you even have a constructive dialogue when you don't know what the policy is?
• 10:07
  SPEAKER_00: Because the president may get a call from someone with an interest in this industry.
• 10:12
  SPEAKER_00: And then while we're talking,
• 10:13
  SPEAKER_00: he'll put out a true social post that completely blows up whatever we're talking
• 10:16
  SPEAKER_00: about.
• 10:17
  SPEAKER_01: Yeah, I mean,
• 10:17
  SPEAKER_01: we could have short circuited the conversation right there and say, well,
• 10:23
  SPEAKER_01: the Trump administration does not support a pause or pacing plan and that sort
• 10:29
  SPEAKER_01: of approach to AI.
• 10:31
  SPEAKER_01: So, of course.
• 10:32
  SPEAKER_01: I've found it surprising how many people believe that's actually possible
• 10:35
  SPEAKER_01: and something that the U.S. should be pursuing with China.
• 10:51
  SPEAKER_00: It's an aspirational goal, but you have to start somewhere.
• 10:53
  SPEAKER_00: I mean, I don't know if you know,
• 10:54
  SPEAKER_00: I posted it late,
• 10:55
  SPEAKER_00: but I actually put out a draft proposal for a US-China SAID, Strategic AI Dialogue.
• 11:02
  SPEAKER_01: I loved it.
• 11:02
  SPEAKER_01: Yes, I read it at 11.30 last night.
• 11:05
  SPEAKER_00: I have resisted using AI,
• 11:08
  SPEAKER_00: but I figured for this one,
• 11:09
  SPEAKER_00: I might as well actually get,
• 11:12
  SPEAKER_00: you know, the AI has a say since it's a dialogue about the AI.
• 11:16
  SPEAKER_00: Absolutely, when in Rome.
• 11:17
  SPEAKER_00: Asher and I had a long back and forth conversation.
• 11:20
  SPEAKER_00: I had some input for some folks who are experts on China and AI.
• 11:24
  SPEAKER_00: And that's what we came up with.
• 11:26
  SPEAKER_00: And so, you know, we'll put it out there.
• 11:27
  SPEAKER_01: Yeah.
• 11:28
  SPEAKER_01: People can read through the sort of agreement that will never materialize in reality.
• 11:32
  SPEAKER_00: But the point was trying to actually get a structure
• 11:35
  SPEAKER_00: where you could have conversations and make sure you have the right people,
• 11:38
  SPEAKER_00: but also start small and with areas where there should be very clear...
• 11:45
  SPEAKER_00: Yeah.
• 12:08
  SPEAKER_01: Right.
• 12:09
  SPEAKER_01: I think that would be a good idea.
• 12:11
  SPEAKER_01: And look, I'm sure some of these conversations will take place next week when she is in D.C.
• 12:18
  SPEAKER_01: The agenda for his visit is Remains completely up in the air and opaque.
• 12:23
  SPEAKER_01: So maybe we'll have more details by the time we podcast next week.
• 12:28
  SPEAKER_00: And to that point,
• 12:28
  SPEAKER_00: you know,
• 12:29
  SPEAKER_00: usually there's a free summit meeting from senior officials,
• 12:33
  SPEAKER_00: you know,
• 12:33
  SPEAKER_00: whether it's,
• 12:34
  SPEAKER_00: you know,
• 12:34
  SPEAKER_00: historically it's been like
• 12:36
  SPEAKER_00: the National Security State Department foreign minister track like Jake Sullivan,
• 12:41
  SPEAKER_00: Wong Yi.
• 12:43
  SPEAKER_02: All right.
• 12:43
  SPEAKER_02: And that is the end of the free preview.
• 12:45
  SPEAKER_02: If you'd like to hear the rest of today's conversation and get access
• 12:49
  SPEAKER_02: to full episodes of Sharp China each week,
• 12:52
  SPEAKER_02: You can go to your show notes and subscribe to either Bill's newsletter,
• 12:55
  SPEAKER_02: Cynicism,
• 12:56
  SPEAKER_02: or the Stratechery Bundle,
• 12:58
  SPEAKER_02: which includes several other podcasts from me and daily writing
• 13:01
  SPEAKER_02: from my friend Ben Thompson.
• 13:03
  SPEAKER_02: I'm an incredibly biased news consumer, so I think both are indispensable resources.
• 13:08
  SPEAKER_02: But either way,
• 13:09
  SPEAKER_02: Bill and I are going to be here every week talking all things China,
• 13:12
  SPEAKER_02: and we would love to have you on board.
• 13:14
  SPEAKER_02: So check out your show notes, subscribe, and we will talk to you soon.