Latest / kill switch / why is the government fighting with claude?
Transcript
- 0:09Speaker 1: Yo, what's going on? It's Texter. So technology does not
- 0:13Speaker 1: slow down for anybody. And sometimes I wish that I
- 0:15Speaker 1: could just talk to y'all live about what's happening right now.
- 0:19Speaker 1: But then we were talking about it, we realized, wait
- 0:21Speaker 1: a second, we can talk to you live. So we're
- 0:24Speaker 1: gonna try something new here. Every once in a while,
- 0:26Speaker 1: we're gonna talk about something that's happening right now and
- 0:28Speaker 1: let people jump into the audience to ask questions, and
- 0:31Speaker 1: we'll bring a guest along. And so what you're about
- 0:33Speaker 1: to listen to is our first episode that we live
- 0:36Speaker 1: streamed on Wednesday, March third. So if you want to
- 0:39Speaker 1: catch the next one as we're recording it and maybe
- 0:41Speaker 1: even ask some questions, you can subscribe to our YouTube
- 0:44Speaker 1: page and you can catch the links to that in
- 0:46Speaker 1: the description. So everyone, this is we're doing something different here.
- 0:57Speaker 1: We're doing the first live episode of kill Switch. We're
- 1:01Speaker 1: gonna probably try to do these maybe once a month,
- 1:04Speaker 1: just to give people opportunity to actually jump into the
- 1:06Speaker 1: chat and talk about stuff that is happening right now.
- 1:10Speaker 1: So thanks for bearing with us while we experiment a
- 1:13Speaker 1: little bit. But today what I'd like to talk about
- 1:17Speaker 1: is what is happening with Anthropic, what's going on with
- 1:21Speaker 1: open Ai, what's going on with the US government. To
- 1:23Speaker 1: talk about it today, our guest is Will Knight, who's
- 1:26Speaker 1: a senior writer at Wired. He's been writing about AI
- 1:29Speaker 1: for a while. He's got a weekly AI newsletter called
- 1:32Speaker 1: AI Lab. Will Welcome to kill Switch.
- 1:35Speaker 2: Welcome in, Thank you for having me. I'm wanting to
- 1:37Speaker 2: be on the first experimental live stream.
- 1:40Speaker 1: Yeah, hey, look, we'll see. Everything's in experiment at this point,
- 1:43Speaker 1: for better or for worse. Sure, So I want to
- 1:48Speaker 1: start this off with a timeline, Okay. On the twenty seventh,
- 1:52Speaker 1: Donald Trump gets on truth Social and says that Claude
- 1:55Speaker 1: is a radical left woke company. Okay, yeah, out of nowhere,
- 2:00Speaker 1: People like, what why is he saying this? Pete heg
- 2:02Speaker 1: Saith gets on Twitter and says that Claude Anthropic, the
- 2:06Speaker 1: company that makes Claude, is a supply chain risk, which
- 2:10Speaker 1: most people have never thought about what that even means.
- 2:13Speaker 1: The next day, Open Ai, who makes chat GPT, announces
- 2:17Speaker 1: that they're working with the Department of War, and then
- 2:19Speaker 1: Donald Trump announces that we're bombing Iran. As I say
- 2:23Speaker 1: that that sounds like a completely unhinged sequence of events.
- 2:28Speaker 1: Please tell me where I'm wrong, because I must be
- 2:30Speaker 1: missing something.
- 2:32Speaker 2: I mean, I think I think there's a lot that's
- 2:34Speaker 2: that's right in there and a lot that's unhinged. But
- 2:38Speaker 2: I mean to sort of set the scene a little bit,
- 2:41Speaker 2: you know. So this goes back. So the use of
- 2:44Speaker 2: Claude in classified military systems has its origins last year,
- 2:49Speaker 2: last summer, Anthropic was the first to get involved in
- 2:53Speaker 2: working with the military. You know, they've pursued a very
- 2:55Speaker 2: business focus, and I think that sort of led them
- 2:58Speaker 2: to focus on the d D, which is like one
- 3:00Speaker 2: of the biggest businesses in the US. So they ended
- 3:02Speaker 2: up supplying Claude their model, a version of that which
- 3:05Speaker 2: runs on the Pentagon's own systems for military use on
- 3:11Speaker 2: classified systems. In many ways, this is totally remarkable because
- 3:16Speaker 2: just a few years ago, everybody in the AI industry
- 3:18Speaker 2: was saying, one thing we don't want to do is
- 3:20Speaker 2: work with the military, and people were staging, you know,
- 3:22Speaker 2: walkouts over this, right, So it shows you how the
- 3:26Speaker 2: tech world has really pivoted to that. What happened more
- 3:29Speaker 2: recently was that its themes and you know, this is
- 3:33Speaker 2: somewhat disputed but there's been reports that when President Maduro
- 3:38Speaker 2: was captured, Claude was used, and somebody at Anthropic may
- 3:42Speaker 2: have raised questions about that. This seems like it snowballed.
- 3:46Speaker 2: And one of the carve outs in the original contract
- 3:49Speaker 2: signed last year there were two. One was against mass
- 3:53Speaker 2: surveillance of US citizen and the others was that the
- 3:56Speaker 2: model shouldn't be used to build fully autonomous weapons. And
- 4:01Speaker 2: the DoD or W said that they wanted to change
- 4:06Speaker 2: the contract essentially, which is an unusual thing to do,
- 4:08Speaker 2: but they wanted to allow all your lawful use. And
- 4:12Speaker 2: to some degree, this dispute seems like it's a little
- 4:15Speaker 2: bit like people, you know, each side talking past each
- 4:17Speaker 2: other to some degree. And I think that the DOW
- 4:20Speaker 2: the Department of War became very concerned about a tech
- 4:24Speaker 2: company saying how it could and couldn't use the technology.
- 4:27Speaker 2: And to be fair, that is quite an unusual idea, right.
- 4:30Speaker 2: It's like somebody selling a missile or a jet fighter
- 4:33Speaker 2: to the Pentagon and then later on saying we don't
- 4:35Speaker 2: want you to use it in this way or that way.
- 4:38Speaker 2: But then again, ai is is a really different technology,
- 4:40Speaker 2: is very new, it's very experimental and I believe anthropic,
- 4:44Speaker 2: especially among AI companies, it's very focused on safety and
- 4:46Speaker 2: they have concerns just about how it could end up
- 4:49Speaker 2: being used.
- 4:50Speaker 1: Yeah, well, so let's jump into this so anthropic you know,
- 4:54Speaker 1: this is a statement that is put on anthropic dot com.
- 4:57Speaker 1: So this is their own, you know, pr statement here
- 5:00Speaker 1: and the title is statement from Daria Ama Day on
- 5:03Speaker 1: our discussions with the Department of War, and it starts
- 5:06Speaker 1: out the first line, I believe deeply in the existential
- 5:09Speaker 1: importance of using AI to defend the United States and
- 5:12Speaker 1: other democracies and to defeat our autocratic adversaries. So already
- 5:17Speaker 1: out of the gate, that's kind of an interesting thing
- 5:20Speaker 1: for again a company in Silicon Valley to be saying,
- 5:23Speaker 1: even just in the past couple of years, really, but
- 5:27Speaker 1: it seems like the two big points of contention here,
- 5:30Speaker 1: there's two. He says, using these systems for mass domestic
- 5:34Speaker 1: surveillance is incompatible with democratic values. So spying out other people,
- 5:39Speaker 1: we're okay with that. Spying on people in the US,
- 5:42Speaker 1: We're not okay with that. Right, And then the next
- 5:45Speaker 1: thing is, quote frontier AI systems are simply not reliable
- 5:49Speaker 1: enough to power fully autonomous weapons both of those sound
- 5:54Speaker 1: totally okay and reasonable. Not only do they sound okay
- 5:59Speaker 1: and reasonable, but I mean, look, I've spent a whole
- 6:02Speaker 1: lot of time around people on the pretty far rate
- 6:06Speaker 1: and you know, for work, right, and they will tell
- 6:09Speaker 1: you I don't want no government spying on me, and
- 6:13Speaker 1: I don't trust these computers. Right, where's the beef? What's
- 6:16Speaker 1: the problem here?
- 6:18Speaker 2: That's a great question. You know. I've spent a lot
- 6:20Speaker 2: of time talking to people in the Pentagon and military
- 6:25Speaker 2: defense companies, and nobody believes that those frontier models are
- 6:29Speaker 2: ready were even remotely near ready being used in autonomous weapons,
- 6:33Speaker 2: And consistently the government said we have no desire to
- 6:35Speaker 2: do either of those things. I think the way what
- 6:39Speaker 2: it really came down to was that they didn't just
- 6:43Speaker 2: you know, take a knee and agree to change the
- 6:46Speaker 2: terms of their contract. And as you say, you know,
- 6:49Speaker 2: it culminated in Trump and Hegxeth and others putting out
- 6:54Speaker 2: these very very you know, as often happens, they were
- 6:58Speaker 2: sort of classifying them all of a sudden as the
- 6:59Speaker 2: radical left, which is it's pretty incredible for company supplying
- 7:03Speaker 2: technology that's working on classified US systems.
- 7:06Speaker 1: Right, So this is the post that Donald Trump made
- 7:10Speaker 1: on his social media network Truth Social and this I'm
- 7:13Speaker 1: reading from the top, and I'm reading verbatim. The United
- 7:16Speaker 1: States of America will never allow a radical left woke
- 7:20Speaker 1: company to dictate how a great military fights and wins wars.
- 7:25Speaker 1: That decision belongs to your commander in chief and the
- 7:28Speaker 1: tremendous leaders I appoint to run are military. And then
- 7:31Speaker 1: he finally gets to what he's talking about. He says,
- 7:33Speaker 1: the left wing nut jobs Anthropic have made a disastrous
- 7:37Speaker 1: mistake trying to strong arm the Department of War and
- 7:41Speaker 1: force them to obey their terms of service instead of
- 7:43Speaker 1: our constitution.
- 7:44Speaker 3: Yeah.
- 7:45Speaker 2: I mean, like a lot of that doesn't really even
- 7:47Speaker 2: make sense. I think Anthropic is not a radical left
- 7:51Speaker 2: organization by a stretch. It's since come out that that
- 7:56Speaker 2: should The DoD was negotiating with that for they were
- 7:59Speaker 2: trying to get this war odding kind of massalage into
- 8:01Speaker 2: a format that everybody was happy with. And it doesn't
- 8:04Speaker 2: seem like that would have been impossible. It feels like
- 8:07Speaker 2: some people within the administration lost patients, And I think
- 8:10Speaker 2: a lot of it came down to just power and
- 8:14Speaker 2: the Department of Defense and the leadership not being able
- 8:18Speaker 2: to handle being denied that by a tech company that
- 8:21Speaker 2: they could very easily see as as you know, part
- 8:23Speaker 2: of this sort of you know, coastal elite and in
- 8:26Speaker 2: the case of Anthropic, woke and liberal. I mean, it's
- 8:30Speaker 2: another reflection of how AI is just is it sort
- 8:32Speaker 2: of also a cultural lightning rod, right, it becomes suddenly
- 8:36Speaker 2: it becomes this kind of left versus right. So, I mean,
- 8:41Speaker 2: you know that that post kind of does unfortunately sum
- 8:46Speaker 2: up the entire situation, which is slightly bewildering if you
- 8:49Speaker 2: know that now they've they've taken the action to try
- 8:52Speaker 2: and to classify, as you said, Anthropic a supply chain risk,
- 8:56Speaker 2: which is something that's only ever been to foreign nations.
- 8:59Speaker 2: And we're doing this to one of the foremost AI
- 9:02Speaker 2: companies in the United States, one of them I supposedly,
- 9:05Speaker 2: you know, should be our national champions. It it doesn't
- 9:09Speaker 2: sound like to me they're really promoting American technology. If
- 9:13Speaker 2: you if you're too keen to do that, I think
- 9:15Speaker 2: you don't. You know, it's a real danger you're going
- 9:17Speaker 2: to harm some of you know, this this nascent, very
- 9:21Speaker 2: important American company also maybe dissuade other companies from working
- 9:24Speaker 2: with the government or the Pentagon if they feel that
- 9:26Speaker 2: that's going to happen.
- 9:27Speaker 1: Right, Yeah, So the supply chain risk thing again that
- 9:32Speaker 1: that's the language that Pete Hegseth used. What does it
- 9:35Speaker 1: mean to call anthropic a supply chain risk? What does
- 9:40Speaker 1: that even mean? Yeah?
- 9:42Speaker 2: I mean, you know, well, the first thing is to
- 9:43Speaker 2: say that this supply chain risk designation is incredibly unusual, extraordinary,
- 9:49Speaker 2: and just the kind of thing that I think the
- 9:53Speaker 2: sort of left field thinking that we're become accustomed to
- 9:56Speaker 2: with the administration because it's usually applied to, you know,
- 10:00Speaker 2: a company that you want to remove from the US
- 10:02Speaker 2: supply chain because you're worried that their software or their
- 10:05Speaker 2: hardware might contain backdoors, might be a threat to national
- 10:09Speaker 2: security because it could be used by an adversary. So
- 10:13Speaker 2: this is the very sort of all or nothing. You
- 10:15Speaker 2: either agree with everything we say, or you are the enemy.
- 10:18Speaker 2: I mean, that's the kind of tenor which sounds familiar
- 10:21Speaker 2: to me generally. Often the most extreme version of that designation,
- 10:26Speaker 2: and we haven't seen the full details of it because
- 10:27Speaker 2: they haven't produced it yet, but would mean that any
- 10:30Speaker 2: company who's working with the d id couldn't use your technology,
- 10:33Speaker 2: which is a vast I mean, Amazon, which has invested
- 10:36Speaker 2: billions in Anthropy, has massive contracts with the DoD so
- 10:39Speaker 2: it's an enormous thing to designate them. It's also doesn't
- 10:43Speaker 2: really make sense because you know, they were also saying
- 10:45Speaker 2: we may invoke the Defense Production Act, which has been
- 10:48Speaker 2: used in wartimes or in actually also during the Cold
- 10:51Speaker 2: War to force companies to prioritize production and supply because
- 10:56Speaker 2: what they were making is so important. So it's it's wait,
- 11:01Speaker 2: I think this may be what, Yeah, this may be
- 11:03Speaker 2: a legal problem that the d ID has, and I
- 11:06Speaker 2: think you know Anthropic is going to as a signal
- 11:09Speaker 2: that's going to take some legal action, So you can't
- 11:11Speaker 2: it's difficult to argue. On one hand, it's so vital
- 11:14Speaker 2: we may invoke the Defense Production Act, but then we
- 11:17Speaker 2: may also designate you a supply chain risk.
- 11:20Speaker 1: Yeah, like you're too dangerous for us to use, but
- 11:25Speaker 1: you're so useful that we want to use your product.
- 11:28Speaker 2: But we want to force you to supply it big one.
- 11:32Speaker 1: Yeah, like nobody can have it, but we want it. Okay,
- 11:35Speaker 1: what are we doing here? So basically the next day
- 11:50Speaker 1: open Ai signs something with the government. How does that
- 11:54Speaker 1: sequence of events? Like, what kind of sense are you
- 11:57Speaker 1: able to make of that?
- 11:58Speaker 2: Yeah, it is very interesting that anthropics chief rival. And
- 12:03Speaker 2: quick reminder that Anthropic was founded by people who left
- 12:06Speaker 2: OpenAI because they said that company was not developing AI
- 12:10Speaker 2: with enough focus on safety. And they're really really bitter rivals.
- 12:15Speaker 2: And as it was going on, Sam al Mun was
- 12:18Speaker 2: working through an alternative deal. The deal that they signed
- 12:22Speaker 2: seems extremely similar to the one that Anthropic we're trying
- 12:25Speaker 2: to hash out, has similar carve outs, so it's sort
- 12:28Speaker 2: of doesn't actually carve them out, but it specifies that
- 12:31Speaker 2: those things that massurbillance is unlawful and that nobody wants
- 12:36Speaker 2: to build autonomous weapons.
- 12:37Speaker 1: So are the details or the terms of the contract
- 12:41Speaker 1: that OpenAI signed with the Department of War, are they
- 12:45Speaker 1: materially different from what Anthropic did? Because this is from CNBC,
- 12:51Speaker 1: Altman said, and I want to read this here quote.
- 12:55Speaker 1: I believe that we will hopefully have the best models
- 12:57Speaker 1: that were encouraged the government to be willing to work
- 13:00Speaker 1: with us, even if our safety stack annoys them. But
- 13:03Speaker 1: there will be at least one other actor, which I
- 13:05Speaker 1: assume will be x Ai, which effectively will say, will
- 13:09Speaker 1: do whatever you want. So the vibe here seems to
- 13:14Speaker 1: be like, okay, well listen, at least worthy adults in
- 13:17Speaker 1: the room. They're gonna let Grop do whatever Groc wants
- 13:19Speaker 1: to do, whatever, whatever Trump wants to do. They're just
- 13:22Speaker 1: gonna let it do it, drop bombs wherever. At least
- 13:24Speaker 1: we're going to be some adults in the room. But
- 13:26Speaker 1: is Opening Eye doing something fundamentally different. Did they seed
- 13:30Speaker 1: more grounded than Anthropic did or is it the same thing?
- 13:35Speaker 2: You know, Yeah, there's a lot going on there. I
- 13:37Speaker 2: think that they they did see slightly more ground in
- 13:41Speaker 2: that they basically agreed to a contract that specified all
- 13:45Speaker 2: lawful use. They changed the contractors make Oh but you know,
- 13:49Speaker 2: to be clear, here's what's what is lawful and what
- 13:51Speaker 2: the deity doesn't want to do. But I think there
- 13:54Speaker 2: isn't a huge amount of difference. I mean, I think
- 13:56Speaker 2: what you're seeing with that that internal statement was really,
- 14:01Speaker 2: you know, Sam open Ai willing to take a bit
- 14:04Speaker 2: of flat because it's just incredibly important to get a
- 14:09Speaker 2: win over Anthropic, to be seen to be very on
- 14:12Speaker 2: board with the US government because you want more and
- 14:14Speaker 2: more involvement, more and more contracts and they've since you know,
- 14:17Speaker 2: open a I since got quite a lot of blowback.
- 14:20Speaker 1: Yeah, well, let's talk about the blowback here. I mean
- 14:23Speaker 1: this is on tech crunch us app. Uninstalls of chat
- 14:27Speaker 1: gbt's mobile app went up almost three hundred per so
- 14:31Speaker 1: people were uninstalling chat gbt and started installing Claude Anthropics Claude.
- 14:40Speaker 1: And so there's a weird way in which there's this
- 14:45Speaker 1: and I really hesitate to call this a game, but
- 14:47Speaker 1: there's there's a weird, really weird way in which there's
- 14:49Speaker 1: this game being played over who gets to be the
- 14:53Speaker 1: contractor for the Department of War. But then also we're
- 14:57Speaker 1: still fighting for consumers here.
- 15:00Speaker 2: Definitely, And it is interesting. It's the sort of perception
- 15:03Speaker 2: of who's the good guy and you had, you know,
- 15:06Speaker 2: Katie Perry was subscribing to claud pro and posting about
- 15:10Speaker 2: it and this, you know, the sort of interesting cultural moment.
- 15:14Speaker 2: And I think the thing that was that struck me
- 15:17Speaker 2: was like, the people really know how Claude is being
- 15:20Speaker 2: used by the military, Do they know what Anthropic was
- 15:23Speaker 2: willing to do? I think it's gotten a little oversimplified
- 15:26Speaker 2: to my mind, and I think that one of the
- 15:29Speaker 2: risks there is that we ought to be rather necessarily
- 15:32Speaker 2: uninstalling something or posting about who you think is best,
- 15:37Speaker 2: really pushing for more of an explanation about how the
- 15:39Speaker 2: technology is being used and developed and tested, like what
- 15:42Speaker 2: is it mean that it's used on classified systems? There
- 15:45Speaker 2: should be more accountability there. And the fact of the
- 15:48Speaker 2: matter is there is nothing legally to putting the issue
- 15:51Speaker 2: of the bonus to once nothing prohibiting the development of
- 15:53Speaker 2: autonomous weapons, and there isn't really a legal framework around
- 15:57Speaker 2: that sort. If they, you know, civilians were affected by
- 16:00Speaker 2: fully autonomous weapons, you suddenly have a situation where nobody
- 16:03Speaker 2: is legally accountable. And so these are things that really
- 16:06Speaker 2: need to be figured out. And that doesn't mean a
- 16:09Speaker 2: fully autonomous web that could be your automated part of
- 16:12Speaker 2: the so called kill chain and so decisions were kind
- 16:15Speaker 2: of automated away. I think that is really important to
- 16:18Speaker 2: have more focus on this and to have more discussion
- 16:20Speaker 2: of it. And you know, one of the reasons, honestly,
- 16:24Speaker 2: as someone who's looked at the DoD and the work
- 16:27Speaker 2: going on around autonomy, is that unfortunately it is the
- 16:31Speaker 2: case that we're just going to have more and more autonomy,
- 16:33Speaker 2: and most analysts expect that they're going to be swarms
- 16:36Speaker 2: of drones attacking systems, and the truth is you just
- 16:39Speaker 2: won't be able to have humans in the loop always.
- 16:41Speaker 2: So this is going to become a really big issue.
- 16:43Speaker 2: And it's not necessarily about anthropics models in those situations,
- 16:47Speaker 2: but the question of who's accountable, how do we know
- 16:50Speaker 2: that the systems are reliable? That is going to become
- 16:52Speaker 2: more and more important. So I think it's good that
- 16:54Speaker 2: the situation is getting some attention. I do think that
- 16:57Speaker 2: the positive thing is that it is important for people
- 17:00Speaker 2: who understand this technology really really well. Do you maybe
- 17:03Speaker 2: have a role in at least influencing how it's used.
- 17:06Speaker 2: And that doesn't mean, you know, like you get to
- 17:08Speaker 2: say yes or no to a mission. It just means
- 17:10Speaker 2: perhaps that you say, have some input on how you
- 17:15Speaker 2: ought to try and deploy it within the Pentagon I.
- 17:17Speaker 1: Think, I mean, how does that functionally work? Though maybe
- 17:20Speaker 1: that's the question here, That is the question, Yeah, can
- 17:23Speaker 1: you have a contract with the government and tell them no,
- 17:28Speaker 1: you can't do this? And I'm asking that partially because
- 17:31Speaker 1: certainly there is a lot of talk that will wait
- 17:34Speaker 1: a second, The iron Strake. Was Claude used there.
- 17:38Speaker 2: Yeah. To answer the first question, I think that is
- 17:41Speaker 2: a really good question. How do companies, how does the
- 17:43Speaker 2: public understand how things are developed, what the rules are
- 17:47Speaker 2: around them. And there are specified rules around autonomous weapons,
- 17:51Speaker 2: around surveillance, but this is also a technology that's different,
- 17:55Speaker 2: that can misbehave in surprising ways and can be kind
- 17:58Speaker 2: vulnerable in surprising ways. So I mean that it needs
- 18:01Speaker 2: a bit more scrutiny. The Washington Posters has reported. I
- 18:05Speaker 2: think it's that it seems very likely true that Claude
- 18:08Speaker 2: was absolutely used in the Iran operation as part of
- 18:12Speaker 2: this system called Maize and which was built by Palanteer,
- 18:15Speaker 2: which includes a bunch of things, not just a language model.
- 18:18Speaker 2: It can do you know, analyzed maps and images and
- 18:21Speaker 2: mission plan and so on. But yeah, as part of that.
- 18:24Speaker 2: And so now, one of the things that was kind
- 18:26Speaker 2: of crazy with Trump's initial statement was that he said,
- 18:29Speaker 2: I'm ordering everybody to stop working with Anthropic because they're
- 18:33Speaker 2: all these terrible people. But there'll be a six month
- 18:36Speaker 2: period of what we carry on doing. Right, it can't
- 18:38Speaker 2: be that bad, and we're going to use We're about
- 18:42Speaker 2: to use them in this you know, incredible operation in
- 18:45Speaker 2: the Middle East.
- 18:47Speaker 1: So yeah, I mean we're laughing. It's terrible. I mean
- 18:51Speaker 1: it's ridiculous and also terrible. I suppose it can be
- 18:54Speaker 1: two things at the same time. And because we're doing
- 18:59Speaker 1: this live, I wanted to take a few questions. And
- 19:02Speaker 1: one of the questions that I got actually previously I
- 19:06Speaker 1: posted on Instagram does anybody have any questions? And somebody posted,
- 19:09Speaker 1: and I think they were kind of joking, but they said, hey,
- 19:12Speaker 1: can we just make all the country's Ais fight against
- 19:15Speaker 1: each other? Can we just have them fight instead of
- 19:17Speaker 1: the real war. But it's kind of funny that they
- 19:21Speaker 1: say that because there's I don't know if you've seen
- 19:24Speaker 1: this article in the New Scientist, but basically different AIS,
- 19:30Speaker 1: and the headline here kind of says it, all AIS
- 19:33Speaker 1: can't stop recommending nuclear strikes in wargames simulations, and the
- 19:37Speaker 1: subhead leading AIS from Open Ai, Anthropic and Google opted
- 19:42Speaker 1: to use nuclear weapons in simulated wargames in ninety five
- 19:46Speaker 1: percent of cases.
- 19:48Speaker 2: Right, Yeah, So I read that paper. It's important to
- 19:52Speaker 2: know that this person was just trying to understand what
- 19:55Speaker 2: the kind of behavior of models would be. Well, I'm
- 19:57Speaker 2: not advocating to do that, and I don't think anybody.
- 20:00Speaker 2: I think actually this is one of the like not
- 20:02Speaker 2: letting AI have a role in nuclear weapon decision making
- 20:06Speaker 2: is one thing that the US and China have actually
- 20:09Speaker 2: agreed on, which is extremely unusual these days.
- 20:11Speaker 1: Right.
- 20:11Speaker 2: So, but it's unsurprising to me that if you have
- 20:14Speaker 2: a model trade on the entire Internet, it's going to
- 20:18Speaker 2: tend toward escalation and not be very very expert in
- 20:24Speaker 2: how to avoid you know, it's not going to be
- 20:27Speaker 2: someone who spent their life worrying about that sort of thing.
- 20:30Speaker 2: And on the question of whether ais could fight each other,
- 20:33Speaker 2: the truth is, like you are already seeing in certain conflicts,
- 20:38Speaker 2: you know, autonomous systems, semi autonomous systems are remotely operated
- 20:42Speaker 2: fighting other semi autonomous systems.
- 20:44Speaker 1: Right.
- 20:44Speaker 2: And some people may argue that that would remove people
- 20:47Speaker 2: from harm's way. That is not the reality of a conflict.
- 20:50Speaker 2: There's going to be plenty of people in harm's way.
- 20:53Speaker 2: And the question is, you know, how do you do
- 20:55Speaker 2: that in ways that are reliable and don't make mistakes.
- 20:59Speaker 2: And there has been talking in recent years of these
- 21:02Speaker 2: sort of things like existential risks, but long before that,
- 21:04Speaker 2: there are lots of systemic dangers of using this technology
- 21:08Speaker 2: in any critical system. So you know, just as you
- 21:12Speaker 2: have hallucinations if you're using a chat, or just as
- 21:14Speaker 2: it gets simple reasoning wrong in a very convincing way,
- 21:18Speaker 2: that is what we should be worried about. I think
- 21:19Speaker 2: in how this and other models language models are used
- 21:23Speaker 2: by the military, by the Pentagon.
- 21:25Speaker 1: You know what.
- 21:25Speaker 2: I once talked to the excuse and naval officer in
- 21:29Speaker 2: the US Navy whose job it was to, you know,
- 21:32Speaker 2: be in command of these nuclear submarines, and he said
- 21:35Speaker 2: one of the first things he did when he got
- 21:37Speaker 2: the job was get permission to go and talk to
- 21:38Speaker 2: his counterpart in China, go meet him and spend time there.
- 21:42Speaker 2: The reason being because if something went wrong, if one
- 21:45Speaker 2: of the systems went wrong with some a person made
- 21:48Speaker 2: a mistake, you know, or say there's a collision between
- 21:52Speaker 2: two vessels, and you know, you've a risk of escalation.
- 21:55Speaker 2: He wants to be able to pick up the phone
- 21:56Speaker 2: and talk to someone he knows. He doesn't want that
- 21:58Speaker 2: to be the first time he's spoken to them, and
- 22:00Speaker 2: so that sort of shows the importance of the human element.
- 22:03Speaker 2: And that's that's not something that those we talk about
- 22:07Speaker 2: a g I and intelligence being sold. But that is
- 22:09Speaker 2: a very human thing that no language want to really,
- 22:12Speaker 2: they don't have that kind of human human intelligence, right.
- 22:17Speaker 1: Yeah, chat GBT can't take groc out for coffee.
- 22:20Speaker 2: Yeah No, well no not yet. I mean, that's a
- 22:23Speaker 2: frightening thought, it is. I don't think anybody would want
- 22:25Speaker 2: to take rock count.
- 22:26Speaker 3: Yeah.
- 22:39Speaker 1: So there's a question also here in the chat that
- 22:42Speaker 1: I want to get to, so b masters asking saying
- 22:46Speaker 1: they read some sources that say that Anthropic was actually
- 22:49Speaker 1: willing to give in to the Department of Wars demands,
- 22:52Speaker 1: but the communication broke down. The CEO just doubled down
- 22:56Speaker 1: on the ethical argument, just as a pr move. I mean,
- 22:58Speaker 1: what your reporting have you? Have you seen anything that
- 23:01Speaker 1: would substantiate that.
- 23:02Speaker 2: I couldn't tell you definitively if they were about to
- 23:06Speaker 2: cave to the d d's terms. I have heard that
- 23:10Speaker 2: they were very close to reaching an agreement which seemed
- 23:12Speaker 2: like sounded to me like a compromise between the two
- 23:16Speaker 2: and it wasn't very far off. I think it's gonna
- 23:18Speaker 2: it's gonna be too easy to get caught up in
- 23:21Speaker 2: the drama of these companies in there and missed the
- 23:24Speaker 2: bigger picture. Which is we'll hang on. Maybe we should
- 23:27Speaker 2: have more of a discussion about how those things are
- 23:30Speaker 2: you what are the benefits, what are the It's important
- 23:32Speaker 2: to say that I don't think you should not use
- 23:35Speaker 2: AI in defense. It's all about how reliably you can
- 23:38Speaker 2: do it, What are the limits? What accountability is that?
- 23:41Speaker 2: How how does citizens sort of know how that's being
- 23:43Speaker 2: used in their name?
- 23:44Speaker 3: Right?
- 23:44Speaker 2: And it you know, Trump wrote in that that message
- 23:47Speaker 2: that he decides how wars are fought, and in some way,
- 23:51Speaker 2: right he decides when we invadeer on, but he doesn't
- 23:53Speaker 2: decide the terms of war, like you know, what is
- 23:57Speaker 2: a war crime? That is decided by society at large.
- 24:01Speaker 2: And that's the sort of a bigger thing that we
- 24:02Speaker 2: to be asking these questions when it comes to the
- 24:05Speaker 2: new capabilities that the AI is going to give governments,
- 24:09Speaker 2: which is they would be able to do new kinds
- 24:11Speaker 2: of massurveilance you could never you know, never imagine before. Right,
- 24:14Speaker 2: you can be able to find patterns you couldn't think
- 24:16Speaker 2: of it, or you'll be able to sort of automate
- 24:18Speaker 2: that much more so, I think, yeah, that's the bigger
- 24:22Speaker 2: thing for me, and I think I should remember that
- 24:24Speaker 2: you know, real life, real human beings at stake here
- 24:27Speaker 2: and that war as hell, and it is something you
- 24:30Speaker 2: want to try and avoid.
- 24:31Speaker 1: Right, Yeah, you were talking about the bigger picture, and
- 24:35Speaker 1: also you mentioned palents here, which, as it happens, I
- 24:38Speaker 1: just a little bit ago talk to mechanic Kelly, your
- 24:40Speaker 1: colleague about talents here, and we actually have an episode
- 24:44Speaker 1: about tech drop in next week. But this kind of
- 24:48Speaker 1: seems to be part of a larger trend, like you said,
- 24:52Speaker 1: big tech companies who were very interested in working with
- 24:57Speaker 1: the government, but specifically working with the government in military capacities,
- 25:03Speaker 1: which feels new and in some ways it's not. Of course,
- 25:07Speaker 1: in some ways it's not. Some of the origins of
- 25:09Speaker 1: things like the Internet, you know, does come from the military.
- 25:12Speaker 1: But in terms of the way that we think about
- 25:15Speaker 1: the culture of Silicon Valley, I think this turn has
- 25:18Speaker 1: been a little bit unsettling but also surprising for a
- 25:22Speaker 1: lot of people. But it does to be a trend
- 25:24Speaker 1: right now. Where do you see this going, say, even
- 25:28Speaker 1: over the next year or so.
- 25:31Speaker 2: I mean, that's a great question. That is a really
- 25:33Speaker 2: interesting question. It's hard to imagine ALI companies becoming disentangled
- 25:39Speaker 2: because they are is so strategically important, and the incentive
- 25:43Speaker 2: for working with the government is enormous. Right, The amount
- 25:46Speaker 2: of data, the amount of compute, the amount of money
- 25:49Speaker 2: that the government has to throw out those companies is
- 25:51Speaker 2: so huge that it would be surprising to me if
- 25:54Speaker 2: everybody said, our conscience is not clear, so we're going
- 25:58Speaker 2: to rein that in So if you imagine that we're
- 26:01Speaker 2: going to have more and more tension, more more conflict,
- 26:04Speaker 2: there will be pressures on them too, for everybody to
- 26:08Speaker 2: use that AI and new ways that might be concerning,
- 26:12Speaker 2: besides the use in military situations, which I think we
- 26:17Speaker 2: should have try and have transparency on. I think it
- 26:20Speaker 2: is really telling that Anthropic had this concern about mass
- 26:23Speaker 2: surveillance because I think language models are preternaturally brilliant at
- 26:30Speaker 2: passing large amounts of text and audio and making sense
- 26:35Speaker 2: of it, finding patterns and making predictions like we haven't
- 26:38Speaker 2: done anything like that, and that's exactly what you might
- 26:41Speaker 2: want to do with in surveillance. So the temptation is
- 26:43Speaker 2: going to be huge, and yeah, I think it will.
- 26:47Speaker 2: I think it will be very interesting to see if
- 26:50Speaker 2: this turns into more of a pushback if more people.
- 26:54Speaker 2: One thing is that tech workers have a ridiculous I
- 26:57Speaker 2: mean especially AI engineers, right has an incredible amount of
- 27:00Speaker 2: cloud these days because they're so valuable. And I think
- 27:02Speaker 2: this is you know, one reason why Sam Alman and
- 27:06Speaker 2: others are going to extraordinary lengths to try and explain
- 27:10Speaker 2: and to a piece their their staff and maybe change
- 27:14Speaker 2: even change tax because of that.
- 27:15Speaker 1: Yeah, well, there were there were a bunch of workers,
- 27:17Speaker 1: I mean hundreds of workers from open Ai and Google
- 27:20Speaker 1: who signed an open letter basically supporting anthropic. There is
- 27:26Speaker 1: something of a little bit of pushback, if I can
- 27:30Speaker 1: call it that, from inside of these companies, and even
- 27:33Speaker 1: you know, at the consumer angle, people uninstalling chat ubt
- 27:37Speaker 1: and installing Claude. The question, though, becomes, are both of
- 27:42Speaker 1: those things enough? Is the pushback from inside the company?
- 27:45Speaker 1: Enough is a bunch of people deciding that, Hey, I'm
- 27:47Speaker 1: gonna uninstall open Ai and use the woke claud which
- 27:51Speaker 1: I think we've established is is maybe not that maybe
- 27:53Speaker 1: not as woke as some people might have be given
- 27:56Speaker 1: it credit for. What does that do? Does that anything
- 28:00Speaker 1: move the needle?
- 28:02Speaker 2: I don't think it does at this stage. I think
- 28:04Speaker 2: the question is does it go from hundreds of workers
- 28:06Speaker 2: to thousands and does the pushback from the public, which
- 28:11Speaker 2: already seems to be fading somewhat, does that turn into
- 28:15Speaker 2: something maybe with further incidents that has a meaningful impact
- 28:18Speaker 2: on bottom line, you know. And it is sort of
- 28:21Speaker 2: hard these days, right for people. It's harder for people
- 28:24Speaker 2: to push back and draw lines when it comes to
- 28:26Speaker 2: the government. It seems like everybody's a bit more curtailed
- 28:29Speaker 2: and scared, right, But at some point that you know,
- 28:32Speaker 2: as we've seen with some of the protests, that there
- 28:35Speaker 2: is a sort of breaking point, and I think it
- 28:37Speaker 2: needs to be a bigger pushback. But I think it
- 28:39Speaker 2: also we probably should have more nuance around what that
- 28:45Speaker 2: pushback here it really is, you know, just saying anthropic
- 28:48Speaker 2: is better. It's actually maybe like maybe everybody should get
- 28:52Speaker 2: together and say all these companies should be saying, hey,
- 28:54Speaker 2: we're not totally sure this stuff is ready even for
- 28:58Speaker 2: half the use cases whatever it might be like, and
- 29:01Speaker 2: then have the government explain to people how it's being mean.
- 29:04Speaker 2: It's one of the things that America can do, you know,
- 29:07Speaker 2: is pushes political leaders in theory to do that. So
- 29:10Speaker 2: I would rewind hope.
- 29:12Speaker 1: For well, thank you so much for coming through and
- 29:16Speaker 1: talking about this. It's heavy stuff, but we got to
- 29:18Speaker 1: talk about it. Thank you for real.
- 29:19Speaker 2: You're very welcome, ye, thanks for having me.
- 29:24Speaker 1: Thank you so much for listening to another episode of
- 29:26Speaker 1: kill Switch. You can email us if you want to
- 29:29Speaker 1: talk at kill Switch at Kaleidoscope dot NYC or on
- 29:32Speaker 1: Instagram or at kill switch Pod, and if you like
- 29:35Speaker 1: what you're hearing, maybe leave us a review. It helps
- 29:38Speaker 1: other people find the show, which helps us keep doing
- 29:40Speaker 1: our thing. And as I said at the beginning of
- 29:43Speaker 1: the show, kill Switch is on YouTube, so if you
- 29:45Speaker 1: want to catch the next one live, the link for
- 29:48Speaker 1: that and everything else is in the show notes. Kill
- 29:51Speaker 1: Switch is hosted by Me Dexter Thomas. It's produced by
- 29:54Speaker 1: Sena Ozaki, Darluck Potts, and Julia Nutter. Our theme song
- 29:59Speaker 1: is by me and Kyle Murdoch from Kaleidoscope. Our executive
- 30:02Speaker 1: producers are Ozma Lashin, mungesh Hati Kadur, and Kate Osborne
- 30:06Speaker 1: from iHeart. Our executive producers are Katrina Norvil and Nikki Ecord.
- 30:11Speaker 1: Catch on the
- 30:11Speaker 3: Next one, Goodbye,