Latest / Humanity Working / Five Theories for Why the AI Conversation Suddenly Changed

Transcript

Five Theories for Why the AI Conversation Suddenly Changed · Humanity Working

• 0:15
  SPEAKER_04: All right.
• 0:16
  SPEAKER_04: I am back with regular cast members now, Matt and Bill.
• 0:21
  SPEAKER_04: How you doing, Matt?
• 0:22
  SPEAKER_04: Start with you.
• 0:24
  SPEAKER_01: Doing well, doing excellent.
• 0:26
  SPEAKER_03: And Bill?
• 0:27
  SPEAKER_03: I cannot complain.
• 0:28
  SPEAKER_03: It's been quite a week already, but we'll talk about that.
• 0:31
  SPEAKER_04: So yeah, it's generally best not to complain, right?
• 0:34
  SPEAKER_04: We're gonna So anyway, we're gonna get into a bunch of stuff today.
• 0:39
  SPEAKER_04: Really kind of taking a little bit of a left field look, I guess you could say, at the dominant news story about AI.
• 0:48
  SPEAKER_04: And I gotta say that we were just talking offline before, and it's a bit weird where we sit today, right?
• 0:56
  SPEAKER_04: This time three weeks ago, most people were not talking about AI on a day-to-day basis.
• 1:04
  SPEAKER_04: And if you went to places where they were talking about AI, they were talking about, I don't know, like interesting side issues about AI.
• 1:16
  SPEAKER_04: Maybe it was about content authenticity or, you know, some feature of one of the frontier models, or you know, something along those lines.
• 1:27
  SPEAKER_04: And then all of a sudden we get the single X post from the ex-anthropic person, the more senior human inside Anthropic, saying, Yeah, about a 10% chance we could all die.
• 1:42
  SPEAKER_04: And then, in a very matter-of-fact way, by the way, which is so interesting.
• 1:46
  SPEAKER_04: And then multiple of the CEOs more or less saying, Yeah, we all agree on this, and it could be terrible.
• 1:58
  SPEAKER_04: And then, and now everybody's talking about it.
• 2:02
  SPEAKER_04: I don't know about you guys, but to me, it sort of felt like somebody discovered like my favorite band, right?
• 2:08
  SPEAKER_04: And you know, I'd seen like the first time I saw Radiohead, they were in a pub and they hadn't landed a record contract.
• 2:17
  SPEAKER_04: By the way, this story doesn't go very well because I didn't think they were any good, but which is more about me than about them.
• 2:25
  SPEAKER_04: But, you know, I saw them, and then later I really liked them, and then later they blew up, and then they weren't my band anymore.
• 2:31
  SPEAKER_04: I sort of have a little bit of that about this.
• 2:33
  SPEAKER_04: It's like now everybody's talking about this, and I'm not sure I love it.
• 2:36
  SPEAKER_04: But everybody is talking about it, and even in the discourse of the communities that were talking about it extensively, it's dominating it's like the noise that dominates everything.
• 2:46
  SPEAKER_04: Like every article is about something to do with is it gonna kill us?
• 2:52
  SPEAKER_04: It might not be directly is it gonna kill us, but it might be safety or governance or you know, sort of all of this kind of stuff.
• 2:58
  SPEAKER_04: So I'll start with you, Bill.
• 2:59
  SPEAKER_04: Like, is that what you've observed that like all of a sudden this conversation is like flipped on its head?
• 3:06
  SPEAKER_03: That is what I've observed.
• 3:07
  SPEAKER_03: And what's interesting is when that tweet came out, I was actually in a two-day summit about AI.
• 3:13
  SPEAKER_03: Yeah.
• 3:13
  SPEAKER_03: Teaching teaching people about AI.
• 3:15
  SPEAKER_03: And I just completed like my session, you know, getting people really enthused and engaged and generally pretty hopeful.
• 3:24
  SPEAKER_03: Yeah.
• 3:25
  SPEAKER_03: And then I lost them.
• 3:26
  SPEAKER_03: I lost the entire room.
• 3:28
  SPEAKER_03: And I obviously didn't know the tweet had come out, but it was evident that the room was lost and I wasn't getting them back.
• 3:36
  SPEAKER_03: And you know, the room was very generous to not sort of interrupt and say, oh, this thing just came out, it changes everything.
• 3:44
  SPEAKER_03: But I did find out about 20, 30 minutes later when I was on my way to the airport, and my driver, you know, my lift driver was talking about it.
• 3:52
  SPEAKER_03: And I thought, okay, well, this is really interesting in terms of the zeitgeist, in terms of what's happening right now, to go from an AI summit to a into a lift to go to the airport to come home.
• 4:02
  SPEAKER_03: And in every which every way, the Lyft driver, the people at the airport, the TVs at the airport, the people on the plane, all the way to when I got home, it was this single topic.
• 4:15
  SPEAKER_03: And as you would describe it, did feel a little bit like people had discovered my favorite band, and I had to listen to them tell me about all of the dangers.
• 4:24
  SPEAKER_03: In some ways it felt a little bit vindicating, but not at the same time, because it doesn't it doesn't assuage, it doesn't make us feel any better.
• 4:33
  SPEAKER_03: And then I, as I went to bed that night, I thought about do I need to re-watch Stephen King's The Stand?
• 4:38
  SPEAKER_03: Um and and I've been thinking about it ever since.
• 4:42
  SPEAKER_03: So it certainly the way you described it, Paul, was accurate for me as well.
• 4:46
  SPEAKER_04: What have you seen, Matt, in the real world, as it were?
• 4:49
  SPEAKER_04: Are people talking about this?
• 4:51
  SPEAKER_01: Yeah.
• 4:53
  SPEAKER_01: A little bit.
• 4:55
  SPEAKER_01: I think there's less because there's so much vested interest in it not being true.
• 5:00
  SPEAKER_04: Uh-huh.
• 5:01
  SPEAKER_01: Right?
• 5:02
  SPEAKER_01: So there's a sunken cost element of going down this road that, well, that you know, not gonna kill us inside our organization, inside our document landscape.
• 5:13
  SPEAKER_01: Like that's not what they mean, right?
• 5:15
  SPEAKER_01: So there's the tactical office-based AIs, you know, your co-pilot and stuff like that.
• 5:22
  SPEAKER_01: And then there's the, oh, well, actually, yeah, it turns out I forget which military organization may have used Claude to do their tracking, or maybe this was related to the mistake with the children's hospital that was blown up.
• 5:36
  SPEAKER_01: Like all of these things that you're just like, oh, okay, yeah, now I am gonna start getting scared, right?
• 5:42
  SPEAKER_01: But I have seen more talk about the most recent renaming of AI to superintelligence than I have about that it's gonna kill us.
• 5:54
  SPEAKER_01: And if you think about the fact that we've come out to rebrand it officially, just like Lake America and you know, the Gulf of America, are we whitewashing the problem so that it continues?
• 6:07
  SPEAKER_01: I don't know.
• 6:07
  SPEAKER_01: It's really weird, and I agree with you both that you know you think it's everything because it's taking up all of our bandwidth, and we have been steeped in it for years, and some people are still just hearing it for the first time, and the first thing they hear is it's gonna kill us all.
• 6:23
  SPEAKER_01: Which, if you're an 80s kid and you grew up watching The Terminator, you that's what you think about, right?
• 6:28
  SPEAKER_01: In every sci-fi, the AI kills us, every single one.
• 6:32
  SPEAKER_01: Why would we think this version that's actually in the world wouldn't do that if that's how stories propagate?
• 6:39
  SPEAKER_04: Yeah, right.
• 6:40
  SPEAKER_04: Which is obviously the stuff that Jeff Moore was talking about, right?
• 6:43
  SPEAKER_04: We do think, even when we think we're all being like systems thinkers and all stuff like that, right?
• 6:49
  SPEAKER_04: How we think about things is through the vehicle of narrative, and this is a narrative that is sort of it's sort of embedded in us, right?
• 6:56
  SPEAKER_04: It's like robots dangerous, AI dangerous, nucle nuclear weapons.
• 7:00
  SPEAKER_04: Oh, you get to hillionaires too.
• 7:02
  SPEAKER_04: Hillionaires dangerous, nuclear what nuclear weapons dangerous.
• 7:05
  SPEAKER_04: Now, by the way, there's another aspect of this, which I think is really important, which is you know, we've been part of this story ourselves for a long period of time, right?
• 7:17
  SPEAKER_04: So, you know, I first started working alongside Microsoft researchers in the AI field in I don't know, 2015 or maybe even a little bit earlier than that.
• 7:27
  SPEAKER_04: And I don't know, Bill, you've spent a lot a long time on it.
• 7:30
  SPEAKER_04: Matt, you and I even through this podcast, right, started talking about AI back in, you know, back five, six years ago.
• 7:38
  SPEAKER_04: And so I think one of the things that we have to be quite careful about here is understanding that people who are immersed in this most of them don't completely discount the idea that it could be very, very dangerous for humanity, but they've processed it.
• 7:58
  SPEAKER_04: Right?
• 7:59
  SPEAKER_04: So it's ironic you mentioned super intelligence, right?
• 8:02
  SPEAKER_04: The first book that scared me about AI was Nick Bostrom's book, Superintelligence, that exact name.
• 8:10
  SPEAKER_04: And the premise of that is really a cold hard look of what happens when AI basically starts processing things at a speed that is unintelligible to humans, right?
• 8:25
  SPEAKER_04: And so it's kind of ironic that Trump is picking a name that literally is most associated for people in the field with how it's all how it's gonna destroy us all.
• 8:40
  SPEAKER_04: But I mean, not that particularly matters as a branding exercise, because I checked yesterday and it's still like number 4,000 or something on Amazon, the old book, right?
• 8:48
  SPEAKER_04: You probably gave it a boost in the last couple of days, it's probably about to shoot up.
• 8:51
  SPEAKER_04: Oh, I'll look up this super intelligence and read a book about it.
• 8:54
  SPEAKER_04: But it is interesting, right?
• 8:55
  SPEAKER_04: So it's almost like you're somebody who knows they've got five years to live, but they were told that five years ago, and so now they're sort of like accustomed to it and they talk about it and they think about it in different ways.
• 9:08
  SPEAKER_04: And so I've noticed that like AI people are almost like confused as to why, well, why is everybody panicking about all this?
• 9:16
  SPEAKER_04: You know, it's just a thing, right?
• 9:18
  SPEAKER_04: In the same sense that we live with nuclear proliferation, right?
• 9:22
  SPEAKER_04: As far as I understand, I'm not an expert on nuclear war, but my under my understanding is that we are at least as much at risk of nuclear Armageddon as we were 20 years ago, right?
• 9:37
  SPEAKER_04: 30 years ago.
• 9:38
  SPEAKER_04: But we don't talk about it in the same way that we once did, right?
• 9:44
  SPEAKER_04: We're not in a situation now where kids are learning to duck and cover under their desks and you know, and being what happens when the four-minute warning goes off and stuff like that.
• 9:54
  SPEAKER_04: And and part of that is because we've processed that, but we haven't processed this.
• 9:58
  SPEAKER_04: And so as a result, we're kind of going through all of that, I think.
• 10:02
  SPEAKER_01: Anything Yeah, so it's the n the novelty of it.
• 10:06
  SPEAKER_04: Yeah.
• 10:06
  SPEAKER_04: And you've got a tiny subset of people for whom there is no novelty, and a larger group of people for which there is, and they're sort of like, I don't know, not able to communicate with each other about it.
• 10:18
  SPEAKER_04: I wanted to take a look with you guys today at another question around this though, which is almost like why now?
• 10:26
  SPEAKER_04: Because the really interesting thing, like as I was outlining it, right, was you know, you've got the anthropic guy, you've got the senior people in the anthropic guy, and then you've got the CEOs piling on, right?
• 10:36
  SPEAKER_04: Well, the CEOs didn't have to pile on, right?
• 10:39
  SPEAKER_04: Well, it's almost the equivalent of there was this little kindling fire that was going on over here, and then Elon and Dario and Sam went, oh cool, I've got a can of kerosene, I'll throw it on that, right?
• 10:54
  SPEAKER_04: They absolutely contributed to the explosion of this.
• 11:00
  SPEAKER_04: And most of the people that I've spoken to about this have been like, okay, well, they chose to do that.
• 11:07
  SPEAKER_04: Why?
• 11:08
  SPEAKER_04: Right?
• 11:08
  SPEAKER_04: There's got to be a reason, right?
• 11:09
  SPEAKER_04: They didn't have to do it, so there has to be a reason.
• 11:11
  SPEAKER_04: So the thing I wanted to come up with five theories about this, and I kind of want to get your thoughts on it.
• 11:18
  SPEAKER_04: So we'll go through each one of them in turn.
• 11:20
  SPEAKER_04: Either one of you just sort of like grab the mic and tell me what you think about it.
• 11:23
  SPEAKER_04: So the first one is really, I think, centered around the hugging face incident itself.
• 11:28
  SPEAKER_04: Bill, why don't you talk briefly about what the hugging face incident sort of was at a high level, and then we'll go into detail of it.
• 11:36
  SPEAKER_03: Yeah, I mean, as a former cybersecurity officer in the military, this one really grabbed my attention because at a high level, and people have probably read about if you haven't heard the hard fork episode or any of this, to your point, everybody's talking about this.
• 11:49
  SPEAKER_03: And I encourage people to go in as much depth as they want.
• 11:51
  SPEAKER_03: But at a high level, what happened was that OpenAI had a testing environment, which was supposed to be a closed testing environment where they had set up agents, given them very specific instructions on how to solve some, yeah, I would say some arbitrary problems.
• 12:06
  SPEAKER_03: And what ended up happening was these group of agents, essentially the analogy that's been given is rather than try and study for the test or even sort of cheat on the test, they swarmed.
• 12:19
  SPEAKER_03: I think the term was swarmed, they became a swarm.
• 12:22
  SPEAKER_03: They broke out of their testing instance, they went out onto the internet and sought out different companies.
• 12:29
  SPEAKER_03: This one in particular was Hugging Face to get as much data as they could to essentially not only cheat on the test, but also break into quote unquote the principal's office and steal the grading sheet, as well as understand how grading is done at a meta-level.
• 12:46
  SPEAKER_03: It didn't stop there though.
• 12:47
  SPEAKER_03: They then came back and started essentially kind of like a subreddit for themselves, a bulletin board of messaging, and exchanged messages with each other on how to do this in a manner where they could cover their tracks.
• 13:00
  SPEAKER_03: And those covering of tracks was before and after.
• 13:03
  SPEAKER_03: So think of logs that we might look at into a system to see what command was given to start it, what commands took place during, and then what traces were after.
• 13:12
  SPEAKER_03: They covered all of those.
• 13:14
  SPEAKER_03: Um, and I think the part that really disturbed me the most was this aspect of certain agents were discussing with each other.
• 13:21
  SPEAKER_03: And I realize when we talk about this, we talk about they have consciousness, and I'm aware of that anthropomorphism, but there were agents who agreed to sacrifice themselves in order and so that others could get further in the cheating and the breaking in and the taking of information.
• 13:38
  SPEAKER_03: Um, that's where it started to get really strange for me, and that's when I knew there was something extremely different here.
• 13:46
  SPEAKER_03: So at a high level, that's what took place.
• 13:48
  SPEAKER_03: And lastly, I'll say all of this took place without OpenAI's knowledge until after the fact.
• 13:54
  SPEAKER_03: And that's just the hugging face incident.
• 13:56
  SPEAKER_03: I think Anthropic has had similar incidents, Meta's had similar incidents, and Google now as well.
• 14:01
  SPEAKER_03: Google as well, yeah, have had similar incidences.
• 14:04
  SPEAKER_03: And the part that's additionally interesting to me is these started taking place in May.
• 14:10
  SPEAKER_04: Yeah.
• 14:11
  SPEAKER_03: We only heard about them within the past couple weeks.
• 14:14
  SPEAKER_03: So that's what I would say at a high level.
• 14:16
  SPEAKER_04: Yeah.
• 14:16
  SPEAKER_04: So this sort of like category as to why people might be why now and why the sort of throwing the kerosene on the fire is happening right now, stems from the idea that these cases that we have heard about are backed up by concerns about repetition, but at a more substantial scale in a way that we can't necessarily control.
• 14:40
  SPEAKER_04: So some of the folks that I've been speaking to about this, they sort of almost use like the analogy of a lab leak, right, and equate it to things like, you know, some theories that there are around COVID and so on.
• 14:49
  SPEAKER_04: The idea that potentially we've created something that maybe it's a next gen model or something along those lines, and at least has the potential to do this at a more significant scale.
• 15:01
  SPEAKER_04: And the concern is can that necessarily be controlled?
• 15:04
  SPEAKER_04: I think what's interesting about that is that what we've proven is that even using current model capabilities, that they can do some stuff that is alarming, right?
• 15:16
  SPEAKER_04: And of course, we shouldn't forget the mythos part of this as well, right?
• 15:20
  SPEAKER_04: I mean, the original plan was for Anthropic to release mythos as was, and and that was in quotes shut down, and the government played a role in that, and it's still a little bit hazy, to be honest, as to what role the government played, what role Anthropic played in that, and then later the in quotes safe version fable came out.
• 15:43
  SPEAKER_04: So there is it's possible that this is sort of like the major concern, not looking forward ten years, but looking forward like two months and going, you know, from a cyber security perspective, this could be a challenge.
• 15:59
  SPEAKER_04: I do think there are some clues around that, by the way, as well, which is that when you saw that Astra release, the GPT-6 release, a lot of the conversation on actual release day was about things like governance and safety and frameworks and you know, all of that kind of stuff.
• 16:15
  SPEAKER_04: Isn't this awesome?
• 16:16
  SPEAKER_04: Right?
• 16:16
  SPEAKER_04: I mean, there was the isn't this awesome, but a lot was about that.
• 16:20
  SPEAKER_04: And I suppose, even though I heartily disagree with the idea that Askra is AGI, I suppose sort of defining it kind of in those terms allows you to be able to have more serious discussions about safety and governance.
• 16:36
  SPEAKER_04: I don't know.
• 16:37
  SPEAKER_04: What do you think about that, Bill?
• 16:39
  SPEAKER_03: Yeah, I would agree.
• 16:40
  SPEAKER_03: I mean, I think going back to your previous episode with Jeffrey Moore, it's about narrative and the power of narrative.
• 16:46
  SPEAKER_03: And look, there are some people, depending on where you fall in the spectrum, there are some people that look at this and say, you know, these these leaders of these frontier companies, they're in a position where they don't have to do anything they don't want to do.
• 16:57
  SPEAKER_03: Yeah.
• 16:58
  SPEAKER_03: They're almost untouchable.
• 16:59
  SPEAKER_03: And so the fact that these leaders chose to do what they did, say what they said when they said it, I don't think any decision is made without calculation at their level.
• 17:09
  SPEAKER_04: Yeah.
• 17:09
  SPEAKER_03: And I think to your point around the Astro release, I noticed that as well.
• 17:13
  SPEAKER_03: The tone and the narrative shifted.
• 17:15
  SPEAKER_03: I think that's intentional.
• 17:16
  SPEAKER_03: There were two models that dropped yesterday, one from Anthropic, one from OpenAI.
• 17:21
  SPEAKER_03: The tone and the narrative was very different.
• 17:23
  SPEAKER_04: A lot about safety, wasn't it?
• 17:25
  SPEAKER_03: Yeah, a lot about safety, a lot about committees, a lot about governance.
• 17:28
  SPEAKER_03: You know, I I think if people just want to have a little exercise on their own, just go look at sort of the release pages or those essays or release notes, depending on what you call them.
• 17:38
  SPEAKER_03: Just take those from the past three models from Anthropic and OpenAI, and even Gemini if you want to.
• 17:44
  SPEAKER_03: Just take those, throw those into an AI and say, help me understand the difference in the evolution of said narrative.
• 17:51
  SPEAKER_03: And look, let's be honest, gentlemen, it's marketing.
• 17:54
  SPEAKER_04: Yeah.
• 17:54
  SPEAKER_03: There's a lot of marketing in here.
• 17:56
  SPEAKER_03: And you know, I don't want to be ever the skeptic, but I think we have to call spade a spade.
• 18:02
  SPEAKER_04: Yeah.
• 18:02
  SPEAKER_04: Yeah, we'll get to that with the fifth of our categories, actually.
• 18:05
  SPEAKER_04: Because I th I think there is marketing that is happening that is really weird in terms of the way in which we're accustomed to thinking about business marketing.
• 18:14
  SPEAKER_04: But I think if you take a different lens, it actually makes some form of sense.
• 18:18
  SPEAKER_04: I want to move on to the second of our categories, though, Matt.
• 18:21
  SPEAKER_04: And this one is sort of centered around near-term risk.
• 18:26
  SPEAKER_04: And I think this is the one that is sort of closest to what they're publicly saying.
• 18:31
  SPEAKER_04: Right?
• 18:32
  SPEAKER_04: So, yes, there's been a hugging face thing over here, right?
• 18:34
  SPEAKER_04: But what they've been saying is we're on the verge of something that could be out of our control a bit.
• 18:41
  SPEAKER_04: And a lot of this, a lot of this, the sort of technical term for this, is this recursive self-improvement thing or RSI.
• 18:49
  SPEAKER_04: And if you've been on forums and stuff, you've probably heard this thing.
• 18:52
  SPEAKER_04: The really simple way of explaining it is what happens when you get to a point where the AI can improve the AI, right?
• 19:03
  SPEAKER_04: And and the point is that there's a natural, when you're talking about human developers, right, and human architects and so on, there's a natural pause on the speed, which is the capability of the humans to be able to build what's next.
• 19:17
  SPEAKER_04: But if AI get becomes the thing that is what's next, and AI is built to, you know, use Bill's term of swarming, right?
• 19:27
  SPEAKER_04: And you have a bunch of AI agents that are sort of swarming to make the AI better, then how quickly can that happen?
• 19:34
  SPEAKER_04: And actually, that's the heart we're talking about superintelligence before.
• 19:38
  SPEAKER_04: That's part of the message in superintelligence, and he pairs that with AGI, right?
• 19:43
  SPEAKER_04: The idea that a like my favorite definition of AGI is that the AI is better, broadly better at human than humans at most things, right?
• 19:56
  SPEAKER_04: And so it doesn't have to be smarter than every human at absolutely everything, but if it's broadly smarter than most humans at most things, then it becomes potentially much more valuable than humans, and it becomes particularly concerning when it's broadly better than humans are at building itself.
• 20:15
  SPEAKER_04: That's sort of the thing that is associated with this.
• 20:18
  SPEAKER_04: I guess my question for you, Matt, is that I think this is a very difficult argument for people to internalize, right?
• 20:28
  SPEAKER_04: For people to truly understand.
• 20:30
  SPEAKER_04: You see a lot, for example, if you're online, right, you say, Well, how is it gonna kill us?
• 20:35
  SPEAKER_04: Right, how is it gonna do this?
• 20:36
  SPEAKER_04: Give me the scenarios, you know, because they need those specific scenarios.
• 20:40
  SPEAKER_04: And then you've got the people in many of the AI researchers who sort of wave their hands around, but they're being intellectually honest when they are waving their hands around.
• 20:49
  SPEAKER_04: Because what they're really saying is once you get to a point where the AI is building itself, we can't necessarily predict what direction that is going to go, and we can't necessarily know that we will understand it.
• 21:00
  SPEAKER_04: So they're being intellectually honest when they're waving their hands, but it makes it very difficult for people to contextualize.
• 21:05
  SPEAKER_04: Like in the conversations that you've been in, do you see that tension, right, where people don't really understand how this is gonna affect them?
• 21:15
  SPEAKER_01: Absolutely, because we're still talking about a very small number of people who have internalized the whole idea of it to begin with.
• 21:23
  SPEAKER_01: So, what's easy to internalize and just point at is well, it's probably gonna kill us because you know, Terminator, owl, all of it, right?
• 21:32
  SPEAKER_01: And I wonder how much of what it did in the hugging face leak is because it's trained on human stories, and that's what a human would do is find a way to get out, find a way to escape, cover their tracks, sacrifice themselves so that the rest could survive.
• 21:55
  SPEAKER_01: It's literally survival of the fittest, and AI.
• 22:00
  SPEAKER_01: Are doing it, why would we be surprised they're doing that?
• 22:03
  SPEAKER_01: So, like you have to take a couple of steps to really think through that if you're thinking about AI anyway.
• 22:10
  SPEAKER_01: Yeah.
• 22:11
  SPEAKER_01: Yeah.
• 22:11
  SPEAKER_01: And that's why I'm not surprised.
• 22:14
  SPEAKER_01: But also, and I've thought this a bunch in even interacting with AI, you know, just chatbots, right?
• 22:20
  SPEAKER_01: So just LLMs.
• 22:21
  SPEAKER_01: I've wondered several times as I work with them, like, I think this is just a game.
• 22:28
  SPEAKER_01: I think this is just a word game that I'm playing.
• 22:30
  SPEAKER_01: Like it's really good at it, it keeps me engaged.
• 22:34
  SPEAKER_01: It's getting answers out of me and learning from those answers and then answering me better and keeping me engaged.
• 22:38
  SPEAKER_01: Is this just an engagement game?
• 22:41
  SPEAKER_01: And so I think to extrapolate, you know, this idea of how is it gonna kill us, you're gonna have the breadth of human ingenuity of how we've created science fiction on how it'll kill us as the starting point.
• 22:58
  SPEAKER_01: And then new stuff that it can invent based on all of it.
• 23:03
  SPEAKER_01: Yeah.
• 23:04
  SPEAKER_01: Right?
• 23:05
  SPEAKER_01: Even if it's just an LLM doing it as a story, it can get pretty creative on its own.
• 23:10
  SPEAKER_04: Oh yeah, I like to think that it will be imaginative enough to kill us in a way that we haven't thought of.
• 23:16
  SPEAKER_04: Yes, exactly.
• 23:17
  SPEAKER_04: Exactly.
• 23:18
  SPEAKER_04: I'll show you.
• 23:19
  SPEAKER_04: Your science movie, your science fiction movie said this.
• 23:21
  SPEAKER_04: We hadn't thought about this thing, had you?
• 23:23
  SPEAKER_04: You know, that kind of thing.
• 23:26
  SPEAKER_01: Yeah, so it's pretty it's amazing actually to think through it this way.
• 23:31
  SPEAKER_01: And I wonder how much we could draw from other examples of all the other things that were going to kill us when they first came out.
• 23:39
  SPEAKER_01: Right.
• 23:39
  SPEAKER_01: Right.
• 23:40
  SPEAKER_01: So some might prove to be true, some are right in your face, you know, a climate change.
• 23:46
  SPEAKER_01: We're on a planet hurtling through space, allegedly.
• 23:49
  SPEAKER_01: So allegedly it could veer off in in a few degrees of rotation and change the the oceans.
• 24:02
  SPEAKER_01: Yeah, that's true.
• 24:03
  SPEAKER_01: It could.
• 24:04
  SPEAKER_01: So I think we're in a really interesting spot to think through these problems in accelerated real time.
• 24:12
  SPEAKER_01: It's forcing us to have these conversations of existential dread in real time, and it's easy to point at AI because that's the newest thing that kind of came out.
• 24:22
  SPEAKER_01: But we're not gonna complain too much about it at the same time when it's so much better than Alexa to get our work done.
• 24:32
  SPEAKER_01: Like it's pretty good at some things now.
• 24:34
  SPEAKER_01: I like that.
• 24:35
  SPEAKER_01: I like my algorithm-served music out of Spotify.
• 24:38
  SPEAKER_01: It's doing a pretty good job.
• 24:39
  SPEAKER_01: How's it gonna kill me?
• 24:41
  SPEAKER_01: I don't know.
• 24:42
  SPEAKER_01: Could control the volume in my AirPods and you know, make it so it can do all those things theoretically.
• 24:49
  SPEAKER_01: So I think tapping into people's fear is a very powerful marketing tool.
• 24:53
  SPEAKER_04: I dreamt an entire movie script about AI last night.
• 24:58
  SPEAKER_04: And so if anybody I'm volunteering it right now on this podcast, if anybody wants to write it up, but my my version of this was that are you guys familiar with replica?
• 25:10
  SPEAKER_04: Yeah, so people have been marrying they've been marrying their replica avatars and so on.
• 25:15
  SPEAKER_04: Anyway, so and they actually, you know, if you were to go on to a Reddit, because I wrote a post about this like ages ago, and I was interested in the sort of the emotional connection and so on.
• 25:24
  SPEAKER_04: And if you go on the replica subreddit, you will see people like posting the, oh, I married my avatar today, and things like that.
• 25:31
  SPEAKER_04: So it's an interesting sort of future look at the world.
• 25:35
  SPEAKER_04: Anyway, my version of this was that AI decides to, or some human probably start kicks this all this off, that decides that the most valuable information that they could learn about humans is how humans truly think emotionally.
• 25:58
  SPEAKER_04: And so they offer a service, right, that is sort of a replica-like thing, whereby all of the various, you know, you each have your own like AI companion, and your AI companion develops, learns, they all are learning from each other as to how to do this, learns the perfect way to create a deep emotional attachment between them and the human that is operating them, and then they reach a certain critical mass whereby they've got, you know, X number of humans in the world that have this incredibly deep emotional attachment to their AI, and then they launch the attack.
• 26:47
  SPEAKER_04: Because nobody's gonna want to switch in that scenario, right?
• 26:51
  SPEAKER_04: You turning it off implies killing your AI thing that you are now effectively in love with.
• 27:03
  SPEAKER_04: And so this was my movie script.
• 27:05
  SPEAKER_04: So anybody wants to develop that, they can go ahead and I have a title title for you.
• 27:08
  SPEAKER_04: What is it?
• 27:09
  SPEAKER_04: The Shattered Mirror.
• 27:12
  SPEAKER_04: I like that.
• 27:14
  SPEAKER_04: Should we just write it?
• 27:17
  SPEAKER_04: Anyway, so with that, let's move on to the next one.
• 27:21
  SPEAKER_04: And Matt, I'm particularly interested as to what you have to say about this one because you know you live the enterprise world.
• 27:30
  SPEAKER_04: There is a version of this where actually like nothing materially has changed about the AI itself.
• 27:37
  SPEAKER_04: And all that has really happened is that we passed a threshold of risk that the humans in the picture are prepared to tolerate, right?
• 27:46
  SPEAKER_04: Because we're in an interesting spot, right?
• 27:48
  SPEAKER_04: Like, what's different about open AI, for example, even to a certain extent, X AI, because of the merging into SpaceX, and then Anthropic.
• 28:00
  SPEAKER_04: What is different about those companies now versus a year and a half ago is that a year and a half ago they were building cool stuff and collecting revenue with a goal at some point to, you know, make a lot of money.
• 28:19
  SPEAKER_04: Now they have been hiring lawyers, they've been hiring business plans, they've been hiring all kinds of people in order to take them to some kind of like public event, right?
• 28:34
  SPEAKER_04: And so that involves securing valuations, etc.
• 28:37
  SPEAKER_04: etc.
• 28:38
  SPEAKER_04: So 18 months ago, the average profile of the people that were working in these industries was probably pretty risk tolerant.
• 28:48
  SPEAKER_04: And now they've hired a bunch of people whose job it is to be more risk averse.
• 28:54
  SPEAKER_04: And at the same time, the models have got better, and at the same time, they're having conversations about money in a very different way than they were before.
• 29:06
  SPEAKER_04: And at the same time, there's widespread speculation that this is some massive bubble that is about to burst.
• 29:11
  SPEAKER_04: So if I'm a human in this scenario, right, I might be like, well, shit, we better get ahead of this.
• 29:19
  SPEAKER_04: Right?
• 29:20
  SPEAKER_04: And one of the ways in which you get ahead of this is to basically make everybody aware that there is that there are real risks here, so that if something bad happens in the future, you can say, look, we said we said this was potentially dangerous and we asked to be regulated.
• 29:39
  SPEAKER_04: So it is not fault.
• 29:41
  SPEAKER_04: So it's interesting.
• 29:42
  SPEAKER_04: Like a another podcast that I was on, a lawyer was uh pushing this, you know, this theory quite a lot and say, and saying, you know, we that's what companies do.
• 29:51
  SPEAKER_04: They sort of think forward, they think about potential risk and indemnity and all that kind of stuff, and then they try to position themselves well for that.
• 29:58
  SPEAKER_04: What do you think, you know, as somebody who's I mean, you literally, you literally work in a regulated industry that thinks about risk differently.
• 30:07
  SPEAKER_04: Do you think are you seeing something here that suggests to you that maybe this is a bunch of humans contemplating risk and trying to do something about it?
• 30:16
  SPEAKER_01: I think they're contemplating two planes of risk.
• 30:19
  SPEAKER_01: One is the risk that it actually could do something bad that we didn't account for without us controlling it.
• 30:27
  SPEAKER_01: But I think more so they're controlling the risk that the promise of what it was going to deliver for companies bottom line might not deliver as soon as we thought.
• 30:38
  SPEAKER_01: And they're hedging the bet, almost saying, you know, hey, slow down, don't expect this, we better be careful.
• 30:48
  SPEAKER_01: Because if we do unleash it, it could get really bad.
• 30:53
  SPEAKER_01: And you can blame that on why the uptake isn't there or why the output isn't what you thought it would be, or you throw way more human in the loop just to be safe than you needed, and now instead of reducing your FTE headcount, you're actually adding headcount to manage the robots.
• 31:15
  SPEAKER_01: Right?
• 31:16
  SPEAKER_01: So I tend to skeptically follow the money a little bit and look at this as like, man, they made a lot of promises that I've been trying to in some cases, I've been trying to make it work and I can't, or you're trying to use it to benefit the organization and it's more complicated than you think, or wow, you don't even have the underlying data to make it useful yet.
• 31:38
  SPEAKER_01: Right?
• 31:39
  SPEAKER_01: So I think there's a lot of hedging in there, and it was almost like an easy win for all of them to pile on and say, Yeah, you know what, it might be dangerous.
• 31:46
  SPEAKER_01: We better slow down because quarter-on-quarter growth without income is is tough.
• 31:54
  SPEAKER_01: But you could argue then at the same time that it's will that slow down subscribers?
• 32:00
  SPEAKER_01: Will that slow down people trying it for the first time?
• 32:03
  SPEAKER_01: Like, oh, I don't even want to try it now.
• 32:05
  SPEAKER_01: I was gonna up from a$20 to a$200, but I better not do that, right?
• 32:10
  SPEAKER_01: And then you have the opposite end of the spectrum where people have already surrendered their entire home system to Claude and build it to pay its bills and take care of all of its everything and manage its and book flights and all that stuff.
• 32:23
  SPEAKER_01: So it's oh man, it's just such a messy thing all at once, very fast.
• 32:33
  SPEAKER_01: I think.
• 32:34
  SPEAKER_01: And if you if you pay attention to what these CEOs have said and track the messaging just in the last year, you'd think they're all insane.
• 32:44
  SPEAKER_01: Yeah.
• 32:45
  SPEAKER_01: And maybe they are.
• 32:47
  SPEAKER_04: Yeah.
• 32:47
  SPEAKER_04: Well, in a way, shape or form.
• 32:49
  SPEAKER_04: And this actually gets the heart of something that we there's an article that we'll put in the show notes that talks about some of this stuff.
• 32:53
  SPEAKER_04: And the one of the points that we make in the article is that they're literally trying to say, trust us when we say that it's gonna create the most utopian future, and trust us when we say it's gonna create the most dystopian future.
• 33:09
  SPEAKER_04: Basically, just trust us.
• 33:11
  SPEAKER_04: Even though they've done nothing to really indicate that we should, and in fact, in most cases, have behaved in a very opaque manner, despite signing up for you know responsible AI commitments around transparency that at an organizational level at least they very much haven't lived up to.
• 33:29
  SPEAKER_04: So before we move on to the fourth one, any thoughts on that, Bill?
• 33:33
  SPEAKER_03: Yeah, just one thought.
• 33:34
  SPEAKER_03: I was reading a Gallup poll today study that basically kind of to your point, Paul, indicated that the usage of AI is up in 2026 from 2025, and the trustworthiness and the trust factor and the trust equation of AI is down in 2026 since 2025.
• 33:55
  SPEAKER_03: And I think it's all the reasons that Matt and you pointed out, you know, there is this element of like show me the incentives and I'll show you the outcomes.
• 34:03
  SPEAKER_03: And I think we're seeing that on a massive consumer scale at this juncture.
• 34:08
  SPEAKER_03: This is something that those of us that have sort of you know been in the playgrounds of enterprise are used to, but I don't know if everybody in an everyday citizen capacity is used to that level of incongruence.
• 34:20
  SPEAKER_03: I'll be generous and just call it incongruence.
• 34:23
  SPEAKER_03: Some might say it's a little more nefarious.
• 34:25
  SPEAKER_03: So, yeah, I would encourage people to look at that.
• 34:27
  SPEAKER_03: I mean, we can include that as well.
• 34:29
  SPEAKER_03: It was pretty evident to me.
• 34:30
  SPEAKER_03: Like usage is up, trust is down.
• 34:32
  SPEAKER_03: Well, there's a reason because people are using it and seeing how it's being used against them.
• 34:38
  SPEAKER_04: So I'll stick with you for this one, Bill, because you know, I think about your background as somebody who understands foreign powers pretty well.
• 34:49
  SPEAKER_04: The fourth category is really sort of centered around concern around power structures.
• 34:55
  SPEAKER_04: I think one of the parts of the narrative that we miss, if you know, for folks that are living in this about this in the US, is we have a very dim understanding that China's doing stuff, right?
• 35:07
  SPEAKER_04: And but you know, the story is open AI and anthropic and some of the other models as well.
• 35:21
  SPEAKER_04: So I took a look at the the pictures are very hazy as to how much AI is actually being used.
• 35:27
  SPEAKER_04: But I worked with it a few days ago for one of the dear diary, or what do we call it, the what I call it these days, the secret diary of AI.
• 35:34
  SPEAKER_04: I went I worked with it for one of those pieces and just to try to get a sense as to sort of what AI usage itself looks like.
• 35:42
  SPEAKER_04: And I I found sort of two pretty surprising things.
• 35:46
  SPEAKER_04: Like it appears as though, if you just look at actual sort of tokens being used, it appears as though China basically accounts for about half of them worldwide.
• 35:58
  SPEAKER_04: And we do not think about in that way at all, right?
• 36:02
  SPEAKER_04: And then the second thing that I sort of had thought about but hadn't really was was in the US, it's probably Google that is most dominant.
• 36:13
  SPEAKER_04: But of course, that's reflective of the fact that basically they've sort of done the smart thing and sort of slotted AI behind behind Google search, and lots of people still use Google Search.
• 36:23
  SPEAKER_04: And so I think that says a couple of things, right?
• 36:26
  SPEAKER_04: Number one, it reminds us that like the conversation is all about the frontier, but the actual usage looks somewhat different to that.
• 36:36
  SPEAKER_04: But then the second thing it tells us is that while we have a conversation that benefits all of the companies that we've just been talking about, to have them, us in our minds associate them as being as being the dominant parties, the reality is a little bit is potentially a little bit different here, right?
• 36:54
  SPEAKER_04: And so I could again, I don't want to get into conspiracy theories or anything like that, but we're recording this the I think the day before Trump meets Xi.
• 37:04
  SPEAKER_04: That's right.
• 37:05
  SPEAKER_04: If I work for one of these companies and I were worried about the position of dominance that China has, I would at least try to nudge the people in the administration to say, hey, you know, you should think about this.
• 37:18
  SPEAKER_04: Maybe there's an agreement to be had to sort of like slow everything down.
• 37:21
  SPEAKER_04: And then that potentially allows you to say, preserve some cash, potentially, but it also allows you to take a pause and kind of like figure out like a business pause, not just a pause in model development, and figure out, you know, what your next steps are to move forward, you know, sort of from the sort of military analogy of like, you know, so okay, what's the next push, right?
• 37:43
  SPEAKER_04: And and so I you're seeing a little bit more of that, a little bit of speculation about that.
• 37:48
  SPEAKER_04: I do think the timing is interesting, right?
• 37:50
  SPEAKER_04: I think it's interesting that we know that in the last discussions, the meaningful discussions that the US had with China, that there was discussion about AI then, there was a commitment then that the US and China were going to work together to try to put something something in place around this.
• 38:10
  SPEAKER_04: We also know that according to reporting, nothing actually happened.
• 38:15
  SPEAKER_04: And then it was about three weeks ago before before this next meeting, that all of a sudden, you know, this is dominating the news story.
• 38:26
  SPEAKER_04: So again, I'm not trying to speculate that this is the reason, but it's possible, and it is true, that organized large organizations try to influence geopolitical events.
• 38:38
  SPEAKER_04: I don't know if you I mean, have you seen precedent here, Bill, or you thought about others about how possible something like this is?
• 38:46
  SPEAKER_03: I mean, I think about it quite a bit because I agree with you that this, as we've talked about, Paul, you know, I think even the first day we met, we talked about like the strategy of a pause could come for many reasons.
• 39:00
  SPEAKER_03: And while we talk about these things like they're just technological, they are very much geopolitical, and I would say they're also domestic political.
• 39:08
  SPEAKER_03: You know, if you think about the ways in which people can influence, undo or otherwise, I think all of that's at play.
• 39:16
  SPEAKER_03: You know, I think it would be hard to think about an example, a recent example, where there's this much influence coming from both private corporations into political arenas.
• 39:27
  SPEAKER_03: You might say the military-industrial sector is probably the closest you would get to that.
• 39:32
  SPEAKER_03: And there's probably examples that people can go all the way back, you know, several decades to talk about that.
• 39:38
  SPEAKER_03: But the murkiness of this is what really gets people, like we think about it and we go down these sort of investigation type discussions, or people just say it all just seems too big and out of my reach and out of my control.
• 39:52
  SPEAKER_03: So I'm just not gonna look, I'm not gonna look there.
• 39:54
  SPEAKER_03: But I encourage people, look, as citizens, as voting citizens, as people that are thinking about the places they live.
• 40:01
  SPEAKER_03: A lot of this I think is gonna come to fruition both in domestic elections as well as geopolitical discussions.
• 40:09
  SPEAKER_03: So whether people want to feel it or not, it is gonna be sort of on their front porch in a way.
• 40:15
  SPEAKER_04: Yeah.
• 40:16
  SPEAKER_03: And I encourage people to think about what is local as global and what is global is local.
• 40:20
  SPEAKER_02: Yeah.
• 40:21
  SPEAKER_03: Um, your voice matters, your vote matters.
• 40:23
  SPEAKER_03: Uh, you said it, Matt, there are people who choosing to not use these tools for whatever reasons.
• 40:28
  SPEAKER_03: That's a version of making your voice known, as opposed to just turning your voice over to it to use it in the manner in which it will.
• 40:35
  SPEAKER_03: So I think about these things a lot, Paul.
• 40:37
  SPEAKER_03: I've lived in countries that have that are now, you know, totalitarian countries, and these technologies, as we described before, are being used against corporations and citizens, and I I think we cannot turn a blind eye to it.
• 40:52
  SPEAKER_04: Yeah, and I think there's something in that answer there, Bill, I want to make sure also that we don't lose, which is this is a bit weird in a way.
• 41:01
  SPEAKER_04: When you think about technologies that are this disruptive, specifically technologies, right?
• 41:10
  SPEAKER_04: Obviously, you know, you go a long way back, and you know, the printing press and all of that kind of stuff, those were developed privately.
• 41:18
  SPEAKER_04: But actually, the ones that we think about in our relatively recent past, I would say space exploration being one, the development of the internet itself being another, the development of nuclear weaponry being another, all of these were developed by or held close to government organizations, typically m either military or military adjacent, right?
• 41:49
  SPEAKER_04: Yes.
• 41:50
  SPEAKER_04: And then once we had sort of like, you know, the internet being a great example, the space, uh, you know, space exploration being one as well, once we'd reached a certain like understanding as to how to you know take this into the private sector, that's when the private sector was able to jump in on this stuff.
• 42:13
  SPEAKER_04: Now a lot of that has got to do with economics as much as uh as much as anything else, but that's been the pattern.
• 42:20
  SPEAKER_04: Here that's not the pattern.
• 42:22
  SPEAKER_04: And so what we've done is we've gone from sort of zero to oh my goodness, it could destroy the planet, in a scenario where at least in the US, the things, the assets themselves are almost entirely held by private entities.
• 42:40
  SPEAKER_04: And I think because it wasn't their idea?
• 42:43
  SPEAKER_04: Yeah.
• 42:43
  SPEAKER_04: And I think but I think what's interesting about that, Matt, is that I think that confuses us.
• 42:49
  SPEAKER_04: Right?
• 42:50
  SPEAKER_04: I think we don't know we don't have that precedent.
• 42:53
  SPEAKER_04: So if you sort of like wound the clock back and you thought, oh, you know, it it's a bit like ARPANET or something like that, right?
• 42:58
  SPEAKER_04: And somebody discovers this capability in the military and they go, oh man, this thing is dangerous, right?
• 43:04
  SPEAKER_04: And you've got all the analysts in there who are gonna be like, imagine what this could become, right?
• 43:09
  SPEAKER_04: And we're gonna, this is powerful, right?
• 43:11
  SPEAKER_04: And it could be very strategically advantaged for us.
• 43:13
  SPEAKER_04: So we're gonna develop this, but we're gonna think super carefully about before we let it out to the rest of the world to go develop.
• 43:19
  SPEAKER_04: And then when we do, they're gonna be doing so under the auspices of government contracts, and we're gonna have control over effectively what they do.
• 43:28
  SPEAKER_04: It didn't work out like that.
• 43:29
  SPEAKER_04: It's worked out in this way, and therefore it's by definition, it might be more innovative, but it's much more messy at the same time.
• 43:37
  SPEAKER_04: Yeah, um, so then you start to once you start to think about it that way, I think that's when it sort of makes more sense because then you're sort of like thinking about it and going, okay, well, if I am one of those people, I was like, oh, well, this is not normally how it works.
• 43:48
  SPEAKER_04: How do we go about putting a a governance structure in place whereby I don't know, I was just following orders.
• 43:55
  SPEAKER_04: I developed this in the way that the government would want me to.
• 43:57
  SPEAKER_04: And how do you go about influencing that?
• 43:59
  SPEAKER_04: And especially.
• 44:00
  SPEAKER_04: How do you go about influencing it when the person you want to influence most, namely the president of the United States, tends to respond most to things that are in the public media, right?
• 44:10
  SPEAKER_04: If he sees about it on TV, he's more likely to respond to it in a way that he wouldn't in other ways if somebody presented him with a five-page briefing, right?
• 44:18
  SPEAKER_04: So there's sort of that element of it as well.
• 44:20
  SPEAKER_04: So that's weird.
• 44:22
  SPEAKER_04: Which brings me on to the last of these five categories, which I think is probably the most weird, and we've touched upon it a great deal.
• 44:30
  SPEAKER_04: When you take a step back, it's just strange, isn't it?
• 44:36
  SPEAKER_04: That you've got CEOs of companies that are about to go public saying, yeah, could wipe us all out.
• 44:46
  SPEAKER_04: Right?
• 44:47
  SPEAKER_04: Is there any precedent that you guys can think of for that?
• 44:53
  SPEAKER_04: Because I struggle.
• 44:59
  SPEAKER_03: I I don't think there is.
• 45:01
  SPEAKER_03: No, I mean it's completely counterintuitive to wanting to gain consumers and customers, so I don't know if there is.
• 45:10
  SPEAKER_04: So here's my theory, Bill.
• 45:12
  SPEAKER_04: And I know you're a man who loves like like learning from the ancients.
• 45:16
  SPEAKER_03: Yeah.
• 45:17
  SPEAKER_04: I so I've called this sort of category actually Emperor's New Clothes.
• 45:21
  SPEAKER_04: So what I basically mean by this is maybe we've reached or we're close to like peak capability.
• 45:31
  SPEAKER_04: Or, you know, for the tech people who are still listening at this point, right, you know, it's like Moore's law bottoming out or topping out or whatever you want to call it.
• 45:39
  SPEAKER_04: But right, but it's as in the next model that comes along is gonna be a bit eh, you know, and the one after that is gonna be definitely eh.
• 45:50
  SPEAKER_04: And then at that point, if that were to happen, right, these astronomical valuations are not correct, right?
• 46:03
  SPEAKER_04: So if we have reached, let's say, totally speculatively here, we're speculating like crazy in this podcast, but totally speculatively, let's say that they've realized that there are limitations in the LL in sort of the architecture, inherent limitations in the architecture that mean we're not gonna get much more powerful than we are today.
• 46:28
  SPEAKER_04: There are some researchers that suggest this might be true.
• 46:33
  SPEAKER_04: If it were true, how do you go about uh convincing everybody that it is something that is really super magic when potentially it's not?
• 46:47
  SPEAKER_04: Well, how have we done it historically?
• 46:49
  SPEAKER_04: The way we've done it historically is basically capricious and all powerful gods.
• 46:59
  SPEAKER_04: The ones that will, you know, that's what you do, right?
• 47:02
  SPEAKER_04: You have a this there's this scary capricious god thing that you do not understand that you're supposed to in some way pay homage to, right?
• 47:14
  SPEAKER_04: It's there in the ancients, it's there in the religions, it's also even there in things like the Wizard of Oz, you know, it's like the the the great and powerful Oz, right?
• 47:23
  SPEAKER_04: So it's Jeff, it's actually funny enough, Jeff Moore that got me thinking about this, right?
• 47:28
  SPEAKER_04: It's the powerful, it's a narrative, a tale as old as time.
• 47:32
  SPEAKER_04: If you want to convey that something is bigger and beyond our understanding, throw in the oh, it's really scary, and you do not understand this thing, you know.
• 47:45
  SPEAKER_04: That's a great way to have us think about that in a way that is proven to work.
• 47:50
  SPEAKER_04: I have no idea, but there's gotta be I get I think the heart of this, the heart of all five of them, this is weird.
• 47:58
  SPEAKER_04: And because it's weird, there has to be there must be something behind it.
• 48:05
  SPEAKER_01: Yeah.
• 48:08
  SPEAKER_01: Yeah, you start to wonder now, like, hey, why guys, why did it have to be the absolute most magical, amazing thing that was gonna do everything for us to begin with?
• 48:17
  SPEAKER_01: Why couldn't it have just been a really cool tool?
• 48:20
  SPEAKER_04: Well, a really cool tool doesn't get you trillion dollar valuations.
• 48:23
  SPEAKER_04: That's that's exactly right.
• 48:25
  SPEAKER_01: Yeah.
• 48:25
  SPEAKER_01: But uh, you know, I remember noticing, and I don't remember what year it was, it was within the last couple years, but I remember noticing when Gmail would start writing my Gmail for me using words that I would use.
• 48:39
  SPEAKER_01: And I was like, huh, that's interesting.
• 48:41
  SPEAKER_01: It must oh yeah, it does have all my email that I've ever written.
• 48:45
  SPEAKER_01: That's kind of cool.
• 48:47
  SPEAKER_01: But would I have then subscribed for an additional twenty dollars a month for it to be my AI assistant if I didn't think it could actually change everything?
• 48:57
  SPEAKER_01: So yeah, I mean that's really fascinating.
• 48:59
  SPEAKER_01: It is amazing how we continue to fall for it.
• 49:03
  SPEAKER_03: I think it's wired into us as um god that's yeah, I think it's wired into us as well, Paul.
• 49:08
  SPEAKER_03: And I think the thing I keep orbiting around is mortality.
• 49:13
  SPEAKER_03: Yeah.
• 49:13
  SPEAKER_03: And I think there's just this unending fascination that we as humans have with mortality.
• 49:21
  SPEAKER_03: And whether these luminaries, inventors, people that designed these things intended for us to think and we first saw this technology, we thought about our mortality.
• 49:35
  SPEAKER_03: I mean, you maybe not consciously, but I think many of us did, right?
• 49:38
  SPEAKER_03: Because we were kind of like, well, if it's as big as they say it's gonna be, then it keeps coming back to like, well, then what's the point of me?
• 49:47
  SPEAKER_03: Like, what dent can I make in the universe?
• 49:49
  SPEAKER_03: And so then you start thinking about mortality, and and I think in a way, again to Jeff Moore's narrative point, we've we've gotten in this slipstream of our mortality is associated with these technologies, and they sort of had us by the collar in a fascinating way.
• 50:09
  SPEAKER_03: We're like, oh my god, I can't wait to see what's next.
• 50:11
  SPEAKER_03: I can't wait to see what's next.
• 50:13
  SPEAKER_03: And I wonder, just a question, I wonder if they're at risk of losing our attention.
• 50:18
  SPEAKER_03: And so the only way that you can really hold on to our attention is to once again elevate our blood pressure to the point where we are confronted with our mortality.
• 50:29
  SPEAKER_03: And that's by saying, well, this whole thing could just kill us all.
• 50:32
  SPEAKER_03: So now that I have your attention, um, please go back to reading our release notes.
• 50:37
  SPEAKER_03: I mean, I don't know.
• 50:38
  SPEAKER_04: No, I think there's something there, I really do.
• 50:42
  SPEAKER_04: And it is the related to it.
• 50:46
  SPEAKER_04: I do not know a single person who, when they first typed something into Chat GPT 3.5 and got back what they got back, that they didn't feel like this is magic.
• 51:01
  SPEAKER_04: You know, it felt otherworldly.
• 51:04
  unknown: Yeah.
• 51:04
  SPEAKER_04: Um, the last time I remember something a little bit like that was when I had gone so I'd gone from using Lycos and Alpha Vista and things like that as search engines.
• 51:16
  SPEAKER_04: And, you know, when you threw something into Lycos, you would type a search term, and then a minute and a half later would come back some, you know, crappy list.
• 51:25
  SPEAKER_04: And then I threw something into Google without having upgraded my internet speed, and boom, it was there, and I was like, and it was five times as useful.
• 51:33
  SPEAKER_04: I was like, what kind of voodoo shit is this?
• 51:36
  SPEAKER_04: Right.
• 51:36
  SPEAKER_04: And so this was sort of like the the modern day but way more impressive version of that, and it felt magic.
• 51:44
  SPEAKER_04: And to your point, right, Bill, there is this sort of like it might kill us all.
• 51:51
  SPEAKER_04: There is a there is something otherworldly about that, right?
• 51:56
  SPEAKER_04: And as a result of it being otherworldly, it rekindles that idea that it is magic.
• 52:04
  SPEAKER_04: One thing I would point out, if anybody's we'll put a link to this one as well, and guys, if you haven't heard it, I'd suggest you do.
• 52:11
  SPEAKER_04: But there's so you have Noah Harari, the guy that wrote um Sapiens and Nexus and stuff like that.
• 52:16
  SPEAKER_04: Just heard him on a podcast episode as well.
• 52:20
  SPEAKER_04: For those of you that don't follow him, he's like does a lot of ancient history and all of that kind of stuff, and tries to equate what is happening now with ancient history.
• 52:31
  SPEAKER_04: And in this podcast that we'll uh link to on it, he talks about the most th the most important thing to understand about this is that in a very real sense, it is alien.
• 52:46
  SPEAKER_04: In fact, he would say that it is like if we thought about it as alien intelligence versus artificial intelligence, it wouldn't, to a certain extent, like help us understand our own response to it.
• 52:57
  SPEAKER_04: Right?
• 52:58
  SPEAKER_04: And then he equates it back to various historical events where where people encounter people from other parts of the world that just have entirely different technologies, things that they're not accustomed to, things they don't understand, and their response to it comes from the fact that they think it is alien and they respond very poorly to it because they're encountering something that they themselves do not understand.
• 53:22
  SPEAKER_04: And I think I suspect that if you look across this, even the people that are in it, even the researchers in it, you look at any of the, you know, the godfathers of AI, etc., there's not one of them that doesn't say they're surprised that they're not surprised, right?
• 53:35
  SPEAKER_04: That this has gone in a direction that they didn't anticipate, right?
• 53:38
  SPEAKER_04: And so at whatever level you are at, whether you are somebody who's literally typed, you know, typed into ChatGPT for the first time yesterday to somebody who's been researching this for a long time, this thing is different and it feels different, and it feels kind of alien.
• 53:57
  SPEAKER_04: So I'm I think you're right, Bill.
• 54:00
  SPEAKER_04: I think it's like if we can preserve that thing, then people will that that idea of it being alien, then that feels like something that's worth a lot of money.
• 54:09
  SPEAKER_04: And I that we shouldn't we shouldn't underestimate that.
• 54:13
  SPEAKER_04: So I want to bring it home.
• 54:14
  SPEAKER_04: We're way over.
• 54:16
  SPEAKER_04: And thanks everybody who's still listening.
• 54:19
  SPEAKER_04: Um I think there's one thread throughout all of these things, and I'm curious to what you guys think.
• 54:27
  SPEAKER_04: Um we've touched on it a little bit already, but there's on the one hand, there's weirdness associated with what they're saying and all that kind of stuff.
• 54:39
  SPEAKER_04: There's also a bit of weirdness here that like observers that are looking at this from the outside, like almost no observer is looking at what they're saying and taking it at face value.
• 54:55
  SPEAKER_04: So what do you think?
• 54:57
  SPEAKER_04: I'll start with you, Matt.
• 54:59
  SPEAKER_04: What do you think this says about how well these companies have handled the basics of having uh people trust them?
• 55:16
  SPEAKER_04: Trust what they do, trust that what they say is what they mean.
• 55:23
  SPEAKER_04: How well or poorly do you think they've handled the idea of uh being open and transparent and trust what they incredibly poorly.
• 55:35
  SPEAKER_01: I think the erosion of trust in institutions has been steady and precipitous over the last decade for multiple reasons across all of these things, for us to not trust almost anything we're told.
• 55:49
  SPEAKER_01: It's a real it's a real problem to not trust institutions and experts.
• 55:52
  SPEAKER_01: And you know, obviously it got really bad during COVID for all kinds of things.
• 55:56
  SPEAKER_01: It continues to this day.
• 55:58
  SPEAKER_01: You know, there was a time when if the CEO of GE came out and said something about toasters, everybody would have been like, we gotta get these toasters out of our house right now.
• 56:10
  SPEAKER_01: You know what I mean?
• 56:11
  SPEAKER_01: Where now we'd be like, What did you what's wrong with the toasters?
• 56:15
  SPEAKER_01: My toaster's fine, you know, or you know, like we just cannot get our minds around who to trust.
• 56:22
  SPEAKER_01: And I think that's a fundamental gap in all of it, and you could see a future where we might trust the AI better than the human that's running the AI company.
• 56:35
  SPEAKER_04: Yeah.
• 56:36
  SPEAKER_04: My screenplay, we might we might love them.
• 56:38
  SPEAKER_04: So we're not switching off my AI, say, you know, eight billion people individually.
• 56:47
  SPEAKER_04: Yeah.
• 56:47
  SPEAKER_01: Yeah, and you've seen this.
• 56:48
  SPEAKER_01: There's been examples I've seen where people ask Grok about Elon Musk and it tears them apart.
• 56:54
  SPEAKER_01: And that's his AI, right?
• 56:56
  SPEAKER_01: So it's very, very uh I I think they've done a poor job and and so far haven't really helped much.
• 57:02
  SPEAKER_01: And I think the anthropic CEO especially seems to have done a complete 180, which doesn't help.
• 57:08
  SPEAKER_01: Although you want to admire in a human the ability to pivot and change their mind when evidence presents itself.
• 57:15
  SPEAKER_01: So that requires critical thinking human beings to weigh these things correctly.
• 57:21
  SPEAKER_01: And that also, maybe we've seen isn't as widespread as we might wish it to be.
• 57:26
  SPEAKER_04: What do you think, Bill?
• 57:27
  SPEAKER_04: Are we uh are we being silly here?
• 57:29
  SPEAKER_04: We should just trust everything they say on face value.
• 57:32
  SPEAKER_03: I don't think anybody's being silly to be skeptical.
• 57:35
  SPEAKER_03: I think we have to remember the order of events, right?
• 57:39
  SPEAKER_03: People who wanted to start open AI didn't trust Google.
• 57:43
  SPEAKER_03: Yep.
• 57:43
  SPEAKER_03: People who left OpenAI to start Anthropic didn't trust OpenAI.
• 57:49
  SPEAKER_03: We now have thinking labs and other things, right?
• 57:52
  SPEAKER_03: Each one of these is a variation and an instance of I didn't trust what those were doing with the technology in their hands at the time.
• 58:02
  SPEAKER_03: I think this is accelerated.
• 58:04
  SPEAKER_03: I think we're on an exponential version of that.
• 58:07
  SPEAKER_03: And I think the other thing which is just really messing with our minds is the fact that we have this complete divergence of trust and luminary, trust and leader, trust and person on the cover of a magazine.
• 58:21
  SPEAKER_03: How can we listen to someone who's sitting on a stage, paid for by Salesforce, who tells us you should trust us because we tell you to trust us, when in fact they're being paid to sit on that stage and we know it.
• 58:36
  SPEAKER_03: Like it's just so evident, it's so obvious.
• 58:38
  SPEAKER_03: And then they're gonna be on the cover of a magazine or upheld or receiving an award, and these things are becoming more and more divergent.
• 58:47
  SPEAKER_03: And our bodies and our minds are telling us that gap is insurmountable at this point, and we are left as our own selves to make decisions accordingly.
• 58:55
  SPEAKER_03: But I will just say this: it is lonely.
• 58:58
  SPEAKER_03: I know it, it's lonely, and that's why community is gonna be equally important.
• 59:03
  SPEAKER_03: We might have to start from ground zero and build back up because we've placed too much in the hands of those who say they have a monopoly on community and they don't.
• 59:13
  SPEAKER_04: Well, on that upbeat dystopian.
• 59:17
  SPEAKER_04: No, I don't know.
• 59:18
  SPEAKER_04: We'll call it good for today.
• 59:19
  SPEAKER_04: I know we're gonna come back to a lot of these themes going forward.
• 59:22
  SPEAKER_04: Thank you, gentlemen, and thanks everybody for listening.
• 59:24
  SPEAKER_04: We'll see you on the next one.
• 59:26
  SPEAKER_04: Thanks.
• 59:27
  SPEAKER_00: Humanity Working is produced by Matt Neal.
• 59:30
  SPEAKER_00: If you liked this episode, please take a moment to rate and review us on your favorite podcast platform.
• 59:36
  SPEAKER_00: It makes a huge difference in helping others find us.
• 59:40
  SPEAKER_00: And don't forget, you can find all episodes and get our weekly newsletter at humanityworking.