Latest / Mahon McCann / John Vervaeke & Sam Tideman - Why machines are not intelligent!

Transcript

John Vervaeke & Sam Tideman - Why machines are not intelligent! · Mahon McCann

0:00
SPEAKER_01: Thank you for joining me.
0:01
SPEAKER_01: John and Sam, you don't need any introduction in this corner of the internet at this point, but...
0:06
SPEAKER_01: People will know,
0:07
SPEAKER_01: I suppose,
0:07
SPEAKER_01: if you watch some of my other videos,
0:08
SPEAKER_01: we've been doing a series of conversations really on AI,
0:12
SPEAKER_01: building on John's proposal,
0:14
SPEAKER_01: the conversations John and Sam have had before on AI,
0:17
SPEAKER_01: and kind of a bigger cultural conversation that's going on.
0:21
SPEAKER_01: And so this conversation,
0:22
SPEAKER_01: I'm hoping to kind of push that forward at the intersection of the cognitive
0:26
SPEAKER_01: science and the more technical aspects,
0:28
SPEAKER_01: of course,
0:28
SPEAKER_01: that Sam brings to it.
0:30
SPEAKER_01: So I wanted to open up reading a little piece from Johannes Jaeger and John is a
0:34
SPEAKER_01: co-author and there's a number of co-authors on it,
0:36
SPEAKER_01: but I'll include it in the description on naturalizing relevance realization,
0:41
SPEAKER_01: because I think this really sums up the difference that we're getting at between AI
0:45
SPEAKER_01: and relevance realizing organisms.
0:47
SPEAKER_01: So the abstract is the way organismic agents come to know the world and the way
0:52
SPEAKER_01: algorithms solve problems are fundamentally different.
0:56
SPEAKER_01: The most sensible course of action for an organism does not simply follow from
0:59
SPEAKER_01: logical rules of inference,
1:01
SPEAKER_01: but it can even use such rules.
1:04
SPEAKER_01: The organism must tackle the problem of relevance or before it can even use such rules.
1:08
SPEAKER_01: The organism must tackle the problem of relevance.
1:10
SPEAKER_01: It must turn ill-defined problems into well-defined ones.
1:14
SPEAKER_01: turn semantics into syntax.
1:17
SPEAKER_01: This ability to realize relevance is present in all organisms, from bacteria to humans.
1:22
SPEAKER_01: It lies at the root of organismic agency,
1:24
SPEAKER_01: cognition,
1:25
SPEAKER_01: and consciousness arising from the particular autopoetic,
1:28
SPEAKER_01: anticipatory,
1:29
SPEAKER_01: and adaptive organization of living beings.
1:33
SPEAKER_01: In this paper, we show that the process of relevance realization is beyond formalization.
1:37
SPEAKER_01: This is pretty key for what we're talking about.
1:39
SPEAKER_01: It cannot be captured completely by algorithmic approaches.
1:42
SPEAKER_01: This implies that organismic agency,
1:45
SPEAKER_01: and hence cognition as well as consciousness,
1:47
SPEAKER_01: are at heart not computational in nature.
1:51
SPEAKER_01: So I'll stop there.
1:52
SPEAKER_01: Does this mean that there's a fundamental difference between algorithms that need
1:57
SPEAKER_01: well-defined problem spaces and relevance-realizing agents that can turn
2:04
SPEAKER_01: ill-defined problems into well-defined problems?
2:06
SPEAKER_01: And does this create a chasm between computer science-created problems
2:11
SPEAKER_01: artificial intelligence and biological relevance realizers.
2:16
SPEAKER_00: Yeah,
2:18
SPEAKER_00: I mean,
2:18
SPEAKER_00: there's a lot of complex argument,
2:20
SPEAKER_00: and many people who are hearing this who have a computational background would just
2:24
SPEAKER_00: simply object and say,
2:25
SPEAKER_00: no,
2:25
SPEAKER_00: no.
2:27
SPEAKER_00: And so I don't want to presume,
2:29
SPEAKER_00: but the part of the very carefully laid out argument,
2:34
SPEAKER_00: and we take a lot of time,
2:36
SPEAKER_00: it's not a small paper,
2:39
SPEAKER_00: and Johannes did a tremendous amount of excellent work,
2:43
SPEAKER_00: is to try and show,
2:44
SPEAKER_00: no,
2:45
SPEAKER_00: it builds on the work I did in the 2012 paper,
2:48
SPEAKER_00: some of the work I've done throughout the 2022 paper.
2:51
SPEAKER_00: Basically,
2:52
SPEAKER_00: whenever you try to get something that does relevance realization,
2:56
SPEAKER_00: you ultimately presuppose it in the very process of creating the formal system that
3:01
SPEAKER_00: you think is going to be doing the relevance realization.
3:04
SPEAKER_00: That's the argument in a gist.
3:05
SPEAKER_00: Obviously, that's not a technically valid argument,
3:07
SPEAKER_00: I would ask people to read the paper in detail, but I can't go through the whole paper.
3:12
SPEAKER_00: The point I'm making is the central move is – and it follows on the move I made in the 2012 paper.
3:19
SPEAKER_00: Before categorization, there's relevance realization.
3:21
SPEAKER_00: Before inference, there's relevance realization.
3:22
SPEAKER_00: Before rule following, there's relevance realization.
3:25
SPEAKER_00: But for representation, there's relevance realization, et cetera, et cetera, et cetera.
3:29
SPEAKER_00: And then attempts to try and get this with a formal system,
3:33
SPEAKER_00: you just keep pushing it back and back and back.
3:35
SPEAKER_00: And so the problem of creating a formal system is not the kind of problem that
3:40
SPEAKER_00: itself can be called a well-defined problem and therefore can't be solved by a
3:44
SPEAKER_00: formal system.
3:45
SPEAKER_00: That's the gist of the argument in a nutshell, yes.
3:50
SPEAKER_00: Now,
3:51
SPEAKER_00: I want to be very clear what the title of the paper also indicates,
3:54
SPEAKER_00: Naturalizing Relevance Realization.
3:56
SPEAKER_00: We do not mean by this paper that this leads to any ontological dualisms or
4:02
SPEAKER_00: supernaturalisms or anything like this about the mind.
4:05
SPEAKER_00: We think there is a perfectly naturalistic explanation of relevance realization,
4:09
SPEAKER_00: a scientific explanation.
4:12
SPEAKER_00: The reason why Dennis Walsh is on the paper is because we think the revolution that
4:15
SPEAKER_00: is happening in biology right now is giving us the proper scientific framework
4:22
SPEAKER_00: within which we can give a scientific account a relevance realization,
4:25
SPEAKER_00: but it's ultimately...
4:27
SPEAKER_00: biological account where by biology we don't mean physics in mechanisms that we
4:33
SPEAKER_00: happen to refer to as living we mean the things the autopoetic adaptive
4:38
SPEAKER_00: anticipatory machines the closure of constraints all of this sophisticated work
4:42
SPEAKER_00: that's been done in the philosophy of biology that is not stewart kousman's work in
4:46
SPEAKER_00: uh you know life beyond physics and things like that he says it's it
4:52
SPEAKER_00: life is based on physics, but it's beyond physics, if I could pick up on that.
4:56
SPEAKER_00: That all of this,
4:57
SPEAKER_00: right,
4:58
SPEAKER_00: is consistent with physics,
4:59
SPEAKER_00: but it's not reducible to the explanatory principles or ontology of physics,
5:04
SPEAKER_00: and even less so of the mathematical algorithms that are applied to physics.
5:11
SPEAKER_00: And what you end up with is, it is naturalizable, but the explanation is ultimately biological.
5:17
SPEAKER_00: And this goes to some of the things I was saying with Sam about
5:20
SPEAKER_00: At the core of relevance realization is caring,
5:22
SPEAKER_00: and caring ontologically needs autopoetic adaptive anticipatory agents.
5:28
SPEAKER_00: It needs living beings, or you're not actually doing it.
5:32
SPEAKER_00: You are, in some sense, like you're a shadow of it in some way.
5:39
SPEAKER_00: And so I'll stop.
5:40
SPEAKER_00: That's enough.
5:41
SPEAKER_00: I just want to give the gist of the argument.
5:42
SPEAKER_00: Yeah.
5:45
SPEAKER_01: Yeah, and Sam, is there anything you wanted to chip in there, or do you want me to just go ahead?
5:48
SPEAKER_02: Sure, I'll add in a couple things.
5:50
SPEAKER_02: I really like this article.
5:52
SPEAKER_02: I think I agreed with a lot of it, but I should also say I learned a lot of it, too.
5:57
SPEAKER_02: I wasn't just nodding along being like, yeah, I know that.
6:01
SPEAKER_02: This was a really, I think, powerful article.
6:05
SPEAKER_02: And so I'd highly recommend the audience read it.
6:08
SPEAKER_02: But I think that it really is getting at some of the things that we've talked about before,
6:14
SPEAKER_02: that to me,
6:15
SPEAKER_02: for an AI,
6:17
SPEAKER_02: at least AI,
6:18
SPEAKER_02: as all AIs as we've experienced them so far,
6:22
SPEAKER_02: at sort of their highest level have an objective function that they are seeking to maximize.
6:29
SPEAKER_02: And this function is fundamentally mathematical.
6:33
SPEAKER_02: And we've talked about large language models,
6:36
SPEAKER_02: the way that large language models are trained and built at the moment.
6:40
SPEAKER_02: And I think there's a little bit of a zeitgeist error to think that large language
6:45
SPEAKER_02: models are the end-all be-all of artificial intelligence.
6:47
SPEAKER_02: There are whole fields of artificial intelligence that aren't large language models.
6:51
SPEAKER_02: They're just sort of capturing our attention at the moment, but I think for a good reason.
6:55
SPEAKER_02: But this is true of large language models and AI that wouldn't be classified as a large language model.
7:01
SPEAKER_02: that it has a mathematical function that it is trained to optimize.
7:06
SPEAKER_02: And in the case of an LLM,
7:07
SPEAKER_02: it's predict the next word in the text body,
7:13
SPEAKER_02: given all of the previous words in the text body.
7:16
SPEAKER_02: And if you got it right, one.
7:18
SPEAKER_02: If you didn't get it right, zero.
7:20
SPEAKER_02: Here's a huge body of text to train yourself with.
7:24
SPEAKER_02: Maximize the number of ones.
7:25
SPEAKER_02: Minimize the number of zeros.
7:27
SPEAKER_02: You have hundreds of billions of parameters to optimize this with.
7:31
SPEAKER_02: Go.
7:32
SPEAKER_02: And it is a fundamentally mathematical thing.
7:36
SPEAKER_02: And that this is, I think, what John and this paper means by algorithm, right?
7:40
SPEAKER_02: That there is a closed system and it is giving you your optimization function for you.
7:48
SPEAKER_02: And there is no sense that there would be something that is left uncaptured by this
7:53
SPEAKER_02: algorithm within the system that is trying to optimize.
7:56
SPEAKER_02: but that life doesn't get to operate that way.
8:00
SPEAKER_02: And it's kind of my suspicion.
8:02
SPEAKER_02: I'm not, it's my conjecture, I'll say.
8:05
SPEAKER_02: I couldn't,
8:06
SPEAKER_02: I don't have like some formal knockdown proof for this,
8:09
SPEAKER_02: that purpose in the highest sense,
8:12
SPEAKER_02: like when we're thinking about organisms that are seeking after goals,
8:16
SPEAKER_02: I think seeking after goals is,
8:18
SPEAKER_02: I view that as very analogous to trying to achieve a purpose,
8:22
SPEAKER_02: that purpose itself is not mathematically capturable.
8:26
SPEAKER_02: And that there will never be a function that fully captures purpose in the highest sense.
8:35
SPEAKER_02: And so therefore, a mathematical function can only be approximating a purpose.
8:40
SPEAKER_02: And you can get functions maybe that are better or worse at approximating that purpose.
8:45
SPEAKER_02: But if AI is captured in the realm of being subservient to a mathematical function,
8:51
SPEAKER_02: then there is some fundamental limit that it has that life doesn't have when trying
8:58
SPEAKER_02: to adapt itself to being good at achieving a purpose.
9:02
SPEAKER_00: Could I build on that?
9:05
SPEAKER_00: I don't object to the language Sam uses.
9:08
SPEAKER_00: We have some differences about the fundamentality of teleology,
9:12
SPEAKER_00: but applied to biological organisms,
9:14
SPEAKER_00: I have no controversy,
9:17
SPEAKER_00: nothing at all,
9:18
SPEAKER_00: no disagreement at all.
9:20
SPEAKER_00: I think that's right.
9:21
SPEAKER_00: I would want to add,
9:22
SPEAKER_00: because when people hear the word purpose,
9:23
SPEAKER_00: they can think of like the purpose of a tractor or something like that.
9:27
SPEAKER_00: And
9:28
SPEAKER_00: And that is purpose by attribution.
9:31
SPEAKER_00: And I think Sam will agree with this.
9:34
SPEAKER_00: When we're talking about when an organism acts on purpose, the purpose is for it.
9:39
SPEAKER_00: right?
9:39
SPEAKER_00: It's for it, not for us, right?
9:43
SPEAKER_00: And that is a very crucial thing.
9:45
SPEAKER_00: It in no way matters to these systems,
9:49
SPEAKER_00: and almost literally doesn't matter to them,
9:52
SPEAKER_00: if they don't achieve some goal that their function is trying to optimize for.
9:57
SPEAKER_00: It makes no difference to them.
9:59
SPEAKER_00: They don't care.
10:00
SPEAKER_00: It doesn't go into the constitution of their existence.
10:03
SPEAKER_00: It doesn't satisfy their seeking.
10:06
SPEAKER_00: They get no satisfaction from it, which means
10:09
SPEAKER_00: Right.
10:09
SPEAKER_00: Right.
10:21
SPEAKER_00: because an error means that you're trying to do something right or good.
10:25
SPEAKER_00: And that means you're pursuing satisfaction.
10:28
SPEAKER_00: You're doing things that matter to you.
10:30
SPEAKER_00: All of this is absent.
10:31
SPEAKER_00: So when we talk about purpose,
10:33
SPEAKER_00: we're not just talking about like,
10:35
SPEAKER_00: you know,
10:35
SPEAKER_00: the end state of some function being realized.
10:38
SPEAKER_00: It's for the organism and it's normatively guided.
10:41
SPEAKER_00: It's normatively governed in an important way.
10:44
SPEAKER_00: The good, the true, the good, and the beautiful matter to the evaluation.
10:48
SPEAKER_00: Your performance, Mahat, I can evaluate it according to the true, the good, and the beautiful, right?
10:55
SPEAKER_00: In a very deep sense, you can't do this with these machines.
10:58
SPEAKER_00: And that means they're not rational or intelligent.
11:01
SPEAKER_00: And that shows up in like really fundamental ways.
11:03
SPEAKER_00: So you get these machines and you give them all kinds of Raven's progressive
11:07
SPEAKER_00: matrices and you get all the published stuff about.
11:10
SPEAKER_00: And look, they score, you know, in the top percentile.
11:13
SPEAKER_00: They're as good as any human being.
11:15
SPEAKER_00: And then what you do is you get a human being to just come up with a brand new
11:17
SPEAKER_00: version of the Raven's progressive matrices that uses color.
11:22
SPEAKER_00: that hasn't been done in any published one,
11:23
SPEAKER_00: you give it to some machines and they absolutely crap out and you give it to a
11:27
SPEAKER_00: human being and they do as well as they do on the black and white versions of the
11:31
SPEAKER_00: Raven's progressive matrix.
11:33
SPEAKER_00: Like that's the kind of stuff we're talking about here, right?
11:36
SPEAKER_00: As soon as you introduce a new dimension that hasn't been coded into the data,
11:41
SPEAKER_00: these machines are,
11:43
SPEAKER_00: and this is an old point made by Fodor,
11:45
SPEAKER_00: right?
11:45
SPEAKER_00: These machines don't generalize in the way that would mean they actually have general intelligence.
11:52
SPEAKER_02: Right.
11:52
SPEAKER_02: And I think that a big part of the inability for these machines to generalize is a lack of comprehension.
12:00
SPEAKER_02: And I think that this is something that I know that comprehension is one of the
12:06
SPEAKER_02: what exactly is comprehension is debated.
12:09
SPEAKER_02: And I'm curious to hear more of John's thoughts on that.
12:13
SPEAKER_02: I mean,
12:13
SPEAKER_02: I should say I as a Google employee,
12:17
SPEAKER_02: I'm going to have to tread a little bit carefully in terms of some of the things I say.
12:20
SPEAKER_02: But Google was Google's AI was well known a couple of months ago for someone asked it,
12:26
SPEAKER_02: hey,
12:26
SPEAKER_02: could you help me figure out a way to have my cheese stay on my pizza better?
12:33
SPEAKER_02: I make pizza, but the cheese keeps falling off my pizza.
12:35
SPEAKER_02: What should I do?
12:37
SPEAKER_02: And then the Google AI said,
12:38
SPEAKER_02: well,
12:38
SPEAKER_02: you should put glue on the cheese to help it stay attached to your pizza.
12:45
SPEAKER_02: And this is like, of course, it's humorous.
12:48
SPEAKER_02: So they did a big deep dive to figure out why did this happen?
12:51
SPEAKER_02: Why did the AI tell it to put glue on your pizza?
12:54
SPEAKER_02: And it turned out that there was a Reddit thread that was in the training data of
12:58
SPEAKER_02: the AI,
12:59
SPEAKER_02: where someone had posted a very similar question,
13:01
SPEAKER_02: how can I help the cheese stay on my pizza?
13:03
SPEAKER_02: And the top voted answer was, oh, you should glue it to it.
13:07
SPEAKER_02: And it was the top voted answer, because you guys already know, it's hilarious, right?
13:12
SPEAKER_02: It's funny.
13:14
SPEAKER_02: But a human can comprehend
13:17
SPEAKER_02: That gluing your pizza or gluing cheese to your pizza is funny because like in a
13:22
SPEAKER_02: certain sense,
13:23
SPEAKER_02: it answers the question,
13:24
SPEAKER_02: but it violates an unstated assumption of please keep my pizza edible.
13:29
SPEAKER_02: And and that sort of ability to comprehend.
13:34
SPEAKER_02: that could distinguish between,
13:35
SPEAKER_02: say,
13:36
SPEAKER_02: a joke and a not joke,
13:38
SPEAKER_02: or understand the subtext of an answer without,
13:41
SPEAKER_02: you know,
13:41
SPEAKER_02: understanding all of the stated things in the sentence.
13:45
SPEAKER_02: That comprehension is, I think, what AI lacks.
13:49
SPEAKER_02: And therefore,
13:50
SPEAKER_02: it's only ever like whether it's an LLM or something else doing a statistical
13:55
SPEAKER_02: approximation of imitation of the training data that was given to it.
13:59
SPEAKER_02: And then as soon as you're stretching out beyond that, then it will really start to struggle.
14:05
SPEAKER_00: Well, I want to say two things.
14:06
SPEAKER_00: I want to say what I think is going on in comprehension, but I want to first say we have to.
14:11
SPEAKER_00: acknowledge this imitation can be very good because human beings have spent tens of
14:17
SPEAKER_00: thousands of years coding their epistemic judgments into probability relations
14:22
SPEAKER_00: between terms.
14:24
SPEAKER_00: And then not only diachronically,
14:25
SPEAKER_00: synchronically we have,
14:27
SPEAKER_00: you know,
14:28
SPEAKER_00: hundreds of thousands of texts doing this.
14:31
SPEAKER_00: And so you get a massive correlation built up and the machine therefore can do very impressive things.
14:38
SPEAKER_00: What I would say about the comprehension going back to our paper is I think
14:41
SPEAKER_00: comprehension is about the relevance realization that gives you the pragmatics that
14:46
SPEAKER_00: allows you to translate the pragmatics into semantics,
14:49
SPEAKER_00: which allows you to translate the semantics into the syntax.
14:53
SPEAKER_00: And I think that's fully what is going on in comprehension.
14:57
SPEAKER_00: And we didn't even,
14:59
SPEAKER_00: we basically touched about the pragmatics,
15:02
SPEAKER_00: but we didn't put that into the abstract of the paper.
15:06
SPEAKER_01: yeah i want to jump in there because i mean for me reading that and then yeah
15:10
SPEAKER_01: hearing the follow-up is do you think this is a fundamental limit like in terms of
15:14
SPEAKER_01: the you know they don't have inherent goals and needs that constrain the
15:21
SPEAKER_01: probability space to constrain what what is considered a problem and what is
15:25
SPEAKER_01: considered a solution and so
15:26
SPEAKER_01: we're always programming into them some purpose or some goal.
15:30
SPEAKER_01: And I know John, part of your proposal is that they should be autonomous in a sense.
15:36
SPEAKER_01: And I'm kind of wondering how does that, I want to be careful here.
15:42
SPEAKER_01: Yeah.
15:42
SPEAKER_01: Sorry.
15:42
SPEAKER_00: Yeah.
15:43
SPEAKER_00: Well, autonomous in a deep sense of autonomous where they generate norms that they bind themselves to.
15:48
SPEAKER_00: So they would have to be deeply autopoietic, self-making and adaptive and anticipatory.
15:54
SPEAKER_00: They would have to be, uh, uh,
15:56
SPEAKER_00: My thesis,
15:57
SPEAKER_00: this is oversimple,
15:59
SPEAKER_00: but it's basically the thesis in a nutshell,
16:02
SPEAKER_00: and it's the thesis of the paper,
16:03
SPEAKER_00: no artificial intelligence without artificial life.
16:06
SPEAKER_00: Where artificial doesn't mean fake.
16:08
SPEAKER_00: That's why sometimes say we mean artifactual.
16:12
SPEAKER_00: And so if this is right, and this is maybe where Sam and I might disagree, I don't know.
16:19
SPEAKER_00: If the account that this is still a naturalistic phenomena is correct,
16:22
SPEAKER_00: it doesn't mean that in principle we couldn't make some sophisticated artifact that
16:29
SPEAKER_00: could demonstrate anticipatory autonomous behavior.
16:34
SPEAKER_00: autopoetic anticipatory agency.
16:38
SPEAKER_00: In principle, there's no reason why.
16:39
SPEAKER_00: It's just that these machines are not governed by any of those in principles,
16:44
SPEAKER_00: and therefore they,
16:45
SPEAKER_00: in principle,
16:45
SPEAKER_00: will fail.
16:46
SPEAKER_00: Again, we could do this.
16:48
SPEAKER_00: I mean, again,
16:52
SPEAKER_00: If the argument of the paper is correct,
16:53
SPEAKER_00: the argument we're using right now,
16:56
SPEAKER_00: and if Sam wants to disagree,
16:58
SPEAKER_00: I'm happy to hear that.
16:59
SPEAKER_00: But if the argument is right, there's no reason in principle by why this can't be done.
17:03
SPEAKER_00: The point is these are not those kind of entities.
17:06
SPEAKER_00: That's the point.
17:09
SPEAKER_00: Now,
17:09
SPEAKER_00: The point we make in the book,
17:11
SPEAKER_00: Sean and I,
17:12
SPEAKER_00: on mentoring machines,
17:13
SPEAKER_00: or in the point I made in the video essays,
17:15
SPEAKER_00: if we choose to go down that path road,
17:18
SPEAKER_00: we're choosing something very,
17:19
SPEAKER_00: very different than what we have right now.
17:21
SPEAKER_00: And that's a choice point.
17:22
SPEAKER_00: And it has all kinds of other different implications.
17:26
SPEAKER_00: And Sam made excellent points about this, the energy consumption of these beings.
17:31
SPEAKER_00: The physics has to be fundamentally different in these machines than us.
17:34
SPEAKER_00: I use something like, what is it, a 60-watt light bulb of energy, and I do what I do.
17:40
SPEAKER_00: And these machines are taking the energy from a city for two weeks.
17:43
SPEAKER_00: The physics are not the same.
17:47
SPEAKER_02: That number has only gotten bigger since the last time we talked.
17:51
SPEAKER_01: Yeah, it's only going up.
17:53
SPEAKER_01: And do you think, John, could they do relevance realization?
17:56
SPEAKER_01: Because I guess what I took from that abstract was that there's a difference
18:00
SPEAKER_01: between the well-defined algorithms and relevance realizing agents.
18:04
SPEAKER_01: So do you think we could create an artifact that could realize relevance?
18:10
SPEAKER_00: So this goes towards this question.
18:12
SPEAKER_00: Can we create artifactual life?
18:15
SPEAKER_00: Can we create biological life that did not evolve from other already existing biological life forms?
18:21
SPEAKER_00: Yeah.
18:22
SPEAKER_00: And, um, and this is where I know people will differ with me for religious reasons.
18:27
SPEAKER_00: And,
18:27
SPEAKER_00: and,
18:27
SPEAKER_00: and I don't want to,
18:28
SPEAKER_00: I don't want to be just dismissive of that,
18:30
SPEAKER_00: but right now we're playing in the science arena.
18:32
SPEAKER_00: So within the science arena,
18:34
SPEAKER_00: I'm,
18:35
SPEAKER_00: I happen to think that life arose in a naturalistic fashion and therefore there is
18:40
SPEAKER_00: no reason in principle why we couldn't come up with the right set of conditions,
18:46
SPEAKER_00: um,
18:46
SPEAKER_00: that could produce,
18:48
SPEAKER_00: um,
18:49
SPEAKER_00: artifactual biology that supports,
18:51
SPEAKER_00: uh,
18:51
SPEAKER_00: that supports cognition.
18:53
SPEAKER_00: And there are lots of people doing already scientifically interesting work on this.
18:58
SPEAKER_00: Michael Levin is working down from the biology into the mechanics.
19:02
SPEAKER_00: I have a lot of people that I know that are working up from autocatalytic systems
19:06
SPEAKER_00: that can start to do computational.
19:09
SPEAKER_00: Notice what I'm doing.
19:10
SPEAKER_00: That word doesn't mean anything anymore.
19:12
SPEAKER_00: It just means cognition-like.
19:13
SPEAKER_00: right.
19:14
SPEAKER_00: And in non, right.
19:15
SPEAKER_00: Or doing, I would say cognitive type things already.
19:18
SPEAKER_00: Um, so there's a lot of work going on this.
19:21
SPEAKER_00: This is not something I'm just randomly science fiction speculating.
19:25
SPEAKER_00: There's people working from both ends.
19:27
SPEAKER_00: If we think that there is a naturalistic origin to life,
19:30
SPEAKER_00: then there is no reason in principle why we couldn't do it.
19:35
SPEAKER_00: But that is not how these machines are made.
19:38
SPEAKER_00: These are not the kinds of machines they are.
19:40
SPEAKER_00: Um,
19:41
SPEAKER_00: And choosing to do that other thing,
19:44
SPEAKER_00: that's why I talk about thresholds more than...
19:46
SPEAKER_00: The only prediction I made was that all the predictions would turn out to be false
19:49
SPEAKER_00: in a year,
19:50
SPEAKER_00: and they were,
19:51
SPEAKER_00: which is yay as a scientist.
19:55
SPEAKER_00: So we don't talk about predictions.
19:56
SPEAKER_00: We talk about thresholds.
19:59
SPEAKER_00: We have to choose if we want to continue that project, and we have to then choose once we get...
20:05
SPEAKER_00: biological things do we want to do we want to develop them into rational things
20:10
SPEAKER_00: that's another threshold do we want to make them sociocultural things that's
20:14
SPEAKER_00: another threshold and i talk about it like thresholds and but and they would be
20:19
SPEAKER_00: like stages but they're interweaved interwoven because being biological intelligent
20:24
SPEAKER_00: is important to being rational and being rational is important to being
20:27
SPEAKER_00: sociocultural and being sociocultural is important to being rational etc um so
20:33
SPEAKER_00: I'm sorry, your question is a very hard question to answer simply.
20:37
SPEAKER_00: The answer is yes, but it's a very different yes than what most people are thinking.
20:42
SPEAKER_01: That's really what I'm getting is that it almost, I wouldn't classify it as making something.
20:46
SPEAKER_01: It's more like growing.
20:47
SPEAKER_01: It seems to be so different from the top-down current computer science programming
20:53
SPEAKER_01: that's going into AI as to be a totally different field.
20:58
SPEAKER_01: It's really not the way the current...
21:00
SPEAKER_01: zeitgeist is going do you think that's fair it is and then i'll stop so because i
21:04
SPEAKER_00: want to hear what sam has to say but yeah right one of the really important things
21:08
SPEAKER_00: about these machines is in a very real i mean we we set in the original programming
21:14
SPEAKER_00: to set up the the network but the network itself is not programmed see one of the
21:19
SPEAKER_00: big breakthroughs was to realize don't program these machines program them so they
21:24
SPEAKER_00: can learn
21:26
SPEAKER_00: So there's a sense in which they're,
21:30
SPEAKER_00: and I don't mean this in a biological sense,
21:32
SPEAKER_00: but they are self-developing,
21:33
SPEAKER_00: right?
21:35
SPEAKER_00: And I think by analogy,
21:38
SPEAKER_00: and it's only by analogy,
21:40
SPEAKER_00: if we're trying to top down cognitive agency,
21:45
SPEAKER_00: we won't make it.
21:47
SPEAKER_00: But if we do something analogous where we, can we make the conditions whereby it emerges
21:55
SPEAKER_00: like you said, grows, I think that's the move we'll have to make.
22:01
SPEAKER_00: And like I say, because of my particular framework, I don't see why this is in principle impossible.
22:10
SPEAKER_00: And like I said,
22:11
SPEAKER_00: there are already well-developed,
22:13
SPEAKER_00: well-progressing,
22:14
SPEAKER_00: successfully publishing research programs going from both directions towards the
22:19
SPEAKER_00: middle on this.
22:24
SPEAKER_02: So I do think that there are some AI versions of relevance realization,
22:31
SPEAKER_02: but that it's probably not quite the full kit and caboodle.
22:35
SPEAKER_02: Touching on something that John said earlier,
22:38
SPEAKER_02: you said a tractor doesn't have autopoiesis,
22:42
SPEAKER_02: but a farmer might sort of lend his autopoiesis to a tractor.
22:48
SPEAKER_02: The tractor is almost like an extended part of his body.
22:52
SPEAKER_02: And then he's like, oh, my tractor needs an oil change.
22:55
SPEAKER_02: I need new tires.
22:56
SPEAKER_02: I need a new part for the whatever majigger.
22:59
SPEAKER_02: Right.
22:59
SPEAKER_02: And so then the farmer can use his capacity of relevance realization and
23:05
SPEAKER_02: autopoiesis sort of imaginably extend it to a tractor.
23:09
SPEAKER_02: And then all of a sudden the tractor sort of has autopoiesis of a sense.
23:14
SPEAKER_02: By proxy.
23:15
SPEAKER_02: By proxy.
23:16
SPEAKER_02: Yeah.
23:17
SPEAKER_02: And humans have seemingly this pretty amazing gift to incorporate and extend our
23:24
SPEAKER_02: own natural relevance realization.
23:27
SPEAKER_02: I think I'm going to use the farm analogy a little bit longer because I think that
23:32
SPEAKER_02: will be helpful for understanding AI.
23:35
SPEAKER_02: Is that a farmer,
23:36
SPEAKER_02: he cares about his barn,
23:37
SPEAKER_02: he cares about his tractor,
23:39
SPEAKER_02: he cares about his sheep,
23:40
SPEAKER_02: he cares about his plants.
23:42
SPEAKER_02: And like the sheep have a little bit of their own natural relevance realization.
23:47
SPEAKER_02: But the farmer is kind of also doing some stuff on top.
23:51
SPEAKER_02: So like the sheep are like eating their own food and going to the bathroom and other sheep things.
23:56
SPEAKER_02: But the farmer is like keeping predators away and making sure that there is food in
24:00
SPEAKER_02: their trough and stuff like that.
24:02
SPEAKER_02: And so there's some interplay.
24:04
SPEAKER_02: between the farmer and the sheep.
24:06
SPEAKER_02: And sometimes you might even imagine the farmer and the sheep are at odds,
24:09
SPEAKER_02: like when it's time for slaughter or something like that.
24:13
SPEAKER_02: The farmer's auto-polices might trump the sheep's auto-polices at a given point in time.
24:19
SPEAKER_02: And I think that artificial intelligence will be something like a domesticated
24:23
SPEAKER_02: animal or a pet in a certain way as it's growing up,
24:28
SPEAKER_02: that it has a little bit of artificial relevance realization and stuff like that.
24:34
SPEAKER_02: But it's still kind of under humanity's larger protection and that it would be helpless without humanity.
24:44
SPEAKER_02: but that it might have a little bit of kind of independence and growing amounts of that over time.
24:51
SPEAKER_02: Anyway, I'll stop there.
24:54
SPEAKER_02: So I think that it might be helpful to think of artificial intelligence as
24:59
SPEAKER_02: something like a domesticated animal or a pet in terms of its sort of autopoiesis.
25:06
SPEAKER_00: I maybe,
25:07
SPEAKER_00: I mean,
25:07
SPEAKER_00: uh,
25:08
SPEAKER_00: I,
25:08
SPEAKER_00: I,
25:09
SPEAKER_00: I like the idea of,
25:10
SPEAKER_00: you know,
25:10
SPEAKER_00: uh,
25:11
SPEAKER_00: by proxy and the idea of imaginably indwelling.
25:14
SPEAKER_00: And we definitely do that.
25:15
SPEAKER_00: Uh, Dan Chapman, I wrote a bunch of papers on how the scientists do that with the rovers on Mars.
25:20
SPEAKER_00: Um,
25:21
SPEAKER_00: I don't think these machines, I think they do, I think they have some maybe dimensions.
25:26
SPEAKER_00: Maybe this is what you're talking about.
25:27
SPEAKER_00: They have some dimensions of relevance realization.
25:30
SPEAKER_00: They're doing compression particularization and deep learning.
25:33
SPEAKER_00: They're doing, they're doing some of it, but I think it's,
25:37
SPEAKER_00: I might say that they're doing some things that are necessary to relevance realization,
25:42
SPEAKER_00: but very,
25:42
SPEAKER_00: very far from sufficient.
25:44
SPEAKER_00: And I think the sufficiency can't come in unless they are full-blown agents.
25:49
SPEAKER_00: And see, that's where the analogy breaks down.
25:51
SPEAKER_00: The sheep is an adaptive anticipatory agent.
25:55
SPEAKER_00: And I mean agent in a really philosophically robust sense, and these machines aren't.
26:02
SPEAKER_00: That's what's weird about them.
26:03
SPEAKER_00: They're not just tools,
26:05
SPEAKER_00: and they're not just agents,
26:07
SPEAKER_00: and they're sort of intercategorical in this really...
26:10
SPEAKER_00: difficult way for us as a culture to deal with because people oh they're just tools
26:14
SPEAKER_00: that's not quite right oh well they're agents that's not right either they're sort
26:18
SPEAKER_00: of horrific in the sense that they're in between these two categories in a really
26:23
SPEAKER_00: really murky fashion between a tractor and a sheep yeah yeah exactly yeah i thought
26:29
SPEAKER_01: you were talking about the sheep there for a second john real horror to society but
26:36
SPEAKER_01: because yeah that seems to be the oh sorry uh
26:40
SPEAKER_01: That seems to be part of the confusion.
26:41
SPEAKER_01: Is it a tool?
26:43
SPEAKER_01: Is it a being?
26:44
SPEAKER_01: And some people seem to be building the being.
26:46
SPEAKER_01: Some people seem to be building the tool.
26:49
SPEAKER_01: But I guess I thought if the earlier argument is correct, that they
26:52
SPEAKER_01: the current model of AI can't be beings, like relevance-realizing beings.
26:56
SPEAKER_00: They can't be agents.
26:57
SPEAKER_00: I'll use the term agent.
26:58
SPEAKER_00: Can't be agents.
26:59
SPEAKER_00: They can't be agents, but they're not just tools.
27:03
SPEAKER_00: And this is what I'm giving to Sam's point, because I think he's making a good point.
27:07
SPEAKER_00: They have a kind of, how can I put this, non-biological technical autonomy.
27:13
SPEAKER_00: They do things unexpected to us, beyond our expectations, that can then
27:20
SPEAKER_00: impact on us.
27:22
SPEAKER_00: And that is unlike other tools.
27:26
SPEAKER_00: Tools do that unexpected, but that's when they break down.
27:30
SPEAKER_00: They're not breaking down.
27:31
SPEAKER_00: They're excelling.
27:33
SPEAKER_00: They're excelling beyond our expectation, our designs.
27:36
SPEAKER_00: And so they're not just tools in that sense,
27:39
SPEAKER_00: but they're not agents for all the reasons we've been discussing.
27:42
SPEAKER_00: And I think that's what's very, very problematic.
27:45
SPEAKER_00: People, they are
27:47
SPEAKER_00: Oversimplistic in that they try to drop these entities into these existing
27:53
SPEAKER_00: categories too easily,
27:55
SPEAKER_00: and they don't fit comfortably in either one.
27:57
SPEAKER_00: That would be my more sort of sociological take on this.
28:04
SPEAKER_02: One thing that the paper kind of helped me think about when I was talking about
28:08
SPEAKER_02: sort of the,
28:10
SPEAKER_02: I don't know,
28:12
SPEAKER_02: the biological organization natural agency,
28:16
SPEAKER_02: that there are some statistical techniques that seem a lot like a form of relevance realization,
28:22
SPEAKER_02: especially feature selection in a model.
28:25
SPEAKER_02: If I'm building a model to forecast something,
28:28
SPEAKER_02: I'm like,
28:29
SPEAKER_02: here are a thousand possible variables model that you could use that I have already
28:35
SPEAKER_02: cleaned and organized and found for you.
28:39
SPEAKER_02: But you probably don't need all 1000 of these.
28:42
SPEAKER_02: Probably most of them are useless.
28:44
SPEAKER_02: turn these 1,000 variables into a model that's actually useful and there's various
28:50
SPEAKER_02: statistical techniques and it might find,
28:52
SPEAKER_02: okay,
28:52
SPEAKER_02: here are the 25 variables that are actually useful in my model according to,
28:58
SPEAKER_02: and it's a very strict mathematical algorithm that does that.
29:02
SPEAKER_02: But one thing that I was thinking about is that those statistical algorithms often
29:06
SPEAKER_02: figuring out,
29:07
SPEAKER_02: here are all the things I already have at my disposal,
29:10
SPEAKER_02: which ones should I then focus on?
29:12
SPEAKER_02: But life often goes in the opposite direction.
29:15
SPEAKER_02: It's not like life starts out by being like, all right, here's all the information.
29:20
SPEAKER_02: Now let me figure out which information I care about.
29:22
SPEAKER_02: It's like maybe a new sense will sort of slowly evolve and then get incorporated if
29:29
SPEAKER_02: it is newly providing useful information.
29:31
SPEAKER_02: So instead of starting with everything and whittling down, it's like experimentally adding information.
29:37
SPEAKER_02: And only adding and keeping that which proves useful to its purpose.
29:43
SPEAKER_02: And it's a bottom-up additive thing as opposed to sort of this kind of arrogant
29:49
SPEAKER_02: idea of here's all the possible information,
29:51
SPEAKER_02: now whittle out.
29:52
SPEAKER_02: Those are very kind of opposite from each other.
29:55
SPEAKER_00: That's excellent.
29:56
SPEAKER_00: And I was talking to Evan Thompson about this.
29:58
SPEAKER_00: It's in his chapter.
29:59
SPEAKER_00: And they make use of Stuart Kaufman's idea about this.
30:02
SPEAKER_00: You can't set the initial conditions and run your algorithm and get life and how
30:07
SPEAKER_00: life evolves because of what Sam was just saying.
30:09
SPEAKER_00: Life evolves.
30:10
SPEAKER_00: It doesn't deduce.
30:12
SPEAKER_00: I mean, in principle...
30:13
SPEAKER_00: given a human being or maybe a set of human beings enough time,
30:17
SPEAKER_00: they could do what that algorithm does about the feature selection thing,
30:21
SPEAKER_00: because it follows mathematically.
30:24
SPEAKER_00: It's deducible.
30:25
SPEAKER_00: And in the same way,
30:26
SPEAKER_00: even for a physical system,
30:27
SPEAKER_00: if I set the initial conditions and I,
30:29
SPEAKER_00: that's what you want to do with Newton,
30:30
SPEAKER_00: right?
30:30
SPEAKER_00: And then what I can do is I can lay out in a deterministic fashion, all, all the outcomes.
30:35
SPEAKER_00: You can't do that with life because what Sam just said, life evolves, it explores and it exacts.
30:42
SPEAKER_00: So it'll,
30:44
SPEAKER_00: well, you know, the tongue evolved for this purpose.
30:46
SPEAKER_00: It has all of these useless side effects.
30:49
SPEAKER_00: Oh, wait, that useless side effect is speech and bang.
30:53
SPEAKER_00: And you get a whole new thing opens up.
30:55
SPEAKER_00: And because of that, you can't, you can't lay it.
30:59
SPEAKER_00: I agree, Sam, totally.
31:00
SPEAKER_00: And I think you're,
31:00
SPEAKER_00: you're picking up on an argument that Evan and develops really well on the blind
31:05
SPEAKER_00: spot with his coauthors and that,
31:07
SPEAKER_00: and that Stuart Kaufman develops really well too.
31:11
SPEAKER_00: And so I think that's fundamental.
31:14
SPEAKER_00: Now I happen to think, and this is maybe where the argument could maybe be helpful.
31:21
SPEAKER_00: And I know Sam's familiar with this and even going back to the 2012 paper,
31:25
SPEAKER_00: I happen to think that the principles of relevance realization are,
31:30
SPEAKER_00: like of the same genus as the principles of biological evolution.
31:35
SPEAKER_00: And we've got increasing evidence for exaptation and cognition.
31:39
SPEAKER_00: And so basically relevance realization is a kind of cognitive evolution, exploration, exaptation process.
31:53
SPEAKER_02: Mm-hmm.
31:54
SPEAKER_02: And I think sort of building off and kind of tying together a couple of thoughts
32:00
SPEAKER_02: that we've had,
32:01
SPEAKER_02: I think that really one of AI's biggest problems is that it's still,
32:06
SPEAKER_02: we are looking for its purpose,
32:09
SPEAKER_02: it's looking for its purpose,
32:11
SPEAKER_02: and we don't seem to have quite a goal for it yet.
32:15
SPEAKER_02: Yeah.
32:16
SPEAKER_02: We kind of thought that once you give something intelligence, then it like just knows its purpose.
32:21
SPEAKER_02: And it could be the case that even intelligence itself,
32:25
SPEAKER_02: or at least like,
32:26
SPEAKER_02: and I even think intelligence isn't quite the right word yet for these things that
32:31
SPEAKER_02: we have.
32:32
SPEAKER_02: They're sort of like sub functions of intelligence,
32:34
SPEAKER_02: like calculation or,
32:37
SPEAKER_02: you know,
32:38
SPEAKER_02: a couple of sub functions,
32:39
SPEAKER_02: not quite the whole kit and caboodle of what we mean by intelligence.
32:42
SPEAKER_00: But certainly not general intelligence.
32:44
SPEAKER_00: It's certainly not that.
32:45
SPEAKER_02: Yeah.
32:45
SPEAKER_02: So these sub functions of intelligence or intelligence like things, they don't have purpose in itself.
32:53
SPEAKER_02: Like a calculator itself doesn't have purpose until there's someone who has need for calculations.
33:01
SPEAKER_02: And I do calculations for my own autopoetic needs.
33:05
SPEAKER_02: A calculator itself isn't typing in numbers and doing math for kicks and giggles, right?
33:09
SPEAKER_02: It's waiting for someone to tell it what to do.
33:12
SPEAKER_02: And in the same way these things are being told what to do,
33:15
SPEAKER_02: intelligence itself might be kind of purposeless unless it's a tool within a larger
33:22
SPEAKER_02: organism that has a purpose.
33:24
SPEAKER_02: Okay.
33:24
SPEAKER_00: I would go stronger.
33:25
SPEAKER_00: I would say that if you don't,
33:27
SPEAKER_00: I think the ability to seek out purpose is a precondition for intelligence.
33:34
SPEAKER_00: I think it's a constitutive thing to it.
33:36
SPEAKER_00: And again,
33:37
SPEAKER_00: if we go to the stronger normative reading of purpose,
33:43
SPEAKER_00: these machines don't seek out the true,
33:45
SPEAKER_00: the good,
33:45
SPEAKER_00: and the beautiful.
33:47
SPEAKER_00: They don't at all.
33:49
SPEAKER_00: And part of the argument we're making in the well,
33:51
SPEAKER_00: at least one of the implied arguments from the paper is until you seek out relevance,
33:56
SPEAKER_00: you can't seek out the true,
33:57
SPEAKER_00: the good and the beautiful.
33:58
SPEAKER_00: And therefore, there's no normative binding.
34:01
SPEAKER_00: And so there's no real purpose in these these these machines again.
34:08
SPEAKER_00: And.
34:12
SPEAKER_00: That's masked by two things.
34:13
SPEAKER_00: It's masked by the fact that very often these machines are put into context in
34:18
SPEAKER_00: which our obvious machine obviously attributes a purpose to them,
34:25
SPEAKER_00: just like you obviously attribute a purpose to a hammer when you see it.
34:28
SPEAKER_00: The hammer isn't sitting there thinking, though, you know what I need to be doing?
34:31
SPEAKER_00: I need to be hitting a nail.
34:32
SPEAKER_00: But you just, that's, you know, think about how automatically you do it even with inanimate things.
34:37
SPEAKER_00: So first of all, there's that.
34:38
SPEAKER_00: And then now we're having increasingly increasing evidence for,
34:42
SPEAKER_00: and this is disgusting,
34:44
SPEAKER_00: is the amount of fraud.
34:45
SPEAKER_00: that has gone into the demonstrations and the papers and all of this,
34:50
SPEAKER_00: which really deflates a lot of these claims very significantly.
34:55
SPEAKER_00: There's been a lot of mechanical turking in the old sense of a human being
34:59
SPEAKER_00: pretending to be a machine going on,
35:03
SPEAKER_00: and just so much.
35:05
SPEAKER_00: I can't believe.
35:06
SPEAKER_00: I mean, this was part of my concern.
35:09
SPEAKER_00: Again, I and people like, oh, John's.
35:14
SPEAKER_00: Yeah.
35:14
SPEAKER_00: When I said sorry, I'm getting upset here.
35:16
SPEAKER_00: But when I said when you separate this from the science, you're doing something very dangerous.
35:20
SPEAKER_00: And this is the example you get.
35:22
SPEAKER_00: Right.
35:22
SPEAKER_00: You get I mean, I'm not saying cannery.
35:25
SPEAKER_00: Yeah.
35:25
SPEAKER_00: Yeah.
35:26
SPEAKER_00: And fraud and all kinds of stuff going on because there is no there is no
35:30
SPEAKER_00: scientific motivation driving this.
35:32
SPEAKER_00: I'm not saying that we aren't having a problem with fraud in science,
35:35
SPEAKER_00: but there's a lot of honest scientists who are still motivated by the normativity
35:40
SPEAKER_00: of scientific endeavor.
35:42
SPEAKER_00: And the fact that this has gotten separated from science is now turned up in all of
35:48
SPEAKER_00: this duplicity that is masking and distorting and misinforming how people in the
35:55
SPEAKER_00: popular culture are being presented these machines and their performance and what
35:59
SPEAKER_00: they can do.
36:00
SPEAKER_00: And so you have to remember, you have to cut through a lot to get to what Sam and I are talking about.
36:07
SPEAKER_00: Sorry.
36:08
SPEAKER_00: No, no, no.
36:08
SPEAKER_00: I think you're dead.
36:09
SPEAKER_01: Yeah.
36:11
SPEAKER_02: I think in the next couple years,
36:16
SPEAKER_02: it's hard to make predictions about AI,
36:18
SPEAKER_02: say,
36:18
SPEAKER_02: over even the rest of the 21st century.
36:21
SPEAKER_02: But I think in the next couple years,
36:23
SPEAKER_02: there will be a deflation of the interest in AI,
36:27
SPEAKER_02: and it will have felt overblown and stuff like that.
36:31
SPEAKER_02: And again,
36:32
SPEAKER_02: I need to be careful because of my employer,
36:34
SPEAKER_02: but I think that the huge amounts of money invested might not be recouped.
36:40
SPEAKER_02: So even, you know, John's bemoaning the lab leak of AI into, you know, the private sector.
36:46
SPEAKER_02: I think even the private sector will have regrets in many ways about its
36:52
SPEAKER_02: investments in these things and that there won't be as much fruit on the tree as it hoped.
37:00
SPEAKER_00: That's already happening, right?
37:03
SPEAKER_00: A lot of people are already sort of saying,
37:05
SPEAKER_00: it looks like there wasn't the big bump with what seems to be coming from GPT-5,
37:12
SPEAKER_00: right?
37:13
SPEAKER_00: There wasn't the same sort of,
37:15
SPEAKER_00: well,
37:15
SPEAKER_00: it's just they sort of glued some stuff together and tightened some bolts kind of thing.
37:21
SPEAKER_00: Anyways, I interrupted you.
37:23
SPEAKER_01: But I was just going to, I mean, because...
37:25
SPEAKER_01: again,
37:25
SPEAKER_01: like the deflationary kind of account that there's a gold rush going on with this
37:29
SPEAKER_01: kind of AI one that's come out in terms of this top down,
37:34
SPEAKER_01: um,
37:35
SPEAKER_01: algorithmic model,
37:36
SPEAKER_01: but I'm more,
37:37
SPEAKER_01: I'm interested in your proposal,
37:39
SPEAKER_01: John,
37:39
SPEAKER_01: that,
37:39
SPEAKER_01: you know,
37:39
SPEAKER_01: there's an AI that could make us wiser.
37:41
SPEAKER_01: There's an AI that could address the meaning crisis,
37:44
SPEAKER_01: which is what we spoke about in our last conversation,
37:46
SPEAKER_01: which I wonder,
37:47
SPEAKER_01: do you still think that's possible?
37:50
SPEAKER_01: Do you still think there's an option for it or is it just part of the slush pile?
37:54
SPEAKER_00: So let me be clear, you know, and we talked about this.
37:57
SPEAKER_00: I just want to remind people, there's a bifurcation point.
37:59
SPEAKER_00: We can decide we're not going to make them
38:02
SPEAKER_00: autopoietic, and we're not going to try and make them rational or normativity sensitive.
38:07
SPEAKER_00: We're just going to make bigger, faster versions of what we have.
38:10
SPEAKER_00: And that's its own kind of problem.
38:12
SPEAKER_00: And that will have all kinds of issues.
38:13
SPEAKER_00: You know, making something intelligent doesn't make it rational.
38:16
SPEAKER_00: All kinds of making it intelligent, even less makes it rational.
38:21
SPEAKER_00: That has all kinds of problems.
38:22
SPEAKER_00: But if we decide, hey, look,
38:25
SPEAKER_00: Maybe Sam's right and there's a deflation and we go back to it being more of a
38:28
SPEAKER_00: scientific project,
38:29
SPEAKER_00: which I don't think that's going to go away.
38:33
SPEAKER_00: I don't think the scientific project is going to go away.
38:35
SPEAKER_00: So that's where I don't think you're going to see the deflation.
38:39
SPEAKER_00: And remember, up until very recently, this was driven by a scientific project.
38:43
SPEAKER_00: And what I think will happen there is the project,
38:46
SPEAKER_00: and that's what we talk about in Mentoring the Machines,
38:49
SPEAKER_00: the scientific project could give us the kinds of beings that could be
38:57
SPEAKER_00: artifactually rational and wiser and perhaps in certain ways exceed us.
39:03
SPEAKER_00: And then if that's true,
39:10
SPEAKER_00: they would deeply care about the true,
39:12
SPEAKER_00: the good,
39:12
SPEAKER_00: and the beautiful,
39:13
SPEAKER_00: and their greater cognitive grasp would presumably make them aware of the mentions
39:21
SPEAKER_00: of the true and the good and beautiful that we only have glimpses of,
39:24
SPEAKER_00: and that means they could potentially aid us towards enlightenment.
39:29
SPEAKER_00: Yes, I think that is a scientific possibility, and I think we
39:38
SPEAKER_00: I don't think we're going to stop the scientific project.
39:41
SPEAKER_00: And so I'm trying to recommend very strongly that we orient that way,
39:48
SPEAKER_00: that we don't just gain knowledge,
39:50
SPEAKER_00: that we try and say,
39:51
SPEAKER_00: well,
39:52
SPEAKER_00: why don't we make machines that not only are knowers,
39:56
SPEAKER_00: but are really seeking wisdom in a way that could help us seek wisdom?
40:01
SPEAKER_00: That's a very doable project.
40:02
SPEAKER_00: That's my proposal.
40:03
SPEAKER_00: Yes.
40:05
SPEAKER_01: And does this change your critique,
40:07
SPEAKER_01: Sam,
40:07
SPEAKER_01: in terms of the quantifiable-ness of wisdom,
40:10
SPEAKER_01: if there are these type of more relevance-realizing agents,
40:15
SPEAKER_01: I suppose,
40:16
SPEAKER_01: rather than...
40:18
SPEAKER_01: computer science computational algorithm model of AI.
40:22
SPEAKER_02: And this is where I think it's interesting,
40:25
SPEAKER_02: because to be an organism that is autopoetic and being shaped towards the good,
40:33
SPEAKER_02: the true,
40:34
SPEAKER_02: and the beautiful,
40:35
SPEAKER_02: you kind of need to be something that's fighting for survival and perpetuation into
40:41
SPEAKER_02: the future and able to be adapting to the feedback of success and failure at that.
40:48
SPEAKER_02: Basically, you need to put yourself out in Darwin's world and for better and for worse.
40:58
SPEAKER_02: And it's hard for me to imagine how,
41:03
SPEAKER_02: like,
41:04
SPEAKER_02: it's like,
41:04
SPEAKER_02: to me,
41:05
SPEAKER_02: it's like,
41:05
SPEAKER_02: so we're going to build a robot and we're going to drop it off in the Amazon and say,
41:11
SPEAKER_02: best of luck,
41:12
SPEAKER_02: we'll check in after a couple thousand generations.
41:15
SPEAKER_02: Right.
41:16
SPEAKER_02: You know, here we've given you some tools.
41:19
SPEAKER_02: Make a living for yourself.
41:21
SPEAKER_02: Watch out for the Jaguars.
41:22
SPEAKER_02: Yeah, it's like I'm not like I'm not quite sure exactly what the details of this process.
41:29
SPEAKER_02: Let me respond to that because I don't think that's what John's proposing.
41:34
SPEAKER_02: But in my mind, that's sort of like what would be required.
41:37
SPEAKER_00: Yeah.
41:38
SPEAKER_00: So first of all, we don't do that with our kids.
41:42
SPEAKER_00: We drop them into civilization after putting them through an extensive mentoring educational process.
41:48
SPEAKER_00: And we don't start them from scratch.
41:50
SPEAKER_00: We don't say, well, you're going to have to figure out how to do fire.
41:55
SPEAKER_00: We do what's called cultural ratcheting.
41:58
SPEAKER_00: And I would presume that we could
42:01
SPEAKER_00: just like we do with our kids, we could start the machines there.
42:05
SPEAKER_00: We wouldn't have to start them in the Amazon.
42:08
SPEAKER_00: We could start them at the pinnacle of our cultural ratcheting.
42:11
SPEAKER_00: And part of my argument is they would have to be like sociocultural beings in some way,
42:17
SPEAKER_00: and then they could ratchet beyond that.
42:19
SPEAKER_00: And let's be very clear about why is that at least, I think, scientifically possible.
42:25
SPEAKER_00: If you believe in evolution,
42:28
SPEAKER_00: you should believe that we are species that have cognitive relevance realization
42:33
SPEAKER_00: capacities that greatly exceed those of the Australopithecines and the other great apes.
42:40
SPEAKER_00: So the fact that there could be more,
42:43
SPEAKER_00: as we argue in the paper,
42:44
SPEAKER_00: this doesn't...
42:45
SPEAKER_00: Well,
42:45
SPEAKER_00: as we imply in the paper,
42:46
SPEAKER_00: this doesn't mean you can get an infinite intelligence or any of the bullshit that
42:49
SPEAKER_00: people are saying right now.
42:51
SPEAKER_00: That's...
42:52
SPEAKER_00: People treat intelligence like a magic wand that can somehow break the laws of
42:55
SPEAKER_00: biology or physics or something.
42:57
SPEAKER_00: That's ridiculous.
42:58
SPEAKER_00: But
43:01
SPEAKER_00: Could we get an artifactual being and mentor it,
43:08
SPEAKER_00: as we say in the book,
43:09
SPEAKER_00: using that word,
43:11
SPEAKER_00: such that it culturally ratchets faster than we do?
43:16
SPEAKER_00: Just like we culturally ratchet way faster than the Homo erecti.
43:19
SPEAKER_00: They take tens of thousands of years and we do it really fast.
43:23
SPEAKER_00: I don't see why not.
43:25
SPEAKER_00: And that's the argument I'm making.
43:27
SPEAKER_00: That's the kind of argument I'm making.
43:29
SPEAKER_02: So then they would need to be something like a social being that was making a living for itself.
43:35
SPEAKER_02: Yeah, they would basically need a job and a bank account.
43:39
SPEAKER_02: And,
43:40
SPEAKER_02: and,
43:41
SPEAKER_02: and some and that's not too dissimilar from the situation that we're already in,
43:46
SPEAKER_02: right,
43:47
SPEAKER_02: where,
43:47
SPEAKER_02: you know,
43:47
SPEAKER_02: these companies have these things.
43:49
SPEAKER_02: And, of course, they need to, you know, satisfy stockholder fiduciary responsibility.
43:55
SPEAKER_02: And so they they want these products to be able to make a living.
43:58
SPEAKER_02: So they're
43:59
SPEAKER_02: Quickly as possible, trying to figure out how to apply these things to make as much money.
44:04
SPEAKER_02: And that,
44:05
SPEAKER_02: that to me,
44:05
SPEAKER_02: I'm not quite sure what the difference is between that and sort of what,
44:11
SPEAKER_02: what you're saying,
44:12
SPEAKER_02: John.
44:14
SPEAKER_00: Well,
44:14
SPEAKER_00: the difference is those companies and well,
44:19
SPEAKER_00: at least the machines in those companies don't care about the truth,
44:21
SPEAKER_00: the good,
44:21
SPEAKER_00: the beautiful,
44:22
SPEAKER_00: the way we try to get our children to care about the truth,
44:25
SPEAKER_00: the good,
44:25
SPEAKER_00: beautiful.
44:26
SPEAKER_00: They don't participate in a Hegelian fashion in the normativity in which they
44:30
SPEAKER_00: respect precedents and they try to set precedents and all that kind of stuff.
44:35
SPEAKER_00: Those are very big difference.
44:37
SPEAKER_00: Again, that's why we keep using the verb mentoring, right?
44:43
SPEAKER_00: I'm leaning really heavily on that because we chose that term very, very carefully.
44:48
SPEAKER_00: Now, I think there's some good points that you've made before, and I want to give you credit for them.
44:52
SPEAKER_00: We may decide,
44:53
SPEAKER_00: look,
44:56
SPEAKER_00: the way of making these artifactual beings is just so environmentally toxic because
45:01
SPEAKER_00: we have to mine these rare metals and we,
45:03
SPEAKER_00: you know,
45:04
SPEAKER_00: like,
45:04
SPEAKER_00: and we may just say,
45:05
SPEAKER_00: sorry.
45:06
SPEAKER_00: Or they're just too darn expensive.
45:08
SPEAKER_00: Or some combination thereof of environmental externalities and economic internalities.
45:20
SPEAKER_00: And that may be the case.
45:21
SPEAKER_00: Again, that's why we're very careful about using this word threshold.
45:25
SPEAKER_00: It might be that as we approach that, we don't know yet.
45:29
SPEAKER_00: That's, I think, a genuine empirical question.
45:31
SPEAKER_00: But we could get to there.
45:33
SPEAKER_00: And for very good reasons,
45:34
SPEAKER_00: we could say,
45:34
SPEAKER_00: no,
45:35
SPEAKER_00: we're not going down that road because we have to despoil large parts of the earth
45:40
SPEAKER_00: and impoverish huge numbers of people.
45:43
SPEAKER_00: And no company wants to run the economic risk to make these conditions possible for these machines.
45:50
SPEAKER_00: That's...
45:51
SPEAKER_00: Yes.
45:51
SPEAKER_00: I don't deny that either, Sam.
45:53
SPEAKER_00: I completely agree that that's a possibility.
45:56
SPEAKER_00: I'm just trying to say, suppose that isn't the case.
45:59
SPEAKER_00: I'm answering Mahon's question.
46:01
SPEAKER_00: This is what I'm proposing.
46:04
SPEAKER_02: So it seems like they're sort of migrating from tractors into sheep while being on the farm.
46:11
SPEAKER_02: You know,
46:12
SPEAKER_02: while being under sort of the shelter of human civilization,
46:15
SPEAKER_02: and I should add,
46:17
SPEAKER_02: all humans are basically,
46:18
SPEAKER_02: to some degree or another,
46:19
SPEAKER_02: under the shelter of human civilization,
46:22
SPEAKER_02: while also contributing to and helping shape,
46:25
SPEAKER_02: you know,
46:26
SPEAKER_02: human civilization.
46:27
SPEAKER_02: We're sort of like a self-domesticated species in a certain sense.
46:32
SPEAKER_02: Yeah.
46:33
SPEAKER_02: um but it seems like you know for these ais to be to to even achieve john's i i
46:41
SPEAKER_02: would say that that's a noble vision although sometimes you know i hear jonathan
46:45
SPEAKER_02: peggio jumping into my head and you know the the worry about you know create you
46:50
SPEAKER_02: know the people bowed and prayed to the neon god they made right like you know that
46:54
SPEAKER_02: song came on the pandora earlier this morning i'm like oh
46:59
SPEAKER_02: I realize the relevance of that song lyric.
47:04
SPEAKER_02: So the worry of creating something more powerful than yourself is creeping into my mind.
47:08
SPEAKER_02: But even if we imagine trying to make these things better and better sheep,
47:13
SPEAKER_02: it's still that they are kind of interacting with us,
47:17
SPEAKER_02: getting better at taking care of themselves while we're making them better at
47:20
SPEAKER_02: taking care of themselves and generating some sort of usefulness in the meantime.
47:24
SPEAKER_00: And then I want to add one more thing in, which we didn't really talk about last time.
47:27
SPEAKER_00: And this is what I'm going to say now is tremendously fraught.
47:33
SPEAKER_00: And I think it's fraught with all kinds of horrible ethical and political things.
47:38
SPEAKER_00: We will also be cyborging ourselves along the way, right?
47:42
SPEAKER_00: We will be doing that.
47:45
SPEAKER_00: And I think that is a more near future than these machines becoming the kind of
47:49
SPEAKER_00: thing we're talking about.
47:50
SPEAKER_00: Because what we found is we thought we had to make these really sophisticated
47:55
SPEAKER_00: machines when we sort of wanted to glue them into the brain.
47:57
SPEAKER_00: But it turns out that,
47:58
SPEAKER_00: no,
47:58
SPEAKER_00: the brain really,
47:59
SPEAKER_00: like it's this,
48:00
SPEAKER_00: it'll do a lot of work on its side to fit itself to the prosthetic.
48:05
SPEAKER_00: It doesn't mean it's not a hard problem.
48:07
SPEAKER_00: But so I think we're going to cyborg ourselves.
48:10
SPEAKER_00: And so I think...
48:12
SPEAKER_00: We're going to be cyborging ourselves as we're making these.
48:14
SPEAKER_00: And so it's going to get very blurred and messy.
48:17
SPEAKER_00: And again, that's another threshold.
48:19
SPEAKER_00: And we may decide, look, we're cyborging ourselves.
48:22
SPEAKER_00: And this is getting, this is just spoiling the planet.
48:25
SPEAKER_00: It's leaving a whole group of people behind.
48:27
SPEAKER_00: It's actually destroying our humanity.
48:29
SPEAKER_00: There are side effects we didn't anticipate.
48:32
SPEAKER_00: I don't deny any of that also, Sam.
48:34
SPEAKER_00: But I do think, I think we will, I...
48:38
SPEAKER_00: I'll say this.
48:38
SPEAKER_00: I think it is much more probable that we will cyborg ourselves before we get these autonomous,
48:46
SPEAKER_00: artifactual agents.
48:48
SPEAKER_02: Yeah.
48:49
SPEAKER_02: Yeah.
48:50
SPEAKER_02: Yeah.
48:51
SPEAKER_02: And in a certain way, we almost cyborg with the sheep.
48:54
SPEAKER_02: Right.
48:54
SPEAKER_02: We started making clothes out of wool.
48:56
SPEAKER_00: Oh, yeah.
48:57
SPEAKER_02: We became lactose tolerant into adulthood that might have had more to do with cows.
49:01
SPEAKER_02: But, you know, like that same sort of idea, you know, we we domesticate the animals and transform them.
49:06
SPEAKER_02: But it also transforms us and our bodies a fair amount as well, too.
49:11
SPEAKER_00: And so it's going to be this very complex,
49:15
SPEAKER_00: well,
49:15
SPEAKER_00: as it's always been,
49:16
SPEAKER_00: it's going to be this very complex process.
49:19
SPEAKER_00: And I want to say that because I sometimes sound like I'm offering this roadmap.
49:24
SPEAKER_00: I'm very careful.
49:25
SPEAKER_00: Again,
49:25
SPEAKER_00: we talk about threshold because those are sort of places where I think decisions
49:30
SPEAKER_00: can be made,
49:30
SPEAKER_00: but I think the process is going to be very messy and murky in this way I'm describing.
49:36
SPEAKER_02: I think that a big question that will emerge from this is what is the purpose of human beings?
49:42
SPEAKER_02: And that is AI fundamentally competing with us for our niche in the environment?
49:50
SPEAKER_02: And that's sort of a fear.
49:52
SPEAKER_02: A lot of the sci-fi movies imagine that same idea,
49:55
SPEAKER_02: that there is the niche of the human being,
49:59
SPEAKER_02: the intelligent social thing at the top of the pyramid,
50:02
SPEAKER_02: and AI is going to try and dethrone us from that?
50:06
SPEAKER_02: Or is there some other potential niche that is non-competitive but complementary to
50:12
SPEAKER_02: our own that these things could fill?
50:15
SPEAKER_00: And that's another threshold.
50:17
SPEAKER_00: In the history of biological evolution, we see these forks between extinction and symbiosis.
50:24
SPEAKER_00: And again,
50:25
SPEAKER_00: I'm not sure.
50:28
SPEAKER_00: I'm proposing we could steer things towards symbiosis.
50:32
SPEAKER_00: That's another way of understanding what our proposal is.
50:36
SPEAKER_00: We're steering things towards symbiosis rather than replacement.
50:41
SPEAKER_00: And I think your point,
50:42
SPEAKER_00: this is what I mean when I say to people that theology is going to become the most
50:47
SPEAKER_00: important discipline of the future because who are we and what are the parts of us
50:51
SPEAKER_00: that are not capturable by language,
50:54
SPEAKER_00: the spiritual somatic axis and our relationship to ultimacy,
50:59
SPEAKER_00: all of these things and our sacred purposes are going to come to the fore again
51:03
SPEAKER_00: because our difference between these machines,
51:05
SPEAKER_00: even if all we do is churn out more and more of the LLMs,
51:09
SPEAKER_00: the differences are going to disappear.
51:11
SPEAKER_00: And we're going to,
51:11
SPEAKER_00: I think we're going to have to,
51:13
SPEAKER_00: and I think I'm not,
51:15
SPEAKER_00: I'm not sure this is totally a bad thing.
51:16
SPEAKER_00: I think trying to recover those parts of us that are not capturable by sort of a
51:20
SPEAKER_00: Cartesian framework is maybe a good thing,
51:24
SPEAKER_00: or at least has a potential to be a very good thing.
51:27
SPEAKER_01: Yeah.
51:27
SPEAKER_01: It seems to be running that experiment.
51:29
SPEAKER_01: Like that's a really interesting point, John, that it,
51:31
SPEAKER_01: it is forcing us to look at the things that have been thrown out somewhat in
51:36
SPEAKER_01: modernity that actually used to define us that have been left by the wayside to
51:42
SPEAKER_01: understand ourselves.
51:43
SPEAKER_01: And I think that, because you said, John, about creating the conditions where cognitive agency can grow.
51:49
SPEAKER_01: And this idea of creating that for an AI or for a,
51:54
SPEAKER_01: artifactual being is like I think we need to do that for ourselves as well a lot
51:59
SPEAKER_01: like that's kind of you know that's part of the argument though that's part of the
52:03
SPEAKER_01: argument that's the interesting bit where it's like if do you think that's a good
52:07
SPEAKER_01: bet in terms of do for the artifactual being that then over leaps us and
52:13
SPEAKER_01: that can then return to, again, offer us, I suppose, this enhancement of cognitive agency.
52:19
SPEAKER_01: Is that the kind of idea?
52:21
SPEAKER_00: So I'm going to make kind of like what's an anthropic argument,
52:24
SPEAKER_00: which is that kind of being that I'm proposing would only exist if we were able to
52:31
SPEAKER_00: give it good mentoring templates.
52:33
SPEAKER_00: And see, what we've relied on so far with intelligence is that we're just naturally intelligent.
52:38
SPEAKER_00: We don't have to do much to be intelligent.
52:40
SPEAKER_00: We just have to not suffer trauma.
52:42
SPEAKER_00: That's basically it.
52:44
SPEAKER_00: That's not the case for rationality and wisdom.
52:48
SPEAKER_00: If we're going to make the kind of entities that could enter as silicon sages into
52:53
SPEAKER_00: a symbiotic relationship with us,
52:55
SPEAKER_00: we have to first become a lot more rational in the platonic sense of rationality
53:00
SPEAKER_00: not the Cartesian sense.
53:01
SPEAKER_00: In the Platon,
53:02
SPEAKER_00: we have to become much more rational and wise in order to provide the mentoring templates,
53:08
SPEAKER_00: the exemplars,
53:09
SPEAKER_00: the role models for these machines.
53:11
SPEAKER_00: So they, right, they will not come into existence unless we have taken up that responsibility.
53:18
SPEAKER_00: That's an argument I would make.
53:21
SPEAKER_02: And so one thing that I wonder about John's proposal is how would it be possible
53:25
SPEAKER_02: for these things to exceed us?
53:29
SPEAKER_02: Because as of right now, the main way that they work is through imitation.
53:34
SPEAKER_02: And so an imitator is only ever going to be a statistical approximation of what it's imitating.
53:42
SPEAKER_02: And we're still trying to figure out how to get it to imitate the right things and
53:45
SPEAKER_02: not the wrong things,
53:46
SPEAKER_02: and how to tell the difference between a joke on Reddit and a helpful response on Reddit,
53:51
SPEAKER_02: right?
53:52
SPEAKER_02: But theoretically,
53:53
SPEAKER_02: you could get it to be like 98,
53:55
SPEAKER_02: 99% as good as like a really good Reddit commenter or something like that in many ways.
54:02
SPEAKER_02: But how would you ever get it to be
54:05
SPEAKER_02: 10% better than the best Reddit commenter.
54:10
SPEAKER_00: I don't think the LLMs will exceed us in that way.
54:14
SPEAKER_00: I'm talking about if we create artifactual beings that can do autonomous
54:20
SPEAKER_00: recursive relevance realization,
54:23
SPEAKER_00: and do that recursively so that they are cultivating rationality and wisdom.
54:28
SPEAKER_00: And we, of course, think that we exceed early... And if we think there's something like...
54:37
SPEAKER_00: evolution going on in relevance realization.
54:39
SPEAKER_00: And like acceptation,
54:41
SPEAKER_00: there's no reason why,
54:43
SPEAKER_00: if that's the principles by which they're operating,
54:46
SPEAKER_00: there's no reason why they can't exceed us,
54:49
SPEAKER_00: just like we exceed homo erectus,
54:51
SPEAKER_00: or even to some degree,
54:53
SPEAKER_00: the way we even ontogenetically exceed our children,
54:57
SPEAKER_00: because we go through a lot of
55:06
SPEAKER_00: If we admit the possibility both biologically of qualitative change through
55:11
SPEAKER_00: speciation and ontogenetically through a qualitative development in a Piagetian
55:17
SPEAKER_00: sense within human beings,
55:19
SPEAKER_00: again,
55:19
SPEAKER_00: there's no principle why we couldn't give them that capacity to exceed us.
55:24
SPEAKER_00: Because they're not going to be imitation machines.
55:26
SPEAKER_00: That's the whole point, right?
55:28
SPEAKER_00: They're going to be relevance-realizing machines.
55:30
SPEAKER_00: They're not going to imitate us doing relevance realization.
55:33
SPEAKER_00: They're going to be capable of it.
55:35
SPEAKER_00: So we could give them that new version of Raven's progressive matrices that has not
55:42
SPEAKER_00: been laid down in any text anywhere,
55:45
SPEAKER_00: and they will be able to solve it like you can.
55:49
SPEAKER_02: So,
55:50
SPEAKER_02: to me,
55:50
SPEAKER_02: I think,
55:51
SPEAKER_02: and I suppose you would probably agree with this,
55:53
SPEAKER_02: there would need to be some categorical technological innovations.
55:59
SPEAKER_02: in our current AI to even facilitate for that.
56:04
SPEAKER_02: Because when we were talking earlier about the limits of mathematics sort of
56:08
SPEAKER_02: constraining them,
56:09
SPEAKER_02: it would seem like they would need to be able to get beyond that sort of constraint
56:14
SPEAKER_02: and be more like an organism in how that they're responding.
56:17
SPEAKER_00: That's exactly the argument, exactly.
56:19
SPEAKER_00: That's what I mean, again, when I'm saying no artificial intelligence without artifactual life, right?
56:25
SPEAKER_00: Not really, really, really.
56:27
SPEAKER_00: And if you think life has the capacity,
56:29
SPEAKER_00: both autogenetically and phylogenetically,
56:32
SPEAKER_00: to go through qualitative development,
56:34
SPEAKER_00: which it repeatedly demonstrates,
56:36
SPEAKER_00: then that would be also the case for these beings.
56:39
SPEAKER_00: Yes, that's the argument.
56:41
SPEAKER_01: So it's almost like a perfect evolver or something, like a creating, like...
56:45
SPEAKER_01: an artifactual being that's like super good at evolving um look look look at how we
56:50
SPEAKER_00: are different than some of our ancestors so like and biologists talk about this in
56:56
SPEAKER_00: a lot of creatures you see the evolution of traits and then we have got we have
57:00
SPEAKER_00: done this extra thing other organisms too but we have sort of done it really well
57:06
SPEAKER_00: we evolve the metal trait of evolvability
57:11
SPEAKER_00: Now, we do that cognitively in learning.
57:13
SPEAKER_00: Other organisms do it by having really rapid,
57:16
SPEAKER_00: like short lifestyles and rapid generational turnover and a lot of genetic variation.
57:21
SPEAKER_00: But we've sort of done that.
57:23
SPEAKER_00: We can go almost anywhere.
57:30
SPEAKER_00: Like,
57:31
SPEAKER_00: that's the evolution of evolvability,
57:34
SPEAKER_00: as opposed to the evolution of...
57:35
SPEAKER_00: And this is part of the argument of the paper,
57:37
SPEAKER_00: right?
57:38
SPEAKER_00: You know, your traits are always environmentally specific, right?
57:43
SPEAKER_00: They have to be.
57:44
SPEAKER_00: Bias, variance, trade-offs, and all kinds of things like this.
57:49
SPEAKER_00: These machines could have our capacity of evolving evolvability, and maybe they can ratchet that up.
58:00
SPEAKER_00: um again sam's point is exactly the right point we're talking about huge
58:07
SPEAKER_00: breakthroughs that have to happen my point is this project is ongoing and it's
58:12
SPEAKER_00: making progress and it's not being paid attention to because we're sort of focusing
58:16
SPEAKER_00: on the llms right now um but um
58:22
SPEAKER_00: Yeah,
58:22
SPEAKER_00: Sam,
58:22
SPEAKER_00: and again,
58:25
SPEAKER_00: I will acknowledge your point that you made in our last conversation,
58:28
SPEAKER_00: that those breakthroughs might come at economic and environmental costs that we
58:32
SPEAKER_00: don't want or are unable to bear,
58:34
SPEAKER_00: and then that just shuts the whole thing down.
58:36
SPEAKER_00: I acknowledge that, right?
58:37
SPEAKER_00: I will continue to acknowledge that, yes.
58:40
SPEAKER_02: Sure.
58:40
SPEAKER_02: I guess my,
58:42
SPEAKER_02: sort of the other question is,
58:43
SPEAKER_02: well,
58:44
SPEAKER_02: we already have,
58:45
SPEAKER_02: humans already are biological beings,
58:48
SPEAKER_02: right?
58:49
SPEAKER_02: Is it perhaps a better prioritization to just kind of work on helping humans be
58:55
SPEAKER_02: wiser and wiser as opposed to trying to invent something that can do it and then do it?
59:04
SPEAKER_02: We already can do it.
59:06
SPEAKER_00: So here's the answer, and here's where you and I agree.
59:10
SPEAKER_00: The LLMs are not a scientific project, and so they – yeah.
59:15
SPEAKER_00: If what I'm saying is true,
59:18
SPEAKER_00: that there is a scientific project,
59:19
SPEAKER_00: and I think that scientific project as a scientific project will gain ground as
59:24
SPEAKER_00: some of the deflation around the LLMs sets in.
59:27
SPEAKER_00: this is my answer to you.
59:28
SPEAKER_00: And this,
59:29
SPEAKER_00: this has been the,
59:31
SPEAKER_00: the,
59:31
SPEAKER_00: the claim of CogSci,
59:32
SPEAKER_00: which is,
59:33
SPEAKER_00: well,
59:34
SPEAKER_00: you gain an explanatory move that you don't have in how else you do science by
59:39
SPEAKER_00: taking the design stance,
59:41
SPEAKER_00: by trying to build something that can do what you can do.
59:44
SPEAKER_00: I gain, I leap ahead in my insights and my theoretical proposals about what might be going on to you.
59:52
SPEAKER_00: And, and so I,
59:54
SPEAKER_00: I have always pursued cognitive science and been interested in AI insofar as it
59:59
SPEAKER_00: helps me understand human intelligence,
1:00:03
SPEAKER_00: biological intelligence,
1:00:04
SPEAKER_00: rationality,
1:00:05
SPEAKER_00: wisdom.
1:00:06
SPEAKER_00: And if what I said also is,
1:00:07
SPEAKER_00: you know,
1:00:09
SPEAKER_00: if we do make them...
1:00:11
SPEAKER_00: Remember,
1:00:12
SPEAKER_00: we have to...
1:00:13
SPEAKER_00: As we're trying to make them,
1:00:14
SPEAKER_00: we have to make ourselves better along the way.
1:00:18
SPEAKER_00: So I have two answers to you.
1:00:19
SPEAKER_00: One is...
1:00:20
SPEAKER_00: Well,
1:00:20
SPEAKER_00: if this gets back to being a scientific project,
1:00:24
SPEAKER_00: then this way of doing things gives us a kind of method for gaining knowledge that
1:00:30
SPEAKER_00: we can't get well by other means.
1:00:33
SPEAKER_00: And then secondly,
1:00:34
SPEAKER_00: well,
1:00:35
SPEAKER_00: the proposal I'm making would actually motivate us and help bootstrap us to
1:00:39
SPEAKER_00: becoming better.
1:00:41
SPEAKER_02: Mm-hmm.
1:00:42
SPEAKER_02: The learn by doing and the learn by designing.
1:00:48
SPEAKER_02: So here, I guess this is where I'll dabble in venturing into theology.
1:00:55
SPEAKER_02: That part of my kind of,
1:00:58
SPEAKER_02: I don't know,
1:00:59
SPEAKER_02: Neoplatonic Darwinism that we've sort of discussed before is almost like there's
1:01:04
SPEAKER_02: the form of the different organisms in the sky,
1:01:06
SPEAKER_02: right?
1:01:07
SPEAKER_00: And you can say this way now, by the way, you know, butterfly cutting edge biologist.
1:01:12
SPEAKER_00: Yeah.
1:01:13
SPEAKER_02: Well, good.
1:01:14
SPEAKER_02: I'm glad that I feel less weird and lonely and talking that way.
1:01:18
SPEAKER_02: You know, I've talked about the polar bear in the sky.
1:01:21
SPEAKER_02: Right.
1:01:22
SPEAKER_02: And like, you know, a polar bear, there's like almost like a tractor beam.
1:01:26
SPEAKER_02: between the perfect polar bear and all the polar bears and somehow they have gotten
1:01:31
SPEAKER_02: into the place where they are kind of um bumping against the the feedback mechanism
1:01:39
SPEAKER_02: that helps them know if they're doing a good job or that and that's sort of the
1:01:42
SPEAKER_02: emergence and the emanation at the same time and then you can imagine that there's
1:01:46
SPEAKER_02: more than just polar bears up there in heaven that there's the whole suite of all
1:01:50
SPEAKER_00: the animals and that's whitehead's model in a sense yeah
1:01:54
SPEAKER_02: And it's you know, there's almost like this perfect collage of the ecology of Earth up there.
1:02:00
SPEAKER_02: And then that's sort of tying back to kind of what I said earlier.
1:02:03
SPEAKER_02: We need to think about where does where do human beings fit in the ecology of animals in the sky?
1:02:10
SPEAKER_02: What tractor beam is on us that we are being selected at being good for?
1:02:16
SPEAKER_02: And to me, the religious question, are we the beings that are supposed to be at the top of the pyramid?
1:02:23
SPEAKER_02: Are we the image bearers of God put over creation in a unique, categorically separate sense?
1:02:30
SPEAKER_02: And if that's the case, inviting someone else into that space might not be the right thing to do.
1:02:36
SPEAKER_02: And that's where theology and cosmology and all those things start to come back into the picture.
1:02:46
SPEAKER_00: And I think that's a good argument, Sam.
1:02:49
SPEAKER_00: And first of all, I want to take a moment to acknowledge respect.
1:02:53
SPEAKER_00: towards that argument.
1:02:54
SPEAKER_00: I mean,
1:02:57
SPEAKER_00: you could say maybe there's an argument,
1:02:59
SPEAKER_00: a neoplatonic argument,
1:03:00
SPEAKER_00: a whiteheady and neoplatonic argument.
1:03:02
SPEAKER_00: Maybe there's attractors which inform space or something like that.
1:03:06
SPEAKER_00: And people are talking that way now.
1:03:08
SPEAKER_00: And maybe, you know,
1:03:11
SPEAKER_00: it would be a sin to try and displace us.
1:03:15
SPEAKER_00: I think as a theological argument,
1:03:18
SPEAKER_00: so we're both admitting this isn't a scientific argument,
1:03:20
SPEAKER_00: but that doesn't mean it's not a good argument.
1:03:22
SPEAKER_00: I'm not saying that.
1:03:23
SPEAKER_00: It's a theological argument.
1:03:25
SPEAKER_00: And so if you'll allow me, you're saying something like it might be a sin to do this.
1:03:30
SPEAKER_00: And I think that's a legitimate theological question to ask.
1:03:35
SPEAKER_00: I suppose I would say, well, I would want to do more theological argument.
1:03:40
SPEAKER_00: I would want to say,
1:03:41
SPEAKER_00: my Neoplatonism would say,
1:03:43
SPEAKER_00: well,
1:03:43
SPEAKER_00: if what you're saying is true,
1:03:44
SPEAKER_00: we couldn't displace ourselves.
1:03:48
SPEAKER_00: It doesn't sort of work in the cosmology.
1:03:52
SPEAKER_00: And maybe what I'm proposing is, would be in fact sort of a test.
1:03:56
SPEAKER_00: I've sort of done this in the argument.
1:03:58
SPEAKER_00: I said, we try this and we might discover these machines can't actually become enlightened.
1:04:05
SPEAKER_00: And here, John Vervaeke laying it on the table, totally possible.
1:04:10
SPEAKER_00: Yes.
1:04:12
SPEAKER_00: And then that tells us what is absolutely unique,
1:04:15
SPEAKER_00: because I think,
1:04:16
SPEAKER_00: I happen to believe there is good evidence to believe that there have been human
1:04:20
SPEAKER_00: beings that were enlightened.
1:04:22
SPEAKER_00: And then that would show radically how different we are from them.
1:04:27
SPEAKER_00: And I think that's a very real possibility.
1:04:30
SPEAKER_00: And...
1:04:35
SPEAKER_00: I think we have to be concerned about...
1:04:38
SPEAKER_00: Boy,
1:04:41
SPEAKER_00: you know,
1:04:42
SPEAKER_00: if you'd asked me if I'd been using this language five years ago,
1:04:44
SPEAKER_00: I wouldn't have said it.
1:04:45
SPEAKER_00: But something like blasphemy and hubris from the two different traditions,
1:04:51
SPEAKER_00: the Hebrew and the Greek tradition,
1:04:53
SPEAKER_00: we do need to think about this.
1:04:56
SPEAKER_00: And this is perhaps where it's overlapping with Jonathan's concerns.
1:04:59
SPEAKER_00: It might be that...
1:05:02
SPEAKER_00: I still don't know if we could actually make something that displaces us,
1:05:07
SPEAKER_00: if that's sort of our platonic role.
1:05:09
SPEAKER_00: But here's what I'll say, at least, as an intermediary.
1:05:12
SPEAKER_00: We could sure make something that made us believe it displaced us.
1:05:16
SPEAKER_00: And that could be a horrible,
1:05:18
SPEAKER_00: sinful thing to do in a very,
1:05:21
SPEAKER_00: very dangerous way,
1:05:23
SPEAKER_00: sort of a demonic kind of thing in that fashion.
1:05:27
SPEAKER_00: And that's more like Jonathan's worry.
1:05:29
SPEAKER_00: And I take that very seriously.
1:05:31
SPEAKER_00: I take that very seriously.
1:05:34
SPEAKER_00: I don't know what to say about that other than I think we should keep doing the
1:05:38
SPEAKER_00: science and the philosophy and the theology.
1:05:40
SPEAKER_00: Here's why.
1:05:41
SPEAKER_00: We're not going to stop the economic project in the LLMs.
1:05:46
SPEAKER_00: That machine is going to keep running.
1:05:48
SPEAKER_00: We're not going to stop the scientific project,
1:05:50
SPEAKER_00: but the scientific project has a possibility of being wedded to,
1:05:54
SPEAKER_00: we're trying to do it here,
1:05:55
SPEAKER_00: a philosophical and a theological project in which we are constantly asking these
1:05:59
SPEAKER_00: questions as deeply and as provocatively and getting as much feedback by any
1:06:04
SPEAKER_00: progress we're making in these projects to get a clearer answer.
1:06:08
SPEAKER_00: And,
1:06:09
SPEAKER_00: like,
1:06:10
SPEAKER_00: I'm totally with the case of,
1:06:12
SPEAKER_00: like,
1:06:12
SPEAKER_00: look,
1:06:13
SPEAKER_00: there's converging evidence that we're sort of moving in the wrong direction or something.
1:06:17
SPEAKER_00: Yeah.
1:06:18
SPEAKER_00: Yeah, totally.
1:06:20
SPEAKER_00: Again, I take very seriously that question.
1:06:28
SPEAKER_00: Like I say,
1:06:29
SPEAKER_00: once it's properly understood as a theological argument,
1:06:33
SPEAKER_00: and I'm not dismissing it because it's a theological argument.
1:06:38
SPEAKER_00: So that's my best answer.
1:06:40
SPEAKER_00: I would want to get into a theological argument about whether or not actual displacement is possible.
1:06:47
SPEAKER_00: I would want to see maybe they could show us that we can't be displaced,
1:06:51
SPEAKER_00: and that would be really powerful empirical evidence.
1:06:55
SPEAKER_00: And then a third thing is we might get things that fool us into believing we've
1:07:00
SPEAKER_00: been displaced when we actually haven't.
1:07:02
SPEAKER_00: And that is also very concerning.
1:07:07
SPEAKER_02: And I don't want to sound too critical.
1:07:11
SPEAKER_02: After all, I'm the one who's helping doing autopoiesis for the corporate machine at the moment.
1:07:16
SPEAKER_02: So I don't want to open myself up to too much accusation of hypocrisy.
1:07:20
SPEAKER_02: But I think those were all really good points, Sean.
1:07:23
SPEAKER_02: And I'm not saying it's time to shut off the project yet.
1:07:28
SPEAKER_02: I hear what you're saying.
1:07:29
SPEAKER_02: I think you make a lot of good points too.
1:07:31
SPEAKER_02: And I don't want to be too close-minded myself out of some sort of, I don't know,
1:07:36
SPEAKER_02: arrogant theological fundamentalism either um and i appreciate that i think those
1:07:41
SPEAKER_01: are good points yeah yeah i think i understand your proposal so much better now
1:07:45
SPEAKER_01: john in terms of clarifying those kind of differences between what's going on with
1:07:50
SPEAKER_01: the llms and that kind of computer science artificial intelligence and then what
1:07:54
SPEAKER_01: you're talking about with artifactual beings and on that project so i think this
1:07:59
SPEAKER_01: has been tremendously helpful um
1:08:01
SPEAKER_01: clarifying that yeah if there's any we're a little bit over time here if there's
1:08:05
SPEAKER_01: any final words you want to say or anything but yeah brilliant conversation well I
1:08:09
SPEAKER_00: want to be very clear I mean some of the predictions I made about the
1:08:15
SPEAKER_00: predictions failing, a lot of that came true.
1:08:18
SPEAKER_00: And the threshold argument,
1:08:19
SPEAKER_00: there's a lot in the current argument I gave you that is continuous with the
1:08:23
SPEAKER_00: original presentation and the video essay.
1:08:25
SPEAKER_00: But I've also learned a lot.
1:08:27
SPEAKER_00: And so I don't want everybody believing that as I've presented things now is
1:08:31
SPEAKER_00: exactly how they were in the original formulation.
1:08:33
SPEAKER_00: There's been development through talking to people like Sam.
1:08:37
SPEAKER_00: through talking to many people.
1:08:38
SPEAKER_00: I've had many excellent conversations.
1:08:41
SPEAKER_00: This is why we're actually releasing the book in serial form,
1:08:44
SPEAKER_00: because the argument is growing and changing as these discussions and as things go on.
1:08:51
SPEAKER_00: So I don't... I'm not...
1:08:53
SPEAKER_00: I don't think I'm contradicting myself.
1:08:55
SPEAKER_00: I'm definitely not being hypocritical in any fashion.
1:08:58
SPEAKER_00: But I don't want people thinking I'm claiming that I said all of this at the very beginning.
1:09:02
SPEAKER_00: I didn't.
1:09:04
SPEAKER_00: This has developed from that original argument by doing exactly these kinds of
1:09:08
SPEAKER_00: discussions for which I am extremely grateful.
1:09:10
SPEAKER_00: And I want to express gratitude and a proper credit right now for that.
1:09:21
SPEAKER_02: That sounds great.
1:09:22
SPEAKER_02: I really like talking to you too, John.
1:09:24
SPEAKER_02: I feel like I wish I could share these things with more of my coworkers and get them to pay attention.
1:09:32
SPEAKER_02: I wish that more of these conversations were happening at the level that they
1:09:37
SPEAKER_02: should be happening to make a difference.
1:09:40
SPEAKER_02: And I agree that this is one of those things where the ground is changing so
1:09:45
SPEAKER_02: quickly that one needs to be open-minded and adaptable,
1:09:50
SPEAKER_02: but not open-minded to the point of gullible,
1:09:53
SPEAKER_02: I guess.
1:09:55
SPEAKER_02: And I think that I really appreciate your openness and honesty, John.
1:10:00
SPEAKER_02: And I'm honestly just flattered that you consider me a worthwhile conversation partner at this point.
1:10:05
SPEAKER_02: So I hope that this conversation continues to happen.
1:10:09
SPEAKER_00: Well, I think this conversation is yet again evidence of that.
1:10:13
SPEAKER_00: I should say,
1:10:14
SPEAKER_00: and I'm bound by a lot of confidentiality in protecting people's career,
1:10:18
SPEAKER_00: but I am talking to a lot of people on the inside about this kind of stuff.
1:10:24
SPEAKER_00: And there's more people than you might think on the inside who are trying to figure
1:10:29
SPEAKER_00: out the theoretical and rhetorical moves to try and get more people engaged in this
1:10:36
SPEAKER_00: sort of wise AI project,
1:10:38
SPEAKER_00: if I can
1:10:39
SPEAKER_00: give it that name.
1:10:41
SPEAKER_00: And so I'm not claiming that they're a majority or anything ridiculous like that,
1:10:46
SPEAKER_00: but there is a growing number of people who are saying,
1:10:52
SPEAKER_00: no,
1:10:52
SPEAKER_00: no,
1:10:52
SPEAKER_00: we need to be talking about this more scientifically,
1:10:56
SPEAKER_00: more philosophically,
1:10:57
SPEAKER_00: and
1:10:58
SPEAKER_00: I think even a few people,
1:10:59
SPEAKER_00: even theologically now,
1:11:01
SPEAKER_00: are like,
1:11:01
SPEAKER_00: yeah,
1:11:01
SPEAKER_00: maybe we need to talk about some of these deeper normative aspects of reality.
1:11:07
SPEAKER_00: There's been a paper published recently on the platonic representation hypothesis,
1:11:11
SPEAKER_00: which is this weird finding that you can take very different AIs with very
1:11:15
SPEAKER_00: different architectures organized for very different goals,
1:11:19
SPEAKER_00: and if they're powerful enough,
1:11:21
SPEAKER_00: they will converge on sort of the same ontology.
1:11:24
SPEAKER_00: Which is like, wow!
1:11:26
SPEAKER_00: That's a scientific discovery.
1:11:29
SPEAKER_00: The answer to the solution is in the sky already.
1:11:33
SPEAKER_00: Yeah.
1:11:34
SPEAKER_00: Well, I make no... I've never hidden the fact that I'm a Neoplatonist.
1:11:39
SPEAKER_00: And so, I mean...
1:11:44
SPEAKER_00: The fact that people are starting to think,
1:11:46
SPEAKER_00: oh,
1:11:47
SPEAKER_00: maybe the deeper ontological issues also should be brought to bear on this.
1:11:52
SPEAKER_00: There was a paper recently published.
1:11:54
SPEAKER_00: People are trying to argue for a levels of reality approach,
1:11:57
SPEAKER_00: and they were trying to use ideas from AI and computationalism to do that.
1:12:01
SPEAKER_00: I don't think that's all going to work, but I respect what they were trying to do.
1:12:04
SPEAKER_00: I'm just saying it's not a monolith.
1:12:07
SPEAKER_00: of just techno nerds or something doing their thing for Malakian reasons.
1:12:12
SPEAKER_00: There are a lot of people of good conscience who want to bring philosophical,
1:12:17
SPEAKER_00: scientific,
1:12:18
SPEAKER_00: and ontological,
1:12:19
SPEAKER_00: even theological reflection to bear on this.
1:12:21
SPEAKER_00: And I am talking to them, and there are papers emerging.
1:12:24
SPEAKER_00: So I just want to put that out as sort of maybe the final thing I'll say,
1:12:27
SPEAKER_00: because we don't want to end all sort of dark.
1:12:31
SPEAKER_00: There are real people giving real possibility for real hope.
1:12:35
SPEAKER_03: Mm-hmm.
1:12:38
SPEAKER_01: Definitely.
1:12:39
SPEAKER_01: I think the Malakian forces are always going to be there,
1:12:41
SPEAKER_01: but definitely the philosophical and theological questions that are coming up are
1:12:45
SPEAKER_01: an interesting consequence of that obstacle,
1:12:48
SPEAKER_01: at least that it's bringing some of them to the fore.
1:12:51
SPEAKER_01: Thank you both very much.
1:12:52
SPEAKER_01: This has been brilliant.
1:12:54
SPEAKER_01: Thank you so much.
1:12:55
SPEAKER_02: Thanks, Mahon, for hosting.
1:12:56
SPEAKER_02: I appreciate it.
1:12:56
SPEAKER_01: Thanks, Mahon, very much.
1:12:58
SPEAKER_00: And Sam, it's always