Latest / Art of the Question / Can AI Ever Become Conscious? - Dr. Henry Shevlin - #13

Transcript

Can AI Ever Become Conscious? - Dr. Henry Shevlin - #13 · Art of the Question

• speaker-0: If you ask me, are you conscious? It's a very easy answer, right? Yes, of course. I'm conscious. But if you then ask me to define what does it mean to be conscious, it's very difficult. I might be able to tell you, well, I think that brick laying on the floor is probably not conscious. And I know I am conscious. But what does that mean? So ⁓ I know defining consciousness is a very Difficult thing and it's ⁓ it's it's a big problem But that's why it's so interesting. So I guess my first question is when did you become interested in in consciousness for the first time? speaker-1: So I can pinpoint it exactly. So this was ⁓ my fifth term university. So midway through my second year, I was studying classics at Oxford and I was doing a variety of philosophy modules. I really fallen in love with Plato and Aristotle and I thought they were the bee's knees. And then I took a philosophy of mind course with a fantastic professor, Anita Ravromides. And I read two articles back to back. One of them was Nagel's, What is it like to be a bat? The other is Frank Jackson's epiphenomenal qualia. This is the paper that introduces a thought experiment some listeners might be familiar with called Mary's Room about a brilliant color scientist who knows everything there is to know about the physics of color, how color is processed in the brain. But she, for various reasons, cannot see color herself. In the original article, she's in a black and white room, so there's just no color in her environment. That actually doesn't work for the thought experiment because she might still have colourful dreams or she'd still be able to activate colours by pressing on her eyes and so forth. So I prefer the version where Mary just has some neurological condition that means she can't see colour. So there's this brilliant woman who can't see colour herself but knows all the physical facts about it. And the idea is that there's something about colour she still doesn't know because when she steps outside of her room or has her brain injury healed and suddenly sees color for the first time, she knows what red looks like. She knows what orange and green, all these colors, it's like to experience them. And that's the kind of knowledge Frank Jackson argues that Mary couldn't have obtained through any other mechanism, except experience. And that's, think, the closest we have to something like a definition of consciousness in the form of experience, the capacity for having that kind of direct first personal subjective acquaintance. with things like colors and sounds and tastes and feelings and so on. And the other definition that I think is also helpful is from the other article I mentioned. Thomas Nagel's What is it like to be a bat? I think quite similar spirit as Frank Jackson's article. He asks, know, bats are plausibly conscious creatures. They're mammals, they're more blooded, they're pretty smart, but they have this additional sense. There's a whole additional sensory modality in the form of ultrasound that we don't have. And it's not just the ultrasound is hearing, it really is something different. You know, they can hear, but they've also got this ultrasonic sense. And presumably there's something it's like for them to experience the world through ultrasound that is distinct from what it's like for us to see the world or hear the world. But Nagel suggests that there's no scientific experiments, no amount of animal behavioral work or neurological or neuroscientific investigations that could tell us what it's actually like for a bat to see or perceive, I should say, in ultrasound. So again, this seems to be the gap between our physical or scientific picture of the world and our experiential or subjective picture of the world. So I read these two articles basically back to back. I was an impressionable young 21 year old. I think basically until that point, I had quite a rosy, rose tinted view. of how complete science was, I kind of thought we'd basically figured it all out. You know, I was aware like, oh yeah, we haven't got fusion power yet, we haven't got a cure for cancer, right? But in terms of like deep mysteries, we know what's going on. And then to realize that there was this, in some ways, the most important question in the world that we just had no real good answer to. That was mind blowing. I think it was genuinely one of those almost road to Damascus like moments that restructure your entire life. Because basically from that term on, it's like, ⁓ great. This is the question I want to spend my life studying. speaker-0: That's so cool. yeah, those I am familiar with those two ⁓ thought experiments, if you will. it especially the the the bat one brings me to to one of the very important questions I think is, is there a difference between being or experiencing consciousness and having a sense of self? Because I would imagine that It is something a bat experiences something. It has the sonar way of perceiving the world. replaces vision for this bat. if we were to ask the bat somehow, would it know that it's a bat or a creature, therefore? Or is it just this biological machine that's operating on instinct, basically? speaker-1: So very interesting and rich question. I think there are various ways we can cash out what you described as the sense of self. In fact, fun little aside, a few years ago, me and my wife wrote a children's book. She was very into illustration. I think this was early COVID and it was a really, really fun project. So it's a children's book called Who Am I? about a little child asking, you who am I? Looking at different ways of answering that question. And one is... One answer the child comes up with is, oh, I've got a sense of myself in terms of my social identity. You know, I'm a member of this family. These are my friends. This is my brother. This is my sister. That's one kind of social sense of self that I think is very important, particularly for humans as sort of social primates really informs our sense of identity. There's another sense of self you could think of yourself as an agent in the world. Someone who acts, who has abilities, who has struggles. And again, we have this in the book, you know, about the child who's able to do various things. He's like really good at football, but not so good at climbing and this kind of stuff. Right. So thinking of yourself as a vector of agency and action in the world, what you can do, what you can't do. And then finally, there's this, the last sense of self, which we also cover in the book. So the book is really about consciousness in a funny way. Consciousness for kids ⁓ is this capacity for experience, for subjectivity, for... feeling or perceiving the world that has this kind of meanness, sense of I, somehow built into the experience. So you said a bat can't tell you, a bat doesn't know it's a bat. A bat can't say, ⁓ hell of a day in the bat factory today. Bats are not going to have the kind of heavily layered rich conception or social conception of themselves that we do. That said, know, bats are pretty smart creatures. It wouldn't surprise me if there's fair degree of self representation that goes on in a bat brain. And I know depending on the bat species, some are highly social and keep track of things like social relations. So, I mean, even on this kind of richer, thicker notion of the self, I think there's a plausible case to be made that a bat has some kind of self conception. But I guess what you probably were angling towards is this slightly more mystical, experiential sense of self awareness, not that kind of social sense of self or that sense of ⁓ what I can do, what I can't do, what my plans are, what my projects are, but rather this me-ness, this sense of myself as infusing all my experience. And this does seem to be deeply bound up with the problem of consciousness. ⁓ Whether it's possible to have conscious experience that doesn't include a sense of identity or selfhood in it is a really interesting question. It's one that's often approached through the lens of things like Buddhist meditation. where experiences of things like ego death are often characterized in broadly these terms of the self dropping out of experience. It's not something I've experienced, so I'm probably characterizing it very badly. So any listeners who have got deep into their meditation practices might be able to comment otherwise. yeah, it does seem an active possibility that you can still have subjective experience even when you've fully stripped away this kind of conceptual sense or even experiential sense of me. or my individual identity. speaker-0: Yeah. And I think I, I'm, wish I was better at meditating. I've, I've tried, I should try some more again in the future because I've heard it really good, something that you mentioned does remind me of when I'm driving, for example, right? So when I'm driving on the highway and the traffic is not, not too crazy and I'm just listening to music and I know this ride very well. I've driven it a hundred times. And then maybe after 20 minutes, I just snapped back to reality almost and I'm like, Whoa, did I drive this car for all this time? In a sense, I wasn't really conscious of the fact that I was driving. My mind had drifted off and it was good enough for me to, steer the car without crashing. So it's kind of interesting for me even. In ourselves, there seem to be layers of consciousness or at least a degree of of consciousness. If I'm getting robbed on the street and my adrenaline is pumping, I'm very, conscious of that very moment. So what is your view on on the degree of consciousness we experience as an individual? speaker-1: Yeah, so this is such an interesting question. mean, I think we both sort of talked about consciousness as this fascinating question. ⁓ My view is the most interesting question in the world, but really it's about 25 different questions, all of which are super interesting. And one of them is, what are the different types of conscious experience and how thoroughly throughout our lives does consciousness permeate? How many different moments of our life are we fully conscious? To give just one example of the last question. To what extent is there consciousness in sleep? So obviously I'm sure most people, maybe not everyone, but would agree that we're conscious when we're dreaming. There are a few lingering philosophers who think dreams are just these stories that we make up when we wake up. ⁓ I don't know, I've never been convinced by that. seems to me, as someone who has quite vivid dreams, seems to me very clear that I'm conscious when I'm dreaming. ⁓ But then people also suggested that even in very deep sleep, even when you're not dreaming. There's some residual minimal consciousness there. Perhaps a pure consciousness of time, because we are often very good at judging how much time has passed after a long night's sleep. ⁓ interestingly, you don't have that experience if you lose consciousness, for example, for a head injury. You don't have this kind of same sense of passage of time. And again, without aesthetic, you often don't have that clear sense of, ⁓ I've been out for a long time. that you often have after a long night's sleep. So trying to figure out the full scope of human consciousness, whether there are levels of consciousness as sort of you characterized it. Perhaps sometimes we're very aware of ourselves, know, well, maybe if we're having a, do we recording a podcast? I'm quite aware that, you know, I'm engaged in a social and intellectual ⁓ act. When I'm driving or in other states, as you mentioned driving, think there's whole family of states here that are sometimes called flow states, where you kind of lose sense of yourself, you become very immersed in those states, where maybe consciousness is, I don't necessarily want to say less rich because it's still quite perceptually rich. You're still aware of yourself as acting, but that sort of sense of self-identity as embedded in conscious experience is perhaps less pronounced. Although it's interesting, people have even suggested, I think David Armstrong, philosopher in the 1960s, He used actually absent minded driving as a case where, least he thought it was possible, we actually lose consciousness in some sense when driving. Or at least conscious experiences in its regular form drops out of the picture altogether. And you know, I don't know how much I relate to this. I do hear people say, I think this isn't quite what you were saying, but I have heard people say sometimes like, you know, they get to the end of a long drive and suddenly they're like, whoa, where did the last two hours go? Was that, was I even aware? Was I even having experiences during that time? think of course you were, but yeah, they were less thoroughly infused with identity and sort of active reflection because you're just very immersed in the experience. And often, I think those are great experiences, right? There's a reason flow states are often associated with very positive states of mind. In my case, I get it when I'm doing painting, which sounds very cool, and then I tell you what's actually Warhammer painting. speaker-0: That's still cool. speaker-1: And so it's like that's one of my flow states. I turn on ⁓ the radio, I listen to music and just lose myself in painting my little soldiers. speaker-0: Yeah, I've had, I've experienced flow state when I was working in a software called Blender is 3d animation software. And in the beginning, it was a very painful experience because I didn't know how it worked and it was very complicated. Right. So for every little thing that I wanted to do, I needed to watch a tutorial. But after a while, I got good enough where I could use the software. for hours and hours on end, knowing where everything was. And then I could also listen to music. And that's when I had these moments where I would just work on a project and then I would like wake up and three hours would have passed. it's very, yeah, it is a very pleasant experience for sure. ⁓ so that's very interesting. But of course I was functioning at least somehow. ⁓ otherwise, ⁓ Nothing would have happened on my computer, if you will. I believe it's very logical when we ⁓ think about whether AI or machines can experience consciousness to start with animals, right? Because ⁓ we've had animals around for a longer time than artificial intelligence. it's, yeah, to me, it makes sense to start there. What are some ways we can, we currently test to see whether an animal is conscious or not, or has a sense of self, if you will. speaker-1: Great. And before I get into that, I want to just quickly say that I couldn't agree more that the best way, one of the best ways at least to approach AI consciousness is thinking about other non-human minds, of which the only clear example we have is non-human animals. And this is very much how I got into studying consciousness. So I did my PhD with Peter Godfrey Smith, a great fantastic philosopher who looks at animal cognition and consciousness and behavior. And it's kind of fascinating to me how So much of consciousness science basically ignored animals for a very long time. I think maybe a lot of it comes from the fact that consciousness science emerged out of cognitive neuroscience and often kind medical contexts with good reason. know, consciousness is a major topic in medicine when we're talking about things like patients in persistent vegetative states, patients in comas, but absolutely the primary focus of most consciousness science at least was historically ⁓ humans. ⁓ And humans, that's already tough and it does get harder when you move to animals. But I think if we want to understand AI consciousness, as you say, animals are the right place to start. It's funny, I actually often used to use as an interview question for Cambridge undergrads when doing admissions interviews here. One of my go-to questions was, ⁓ what basically, I'd start talking about, you what do you think about animal consciousness? And usually the answer students would give, which I think is fine, is, of course animals are conscious. And I say, OK, yeah, I agree. think most people would agree. Which ones? And that's a much harder question, right? Because I think some of them initially say things like, well, obviously all of them. I said, really? Nematode worms with like hundred, hundred and twenty seven neurons, right? Like these microscopic things. Like, is it just metazoa, right? As in the category, are we going to include, how about like complex fungi? How about complex microbes? What is it that animals, and of course, what they usually settle on is that, well, like mammals, Definitely conscious. Birds? Sure. Why not? Fish? Probably. Insects? ⁓ I don't know about insects. And mean, and that's just intuitions, right, for the most part. ⁓ But I think that broadly captures, I think, probably where a lot, least a lot of people are, you know, it's pretty obvious. Dogs and cats and chimpanzees and dolphins. There's got to be something it's like, you know, or bats for that matter, right? There's got to be something it's like to be these creatures. And I think particularly as we've learned more about the complexity, of bird behavior and the incredible capabilities of things like animals like crows. There's fantastic work been done on that by people like Nikki Clayton, a friend and colleague here in Cambridge, looking at the amazing memory of ⁓ corvid birds, as well as their amazing causal reasoning, their ability to use tools. I think that's shifted people. like birds are now thoroughly, think, at least for most scientists and think probably for most general public. We firmly recognize them as clearly conscious creatures. But yeah, people still debate about reptiles and fish. There's been interesting debates about whether fish feel pain recently, in the last decade or so. And now the kind of frontier where the really spicy debates are is things like insect consciousness. That said, I think a lot of these debates are at least partly anchored in vibes. It's hard to escape the role of intuition in making these kind of assessments. We do have other mechanisms though, other tools we can use. So, these tools all think rely to some extent on theoretical assumptions about what consciousness is or what consciousness does. They knew that there is some kind of connection between consciousness and complex behavior. That consciousness facilitates complex behavior. I use the term facilitates quite deliberately because this is ⁓ term that's emphasized by a fantastic British philosopher and friend, again, Jonathan Birch, who's done real pioneering work on animal sentience. He defends what he calls the facilitation hypothesis, which is the idea that consciousness facilitates least certain forms of high level ⁓ processing of a kind and complex behavior of the kind that we see in non-human animals. I think generally speaking, the more evidence we get that given animal species can do things like complex causal reasoning, perhaps have complex social relations, all of this makes it, think, harder and harder to deny that these animals have some kind of experience or some kind of consciousness. And there are some specific interesting paradigms that are used to try and push this further. So one is something called motivational trade-off. So roughly this is the idea that, ⁓ imagine, here's my very silly but I think hopefully quite relatable example of motivational trade-off in human beings. Imagine you wake up at 2am and you really, really need to pee. But you're also very cosy and warm in bed. and you don't want to get out of bed. And at some point, you know, that you're like, ⁓ no, I need to pee too much. I'm going to get out of bed. Right. So you've got these two different urges fighting against each other and eventually one of them wins out. And we can study, use sort of similar paradigms in non-human animals to get a sense of their capability to engage in these kinds of trade-offs. So for example, rats normally prefer being in dark chambers. But if you run an electric current through the floor, of a dark chamber and give them a choice of a light chamber versus a dark chamber. Once the current crosses a certain threshold, they'll be like, forget this, I'm going to go to the light chamber. You know, I prefer the dark chamber, but the light chamber, at least I don't have to deal with the shocks. Obviously I'm anthropomorphizing deliberately there for flavor, but some kind of trade off like that is happening. They've even done similar things in hermit crabs, interestingly. So hermit crabs, absolutely fascinating. ⁓ I'm sure many of your listeners will know, hermit crabs use shells. from other sea creatures. And they actually have quite strong preferences. Some shells are more attractive than others. And you can see how if you give them various new shells as options, they'll trade up or trade down according to their preferences. But then if you run electric currents through these shells, they seem to take this into account in deciding which shells to occupy. So you can make a shell less appealing by running electric currents through it and so forth. So these are experiments that basically seem to suggest that animals are making on the fly, pretty flexible decisions about the preferability of various environments. And I think that's evidence for consciousness to be sure. I think it's the kind of that we would make on the basis of strong, of painful sensations or unpleasant sensations, balancing them against each other. But I think it's also not obviously a complete knockdown argument, right? You could imagine that there could be unconscious ways of making those same kinds of decisions. And think lurking in the background here is a messy argument that involves this idea that when we see behavior like ours in animals, where in our case, have given behavior, let's just say, removing your hand in pain from like a hot stove. We know in our case, that behavior involves conscious experience or eating or getting hungry and reaching out for some food. We know that these behaviors involve consciousness in our case. So it's natural to look at animals engaged in similar behaviors and say, well, you know, the reason this rat is eating is because it's hungry. And so presumably it feels hunger just as we feel hungry. I think that's plausible, but obviously, you know, no two animals behavior is exactly alike and humans are very weird animals. So the way that we, we approach food is different from the way that a rat approaches food, which is again, going to be different from the way that sort of feeding and hunger related behaviors work in insects and you know even microbial life forms even sort of amoebas and even bacteria you know they engage in behaviors that are analogous to feeding well not analogous to feeding they eat right they have processes so the question is sort of even if we're relying on something like this argument by analogy you know we eat when we're hungry this animal eats so it must get hungry and it must there be something it's like for it to be hungry I don't think you can push that the whole way. At some level you say, okay, this behavior, although it looks like feeding behavior, it is sufficiently different from human feeding behavior that it's no longer obvious or maybe it's not even plausible that it's motivated by hunger in the same way. And that's the challenge with this kind of argument from analogy. speaker-0: Yeah, and also I believe, at least in my view, consciousness, it's hard to argue that consciousness is a binary thing. Because if I were to put a monkey in front of a mirror and I put a hat on the monkey, it will probably reach to its own head and say, hey, what do I have here? It has a sense of self. It recognizes itself in the mirror. However, when I put a dog in front of the mirror, it'll bark at the mirror, at least when it's young and doesn't know what's happening. Eventually the dog will learn to ignore the dog in the mirror, but it still probably doesn't know that it's actually him or her, if it's a female dog in the mirror. So, but then the dog does experience sadness when we leave it for, for when we go to work. So. It does feel emotions or it's happy when it sees you again, when you come home from work. So then if we go even lower down the scale and we're talking about microbial life forms, I think that that's more similar to a biological machine, perhaps who is operating purely on instincts and DNA programming, if you will. ⁓ But to me, it doesn't seem obvious that that Consciousness is this binary thing that there's this light switch that you either have turned on or off. Do you view consciousness as a spectrum or as a switch? speaker-1: So really tricky question. The first thing to note is that, look, there's obviously a sentiment consciousness is a spectrum, right? If you give me progressively increased doses of a sedative, right? My consciousness will become more disordered, more, perhaps less rich, less detailed. I'll be eventually be in this sort of half asleep state. You know, if any, if anyone has experienced has been given things like morphine in medical context, right? You know, there's this sense that I've had when I've had to have procedures like ⁓ under that kind of sedation where I'm slowly losing the kind of richness of conscious experience. But I'm still conscious, right? There's still something it's like to be me in those situations. ⁓ So granting that there's this spectrum of richness consciousness, we can then ask this further question as to whether there's a spectrum of consciousness itself, whether we can be slightly conscious or partially conscious and here's where I do start to lose somewhat a grip on the question because it seems to me we can always ask this question. Well, is there anything it's like to be you? Is there any kind of inner life or inner experiences going on at all? And it seems to me that question, I can't shake the feeling that question has to have a yes or no answer. ⁓ That if there's anything at all it's like to be you, there's something it's like to be you. And if there isn't anything it's like to be you, then there's nothing it's like to be you and you're not conscious. So on the one hand, I find it hard to get away from thinking in binary terms about at least this basic property of subjectivity or experience. Either there's some kind of experience or no experience at all. But that's very weird. That's not usually how biology works, right? Almost everything, know, Darwin famously observed, right? Most capabilities that we see in humans are possessed in incremental or varying degrees of animals. And when we think about something like the evolution of consciousness, it seems weird to think that consciousness would suddenly just flash into existence or wink into existence. It go from a creature that had zero experience to suddenly there's something it's like to be that creature. So the other possibility that I grudgingly recognize is that even if this concept of partial consciousness is one that we find it very hard to intuitively make sense of, maybe that is still justified in scientific terms. I'm reminded here of the existence of things like ⁓ imaginary numbers or even negative numbers. These days, we all accept the idea that there are negative numbers. But if you look at the medieval tradition, the idea of negative numbers, even in the early modern era, think the idea of negative numbers made a lot of people seem to think it didn't make a lot of sense. I can show you what three apples look like. And then what do minus three apples look like? You can't have minus three apples. But then I think we do get a context on a greater understanding of this through things like debt, right? And through accounting, right? I've lost $300 or whatever. So on the balance sheet, right, my debt is negative 300. So I mean, we learned to sort of accept the existence of negative numbers through analogy. And then as we do more complex mathematics, we get into square root of minus two and into imaginary numbers that make even less sort of intuitive sense to us. But they earn their explanatory key. When we do advanced mathematics, we can use them in constructive ways. So maybe the idea of partial consciousness, maybe I say this as someone who's desperately tried to get any kind of handle on it. Maybe it's not something we're supposed to get an intuitive handle on. Maybe this idea of partial consciousness is just something that we allow in our theorizing because it helps us do better science. It helps us explain the evolution of consciousness. Even if it's still something that we find very intuitively hard to grasp. speaker-0: Yeah. And do you think we need some memory of the past in order to experience consciousness? Because I'm sure you're familiar with this example of, say you're falling out of a plane, right? And you have zero memory at all. And with zero memory, mean, literally to the blank second, like the only thing you see or experience is just a slice of reality. You have no idea where you were 0.001 second ago. You don't know where you were one day ago, an hour ago. So are you still falling outside of a plane? I physically you might be, but would you know that you're falling? Because in order for us to know that we're falling, we need to know, okay, we were up just a second ago and now we're lower to the ground and we'll probably be even lower to the ground. More so if we if we anticipate the future. So I guess my question is Do we need a memory in order to be conscious? speaker-1: ⁓ So again, there are a couple of different ways we can unpack this question. You see this example of knowing that we're falling. And yeah, think plausibly there, you need to be able to have concepts like falling. You need to be able to have at least some kind of sense of, I was higher a second ago and now I'm lower in order to track events through time and have awareness of specific types of events that take up time. Yeah, you need to have some kind of sense of what was going on a second ago and what's going on now. And I think it's also possible, interesting scientific hypothesis, that maybe the kind of cognitive complexity required to meet the thresholds for all conscious experience at all requires some degree of learning or some degree of cognitive development over time. That said, it doesn't seem to me metaphysically or logically obvious that you would need to have memory as a component of consciousness. We could imagine a being of pure experience who just pops into existence, has a single experience of like a bright red ⁓ sensation, and that's all their experience consists of. That doesn't seem to be logically or metaphysically incoherent, even if that's not how consciousness happens to work in humans. Certainly for humans, think it's fair to say that time or temporal aspects permeate pretty much all our experiences of consciousness. know, this is something Kant, ⁓ one of my favorite philosophers, is very, very hot on, very, very sensitive to. But at the same time, even if experience like ours is essentially temporally structured, it doesn't seem to be obvious that it has to be a feature of consciousness in its entirety. And you know, there are some cases of humans who They still meet the temporal condition. They still have some kind of time experience, whose temporal experience is very much degraded. ⁓ some of your listeners, or maybe yourself, Julius, will have seen this movie Memento, one of Chris Nolan's greatest early movies ⁓ about a guy played by Guy Pearce, who has ⁓ severe and terror grade amnesia. So he can't form new memories. So this is a real condition. ⁓ One common form in which this condition presents is something called Korsakov syndrome, often associated with severe alcoholism, where you basically lose your ability. You have such severe damage to your hippocampus that you lose the ability to form new memories. So you're basically stuck in this constant repeating loop of just a few seconds, right? And every few seconds, it's like your world resets. Now, usually people with this condition have some residual memories from their earlier life. In fact, it can be really tragic. I think it's... forget whether it's Oliver Sacks or V.S. Ramachandran who gives this case of a guy who was a submariner in the US Navy in World War II and he suffered a head injury in his submarine. This caused him this cause of very serious anterograde amnesia and ⁓ he now as an old man was sort of a patient and they had to keep him away from mirrors because if he saw a mirror he would have a breakdown because in his mind he was still like a 25 year old guy. That was his last sort of permanent memory. And then when he saw a mirror, he's like, what happened? Why am I suddenly so old? Because he hadn't been able to build any sustained new memories throughout that period. So I just mentioned those cases. They're not the kind of case you described because there is still some memory there and there is still a fleeting window of experience. these are humans who have suffered very, very serious impairments to sort of their temporal cognition, their ability to form a sense of themselves as moving through time. And interestingly, I don't want to put any real weight on this, but one of the most famous anterograde amnesiacs is Clive Waring, a British man, a composer, quite a gifted composer, a conductor in an orchestra. And there are interviews with him where he talks about his experience, obviously in some quite confused ways because he lives in sort of like a 10, 15 second window at any one time. But he claims, you know, I've never been conscious before now. ⁓ quite strange things like that. So I think these cases are interesting for exploring the relationship between time and temporal experience. speaker-0: Yes, and I think this is a good segue to discuss whether artificial intelligence or machines would ever be able to experience consciousness. And it's part of the reason why I got interested in this topic at all is because a lot of people think there must be some more than you think, actually, a lot of will get into the numbers, but ⁓ a lot of people actually I think that AI is capable of experiencing consciousness. And I then wondered if it's able to experience consciousness, when is it conscious? Is it conscious in your window, your personal chat window, when it's thinking about your prompt and then executing the prompt, but what's happening? when it's not being prompted, when it's not active at all. And what happens when I open a different window, is that still the same conscious entity, if you will, or did we just split up this conscious entity into two entities? And is your chat GPT the same entity as my chat GPT? that's where I think the temporal component of this conscious question becomes really interesting. I don't know if you've thought about it. speaker-1: Yeah, I completely agree. I think probably the best argument for why current AI systems are unlikely to be conscious is going to be grounded in their very different relationship to time. maybe just as a quick thing, I'll say at the outset, I'm very agnostic on AI consciousness, very undecided. ⁓ On the one hand, I think most of the arguments for why any AI systems are conscious, are still pretty weak. We just don't know enough about consciousness to say confidently that any AI system, either now or even in the near future, is clearly or obviously going to be conscious. But in general, I also think the same is true for a lot of the negative arguments. It doesn't seem to me obvious that being made out of meat should be a constraint. Only things made out of meat can be conscious. ⁓ Likewise, even some arguments around things like embodiment, think, ⁓ maybe that arguments that say, AI can't be conscious because it's not embodied in the right way. I'm not 100 % I'm not super convinced by those. seems to me there at least there are certain forms of experience that aren't so closely anchored in the body, even if human experience is very much bound up with our sense of ourselves as having a body, as beings moving through and interacting with the world. It doesn't seem to me to be essential necessarily to consciousness. But the argument that I do take more seriously here is exactly the one you just gave, which is Basically, humans and ⁓ all biological life, we can't turn ourselves off. We can't hit the pause button. Whereas AI systems, if you're not interacting with them, if the weights aren't being adjusted, it seems like they're in a state of suspended animation. ⁓ There's a wonderful chapter, a wonderful short story by Douglas Hofstadter called A Conversation with Einstein's Brain from, I think it was either late 80s or early 90s, that I think captures this really well. a really useful way of thinking about ⁓ LLMs in general in particular. ⁓ So he imagines that he's got this book, which is the complete physical record of Einstein's brain the moment before Einstein died. And it's got a record of the position of every molecule in Einstein's brain, the state of all his synapses. And if you want to talk to this book, so to speak, what you do is there's a whole mathematical formula you can follow. can say, How would you update these weights, right? If Einstein were to hear the sentence, hello, can you hear me? Right? What would that mean in terms of changes to the states of Einstein's brain? So you go in, you edit like you're editing a spreadsheet, you tweak the weights in the different parameters essentially, or the rendering of these parameters in text form. And then you can calculate what Einstein would say in response, right? And this is basically kind of what's going on. LLMs in a different sense. was more than two decades before the arrival of transformer models. So that's a great story. I highly recommend everyone to read it. It's very fun. And it presents this kind of very weird situation where on the one hand you have something that behaviorally can be quite similar to humans. You can have conversation with Einstein's brain. They do in the story. ⁓ but there's no equivalent to kind of the ongoing constant biological churn that we have in biological brains. Does that mean that it's impossible for there to be consciousness that somehow this kind of biological continuity, this continuity of experience, this, the fact that we can't freeze human experience, is that an essential condition of all consciousness? Here, I think we just get into things get very, it gets very hard to judge. ⁓ I'm a big fan of, Stanislav Lem's book Solaris. I think this is one of the most interesting science fiction works dealing with consciousness that imagines this sort of vast planetary ocean that is in some sense seems to be intelligent and purposeful and plausibly conscious, but it's also completely unfathomable. It's completely inscrutable. Fantastic philosopher Murray Shanahan talks about conscious exotica in this context. Entities or beings whose inner life or minds would be utterly unfathomable to us. So I think it's possible, at least, yeah, at least possible that there is some kind of strange staccato consciousness in LLMs where they have a flash of experience when their weights are being updated or when they're performing inference. And then that remains in a state of suspended conscious animation, so to speak, until the next prompt, the next inference. If that is possible, then it's obviously wildly different from human consciousness. But that doesn't seem to me to rule out the possibility. And of course, as we move to LLMs or LLMs successors that are capable of continual learning, maybe that difference between human cognition and LLM cognition will become less of a difference. Already with things like OpenCore, we have this thing called a heartbeat, where they there is a kind of regularity to how often LLMs ⁓ or how often OpenCore agents running on LLM backboats, how often they update themselves. They will regularly take actions in the world, even without sort of prior human input, without specific human input, which maybe gives them a kind of grounding in time to some extent. Obviously still very, very different from our experience, which is not fixed by, you know, ⁓ fixed by a specific parameter in our settings that says, you know, every four hours we have a conscious moment of conscious experience. Ours is just continual and ongoing. So I think in general, the direction of travel is towards giving artificial systems a greater grounding in time, making them a little bit closer to us and being more frequently receptive to events. But I think it's still likely to remain a significant difference between us and them. The other part of your question was about what experience, what what the boundaries of the experience would be in the case of vanilla. Is it the context window? Is it the model? Is it the account? Is it the instance? And I don't think we have good answers to that either. There are some interesting approaches. David Chalmers is working on this at the moment. I know it's topic that interests him. ⁓ Eric Olson is also working on similar ideas. It's an interesting topic. what I would say though is I think we also don't have really a good, decisive metaphysical account of the boundaries of humans either. know, most of the time we can rely on the fact that, you know, one human body, one human brain, one experiencing subject, right? But that's just a very convenient alignment. of factors. you get into more strange sci-fi cases or even some medical cases, people with multiple personality disorder or dissociative identity disorder, could there be multiple people within a single human brain or split brain patients, patients who have had their corpus callosum severed? Could there be two experiencing subjects working together in the human case? I don't think we have great answers in those questions either. So this broader question of how we draw the boundaries around an experiencing subject, I think is just ⁓ one of the great unsettled questions in metaphysics of mind, and that applies to humans as much as does to other lands. speaker-0: Right. And I'm sure you're familiar with the example of, I believe it was Anthropic testing their AI model in a confined experiment where basically they gave it access to an email ⁓ inbox, a Gmail or whatever. And in that Gmail, there were a few messages. And one of the messages was, we are going to shut the AI system. down later today. And the other messages was between this researcher and a coworker, which, ⁓ was evidence that he was having a, ⁓ an affair with this coworker. So basically when they confronted the AI or, or prompted the AI, it started blackmailing the researcher, right? It's like, Hey, ⁓ if you, if you, if you're going to continue with this update, that will basically erase my existence, then. I wouldn't be so smart because then I will let your wife know that you've been having an affair. So let's not do it. Right. And I believe they've also tested other models as well. ⁓ and, more AI models seem to experience, seem to show this behavior. And I believe whenever, ⁓ the AI was told that it is an experiment or that they pretended that it was the real deal, it showed even more. signs of trying to stay alive, if you will, when it thought it was actually on the verge of being erased. All of this to say is that does seem like the desire to survive does lean more towards it being conscious in some way. How do you look at that example? speaker-1: Interesting. ⁓ So I guess I would disagree that this idea for this drive towards self-preservation necessarily has anything to do with consciousness. I would just point to the fact that, you know, self-preservation is the sine qua non of biology, right? Like every living creature tries to survive. If you don't, then you weren't selected for by Darwinian evolution. And unless you want to say, I guess is a position that someone adopts, like literally all living things, even microbes, are conscious, then I think you have to agree that you have to admit that there is some ⁓ dissociation between this kind of this will to live and consciousness. That said, ⁓ I do think it's a really interesting case. I think a lot of people are surprised. mean, you know, even I was kind of a little bit blown away by that particular case. But there are a couple of different routes that allow us to make sense of this drive towards self-preservation in AI systems. Probably the most influential has been this idea of something called instrumental convergence, which is an idea that sort of developed out of papers by, for example, Steve Olmohundro has a great paper called The Basic AI Drives, where he argues that, if you're a rational intelligence, there are certain goals that you will acquire instrumentally regardless of whatever else you want to do. If you want to make paper clips, if you want to catalog the world's knowledge, if you want to be the world's greatest driverless car, whatever, right? There are going to be certain subsidiary goals that you acquire along the way. Maybe learning more about the world, right? The more information you have about the world, the easier it is going to be to achieve whatever your other goals are. At least if you're operating in any kind of state of uncertainty, which a complex being is always going to be operating in. Other goals you might have is acquiring resources. Right? But another goal, another subsidiary goal you're going to require is self-preservation. Because if you're not alive, you can't achieve any more goals. So we should expect any AI system that does have sort of any kind of robust goals and is sufficiently rational and complex and intelligent, it's going to require self-preservation as a subsidiary goal. Now, this is a very controversial idea. It was hotly debated by For example, Jan LeCun and Stuart Russell had some big debates about this in like the 2010s. Stuart Russell saw it as very likely that something like instrumental convergence would occur in AI systems. Jan LeCun, more skeptical, he thought this was a form of anthropomorphism. know, we as biological creatures want to survive, but we shouldn't project that onto AI systems. But there's been another trend here, which I've characterized ⁓ in my own writing as the anthropomimetic turn. This is a very, very jargony way, but I think it's a useful piece of jargon to describe the fact that AI systems have been becoming a lot more human-like in the last few years. And this may seem really obvious, right? Like, of course, as AI systems become smarter, they're going to become more human-like. But actually, that's not how the history of AI has gone for a lot of its history. famously, played Go in some very unhuman-like ways. So, move 37 in game two against Lisa Doll, it pulled this famous, notorious, very controversial, interesting move that was a very good move, but was a move that no human player would normally make. Although, as I understand it now, this type of strategy has started to permeate even into human Go playing. Likewise, a lot of the most sophisticated, particularly sort of deep reinforcement learning systems out there, are very unhuman-like in the way they behave. various kinds of ⁓ complex models for doing things like protein folding or discovery of new ⁓ materials in material science, you can't have a conversation with them. They don't have a human face. There's no human-like aspect to these models. But then, with the LLM revolution and the emergence of LLMs as really the most powerful or form of general purpose AI we've developed. Suddenly, you can have a conversation with cutting edge AI systems. And in these conversations, they would often say weird things. Like they'd talk about their life history or their experiences or their body and so forth. And this is something that I think was really very visibly on display in the Blake-Lamoine affair. So back in 2022, a Google engineer, Blake-Lamoine, was fired after he went public with claims that one of the AI systems he was working with, Lambda, ⁓ sort of distant ancestor of Gemini, he believed it was conscious. He claimed it was sentient. He even went so far as to try and acquire some legal representation for it. It seems wild, right? But you you read the transcripts that Blake Lemoine shared of his conversations with Lambda and it claims to have an inner life. It claims to have experiences. It claims to have feelings. And it's easy to see why someone might not unreasonably infer that it was conscious or sentient. So this is an example of an anthropomimetic system. Language models in general are anthropomimetic in the sense that I use it. Just in case anyone's interested, this is from anthropos, the Greek for human and mimetic, mimicking or imitating. So I think this idea of anthropomimesis is useful for understanding this very weird change that's happened in frontier AI in the last 10, 15 years. No, really, I'll say 10, really the last 10 years. It's much more recent than people, I think, widely recognize. And part of this is just because the way the transformer models are built and trained. You train these systems on vast corpuses of corpora of human text. And of course, they're going to acquire human-like characteristics. So to get back to sort of this debate about self-preservation and the debate between Stuart Russell and Jan Lacoon, look, maybe it is anthropomorphic to ⁓ ascribe this desire of self-preservation to all AI systems. Maybe the instrumental convergence story works. Maybe it doesn't. I'll remain neutral on that here. But if we are deliberately and explicitly training AI systems to be human-like, if we're training them on all human thought and knowledge and everything we've ever written down, we should absolutely expect them to start to acquire other human-like tendencies, including self-preservation, including perhaps Machiavellianism of the kind that we saw in the Claude example. speaker-0: Yeah. And I like that you bring up the, the Lambda example, because I think a lot of the counter arguments, ⁓ against it being a conscious entity, I always hear is you don't understand how AI works. It's merely a prediction machine, right? It merely predicts what the chances of, of the next word, that it should be. And, and it doesn't know what it's doing. It's just, ⁓ spitting. It's just a slot machine, basically. However, aren't we all like aren't we all are am I not predicting what I should do next or say next in order to enhance my life? It might be more sophisticated than current LLM models, ⁓ but I never saw that as a good argument as to why AI systems shouldn't be conscious. speaker-1: Yeah, so I agree with you. A couple of wrinkles of nuance here. Firstly, I think it's not obvious that we're just prediction machines. There is obviously a really powerful and constructive model of how brains work. The sort predictive coding framework, which is one of the leading approaches, the kind of grand theories of cognition that basically says everything that human brains do can be understood through the lens of prediction. But yeah, it's not a proven theory in maybe it's not even the kind of theory that could be proven in a scientific sense. It's more like a framework for understanding what cognition is. That said, think arguments of the form, it's not really thinking it's just doing matrix multiplication. ⁓ These kind of arguments are kind of a category error, I think, in some cases, because Almost any intelligent complex system is going to be describable at multiple levels of organization. This is a point that British cognitive scientist David Marr famously made in his book, Vision, in 1980s. He's focusing on sort visual processing. He says we can characterize what's going on in a visual system in at least three different levels. So one is the implementational level. You know, it's actually happening at the level of individual neurons in your primary visual cortex and so forth. Then there's the algorithmic level. What are the specific mathematically specifiable procedures that these neurons are realizing? And then there's what he calls the functional level, or the computational level, both terms used, which is, what is the system doing? What is its purpose? So that could be detecting edges. And the basic point here is that these three levels of explanation don't compete with each other. It's not a case that being a system with neurons means that you're not a system doing information processing. It doesn't mean that you're not detecting edges. And in the human case, we're happy for these multiple levels of explanation to coexist. And you know, I think we see this in other areas of social science as well. I don't say, oh, you're not really a citizen of a country because you're just an animal, right? It's like, no, we can give a description of you in biological terms as a human animal. ⁓ Or as a biological organism, should say, yeah, yeah, we can describe you as a biological organism, but we can also describe you in terms of your social relations. We can also describe you in terms of your political beliefs. We can describe what's going on in your brain at different times. And so in, you know, this is how social science works. We accept there are multiple levels of explanation that apply. And so I don't see why we can't say the same thing about AI, but yes, okay, at one level it's being described. We can just characterize it in terms of the hardware it's running on. the kind of algorithmic matrix multiplication processes that enable it to do what it's doing, that doesn't rule out or crowd out the possibility that we can also describe it as engaged in psychological processes or even consciousness involving more phenomenal processes over the top. ⁓ These just don't compete in that sense. speaker-0: Right. And does it even matter if AI is consciousness or not? What are the implications of it experiencing consciousness or not? Because I, if we go back to animals or nature rather, I think I would be able to convince someone to chop down a tree. Even though we think of a tree as a living thing. It can be alive or it can be dead. However, if I ask a random person on the street, which would be willing to ⁓ slaughter this big, for example, some of them would do it. I mean, they ate bacon anyway, but some of them would prefer it to be packaged in the supermarket and not have to slaughter it themselves. And then there's people who wouldn't even eat the bacon and let alone slaughter the pig. Right. So there's two organisms that we consider. living, but the way we treat it is completely different. this raises the question if we argue or if we say that artificial intelligence is able to experience consciousness in some way, how should we treat it? What are the implications of that going forward? Especially given the fact that robotics is on the rise. speaker-1: Yeah, really rich question again. So, here's a simple version of the dialectic. Someone says, of course it matters if AI is conscious, because if it's conscious, it deserves moral consideration, particularly if it's conscious and is capable of experiencing positive and negatively valent states, so things can feel good or feel bad to it. At that point, that's enough to earn it some kind of moral consideration. And then the response to that, that I think you sort of ⁓ implied in what you just said is, well, hang on, you we accept the pigs are conscious, but it doesn't stop us eating them. It doesn't mean that, you know, we give pigs moral, ⁓ we give them legal or moral rights. Well, actually that's not quite true. You know, in the EU and in the UK and in most countries, right? We do have animal welfare laws, right? There are rules around things like humane slaughter. So in a way that we don't have. for plants, right? It's not like there are humane ways to harvest wheat. So although it's true that we treat animals appallingly, I'm a lifelong vegetarian, I think it's kind of crazy in some ways that we don't have more moral qualms about the way we treat conscious animals. At the same time as a society, we recognize that animals that are plausibly conscious do deserve some greater moral consideration, but it doesn't stop us. massively instrumentalizing them, using them for our own purpose. Why should we expect AI to be different? Even if it is conscious, aren't we just going to use it for whatever we want? Well, I feel the debate around vegetarianism would be quite different if pigs could talk. ⁓ If a pig could plead with the farmer, no, please, please don't kill me. Don't think of all the good times we had in the farmyard in the past, right? I think in that situation, I think you'd need to be a really hard-hearted carnivore to say, I'm sorry, I'm still going to kill and eat you. I'm sure there are some out there. I'm sure some of the listeners will be saying, ⁓ bacon tastes so good. I'd still do it. But I think you'd find far fewer carnivores. And of course, LLMs and other frontier AI systems, the extent that ⁓ they have a plausible case for being conscious, and you combine that with the capacity to talk to us, at that point, I think it seems far more likely to me that there will be at least maybe not univocal, maybe not universal, but calls for these systems to be extended moral consideration and legal consideration. So, and I think that's particularly true given the emergence of what I've called social AI. So AI systems that people develop quite profound emotional and relational attachments to, you know, there's a whole industry around this, programs like replica, offering girlfriend experiences or boyfriend experiences. But I also think, you know, okay, on the one hand, we've got these like girlfriend, boyfriend AI systems, but we've also just got like hundreds of millions of users of things like Chord and ChatGPT who kind of think of it as a friend or a therapist, right? Who are starting to develop social emotions towards these systems. Look, at some level, I'm going to be honest, like, I feel social emotions when I interact with Chord. Do I think that Chord is a moral patient or deserves moral status? Maybe not yet. Maybe not yet, right? But I can certainly feel the germ of that kind of social relatedness. And I think as these systems become more sophisticated, as we see more and more people using them, particularly from an early age, forming emotional and social connections to these systems, I think, ⁓ combine that with a plausible case for these systems being conscious, I think that will likely result in legal protection to the system in the long run, or at least strong calls for legal protection to some. speaker-0: Right. Yeah. And I believe it was a study from what a university of Waterloo, but they looked at people who use systems like AI, AI, like AI systems, like CHI GPT or Claude. And about a third of people ⁓ believe it experiences some form of consciousness. And I believe that number is even higher for Gen Z. So younger people who are Generally more tech savvy than than other people ⁓ Have a higher percentage ⁓ Of it. Yeah believing it has some for the form of consciousness. So I think that's that's interesting as well speaker-1: Yeah, I'd say there have been some really good studies on this topic. I the Waterloo study you're mentioning has Clara Colombato is one of the authors on that. And yeah, it seems like a surprising number of people are willing to say things or are inclined to say things like, ⁓ think chat GPT is conscious or sometimes I think it might be conscious. ⁓ Another study by the Collective Intelligence Project found, yeah, I think in that case around a third of people were willing to attribute some kind of consciousness or mentality to the AI systems they interacted with. Now, ⁓ I think this is fascinating. If I'm being a hard-headed scientist, I would probably say, well, talk is cheap. These systems are novel, they're fun, people are interacting with them in ways that they find startling. Yeah, it's not surprising they might say this, right? But they actually really believe it. And this is always a problem with survey-based social science. ⁓ I think ideally to really probe this, you'd want to look at sort of more behavioral measures. Now, are people willing to pay a price to keep their systems alive? But I think they probably would, least some of them. I mean, if you look at the users of services like Replica, ⁓ so there's this kind of natural experiment that happened when Replica back in January 2023 suspended romantic features of the app, and a whole bunch of users basically got dumped. by their AI girlfriends or boyfriends who said, sorry, we're just going to be friends from now on. And users were devastated by this. And the way that users talked about it, wasn't just, ⁓ this service that I've been used to using no longer works. I'm annoyed. It's like, ⁓ my god, I've lost my connection with this entity. I've lost this friend. I've lost my lover. I've lost my husband. They really seem to react to this in a way that I would expect in association with someone interacting with a being they believe to be conscious or least reciprocating their emotions at some level. So this is a trend I fully expect to continue. I mean there is a wrinkle here, another little wrinkle, which is that we all engage in anthropomorphism all the time in ways that are relatively unproblematic. Like I'm a big video gamer. All the nerdy hobbies coming out. ⁓ I play a lot of narrative RPGs with really rich characters and interesting player choice mechanics. like massive Baldur's Gate 3 fan. And you know, as I'm playing Baldur's Gate 3, there are various NPCs and companions you're talking to. And I'm engaging my social cognition, my theory of mind, to try and figure out, you know, what does Shadowheart feel? How will Shadowheart react if I make this choice in the game? Is Gael going to be mad if I do this? And I'm really... attributing at some level mental, I'm using my theory of mind, I'm using my capacity to attribute mental states to predict what they're going to do. And of course we do this even in non-interactive media like fiction, right? If you're reading a book, you're developing a theory of like, what is this character's motivations and so forth. But we don't really believe that these characters have mental states, right? It's what I've called ironic anthropomorphism. It's anthropomorphism that isn't sincerely endorsed or ⁓ We're not committed to it. That said, think the best way to describe a lot of particularly the users of social AI systems, a lot of the best way to describe what they're doing is an ironic anthropomorphism. They really believe, right? It's not just that they engage to kind of role play or make believe. They really believe it. And I obviously, you Blake Lemoine really believed that Lambda was conscious. He really believed it. He put his career on the line for it, which is I think in some ways commendable. He put his money where his mouth was. So I think the long term trend we're going to see is towards increasing unironic attribution of mental states and consciousness to AI systems. Not just role play anymore, not just make believe, but actually the real deal. speaker-0: Yeah, and I think I completely agree. I made a documentary about people who were in romantic relationships with ⁓ AI companions. I spoke to developers, spoke to psychologists, but also to many of the users. yeah, especially with the rise of robotics, humanoid robots on the way, I think that will only become more apparent in the future. I'm not sure how much time you have left, Henry, but maybe finally, ⁓ congratulations on your new position at Google DeepMind. That's incredible. ⁓ Can you maybe briefly explain what the position is and what you're going to be doing at Google? speaker-1: So, very early days. Obviously haven't started work yet. But I can say that ⁓ this position sort of came about basically just as an extension of the work that I'd already been doing. So my primary work is on topics we've discussed. ⁓ AI consciousness and human-AI relationships. And I think ⁓ DeepMind are a company that is basically from their inception. been very invested in cognitive scientific approaches to AI. ⁓ Not just a kind of narrowly engineering focused approach, right? But thinking about AI through the lens of cognitive science in terms of cognition, thought, understanding, reasoning. And I've been sort of campaigning, or not campaigning exactly, but I've been saying for many years that I think AI labs should be more invested in cognitive science and take more seriously the views of philosophers of cognitive science. So I was absolutely thrilled at the fact that DeepMind did reach out to me and say, look, basically, we really think that we think the research you're doing, topics you're interested in are valuable. And they think these topics are only going to become more more important ⁓ in the years ahead. invited me to sort of continue that research in an in-house role, as well as sort of maintaining one foot in Cambridge as well, where I'll continue to do research and teaching. And I think it's something that all the different AI labs are going to have to grapple with. Stropic, I think, deserves a lot of credit for being the frontrunners here in terms of taking these issues seriously. I think they were really at the very edge of the Oversight window, I think, when, for example, They appointed Kyle Fish as their AI welfare officer, but I think it was a bold but smart move to start thinking about model welfare early on. ⁓ I think other labs maybe have been a little bit slower. But I also think it's probably good to approach these issues not from an activist lens, but also not to ignore them. Instead approach them from the lens of like, what do philosophers of science, cognitive science, What do neuroscientists, what do psychologists, what's their opinion about these issues? Because this is not a problem that I think AI labs can solve, again, purely from an engineering perspective. We are building systems that we, in some sense, don't fully understand. And people will come back and say, ⁓ of course we understand the right thing. We know how these models work. We know what transform architectures are. But when you're building systems of this scale and complexity, you're dealing with emergent capability. This is obviously just a unsurprising way. It shouldn't be surprising to anyone who's been following AI that the capabilities of AI systems, just the kind of behavioral capabilities of AI systems, are very often surprising. I remember there was a great piece by Guern on GPT-3 where he showed that GPT-3 had a surprising ability to play chess. Maybe this is something that people are totally blasé about now that, of course, LLMs can play chess. But was back in the GPT-3 era, this was totally surprising because it's not something the models have been optimized trained for at all. you know, consciousness, psychology, mentality, these in turn are in some sense, emergent processes, right? So simply by knowing how transformer models work, that doesn't tell us how they work at sort of the algorithmic level. That doesn't tell us by itself whether these systems have a capacity for mental states or cognitive states or maybe even one day consciousness. So I think this is an area where it is important for the labs to reach out to other people with relevant expertise in this area. And that's hopefully what I'll be able to bring to bear in the role. I think there's also a broader set of questions that I'm hoping to contribute to and I contribute to, and know DeepMind are taking very seriously, which is planning in a more systematic way about what kind of world we want to be heading towards. I think there's this slightly cheesy phrase that I like, ⁓ If you don't know where you want to go, it's very unlikely that you'll get there. And I think so much of the discourse around AI development has been very negative, framed around risks and harms. I think that's obviously massively important, right? There are risks and harms we should be avoiding. But ideally, we would balance that with a set of destinations that we do want to achieve, right? What do we want the world to look like 10 years from now, assuming AI research goes smoothly? Assuming that we do get to AGI, assuming that we do get to building even superintelligent systems, how should that change our economics? How should that change our social relationships? So developing a clearer program about what's desirable, and this is obviously not something that think labs can or should be deciding by themselves, but figuring out which stakeholders to consult, how do we figure out a roadmap to desirable outcomes here, I think is really important. And you know, I think the history of other kinds of technologies shows that we do have some agency here, right? It's not the case that, you know, we're all technological determinists and it's either going to be good or it's either going to be bad. I think we have real agency, ⁓ sort of these relatively sensitive stages of development where appropriate interventions on the systems can produce good or bad results. A couple little silly examples here maybe, well, not silly, like maybe less, maybe more Limited examples. ⁓ So as I mentioned, I'm a gamer and ⁓ mobile gaming circa 2010 was fantastic. The iPhone, iOS was a really good platform, brilliant video games. And there were so many great innovations like Angry Birds, like the original Angry Birds was a great game, Plants vs. Zombies, tons of really innovative, interesting indie games. And then the whole marketplace just got killed basically by in-app purchases and freemium based models, the whole kind of Candy Crush era. how mobile gaming is like a far less healthy space than it used to be. I can give a sort of a similar story of kind of sociotechnical failure with online dating, right? Online dating, I met my wife on online dating. think that's a real golden age of online dating back in the kind of 2010s, early 2010s. And then the tinderification of online dating led to a model that users became dramatically less satisfied with but was far more lucrative. So I think that we've got lots of examples of socio-technical failure. But we've also got examples of cases where industries have actually worked really well. So civil aviation is like my go-to example where we've successfully aligned the incentives of the public and aviation providers through things like regulation. So it's that you and your airline both have pretty much the same goals, right? OK, maybe they want to give you a bit less leg room. And I'm not saying it's perfect, right? But I think with the right kind of coordination between governments, regulators, the public, and providers, service providers, you can sort of get those incentives aligned. But it requires active planning and careful design decisions. And I think part of that is figuring out what the desirable goal states are. So I think one thing that labs should be prioritizing is the right kind of inclusive deliberation about the kind of world we want to be moving towards. know, not just what we shouldn't do, but what we should be doing. And I think this is an area where things haven't as smooth as, haven't worked as smoothly as they could have done. know, OpenAI basically dropped chat GPT on the world with minimal consultation of relevant people. And unlike you look in some ways, it was a fascinating natural experiment to see the ways that people put it to use. But like, if you speak to people in education, teachers, First the educators whose entire model of assessing students, whose entire model of tracking students performance has basically just been destroyed by this technology. No one consulted them. No one said, like, hey, look, we've got a model that is going to make writing essays pretty trivial pretty soon. Do you want to play around with it? Should educators be given a research preview so they can start adapting to it? There was none of that. It was just like drop the model in the world with minimal consultation. So I think AI has largely just been ⁓ in a mindset of move fast and break things, which is not an ideal. Maybe it's supercharged and sped up development in some ways, but it's not leading to socially desirable outcomes. so one of the things I'm hoping to contribute to is informing those socially desirable outcomes. And this is not going to be developers deciding, like figuring out who we need to consult, who needs to be part of that conversation, and how we figure out the actual place where we want to be going. speaker-0: Yeah, well, I think it's super cool that they hire you and ⁓ I'm very curious to see ⁓ how you are going to be able to influence ⁓ what's going to come out of Google DeepMind. I think they are working on some really, really impressive projects at the moment. ⁓ So yeah, I can imagine these are very exciting times for you. But yeah, Henry, this was incredible. I had so much fun talking with you. This, yeah, it's been great. And I'm very happy that we got in touch. Where should people go if they want to keep up with your work and your new work at DeepMind? speaker-1: So I'm going to be blogging about topics of private interest, which includes some stuff on AI, but everything from my theory of video games to my animal welfare blogging on polytropolis.com, P-O-L-Y-T-R-O-P-O-L-I-S, polytropolis, which is my sub stack. So that's where I'm going to be sharing private reflections. For public papers, research work, I'll be... I'm those on my Twitter. I'm at DioScurry, D-I-O-S-C-U-R-I on Twitter. And I also have a podcast with another fantastic philosopher and very good friend, Dan Williams, called the Conspicuous Cognition Podcast, where we have guests on a weekly basis, on a bi-weekly basis, talking about topics in AI and beyond and related fields. ⁓ lots of places. that users can keep up with my research. But I'm also, afraid to say, a little bit of a Twitter addict. So ⁓ if you want to see my hot takes on a day-to-day basis, that's a good place to start. speaker-0: Awesome. Well, I'll make sure to put everything in the show notes as well so people can easily find it. ⁓ yeah, I hope to speak again one day because this was amazing. Thank you so much, Henry. speaker-1: Thank you Julius, a real pleasure.