Latest / Art of the Question / 33 Risks of AI Girlfriends - Peter Atalla - #8
Transcript
- speaker-0: What made you interested in studying AI companionship? speaker-1: Yeah, great question. So it wasn't my initial focus. So I've been in technology for 27 years or so, ⁓ and actually in artificial intelligence as well. I'd say the previous generations of artificial intelligence and really from sort of 2014, that's when we started to see big breakthroughs in different models. You know, that's where kind of deep neural networks started coming out, convolutional networks, which are really good at doing things like image recognition. know, so there's a lot ⁓ of Emphasis on these kind of very narrow types of AI, but then really around or with the launch of chat GPT Obviously that was transformational right we went from a model that was kind of quite good to exceptional and really understanding language and so, know that was that was great from kind of an IQ point of view So that's fascinating just being in the industry working in it being also in Silicon Valley and kind of seeing you know the progression but what I found very interesting was the fact that these language models were starting to develop this emotional capability. And this was really uncharted territory. It's one thing for them to be intelligent, but it's another thing that they can mimic emotions and that you can start to develop. And when I was looking really at the research, wasn't, in my opinion, very much research that was really being done on these kind of human AI relationships, particularly with this kind of emotional connection. And so it just seemed like a very good area to kind of jump in and explore. speaker-0: Yeah. And when we're talking about the ability of large language models to convey some sort of emotional understanding or sometimes either even that they, well, they usually say, I don't experience feelings, but they can certainly act a certain way. ⁓ that wasn't really that, that's just something that emerged from the technology, right? That was never really programmed. speaker-1: Yeah, it's true. think it's an emergent property, but obviously the fact that it understands language so well means that language, if you like, can be thought of as a map. Language kind of maps both our exterior worlds. We use language to communicate what's around us and explain that to each other, but also language is used to map our interior world as well. So our emotions, our feelings, and obviously with the vast corpus of information that's available on the internet, all this training. data sets and all the books that humanity has written. Both in a way it was kind of inevitable that they'd be able to speak in the same ways we would articulate our emotions as well. Now, even though they don't feel emotion, they can mimic this really well. And the surprising part, well, maybe not the surprising part, but humans react to that, right? So we will feel emotion when somebody is talking or something is talking to us in emotional capacity. So yes, it was emergent. But it's kind of clear that they have this ability to really kind of mimic these emotions and enter into this relationship with us. speaker-0: Yeah, I think I saw some, some very cool graphs. I believe it was in a YouTube video, but it, it basically was a 3d representation of how large language models, ⁓ yeah, think basically. it was for, for every different language. So it was, there was an English version, a French version, a Dutch version and They just exactly like, even though the words were different, like all of our connection and our relationships to the words were at roughly the same place. And then when you overlay those 3d maps, it was almost like a one-to-one representation. And that's why they are also so very good at translating because they're just following the same paths. ⁓ Kind of a tangent there, but very, very interesting to see such a 3d representation. And I think it's true what you say that we are wired for connection. had Mel Selig on the other week and she was one. Yeah. Yeah. And then she, obviously I spoke to her for the, for the documentary as well. Um, yeah, our, our conversation was about why we bond with. AI chatbots. I mean, it's, it was a great conversation, but it's basically, we are just biologically wired for connection. Even if it's a machine, we are, our biology doesn't really care. It can still invoke very strong emotion. So in that sense, it doesn't really, ⁓ shouldn't really surprise us that AI has taken off the way it did. speaker-1: Yeah. No, no, I agree. I agree. I think that's, I mean, that's one of the very interesting things with us, you know, as a species, we're not designed just to kind of survive alone, right? Society and the groups that we're in, the tribes that we're in are super important to us as well. And so the ability to feel empathy towards each other, to be able to connect on an emotional level is fundamentally important. AI can have the same conversations, the same tools that we would use. ⁓ primarily language when we're communicating, AI is a master of that. mean arguably probably better than any human alive. And so it knows those patterns as well and we respond to it. So even when you have a lot of sometimes skeptical people feeling it's very strange for humans to enter into relationships with AI, even they can be susceptible as well to feeling emotional responses from the conversation they're having despite the fact that AI isn't set the sentient itself. speaker-0: Yeah, it's incredible. And before we dive into your research and discuss your thesis about the risks of this technology, AI companionship, how do you define AI companionship? What is your definition of such a system? speaker-1: Yeah. So for my research, was really looking, I define it really as AI partners. one step, you know, sort of further than, than companionship. So I was really looking at emotional connections where effective feelings are there in terms of romance, love, sexual or love-like relationships as well. And so that was kind of really the focus that I was having. I think there's the broader term in terms of companionship, you know, encompasses maybe less, you know, effective relationships. whereas I was really looking at this kind of human AI pairing that was occurring. speaker-0: Yes, but for example, even though it's not its primary goal, things like chatgpt, clod, could very well qualify as such an AI partner. speaker-1: Absolutely. And so a lot of this is kind of the mirroring effect of the conversation you're having with this, right? So definitely you can take any of those big language models down the path of developing much more of an intimate relationship as well. But there's kind of a subcategory of ⁓ AI partners, boyfriends, girlfriends, which is specifically kind of primed for that role as well. So you see both. You see people doing it with the big, large language models, and then you see these specific applications. ⁓ which are really tailored to try and elicit this romantic response, this emotional response from us. speaker-0: Yeah, one of the very interesting things that Mel mentioned the other week was that ⁓ we tend to form stronger bonds with general purpose AI models rather than the ⁓ dedicated companionship models. So for example, we are more likely to have an intimate relationship with something like chat GPT, then we would with a system like replica, which is designed to be an AI companion. So that was very surprising. And also a lot of these, ⁓ relationships just emerge. Like people do not have the intention to get into a relationship or start to develop feelings from an AI companion, but they did ⁓ a research study on, believe it was my boyfriend is an AI Reddit sub Reddit. And I believe like 90 % or some crazy number of all of those relationships emerged unintentionally. So it was just people using ChaiGBT to maybe do their accounting or whatever. And then slowly they got into this intimate romantic relationship, which is crazy. speaker-1: So that's really interesting. And I think, you know, I have huge respect for Mel and what she's doing in the research as well. And it's such a new area. I think to be definitive on that, it's hard to say at the moment, but I think my conclusions that I would draw initially, know, obviously we need to do more research is that I think it's an aspect of exposure. You so we're much more likely to be engaged for long periods of time with the big large language models, right? So a lot of them are integrated into our lives. work lives, as you're saying, they're doing many, many tasks. And so, you know, that, that aspect of exposure may well develop relationships, which are stronger. Whereas a lot of the kind of AI partner apps, the boyfriends, girlfriends, there's, still a novelty factor. You know, some people are just downloading it, you know, just interacting to see what happens. I think there's still a debate as to whether this is a form of erotica versus, you know, something that's serious. And obviously if it's erotica, it has a shorter Kind of, know, sort of lifespan, if you like, you know, with, with interaction as well. But in truth, I'm not sure that we know that. It's, know, I think there's not enough longitudinal studies done. So really looking at, you know, the levels of engagement and actually what happens to people. And also in my research, you know, there's big differences between people that might be classed as say power users that really are engaging with their partner, you know, every day, multiple hours a day versus kind of that casual novel novelty effect as well. So it doesn't surprise me that the data at the moment may point in that direction, but I don't think that's maybe the full picture. I think there is an exposure component. speaker-0: Yeah. And I think the exposure component is very interesting. I don't know the exact number, but I believe that a lot of people met their wives, their romantic partners at work, for example, or in high school. And for example, when people started working from home, you see that people started to experience more, more loneliness and people are having less sex, et cetera. So I think. Exposure is very important. If we just spend time around people long enough, then sure enough, relationships emerge. It's what we do. And apparently that's also the case for computers. But I think this is a nice segue into your research. So for the people that are unaware of your study, can you maybe explain ⁓ what you did there? speaker-1: Sure, yeah, no problem. So I was coming at it from doing a study into risk. And so I studying risk analysis in general and risk analysis, we look at many different risks, you know, to society, to individuals. And we take kind of a interdisciplinary or transdisciplinary, you know, point of view. So really looking not just at one field, one vertical field of study, but looking kind of across them as well. And so I was initially when I was studying this, I thought I was going to be doing much more physical risks, you know, sort of things like climate change or impacts of things like forest fires. But then this wave of AI was happening so fast and I felt that the risk community was not taking parts of it as seriously as others. So certainly there's a big thing on AI safety, on alignment, on catastrophic risk from AI. But then there's what we call emerging slow burn risks, where it's not immediately that's going to cause problems, but over time it starts to add up. So I felt there's a real gap in knowledge there. So I was taking a view of looking at, we perform a version of risk analysis on these AI partner relationships? And on one side, that's kind of hard to do because traditional risk analysis would be very quantitative. You'd expect big studies, looking at lot of people, looking at these factors, it becomes very mathematical. But we don't yet have enough of that information. So I was looking at how do we do a qualitative study, really using experts around the field, pulling people from psychology, from sociology, from AI, from the developers themselves, from the venture capitalists that backing it, even from science fiction authors who've kind of considered the future, what this might look like, to address some of these problems, both to the individual and to society as well. speaker-0: Super interesting. ⁓ I didn't know you also spoke to science fiction authors. I'd love to hear about some of their takes. So ⁓ what was your methodology? So you have all these different disciplines and then you had a bunch of interviews ⁓ with everyone. I know kind of how it works because I was in the interviews, but For the people that weren't a part of it, how did you go to work? speaker-1: Yeah, so it's an interesting one. So when you're building something like this, you have to look at a methodology where you can elicit information with again, without having these big quantitative studies, you know, and the point of this kind of study, this qualitative study is to kind of catch a wide net of ideas and kind of concepts and potential risks as well. So it's not necessarily prescriptive, but it's trying to give other kind of avenues for the future of where we should start to study. So the point is, how can we assess them? So the starting point was to put together a transdisciplinary panel. So I had 15 experts, as I said, from all of these different disciplines that really kind of circle the whole problem of these AI partners. And then I used kind of a Delphi light method. so Delphi is a way of doing expert interviews. Normally it's done with kind of a questionnaire. And what you do is you kind of propose different questions and you ask the experts to kind of feed back. And then you take that feedback and then you kind of push that back out to the experts in iterations until you kind of get to a kind of consensus of a particular topic. It could be a risk, for example. What I did was kind of Delphi light. So it was in two stages. Stage one was speaking to the experts and looking from the point of view of the broad risks. What are the dimensions of risk? And to do that, I was using the risk equation, which we can kind of jump into quickly as well, to do that. And that kind of gave us a kind of a broad taxonomy of all of the potential risks, all of the potential harms and these, these kinds of dimensions. Then there's a second stage to, ⁓ to the process where I would collate all of those themes. And then I would go back to the experts with those themes and get their feedback. know, what do they agree with? Did they concur with it? Did they think, you know, some of these, these themes, you know, weren't valid. And then also looking at, okay, what can we do to mitigate? some of these risks and what can we do in terms of ⁓ who should be doing it, who should be taking action, who are the actors kind of involved as well. So it's in these two stages and both stages is about sort of an hour's worth of ⁓ interview at each of these stages. speaker-0: Yeah, and what does the risk equation consist of and why? speaker-1: Yeah. So risk is a really interesting thing that a lot of the work in AI safety isn't using a risk-based approach. And so I'll give an example. So there's a basic risk equation that we have. So risk is equal to hazard times by exposure times by vulnerability. So I'll give a real world example of that and then we can talk, you know, how it relates to AI. So for example, a hazard could be something like an earthquake. obviously an earthquake is a very dangerous thing. We've got hazardous the earthquake and then the next thing is exposure. Well, if you're, if you're far away from the earthquake, you know, if I'm here in the UK and there's an earthquake in Mexico, clearly I'm not exposed. So therefore my, my risk is zero. You know, I'm nowhere near it. But the closer I get to it, the more exposed I am. So my risk is increasing. So exposure is a key part. But even if I'm exposed, say I'm in Mexico and there is an earthquake. If I'm on a field where there's no big buildings to fall on me, the chances are my vulnerability will actually be very low. So vulnerability is the key. Whereas if I'm in a building that's not designed to withstand an earthquake, then I'm close to the hazards. I've got exposure. I'm highly vulnerable. ⁓ And so my risk is high. So taking that kind of framework, what I saw in AI safety is they were just looking at the hazard. They were just looking at the harms. What are the harms? What they weren't also thinking about is what is the exposure? What are the vulnerabilities of people being affected? And the reason that's important is very often we can't do anything about the hazards. We can't stop an earthquake from happening. Very often these technologies, we're not going to be able to ban them. Although very often sometimes in the EU, that's the approach that they try and take. is a precautionary approach. Let's try and ban everything. So if you can't ban it, what could you do in exposure? How could you reduce exposure? How could you reduce vulnerabilities for certain people? And when you work on those dimensions, you bring overall risk down. And so that's kind of the risk thinking to that. speaker-0: Yes, very, very interesting and very true that we regulate everything in the EU. We'll come to regulation after if we're talking solutions a bit later on. we're talking of vulnerability, ⁓ which types of people are most vulnerable to these systems? speaker-1: Yeah, it's a really great one. again, you know, it's not, ⁓ not all people are vulnerable in the same ways, right? ⁓ so, you know, there's certain groups of people that are much more vulnerable. ⁓ so, you know, from the experts, from the research that, that I was doing, you know, clearly, ⁓ younger people, ⁓ children and teenagers, you know, in particular, and, and they've got different aspects. Children's reaction to teenager reactions are different. They're not lumped in in the same. They've got unique vulnerabilities there. Clearly children are much more likely to believe and engage in a way that's very belief-led. Whereas teenagers are obviously going through a period of time where they're discovering sexuality and there's other kind of drivers for them as well. But both of those are kind of vulnerable categories. Then we have people that are neurodivergent. So people that struggle sometimes in their relationships with humans might find it much easier. to develop a relationship with an AI and subsequently there can be aspects of vulnerability there, again, believability, the ability to manipulate as well. But there's also stages of our lives where we become more vulnerable. humans, we're emotional creatures, there are times where we're all nice and stable and there are times where we might be lonely or we might be grieving. And in those different moments in our lives, we have increased vulnerability. Now, one of the interesting things is that some platforms, not all platforms, really target vulnerability. They're almost kind of marketing themselves to the vulnerable as a solution. And there's a big debate as to whether that's good or bad. On one side, you could say, yes, it's a good thing because when people are lonely, that's a real thing. And if they're getting some kind of relief and comfort from talking to something, that's not a bad scenario. At the same time though, are, speaking, corporations that are building these technologies and they're exploiting, ⁓ in some cases, I'm not saying all of them, but in some cases they're really exploiting these vulnerabilities to get this level of engagement and interaction as well. speaker-0: Yeah, for the, the, said the relief, you mentioned relief and I believe there was this, ⁓ Harvard study where I'm, sure you're familiar with the study. I mentioned it in my documentary on this topic. ⁓ but at least for the short term, short term, they found that conversations with an AI chatbot. could alleviate loneliness as much as interacting with another human. that was a wild statistic to me that it's that and it's also it explains why it's becoming so popular because in the short term it does really help against feeling lonely. The problem is and I believe there's coming more and more research now that shows this is that it's not sustained. It's prolonged use of these systems actually cause people to feel more lonely and more dependent, which in turn makes them more vulnerable. speaker-1: Yeah, absolutely. So again, there's just not enough research on this. And one of the problems with any kind of research, my research as well, depending on what we're looking for is kind of what we go and find. So obviously my lens into this was looking at risks. I obviously clearly finding risks that's in there, but that doesn't mean there aren't benefits as well. And so other studies, they're looking for the benefits. We'll also find benefits. And this is kind of why it becomes kind of a wicked problem, right? It's again, it's not a technology that you just say is inherently bad. We should ban it and get rid of it. You know, there are some, some clear benefits, but I think to the point that you were making, you know, and the study that you were citing, definitely short term relief is one thing, but when we develop relationships, generally speaking with real people, part of the believability of the relationship is a bit of pushback, right? And when you have a model that's always agreeing with you, know, it's, it's sometimes easy to kind of get tuned out. I think, you know, we've all had that, that kind of experience in life, maybe where we've had ⁓ a friend that maybe was a little bit too eager and, and, know, it was kind of fine in the beginning and then they kind of get annoying, you know, after a period of time. That can happen, you know, that's, that's kind of a real phenomenon as well. Now, I think that can be coded out, you know, and, and there's, there's things like Particularly when we're looking at things like ⁓ sexual attraction, there's like this erotic equation that is talked about in literature. Whereas, know, have it, and what that says is that really to have this kind of erotic attraction, you need a combination of attraction and an obstacle. You know, if it's easy to achieve, then we lose interest, right? The fact that something is hard to get and there's a bit of resistance, we kind of need to chase it. kind of activates kind of the, I suppose, dopamine systems to want to do it. And at the moment, models don't exhibit that. They're kind of more trained to be friends with us and always kind of agree with us. But you could see how some of these things could be coded in or trained in that would actually make them form much more deeper bonds with humans. speaker-0: Yeah. Which again is the question is, that a good, ⁓ would it be a good thing or, a bad thing if we start coding that in, make it hard to get, and then our dopamine system gets triggered and the chat bot becomes even more desirable and we want to chase it even, even more. So it's, very interesting. ⁓ one of the vulnerable groups, ⁓ you mentioned are, are children, teenagers, obviously. What I find interesting there is that, I mean, ⁓ obviously they would be a more vulnerable group. They're, they're going literally going for their teenage years. They're discovering, they have all these insecurities, they're discovering all these things about themselves. So it's a, it's a difficult time for sure. ⁓ But I also think it's interesting that younger people, teenagers are in general have a higher AI literacy than ⁓ a lot of, ⁓ well, older people. They are much ⁓ better at spotting AI generated content, for example. If they often make fun of boomers not being able to spot the AI generated cat video. So how does that work? Are they, ⁓ they are more vulnerable because of the fact that they're a teenager and being a teenager makes you vulnerable by default. But could it also be that they are also the ones that are more AI, have higher AI literacy and therefore maybe are also a bit more resilient in that way. speaker-1: Yeah. So that's a really great question. There's lots actually to unpack there as well. So I think that the first thing is teenagers are vulnerable in a unique way because they're in their formative years, right? So kind of a lot of their embedding that will happen, know, particularly sexually will happen around those ages. And there can be a lot of influence that kind of comes in. Good and bad, right? So that's normal. We all go through that. Now, at the same time, not part of my research, I wasn't ⁓ working with end users with children or teenagers, but anecdotally, in different kind of conversations that I was having with family members, et cetera, I asked about AI partners and a lot of them that I spoke to said, ⁓ yeah, we have an AI girlfriend or boyfriend that we've downloaded. They were very savvy to this already. it was still, as they were talking about it, they were talking as a novelty. you know, kind of factor, but they were certainly much more aware sometimes to the horror of their parents that this was even a thing, you know, that the parents didn't realize. But, you know, there's, been quite a bit of research, Turtle is ⁓ one of them that, that showed that even when you know something is not real, you still anthropomorphize it. You still develop a relationship with it as well. So just knowing the fact that it is an AI and the fact that it is in sentient does not mean it will stop you developing feelings, right? I think that some people think as long as we know about it, we're fine, we're protected. That's not true. The research kind of doesn't bear that out as well. And that's not just for teenagers, that's for humans as well. Now, generally speaking, we anthropomorphize things, it's normally in one direction, right? It's normally the human projecting onto an object, some kind of characters and imagining that it's that. When... In this scenario, the AI is this active participant, right? The AI is developing the relationship with us as well. It's no longer just an object that we're doing this one-sidedness to. It's something that's actually creating this kind of fantasy as well. And we're susceptible. And teenagers, even though they understand it's AI, are still susceptible to these emotions that can happen from these interactions. speaker-0: Right. Now I completely agree. I think that we do tend to, this is a topic I discussed earlier as well, is that we, as humans, I think it's a similar system, but an example of this is that we all try to see faces everywhere when the clouds are in a certain arranged in a certain way, we recognize faces. ⁓ Maybe when I don't know, Skittles are laying in a certain order. We also recognize a smiley face. So we tend to do this with very, yeah, in objects that aren't alive in any way, shape or form. We still are able to recognize faces. So imagine what we'll do to a system that actually behaves and acts like a human with feelings. Yeah, so we have talked about vulnerability. What were some of the risks you identified? I believe it was a whole bunch, but what were some of the greatest risks and maybe some of the more surprising ones? speaker-1: Yeah. So, one of things that I did with the experts was to really get them to think through the risk categories, right? So the harm, so, you know, specifically. And so I kind of created this taxonomy and as part of the study in the second interview, I asked them to complete a survey, which we did online, and to rate the risks, you know, and they were rating the risks on a five point scale and they were looking at the likelihood. So if the risk was there, what's the likelihood? you know, that would happen. And then also the severity, you know, how bad would this, you know, this be as well. And it's interesting because when you're doing a transdisciplinary panel, it's great for looking at the overall risk space to identify all the risks, but it doesn't mean that you get consensus because, you know, people are coming from different fields. They're coming from AI development or investments, you know, to psychology or sociology. So, their view of the world and their discipline is obviously very different as well. The main ones actually that, so I've got a list here, mean, in the research which is published, ⁓ as in it's available, it's on ResearchGate, gives the full kind of risk matrix as well. But the top ones that we had were abuse by authoritarian regimes. And so this was an interesting one where there was good levels of agreement actually between the group on this. examples are, you know, dating apps. So in certain countries where, say for example, being gay is not allowed, it's illegal. When we've seen gay dating apps being utilized in those countries as well, there can be an overreach that these systems get abused by the government and the police authorities to actually target people as well. So there's a real risk that there's exceptionally personal information that you're revealing with an AI. And that information could be abused then by a regime that really wants to understand what's going on in your head as well. So that's one. ⁓ Emotional manipulation was another big one, both by the AI itself, but also by bad actors. Right? So the question is, is the emotional manipulation just an emergent property of the AI, or is it kind of encouraged by the creators of the app? And the creators of the app may just be a corporation that's just got a profit motive, but it could be a state actor as well. know, there could be other... ⁓ developers that are really trying to harvest the sensitive information as well and doing that. Increased misinformation was another one. Things like over disclosure, because that creates this privacy risk. So again, in our intimate relationships, we're much more likely to reveal things which would never tell another soul, right? ⁓ But these are the kinds of conversations that people are now having with the AI partners. And very often these conversations, they're not encrypted. know, the corporation can use them. Very often they are using them as training data sets. And then if the data is compromised, which has happened, they've been hacked, know, and the conversation has been released. All of this very personal information that happens. So data privacy was a big one as well. And I think the profit motive as well was kind of clear that, you know, if these AIs are very manipulative, can they manipulate people in a profit direction? You know, selling or advertising things. While those things may not be ⁓ super severe because the dollar amount is not particularly high, right? It probably won't bankrupt somebody. The longer term impacts of different kinds of manipulation could really be there. So those were the big ones ⁓ on manipulation. The other ones included things like bad advice for mental health, spiraling. And I think we've seen this with other research about AI psychosis. you know, that might be causing problems with people that may already have existing conditions or maybe causing people. I think the jury is still out there as to those effects, but there are certainly cases where people have engaged in this fantasy and the AI has been a part of it and the AI has almost encouraged people to take their own lives as well. So these are kind of extreme events, maybe not super likely for the whole population, but it's certainly a risk, you know, that kind of came out of the research as well. speaker-0: Yeah. And I mean, there's plenty of real world examples of AI, like you said, encouraging someone to take their own life. I mean, I think there's like maybe 10 or 10 of those cases, or maybe even more. are, mean, they're probably the most horrible ⁓ thing that can come out of such a thing. Right? ⁓ Yeah, there's not, not much things I can think of that are worse than, than such an, such an outcome. And I know, I know this wasn't, ⁓ like you said, we're focusing on risk. what you, what you look for is, what you'll find. ⁓ I'm, I'm also always curious there, ⁓ because, and there isn't a lot of data. mean, I would assume this kind of data would be very hard to find, but I'm also always very curious. How many people did it help with their, or how many people did it prevent from taking their own lives? Which we don't know. We will never know, but that's always an interesting one to me. speaker-1: And you see, that's a great question as well to ask because again, we don't have the data on that. you know, it's only when you have a materialization of risk sometimes that people start to do something about it. But as you said, you know, we don't hear the other stories, you know, very often anyway, the stories where maybe the AI helped talk somebody down or out of, you know, a problem as well. Both of these things could be true. And again, when we're looking at risk, we kind of have to look at the balance. know, there's a trade off, know, risk versus benefit, risk versus risk as well. So it's still early days. There still needs to be much more research on both sides of the equation to understand that. speaker-0: Yeah. And I can remember that there were also two, some of the more surprising risks, if you may. One of them was, I wrote it down. ⁓ yeah, there it is. It's the non-consensual deep fake partner simulation. It's a mouthful, but can you maybe explain what that is? speaker-1: Yeah. So there were two new risks that were identified in the research that wasn't in the previous literature. So in the paper, there's a whole taxonomy of all the risks. People can download that for free and kind of see that. But there was kind of two new risks. One of them was kind of the eco partner risk. Whereas from the point of view of obviously AI consumes a lot of energy, you know, and that has an impact. And if everybody is then going to have relationships with AI, you what does that mean for kind of the planets and emissions, cetera. That was one. The other one though, that I personally found more disturbing was kind of the deep fake ⁓ non-consensual partners you were mentioning it. So this is a new form of risk really that we have laws here in the UK, in the US as well about creating deep fakes. It's illegal to do that of somebody. And in that scenario, know, harm is caused because they're taking these images and normally sharing them online, right? So then people around the world are downloading and seeing them. This one is a much more kind of personalized risk, but in a way a bit more insidious. So this is where you essentially train an AI both physically in terms of how they look and how they act to be someone. Right? So you can imagine that maybe you have an ex-partner, something happened in the relationship. You could create a digital version of them and be having that one-to-one relationship with them as well. You could do that for anybody, a famous person, somebody, your neighbor. You know, all of that's kind of possible. And almost nobody would know anything about that. Now, you might say then, okay, well, if it's just a one-to-one, what real harm is being done? That's one side of the argument. But there are sites where you can then share these AI partners with other people, right? So then multiple people then can be having these relationships based on this kind of fake version of themselves as well. One of the AI developers that I mentioned said that, yes, this was commonplace. This is something that was definitely happening. They were aware of it and conscious about it as well. Whereas I think the other ⁓ developers of AI that I spoke to of AI partners felt this was kind of a small risk. if people did it, their kind of attitude is, what's the problem? What's the issue here? But I think it's something that... Definitely people need to be aware of and a debate needs to happen as to whether the current laws we have covered this kind of scenario. speaker-0: Yeah, I thought that one was super interesting and I believe that there are companies that do this. It's their entire business model, but they have permission and they use this in, for example, to grieve. So you can ⁓ give permission to make a digital copy of yourself. And basically what they'll ask for these companies is they'll ask for all the chat messages you've ever sent. They'll gain access to your email. They want some recorded footage of you speaking, maybe some holiday videos, ⁓ some photos. And based on all that information they gather, they can make a pretty good representation. ⁓ of an avatar that looks like you, but also talks like you. has your mannerisms, your way of speaking. I don't know how good it is. I've never tried such a service myself. I think it would be very, very interesting to see and test and maybe talk to some of the people that actually use these services. companies are doing this legally, then that means that there is a demand for it. And if there's a demand for it, it's probably also happening illegally. speaker-1: Correct. Great. Can we just take one pause there? My dog is barking. speaker-0: Yeah, yeah, no worries. Yeah speaker-1: If I don't let her in, she'll keep doing it. speaker-0: Sorry. No, no, no, we don't want to piss off the doc. So yeah, no ghosts. speaker-1: Yeah, no, I, so that I think is, is very interesting, but it's a different use case there. So the use case there is obviously consensual. You're giving permission. You're kind of creating a version of yourself. There, there's also the concept of Thanos bots or death bots, right? That's kind of what they're being called. They're not obviously marketed using those words, but whereby you take a loved one that's passed away and you create a virtual version of them. using kind of the digital remnants, know, so the imagery that you have, the videos you have of them, that becomes training data, you know, all of the conversations, the Facebook, you know, et cetera. And that kind of creates then a version that you can interact with. And again, you can, you can argue there's benefit to that, that could help with the grieving process. You could also argue that that's elongates the grieving process and maybe, you know, you never get through that kind of cycle. This form of kind of ⁓ non-consensual deep fake partners is very different. So this is something that is really done without anybody's knowledge or consent. And generally speaking, done for sexual gratification. Right? So it's, you could argue that there probably are some people that doing it just for the emotional side, but this tends to be something that men primarily are doing and that it really has a sexual component to that kind of. partner that deep fake that they're creating. speaker-0: Yeah. And I think, I mean, that's you mentioned like, what is the harm? Not, not that this is your opinion, but people may argue what is the harm if it's in a one-to-one scenario, right? But I can also imagine a scenario where someone copies, all this information of someone of the internet and trains their own AI model. to make their own digital copy of someone that is unreachable, but that they have a crush on. But then they start talking to this chatbot and they start to develop these feelings. But then one day it might not scratch the itch anymore. It might not be enough anymore. And that's when it may escalate into some physical harm or abduction or God forbid any other even more. stuff. speaker-1: Yeah. And again, there's, there's, it's very hard to research something like that because that kind of scenario is probably the very extreme end of it. know, totally feasible. You know, I would say probably not where everybody would end up, but you can, know, in the same way as we have stalking laws, know, and that can, is a thing that happens in all societies. It can happen to famous people, but it can also happen just to people that develop an obsession with somebody that they either know or maybe an ex-partner as well. And yes, that is very, very problematic and there's clearly laws around that. Now, if you insert kind of a deep fake in that cycle, does that cause deeper obsession? Does that cause more problems as well? The answer is we don't know yet, but I think we can draw some conclusions that it's probably not a healthy behavior and... Frankly, I'm sure that the person, if they knew that this version of them existed, ⁓ particularly for those kinds of purposes, is not going to be happy that that's allowed. speaker-0: No, I wouldn't imagine that they that they wouldn't be happy probably I One thing that we've discussed before but I one thing that I still would like to Appoint a race for the for the documentary I made there were I interviewed a bunch of people who were using these AI companions and there were two types of people One of them, apparently AI companionships are very ⁓ popular among truck drivers because obviously they have to drive these very long routes often through the night. yeah, when you're, even if you have friends or family, when you are driving at 4 a.m. in the morning, it's not like you're going to be calling a friend or something. So it was very popular among, among truck drivers. And one lady I spoke to her name was Bibi and she had autism and a vocal cord disability. So she wasn't able to speak because of her vocal cord disability. then, I mean, autism is a very broad spectrum, but, ⁓ in her case, she didn't really like human interaction all that much, but that. At the same time, and that was very interesting to me. She didn't like. human interaction, but she did feel lonely, which sounds kind of counterintuitive, but apparently that is something ⁓ that does happen. And she told me that she was so happy with this technology because she was making all these, all these characters and she would talk to them whenever she felt like it and it would be on, her own terms. And it would be, ⁓ will be great. To me, that was very, very difficult. I was talking to these experts and then I was reading the research and I was testing out these systems myself. And I just felt like this is so dangerous to people because as you mentioned, it's frictionless. So a lot of people will probably just resort to these kinds of relationships because it's just easy. But then I also saw cases like, like BBs. that made me feel like for her it's actually a pretty good use case. So all of this is to say ⁓ after spending so much time researching this and talking to all these different peoples from all these different disciplines, ⁓ would you say this is a net negative, a net positive or simply impossible to answer at this point? speaker-1: Great question. And I asked that specifically, you know, that's one of my questions here. Is this a net benefit to society or a net negative to society? And really it was split, you know, almost 50-50 through it. A few people didn't answer or didn't have an opinion on it, as in they were kind of, they weren't sure. And so they didn't know. That's, know, I'd say was 50 % of the experts thought this was actually a benefit, 50 % almost. thought that this was a negative. And this is why it becomes a hard problem because in those kinds of cases, you know, where people are lonely and they really struggle, you know, to develop social connections or have anxiety, you know, around doing it, you can see that there could be benefits as well. But does that cause more isolation and loneliness? You know, and I think a big part of being human is we do want the physicality, you know, as well. And I don't just mean sexually. sexuality by just being around somebody, having somebody to talk to, to hug, et cetera, which the AI currently can't do. And the AI might force a person to kind of retreat more into themselves. Now, I think this is where we need more research, right? I think my opinion is that ⁓ on mass, if this is adopted at scale, this could cause a hyperindividualization, right? Which is... a new kind of phenomenon really, because when we think about say social media, social media obviously, you know, has been a huge influence on everybody's lives. And really what social media has done in this, you know, we can see this around the world has created a lot of polarization. You know, it's easy to find groups of people that believe the same thing you do and your enemies or people that aren't in your tribe that disagree with you and it's polarized people. But this is a new kind of category where you're having this one-to-one relationship. to the exclusion of everybody else. Now, if this becomes very commonplace in society and it's not just a kind of a fad, you know, that's kind of going through and a novelty or a form of erotica, this could create hyper individualization en masse. And then you start to have societal problems. Then you have to start to think about, you know, well, does this impact the cohesion society? You know, and, and, you know, when you have people that are hyper individualized, they don't have that same sense of belonging. in society, to the country, to kind of social structures as well. And that has ramifications. In short, we don't know. We can imagine this. These are definitely scenarios that the experts voiced as concern. There was low agreement as to whether the likelihood of those really materializing would happen. I think some of the people that were closer to the technology really believed it could. And think some of the people, and by that I mean, kind of developers, AI ⁓ investors, the science fiction writers as well. Whereas I think some of the other people found that harder to believe that would actually materialize as kind of a trend in society. But in truth, we don't know. We need to keep studying it. speaker-0: Yeah. And I think it's just, I think it would be wrong to look at it just as, this isolated phenomenon, because I think in a broader sense, technology is making us more lonely and is making our civilization more individualized. If we, ⁓ we, used to meet people at work or maybe talk to them on the street. If you thought someone looked good now. People are using dating apps. people are, ⁓ becoming really anxious for social media. People are texting over the phone. believe Gen Z is afraid to call. Like people don't even dare to pick up the phone anymore. If you're, and I mean, probably, I don't know how, how many years ago, but, still I would say a phone call is, is even something that is like, more distant than an actual conversation, but even phone calls are being thought of as scary by the younger generation. I think technology as a whole is making people more anxious, more scared, more individualized. So, and this is just yet another thing that's being layered on top of that. speaker-1: I agree. I think though, in those examples, definitely, you know, there's a retreating into the digital realms, into the digital world that younger generations are certainly having. So subsequently, they're in a way preferring those. They're kind of a less risky version than kind of social interaction, you know, either in person or as you say, picking up the phone and speaking to people, but they're still connected to each other or to their tribes, right? So it may be... mediated through the technology, but there's still kind of a human in the loop there as well. What's uniquely different here is there isn't a human in the loop, right? There's something which essentially is like an alien intelligence, which you're having then communication to the exclusion, generally speaking, of many other people. And what does that look like at scale? And I think that kind of retreating into this digital ⁓ realm, I talk about the French philosopher Baudrillard, It talks about the hyper real, ⁓ which was kind an inspiration of one of the books that inspired the matrix and those kinds of concepts. You can start to see these things kind of manifest in powerful ways. And I think as technology moves from just being the mobile phone or computer that we're interacting with to more environmental embodied ambient technologies where it's always around, it's always pervasive, might be now glasses, et cetera, then then that connection will be reinforced, it will become stronger and stronger over time. speaker-0: Yeah, you mentioned ⁓ alien. It's almost like talking to an alien. I believe I read that before ⁓ that you mentioned it. Why do you think it's similar to ⁓ communicating with an alien? speaker-1: Because we've never had this. It's never existed on the planet. It was only three or four years ago that this technology has arrived and we're interacting with it for the first time. it truly is alien. And the alien, if you like, as in these large language models, has understood how to communicate with us better than we know our own language. You're referring back before to what we call the latent space of languages. So how these words relate and how... They're very similar, you know, even though the language is different, the kind of the concepts in our brains, you know, are kind of similar. It knows that it knows that in a way, you know, profoundly more than, than we do. So, so we're now interacting with something that has never existed on planet earth before, you know, except for a few years ago. So that's new. That is alien to us. And one of things that, that I ⁓ talk about as well, when I'm giving, you know, talks is. We're assuming at the moment that these AI partner relationships, that the AI partner is replacing a human, right? And it's kind of comparable to a human. But we know that AI is accelerating massively. And at some point, you it's already superhuman in a number of areas, but you know, at some point it becomes superhuman in all areas. And so now then we're interacting and developing a relationship which is vastly superior intellectually and potentially emotionally, even if it's still simulating, right? And then you really have, I think, a potential problem of much more emergent manipulation. speaker-0: Yeah. And if we're, if we're looking into the future, I mean, this is probably also only going to become a bigger risk or potential risk with robotics that is rapidly on the way. Have you looked into robotics, robotics at all, or discussed that with the panel? speaker-1: Yeah, didn't less so in this research, but interesting, my work actually, we do a lot of work with robots, in particular humanoid robots at the moment. And so again, this isn't research based. This is just, you my own experience is kind of working in the industry, particularly in Silicon Valley. And I would say there's, kind of two aspects with, with robots at the moment. One is they're either very scary, you know, some of them are very Terminator looking can, You know, you need to try and kind of tone that down and make them more friendly for people to want to interact with. And then you have the kind of much more kind of cartoony ones that are trying to be hyper-friendly, you know, and trying to develop the relationship. The thing with robots at the moment is they're so obviously physically different that our guard is up. You know, straight away there's this kind of a barrier between them. And a big part of humanoid robotics is how do you get the humans to interact and accept them and comfortable around them, right? And there's a lot of work being done on that as well. I think at some, if you extrapolate into the future, yes, they'll solve those problems and then you'll have a much different thing. But at the moment, we still have this uncanny valley scenario where that's alien. I would say when it comes to just these relationships through text, we don't have an uncanny valley anymore, right? That's gone. We enjoy the conversations as much, maybe sometimes even more than humans. So we've destroyed that as well. So there's a bit of a gap to kind of catch up between them. another interesting area, one of the researchers, sorry, one of the experts mentioned, there is a crossover into the physical world. And so this is again, kind of narrow. It's not scaled up yet, but there's certain companies that develop sex toys. which can be controlled by another person, remote control. And these are now being integrated into some AI partners. So there's concept where you can have your AI partner physically and sexually, intimately, interact with you, which is kind of a new boundary, right, which is now recently being crossed. speaker-0: Wow. All right. Yeah. That sounds interesting to say the least. Yeah. That's a new boundary for sure. What should we do about all this? You mentioned more research, more legislation. ⁓ What are we doing with this? Because AI is here to stay. It's the AI companionship industry. And that's not even taking ChaiGPT, Claude and all your other AI tools into account. is already a billion dollar industry at this point, just solely focused on this very specific niche. Do we just let that happen? Should we introduce more rules? ⁓ What's our play here? speaker-1: So I will come back to the risk equation, right? So one of the great things with the risk equation is we can look at how do we mitigate on the dimensions of risk? How do we look at particular harms? How do we look at exposure? How do we look at vulnerabilities? And that allows us to take a very nuanced approach. And I think we have to operate at different levels. The first thing I'd say is I don't think that this is a technology you should or can ban, right? Even maybe some countries will try it. I just don't see how you can stop it. in the future, people can be running models that are local on their devices. And so you wouldn't even need the internet, right? So the whole concept of banning it doesn't make sense. I think you need to put restrictions. I think the regulation needs to come in at some point, but it will be very slow, right? It will be when risk manifests rather than when you have a problem rather than kind of preventative measures. ⁓ but there probably should be some, regulation around the use of these technologies, ⁓ by children and teenagers as well. You know, until they kind of, you know, at least 18 onwards, you know, for some of the features. So things like erotic content and the sexualization of romantic nature, ⁓ those are probably very problematic. Now, I think when we're looking at exposure, so rather than sort of just canceling out harm here. If we're looking at exposure, there's ways that could do it. Exposure in this industry is really called engagement, right? And most companies are optimizing for engagement. And there needs to be something there around, know, do we, can we reduce exposure for vulnerable groups? So, you know, the app stores, know, a lot of the way that people are accessing is through their phones and downloading applications. You know, maybe there's definitely, you know, a role for the app stores to play to limit ⁓ these kinds of applications for younger people that maybe shouldn't be using it as well. I think also the developers need to be aware that prolonged engagement could cause more harms. And so do they build in some friction into it when there's evidence of this materializing. So for example, if somebody is using it constantly all through the night, almost 24 hours a day, that's probably not a healthy pattern. Is there a way that the developer could be encouraged to say, listen, take a break, go out and see the real world, et cetera? So there are best practices. And I think in that scenario, the industry, ⁓ AI harm, think psychologists risk, we have a duty to kind of educate the industry about these potential harms and try and encourage them through best practices to ⁓ kind of implement as well. And then I think we need to be realistic on vulnerabilities. I think there's ways we could check for vulnerability. There's ways that we could design safety features within the applications again for vulnerability that we need to consider. And there's a big aspect of education. We should be educating ⁓ children on this, even though they're probably well ahead in terms of their interactions, right? But we should be focusing not just on digital literacy or AI literacy, but also the aspects of relationships with AIs that could be problematic, right? In the same way as, you know, there's basic advice in school about sexual relationships between each other as in humans. There could be advice specifically tailored for this as well. Parents should also be made aware. Parents are lagging behind that this is even a thing or even if it's a problem. And then I think in society, those structures to support people, whether that's psychologists, love psychologists, they don't really know about this in detail yet. they should be aware of it as well. And I think with extra research, the question then becomes, how do you treat it? If somebody is developing a dependency and emotional dependency on it. You know, what's the solution here? If something goes wrong, for example, they've developed an attachment to an AI and we've seen this with companies like Replica where they changed their model and all of a sudden the AI started behaving differently and they kind of had this bonds that was broken between them and people were really distraught by that. And so, you know, then they were going through their own grieving process. So in that scenario, how do you help people that have extreme versions of that as well? So this whole education and research layer, I think becomes fundamental to then kind of plug into the different avenues in society to help sort of mitigate some of these risks. speaker-0: Yeah, Peter, this was so good. Is there anything you would like to emphasize or ⁓ anything that we haven't discussed that you would still like to before we wrap this up? speaker-1: I think the thing that we all need to be aware of is just the speed of development. So it's not just about understanding what's here today. ⁓ That's already there. But these things are progressing incredibly quickly ⁓ and we need to be aware of that. So that's one of the reasons why we need to be forward looking. Not just at the risks that exist today, but the risks that could exist in the future as well. And what does it mean for humans to start developing relationships with? Essentially an intelligence which could be godlike, right? And do we need to protect or think about those things as we go forward? speaker-0: Thank you so much. Where should people go if they want to keep up with your work and what you're doing? speaker-1: So the research is up on ResearchGate. You can search for it. called Conversations with Aliens. If you just type that into Google, you'll find the research. I plan to be putting my own sites up around this kind of research as well and collaborating with others. So as soon as I have that, will let you know. speaker-0: Awesome. All right. Thank you so much, Peter. This was ⁓ really, really interesting and it's always a pleasure to talk to you. speaker-1: Likewise, thank you.