Latest / Art of the Question / How AI Could Help Us 'Talk' to Animals - Dr. Elodie Mandel-Briefer - #15

Transcript

How AI Could Help Us 'Talk' to Animals - Dr. Elodie Mandel-Briefer - #15 · Art of the Question

• speaker-0: How did you first get interested in animal communication and why? speaker-1: So that's a while ago now, I think 2005 when I started my masters at the University of Paris South. I basically was very interested in animal behavior and my main aim was to study animals in the wild, so in their natural environment and not in a lab. And by chance, actually at the University of Paris South, there was a brilliant team called the Biocoustics team that is still there, who were studying birds and all kinds of animals in the wild. So I kind of... fell into it because the team was there and I started working with these great people on, actually on scale arcs during my master's and PhD. So I started off working on bird songs and then I moved to mammals later on. speaker-0: Super interesting. ⁓ why is it so important for us to understand how animals communicate? speaker-1: I mean, for many reasons, first for science, know, like fundamental knowledge to understand, ⁓ for example, how communication evolved, how animals ⁓ communicate and how that evolved into human language, for example, and very complex ⁓ communication we use as humans. So we can see some kind of building blocks through the ⁓ animal kingdom with some species with very complex communication and so a bit more simple. ⁓ but also just to understand animals around us. ⁓ so in my case, I study both, ⁓ wild animals and domestic animals. so if you focus on, for example, pets or, or, livestock, then it's quite important to understand them, to, understand their needs better and, and how we should, ⁓ treat them for the best, like, ⁓ with the best level of welfare. speaker-0: Yes, of course. why do we think animals communicate? it just a simple, is there ⁓ an evolutionary case for communication? Are you more likely to survive as a species if you're able to communicate well? speaker-1: Yeah, definitely. ⁓ if we focus on many types of communication, like visual communication and chemical communication, ⁓ I study mostly acoustic communication and that has evolved mostly in species that move in three dimensions. So marine mammals in the sea, ⁓ even fish actually, many fish communicate with sound, which most people don't know. Birds that move in three dimensions and us like ancestors. ⁓ So it's mostly in species that move in three dimension and cannot see each other often. So they need to communicate with sound, is better in these conditions than visual cues, well, visual signals. And it also evolved more in species that also in dark or forest environments. And then what do they communicate about? it's really, I mean, it's really, they communicate about what they need. usually species don't evolve more things than what they need. ⁓ So they can have all very sophisticated things like echolocation in bats, which for example, we don't have. ⁓ But in general, like they won't start, you know, talking as we do if they don't need to. So they communicate mostly about their needs and what is your intention of behaviors. Sometimes external events, but that's quite rare. Like if there is a predator coming and which type of predator is coming. Yeah. So it really depends on the species, it's very different from human language, let's say. speaker-0: Yeah. And how do we determine ⁓ an animal's needs? Because that's what I find so interesting about this whole field of work is that the whole, big problem is that we can't communicate with the animal. So we have to make certain assumptions about their feelings ⁓ and about their communication. Therefore. So that seems like ⁓ a difficult problem. I was ⁓ curious, have you seen the movie or read the book Project Hail Mary? By any chance? It's one of my favorite books and it's currently in theaters because they made a movie as well. But it's basically a story about a guy ⁓ who goes into space on a spaceship in order to save ⁓ planet Earth. And because there are little microbes eating away at the sun. And ⁓ he's trying to figure out why they're eating the sun. And then as he goes to a different star to investigate, he meets an alien from a different planet, different civilization. And they're trying to communicate to each other because they have the same problem. Their son is also being eaten. Right. So a big part of the book, the movie, not so much. is about them trying to figure out how to communicate with each other because they, and they start with similarities, right? So they start with the periodic table. They start with things like time, they use a clock and soon they'll, they'll figure out ⁓ a translation machine, they have a similar starting point. They, they understand, okay, we have the similar context problem, which is the microbe. are eating our stars. So then they use the periodic table and things like time as their context in order to build from there and start communicating. All of this is to say when my cat is meowing, I assume he wants food. When he's pressing his paws against the door, I assume he wants to go into the next room, but it's always an assumption. So I guess my question is how ⁓ do we solve that problem? speaker-1: Yeah, that's a ⁓ tricky one, but we basically solve it by studying animals, observing the behavior of animals. So very good starting point for at least wild animals would be to study them in the wild. And then we basically look at when do they vocalize and what is the context. And then we can... code it in very different ways and like very specific ways. And then we can see, so for example, if one call is always emitted during an agonistic interaction, then we know that it means back off or it depends if it's the aggressor or the other, the one being aggressed that is making the sound. But it's basically an agonistic call. If it's made during affiliation, it's an affiliation call. If it's made when they get separated or reunited, then it's a contact call. the basically the meaning is. I want to keep contact. ⁓ So you can basically by observing the context in which the call is produced, you can kind of figure out what is its meaning. I mean, it's broad meaning in terms of context. then of course, like there is also many more things we can do because sounds ⁓ might be produced in one context, but actually often encode much more information. Because they can encode always information about, of course, the species, because species have different sounds, about the, maybe the population or the individual itself. They very often have different voices, like we do. The sex, often males have lower pitch sounds than females. So they also, in addition to the specific context in which they are produced, they also encode much more information than that. including also what I study ⁓ the most in my team is the emotion, so the internal states. So the emotion and motivation that affects their sounds a lot ⁓ for obvious reasons because of the way sound is produced. ⁓ So in most mammals, like sound is produced like we do, we have vocal folds ⁓ vibrating with the air coming from the lungs and then that makes the pitch, so the lowest frequency of the cell. that you perceive as speech and then it's featured in the vocal tract and basically if you are stressed or have certain emotion then your vocal tract will get more tense and you know like everything will maybe you will articulate more like animals will open their mouth more maybe and that will change the output so the features of the sounds and so that's one way to to study so study the context and study also all the features that are encoded in the acoustic structure of the sounds. ⁓ And then to make sure that we are correct, ⁓ the step that is really needed, which some people also often forget, is that you need to ask them if they perceive that information. So you need to play the sounds. So people have been doing that for many, many years. So you use a speaker, then you play the sound to them, or you play two different sounds that you think have different meaning. And then you see the response to kind of validate what you had in mind. speaker-0: super interesting and what is some of the most sophisticated forms of animal communication that we are aware of? speaker-1: ⁓ Yeah, so there is more and more every year and actually increasingly in the past five years or so a bit more. ⁓ I mean, so the first very interesting calls that were discovered were the calls of vervet monkeys and meerkats, for example, that are different depending on the predator. Because before that, people always thought that what animal calls are just emotionally driven or motivation driven and they don't. ⁓ give any information about something external to the name animal. But then some very famous ⁓ primatologists, found out that vervet monkeys, when there is an ⁓ eagle coming, so a predator coming from the sky, they will give one certain call and that will induce, if you play that sound, will induce, I think, looking up. ⁓ Or if you play sound of predator coming from the ground. that will induce a response of going up. So they will produce a different call when a predator is coming from the ground. And same for the meerkats. So they have very specific calls for specific types of predators. And if you play them, then you see that differential response. That's what we call functionally referential calls. And so it has been found in few species now, but not many. So that's the closest to human words you could have. ⁓ but recently also more and more people are figuring out that, ⁓ some species actually combine, ⁓ sounds as well. So they can combine sounds. Each of them has a specific meaning and then they can combine them and the combination will change the, ⁓ the meaning. Wow. can hear my dog outside. speaker-0: Yeah, I can hear some, some dog sounds and speaking of, I'm wondering what, what is trying to communicate there. ⁓ speaker-1: He's very vocal and he's just bucking at everything that passes through. speaker-0: No, yeah, it's, it's, it's fine for now. I think it shouldn't matter too much. Given the fact that there's pretty good audio cancellation there, but most we'll see. that's super, that's super interesting. And what about, animals naming each other? Because I'm not sure if it's true, but I believe I read somewhere once that for example, whales, have names for each other, that true? speaker-1: whales I can't recall the paper on paper on whales but definitely dolphins have names for each other. Dolphins have very specific whistle, but their own specific whistle. And when they want to address each other, they basically imitate each other's whistle. So if we were dolphins, then I would say Julius. And you would always say Julius, Julius, Julius, Julius. And when I want to address you, would say Julius. So they kind of imitate each other's whistle to address each other. This has also been found in orange-fronted conures. that they don't have their... ⁓ Yeah, they do match each other's skull to address each other. ⁓ And this has also been hypothesized in elephants, but I think then disproven and also in common marmosets, the small primates. There is some kind of either matching, so it can be a bit different. So either they have their own weasel and then the others match them or they have actually a specific They found that like several animals use the specific, the same specific sound when they want to address individual A or individual B. So there is some evidence for name like calls. speaker-0: That's so interesting, especially what you mentioned about having a different call for a different type of predator. And then also some animals that we think have ⁓ some form of names for each other. ⁓ That implies pretty advanced communication, much more so than just ⁓ a random scream, here's a predator or ⁓ here's food or something. so that's, that's, that's really, that's really fascinating. And do they then also, the examples you mentioned because human language, spoken language, ⁓ written language, if you will, ⁓ we have syntax, right? So we have a certain order of words. And even if we use the same words, we can use the same words and switch the order around or even. pronounce the same words, have sarcasm ⁓ and very sophisticated ways of using certain words. But if we're just looking at the order of sounds, ⁓ is that also the case for these types of calls where they're signaling a certain predator or names for example? speaker-1: Yeah, so that's ⁓ what I was mentioning before with the call combination, because people found that in, for example, in baplers, like in some birds and also ⁓ in some primates, if I'm not wrong, in more recent years. So people found that if they use a different order, so they can use some ⁓ small sound before another sound that will change the meaning of the next call. And sometimes also the order is important. So if you have two calls in one specific order and you change the order around, that gives a different meaning. But in terms of syntax, so we see that a lot in birds, but it's a bit different than human syntax. we use, at least used to call it syntax. think now it's more called phonological syntax or close to phonological syntax, but basically, and that's what I studied in Skylox during my PhD. So they have extremely sophisticated song with many, different, what we call syllables, like small units in. ⁓ And the difference with human language is that each unit doesn't have a meaning on itself. It's just a small sound and they put them together. But in many birds, including skyhawks, so they have a very specific way of ordering them. So very specific syntax, if you want. especially Skylarks, they have extremely large repertoire of up to 300 different small units per per male. and then when they come from the same, ⁓ region, ⁓ same location, so they will order them. I mean, some of these, syllables, they would order them in a very specific way, thick way that makes us, sent a phrase if you want of, ⁓ about seven syllables on average. ⁓ and so each male of the location will sing. these at some point in the song that this can be up to 40 minutes long, they will place this phrase. ⁓ And what I found is that so made some different locations can share syllables, actually quite lots, but they will never put them in the same order. So it's only mails of one given location that we make them in the same order. And they can have many of these location phrase if you want. ⁓ actually, interestingly, if you take So when you play the song of a bird from another location, they will be very aggressive to it because it might be a bird trying to look for a territory and stealing their own territory. So they actually react much more. But if you take these phrases and you place them inside the song of another, like we call that a stranger, like a bird from another location, then you will kind of fool them into thinking it's a bird of their own location. And even more interestingly, so if you do that, but then you mix the syllables around, then they don't recognize it anymore. So it's really the order that is important for them. speaker-0: So interesting. speaker-1: Similar and not to human language at the same time. speaker-0: It's kind of a, they have their own slang. That's so cool. And I mean, you, I, in my preparation, I indeed saw that you did your PhD on, Skylarks and we're, now talking about the very interesting results of your study, but how do you go about studying this? Like how in practice it seems so speaker-1: Thanks, Zaki. speaker-0: difficult to me to figure this out and to conduct such a technical study on animals. It's very incredible. speaker-1: So, yeah, so my PhD supervisor, Thierry Aubin, had done his own PhD a long time ago on Skylux as well. So that's, I think they were very interested in the team to continue that work. ⁓ And so when I arrived there, I mean, the first thing I want is to record many birds from many different locations around university. And then a very difficult task, especially at that time, that was around 2005, like we didn't have much AI there. so I basically had to do all the labeling of the C labels by, ⁓ by I, so going over is very long songs, looking at the shape of the C labels and then finding which one are similar or not. that was very young. And then I had like many episodes where people were like, ⁓ but I'm sure we can find a way to do it automatically. And I would send my songs away for hoping for that. then people would never come back to me. So I ended up, luckily I didn't wait. So I just ended up doing everything visually and, I don't know if now I could still recognize them, I could still figure out the, because at some point you know them by heart. So you figure out the shape and especially the phrases. Yeah. So you do lots of recording and analysis and then playback. when in the field and then play the sound to the birds and looking at how they approach the speaker. And then moving all the syllables around, like doing these artificial sounds and playing them, playing that to them. But yeah, it was a lot of time in the. Indefinite is nice. speaker-0: Super cool. before we make the segway into artificial intelligence, because that's something I'm really interested about, what about monkeys and primates? Because they seem like the most, like one of the smartest animals that are also very similar to humans. can, I was in Indonesia, I did a video about this actually, ⁓ where there's, ⁓ Macaques, monkeys at the Uluwatu temple in Bali, Indonesia. And these are one of the only, I believe, if not the only monkey species that learned how to trade valuables. They steal it from tourists and then they trade it back for food. And they are so advanced that they know which items are more valuable to us. So they know if they have a phone, they won't give it. back so easily, maybe if they'll steal a flip flop, they'll know like they want this back, but not as bad. And they have different things that they like better. There's incredible studies done on this as well. but pretty much their highest, ⁓ they, think eggs are, are, the most tasty, most nutritious food. So some of them want They'll only give the iPhone back for example if you give them an AK and they wouldn't want to trade it for an Apple for example. So all of this is to say is we are aware of some incredibly sophisticated behavior in monkeys. We've seen them play video games even. ⁓ I don't feel like they're mentioned all that often when it comes to animal communication. We often speak about dolphins, ⁓ about birds ⁓ and other animals, but for some reason I don't really hear all that much about monkeys. Why is that? Do they just not communicate that much or am I just not aware of the studies? speaker-1: So, I mean, what you refer to is the long tailed macaques, have close collaborator, Malena Fries-Hansen that I've been involved in some studies with. So I've seen these videos quite a lot also when she comes to teach in my courses. She actually built a whole... project called the Long Tail Macaque project where she was trying to help the species because they are very, you know, they're around the temples, they're around people and people think they're very common, but actually they're not. Then the populations are in steep decrease. there's a lot needed to help with the conservation of the species. Yeah, but they're very smart. They've learned to trade. But in terms of communication, yeah, surprisingly actually the closest relatives, mostly the apes ⁓ and also some, primates. They, I mean, they don't learn most of them. And there is some evidence from some vocal learning in, for example, common marmosets, but in most other species, they don't learn the vocalizations or they can't really, they can do accommodation, which means like matching each other's frequencies and so, but not really produce. and early new sounds ⁓ like dolphins or whales or elephants can do. So they have kind of limited capacities for vocal learning, which is why I think they haven't at least studied so much. But then in the past years, there's been a lot of work done by colleagues on bonobos ⁓ and also chimpanzees showing that actually it's not true that they are so basic, they can actually also combine, have some at least rudimentary. So combine calls and now people are starting to study them in much, much more detail using many more parameters to try to understand the context of production of sounds. yeah, so I think the main reason is for that because they have quite limited vocal learning abilities, ⁓ now they've been, so they've been a bit ignored, but now they've been studied more and more. speaker-0: Very interesting. I'm very curious what comes out of that because it seems such a logical place to start, right? Because they're so similar to humans that we kind of evolved from a certain strain of monkeys, some predecessors there. So it would make sense that they would also have the most sophisticated ⁓ way of communicating. So curious to hear about that. speaker-1: But yeah, the vocal repertoire quite limited surprisingly. So some people think that maybe human language evolved from sign language because they have, these species have quite evolved ⁓ sign communication, like visual communication, but not really vocal. speaker-0: Interesting. Okay. Well, we'll have to see what comes out of the studies, but we've briefly touched on it already. ⁓ you, ⁓ did this, ⁓ your PhD on, on Skylarks. was amazing work, but it was also kind of tedious work. and then comes artificial intelligence. ⁓ of course we've seen some incredible developments regarding AI over the past few years. ⁓ When did you first ⁓ consider to start using machine learning in your work and why? Can you take me back to that moment? speaker-1: Yeah. I mean, regarding the, the Skylarks, I was done with Skylarks after I moved to, so after my JG, I moved to London to work with on the, the Fido deer and goats. But then there was one colleague, Dan Stubble, who was an university there and was developing methods to automatically label songs. So I gave, gave him my songs and it kind of partly worked. guess now if we try now, it would be much better, but I think people still struggle with the... this kind of labeling of ⁓ bird calls, especially when it's, know, in setting with noise and background noise and wind and all these things. And when there's so many CBA bolts, it's actually very difficult even for AI. ⁓ But then I started to use, ⁓ I mean, I really started to use AI much later ⁓ for a project on pigs. after goats and fat a deer, I moved to Switzerland, like every academic, I went to all kinds of different places. In Switzerland, I was comparing domestic and wild species in terms of how they express their emotions. So horses and Shevaskis horses and wild boars and pigs to see how they express emotions. That led me to a project on pigs where we then gathered other collaborators. So that's a project called Project Sunwell where we try to develop a tool for automated emotion labeling in pigs. which would be very useful for the welfare. Because when we assess welfare nowadays, it's clearly defined as, at least by researchers working in the field, as with emotion at their core. So the emotion is the most important part of the animal, the welfare of the animal. But then when we monitor welfare, very often, especially through automated monitoring, you only monitor the health of the animal, like the physical health and not mental health. So we really wanted to ⁓ speaker-0: Yeah. speaker-1: build this tool for farmers, which we didn't manage to in that timeframe, but at least we managed to show that it was possible to have an AI that can classify the calls of the pigs based on whether they are positive or negative. So we had a huge database of calls from piglets to slaughterhouse pigs collected by all the teams. And then we knew, I mean, we had Assume their emotions based on the context. So if it's a context that is pleasant for the animal that they seek for and increase their fitness, then it's labeled as positive. And if it's a context that is unpleasant, in the fitness of the animal, it's labeled as negative. ⁓ And then we train the algorithm to recognize the image of the sounds. So it make an image which we call spectrogram, the frequency versus time. And then it's trained on the, it's basically image recognition. And that worked with the 92 % accuracy, so quite well across all core types and all ages of pigs. So that's the first time I really use AI in my team. then afterwards we've been using it more mostly for finding out the size of the repertoire of even other species. that was after I moved from Switzerland to Copenhagen. Now I work also on other wild species, including rhinos and zebras. And so we've used it mostly on zebras to find out how many core types do they have ⁓ by using all kinds of machine learning classification. speaker-0: Super interesting. And if we go a little back to the, to the pig study, ⁓ you mentioned labeling data there. ⁓ I assume that's one of the biggest challenges in using artificial intelligence is having a source of truth, right? Because, if we, yeah, it's, it's with any AI, if the data that you train the model on isn't perfect or isn't good enough. then the output isn't going to be very reliable, isn't going to be very, very good. So how do you go about making sure that the data you put in this model is accurate? And how did you achieve that 92 %? Because that's really impressive. speaker-1: Yeah, that's an important question. many people, at least outside of the field of, know the biology themselves, would say, haven't studied animals. So they think that, you know, you can just record animals, even like marine mammals, with a microphone, leave it there and then throw machine learning to kind of posture it and then figure out what they are saying. it's, It's definitely not true. mean, you really need to know what they're doing when they produce the sound. So who is doing producing the sounds and when and what is the context and what is the kind of infer meaning? ⁓ So first you need to have the behavior, maybe also the physiology at some points. ⁓ We had on the peaks, we had some with the heart rate monitor so we could see also how intense are their emotions. ⁓ So you need information on what the animals is doing. ⁓ also potentially how the other animals are perceiving these sounds. And then as you said, you need lots of manual labeling ⁓ before you can train an algorithm. And there's two types of algorithms. So the first one is that you have an algorithm that can find the codes, so the textile. And that's very tricky for pigs. Actually we have an... I mean, I some colleagues in Switzerland who seem to have solved the solution now and many people working on it as well, but we couldn't do it through the Soundwell project, for example, because when peaks grunt, it's so low in frequency that it's completely buried in background noise, especially in farms where it's very noisy. So the detection of high pitch sound is easy, like the screams and squeals, but the grunts is extremely difficult. So that part is very tricky, but once you have detected the sounds, then the classification, at least now we can We can achieve it because we have this database that we also put online now. So it's free, free available online. But yeah, we needed to achieve that. had, needed lots of recordings, which is, ⁓ was feasible in that project because we were six different teams that had worked on PICS before. ⁓ So we all had existing recordings, some from very, very long time on a lot of animals. And then we kind of gathered that together. And we added some situations we thought were missing. some French team went actually to slaughterhouses. We didn't have slaughterhouse recordings. So we could get that. Also, we added more positive situations like pigs running freely in a corridor or enrichment. So we added some situations we thought were missing. And then we eventually ended up with 7,000 calls, which might not seem. That high actually, if you really want a tool, need many more than that. But at least for training the algorithm, was some 7,000 good quality calls that were labeled as based on their emotions. Yeah. speaker-0: Yeah. And I mean, so as I understand it, ⁓ after you trained the model, ⁓ you, tested it and it was able to, reach a 92 % accuracy. but it also tells me something about other than AI is able to predict the, ⁓ the nature of the call, the, emotional state of the, of the animal. ⁓ also the fact that they just have these. that pigs have this range of emotions, that they do have different calls for different scenarios, which to maybe to some people might also be surprising because ⁓ contrary to monkeys, example, pigs seem ⁓ relatively simple animal, I may. I don't want to be disgraceful to all the pigs out there, but it's... ⁓ Yeah, surprises me that they have such sophisticated communication that it's ⁓ specific enough for different scenarios. of course, the humans, a pig squeal sounds like a pig squeal. Maybe a distressed one sounds a little different, but yeah. speaker-1: Yeah, so that's two kind of misconception, I would say. So one is that pigs are simple, actually they're very smart. They actually have been shown to have lots of cognitive, high cognitive abilities and at a level of a dog at least. So they're actually very smart among farm animals. I mean, I think all farm animals are much smarter than we think. I've studied goat cognition in particular and goats also really stand out in terms of being very smart. ⁓ But then also the other misconception is that emotions are, some people will be still surprised that animals have emotions, but actually emotions are actually very basics. ⁓ It's been shown that at least the, what we call the primary emotions, like basic emotions, like fear and lost and care and all these things, ⁓ they've been shown in rodents to be at least, and probably in old vertebrates to be encoded in ancient regions of the brain. ⁓ So basically that means that all vertebrates at least have these basic emotions and it's quite logical because it helps them to survive in the wild. it guides their decision into whether they should approach or avoid a stimulus and what to expect in the environment. And without emotions, wouldn't be as flexible as that. ⁓ That's clear that all vertebrates at least or even more, ⁓ even there is like emotion-like phenomena also in insects like bumblebees and honeybees and drosophila. ⁓ So they definitely all have emotions. Now we don't know, we can not know for sure how conscious they are about their emotions, but it's quite the expectation is that it's ⁓ more for continuum. So some species are very conscious about their emotions. So they have what we call feelings, they consciously process their emotions and others might be a bit less conscious. about emotions. speaker-0: That's very interesting. ⁓ did an episode which came out yesterday actually with Henry Schevlin, which is a philosopher. And I spoke to him about ⁓ whether it's possible that AI can ever be conscious. So we spoke about the definition of AI, but we also spoke a lot about animal consciousness because that's a logical starting point, right? If we're looking to determine whether something is conscious other than a human being. So I thought animals would be a logical starting point to determine that. And I spoke to him about the question, do you think consciousness is a spectrum or a binary thing? So you either are conscious or you're not conscious or are some beings more conscious than others? You seem to be suggesting that it is a spectrum based on the studies you read, your knowledge. speaker-1: I mean, in terms of what I've been, because I also have been discussing through my work on emotions about consciousness a lot and being part of workshops on consciousness. And actually some philosophers that were part of this workshop will actually place the consciousness limits like at the plants level. Like it's really different between people, someone like put it very high and so below. But I mean, like everything in evolution, it seems like we... always start thinking that it's binary, but then we always figure out that things are more of a continuum, which is actually logical. So I would think that considering how evolution works, makes more sense that it would be a continuum, like a spectrum of more conscious and less. speaker-0: ⁓ Yeah, think I agree. mean, even if I look at myself, when I'm dreaming, I'm kind of conscious, but I'm not as conscious as I am right now at this very moment during this conversation. I think even in myself, there's a spectrum of consciousness. speaker-1: Yeah, actually we tend to always think that humans are like the conscious creatures and are so conscious, but actually there is lots of research showing that we are very unconscious as well. there are many, many things we do without even thinking about it, or we do automatically. we often, yeah, a lot of the things we do are actually through our own conscious. ⁓ speaker-0: Yeah, it's a super interesting topic. getting back to ⁓ machine learning and animal communication, I believe after you finished the big study, you also did a study on more ⁓ generic ⁓ or multi-species animal communication and using an LLM to do some analysis there. Can you tell me about that one from last year, I believe? speaker-1: Uh, yes. Yeah. So, um, so basically, yeah, my main focus in the past 15 years also was, uh, as I mentioned before, like vocal expression of emotions. And I was comparing species in how they express emotions. Cause I was interested in whether we all express emotions in the same way. If we express emotions the same way as animals, which means we could understand each other, et cetera. So I've been comparing many, mostly ungulates, also a bit more, um, birds and. speaker-0: Mhm. speaker-1: and also common marmosets, but mostly ongulates. So over the years I had built this database of seven, I mean, many calls emotionally positive and negative of seven species of ongulates. And so I have a former PhD student who trained an algorithm to see ⁓ if we could differentiate positive from negative calls of all these species of ongulates by kind of throwing them together in the algorithm ⁓ to summarize it very briefly. So it's an algorithm that didn't work on the image, but on the extraction of code features. So it will extract lots of features of the sounds like duration, frequencies, and all these things. And then he trained them by species and also with all the species together. And actually we find that there is enough in common in how species at least ungulates express emotions that we can train kind of, we called it like an ungulate translator, like we can be. angulate emotional translator that kind of can differentiate between the emotionally positive and negative cause of all these species. speaker-0: Super interesting. And ⁓ we're talking a lot about determining ⁓ the emotional state of animals, right? ⁓ Which is incredibly important and incredibly interesting. But I think what some people may misunderstand about using AI to use AI for animal communication. is that we may someday have some sort of translation tool where we would be able to talk to our cats. You sent me the article where they discuss cats GPT. And yeah, it's not as simple and it's at this point very unlikely that we'll ever get to a stage where we have a machine that we can just communicate to our cats like, Hey, how was your day? Do you want any more tuna or something like that? Because there are certain problems that we would need to solve and they might be unsolvable. Can you maybe explain what some of those difficulties are and whether it's likely that we'll ever get such a translation tool? speaker-1: Yeah, I mean, I think we can have a translation tool, but not in the sense, as you mentioned, like asking a question in human terms and to the animals that would translate it, because just because they don't have these words in their own language. So I think we could have potentially a tool that ⁓ definitely decodes the emotion of the animal or maybe what they want, like the motivations. ⁓ So I'm just thinking back about the peaks. So we could differentiate with nine, we could trade an algorithm with 92 % accuracy to differentiate positive and negative calls, but also with 82 % accuracy, which of the, how many contexts did we have? 19, I think 19 different contexts. It actually was. So the calls actually differ between these 19 different contexts. So you could basically have a tool that can tell you. I mean, you can obviously see the context as well, but I mean, can tell you kind of what's the emotion of the animal at that time. ⁓ And also maybe what they want. ⁓ But then you could maybe talk to them with their own sounds, because now we can synthesize animal sounds very well. So you could potentially reply in cat meow, like to the cats. and then communicate with them like that. So that would be one way. ⁓ So definitely decrypt what they're saying in terms of emotions and maybe motivations and then talking back to them in their own sounds. But of course, when we talk about pets, we can already kind of talk to them because they actually very smart so they can learn so many words. It's been shown that. dogs, there's some very skilled dogs that can learn up to, I think, 1,000 different object names and remember them. So they can, they're very good. They're much better at decrypting out ourselves, I think, than we are decrypting themselves. So they can read us very well. ⁓ And they can also manipulate us into giving them food or understanding what they want. So we already have this communication, but of course, it'd be nice to have a kind of emotion translator because we Sometimes can get you wrong as one. speaker-0: Right, so if I understand correctly, ⁓ by default a cat wouldn't have a word for tuna, it would just know the emotion, I'm hungry, probably. speaker-1: I mean, if it really likes tuna, it might make a different meal when there is tuna because the emotion is different likely. they will have a very strong, ⁓ and it's been shown actually that cats, domestic cats have different sounds than the wild ones. I think when they're adults, usually they don't ⁓ meal more. Now I can't remember exactly the findings, but they... They basically mix some sounds to make them very appealing to us to ⁓ serve them or give them what they want. So they kind of manipulate us. But I think they would, they would, they could definitely have a very specific meal for tuna because they like very much tuna and it would be very aroused and positive. speaker-0: Interesting. But I guess what I'm trying to say, let's say there were all sorts of different five different plates in front of a cat. One of two now, one is Salman. One of them is mackerel. ⁓ One of them is the sea bass. I don't know, just a variety of different plates. One of them is chicken. ⁓ it would, and I hope I'm not disrespecting any, any cats here, but I would assume that it just has an emotion for food, might be super excited for the tuna one, but until we learn it that this is a tuna and this is a salmon and this we learn to associate it with different like the dogs you mentioned, right? That they could learn up to a thousand different things based on what we teach it. Then if we want to communicate with the, with animals better, then it would still require each individual animal. to learn something more about the context we gave it, rather than that we just have a tool that immediately understands what they are saying, because they don't have, probably don't have the same expressions for the difference between tuna, salmon, sea bass, mackerel. Like that's something we gave and to them, it might be just nice smelling food. speaker-1: Yeah, exactly. So we can probably teach them to react in different ways and ⁓ very easy. There is training methods that you can use. But otherwise, naturally there is no need for them to evolve a different call for tuna or for whatever. Actually, some animals have a specific name for food. When there is food, they can give food calls, which are the same as the predator calls. So it's like call about the presence of food and they... Usually give that in species where they need conspecifics to come to help defend the food or tear it apart or these kinds of things. but it would be, but actually what is, because now, you know, we always think about translating and being able to communicate with them. But now there is also another trend, which is to use, for example, these dog buttons for dogs. We actually train them to push button to tell you, want to go for a walk or I want to... So that's kind of... turning things around and training them to ⁓ produce our own sounds with what they can. So by pressing buttons, that's my real way to actually communicate. speaker-0: I've seen that, but I'm always curious because I've seen videos of a lady with like 200 buttons on the floor and her cat somehow knows which ones to press and it looks like the cat is definitely making certain decisions about where to walk and then pressing that very specific button. I've seen those videos with dogs as well, but I'm always curious. ⁓ Are we sure that's like, does that work? speaker-1: Yeah, so that's the big question. is a team in the US that is actually studying that scientifically now for a few years. So they're still at the earlier stages, probably need more research. But so far they found that they do seem to know what they are communicating. So it's not random. And it's not only what the owner is saying. So they don't only just imitate the owner. And they seem also to combine them in specific ways. ⁓ And then there is like some vague evidence also that is more sophisticated buttons like help button asking for help. That it might actually be a concept that they understand, but in some of the subjects and others, so it's still at early stages. But I think 200 buttons maybe is a bit much, but at least if you have these two buttons, I mean, through just association between the sound and the outputs, animals learn very well by associations. You can train even a fly, I think, like just learning to associate one thing with one location and one sound with one output. So that's how it works. then ⁓ so they learn and then they can basically ask us what they want. I even heard about, ⁓ not scientifically, right, but apparently videos of like cats communicating with dogs. So the dog asking the cat for the ball that is up the tower. And yeah, that's really cool. speaker-0: Really? That's awesome. That's really incredible. If we are talking about this gold standard translation machine ⁓ that we listen to what an animal has to say, the AI understands it and then produces the sounds that an animal can understand. Which animal species do you think is most likely? to be deciphered first, if at all. speaker-1: I would say, yeah, I mean, it would be the animals that have been studied lot. So maybe pets, dogs mainly are studied quite well. Cats a bit less, I would say. But otherwise species for which people have a lot of recordings. So for example, bottom of dolphins, there is a team in the US that has recordings for like 40 or 50 years. ⁓ actually won this price among the committee of this price called the Cooler Dolitteral Challenge. don't know if you've heard about it, but it's a price that is given every year. ⁓ So the winner gets a hundred thousand dollars for the research and then the one who will really crack the code will get more ⁓ like the big price. ⁓ And so this team won the prize last year and they have like 40 years or more of data. So I think on this kind of species where we know. start to know the system really well and have lots of recordings and training data and all these things. That's probably the species that we'll first be able to really communicate with in that way. speaker-0: Yeah. it's, it's, but dolphins are also one of the smartest species that we're, or, a very smart species at least, correct? speaker-1: Yeah, they're very smart. ⁓ speaker-0: Yeah. So it may, to me, it makes sense that the smarter, an animal species is the more likely that they are to develop a more sophisticated way of communicating, or is that a false assumption? speaker-1: ⁓ I mean, I guess it's true, but it's very hard to assess how smart a species is compared to another, just because we can usually not run the same tests. So ideally you would want to run the same cognitive tests on two species to compare them. But of course, you know, you cannot ask, I don't know, fish to climb a ladder or these things. So it's very hard to design a test that fits the species and it's the same across. ⁓ So it's always very hard to assess the level. And we often have false assumptions. for example, ⁓ honeybees, I've just read a paper yesterday that honeybees can count and have a concept of zero. you know, actually bumblebees and honeybees are among the smartest. Eventually they beat all the tests. They can even play and they play on their own. ⁓ so we, you know, if you don't know, you might think that these kind of species are not the smartest, but actually they seem to be on many levels. So there is a bit of misconception in which species is smart or not. And very often, like as we talk with the pigs, we think domestic animals are not smart because we don't want them to be smart because we want to be able to treat them as we want and eat them. At least, well, not me because they don't eat meat, but other people. ⁓ But we kind of want to downgrade them. And actually they are extremely smart on many levels. ⁓ But there is definitely research showing that What might drive complex communication is the size of the group. So species that live in complex social groups where they have to kind of separate and reunite and maybe communicate about many different things. they have all very complex communication. So it could be birds. There are some birds that are extremely, have very complex communication in that sense. Or it could be ⁓ dolphins, definitely orcas, all these species. speaker-0: Cool. And what about, it just came to mind, ⁓ an octopus. ⁓ octopus is always being named as one of the more intelligent species. But I feel like it's kind of trapped in its own body because a dolphin or a whale is able to make very loud, produces a very loud sound, right? Or a bird can sing and make sounds as well. But... Does an octopus even make sound? ⁓ so what kind of sound or how can they communicate? speaker-1: Yeah, they don't make sound, or at least that we haven't discovered yet, but I don't see how they would produce sound though. But they use visual communication a lot, I think they can use different patterns on the skin also. And they also have signals in this school, the little challenge last year, I think we had one proposal which was about a certain way of communicating with their tentacles. So they would make a certain sign. ⁓ when they kind of meet each other. So they thought it's like a sign to say hi or this kind of thing. So they probably do communicate more in visual, with the visual channel. speaker-0: Super cool. ⁓ And you briefly mentioned ⁓ something about zebras. I was just curious. We've discussed your previous work, ⁓ but what is it that you're currently working on? speaker-1: So, yeah, so since I'm in Copenhagen, so that's now seven, since seven years now, we also study African mammals. So we had a PhD student looking at zebras and another one at rhinos. And we're mostly interested there at how they use their sounds to communicate in the group and how they, if they use sound to decide when to depart and when to stop as a group and how the vocal communication is linked to the social network. ⁓ So this kind of question, so we mapped their repertoire and then we looked at if they use, when they make snorts, so snorts are not really for communication, mostly for cleaning nostrils, but maybe we thought they use these, when they hear these sounds, it means, okay, we're about to depart or someone wants to go. ⁓ And actually we didn't find evidence suggesting that they do use sounds, they use more visual cues. But that's kind of what we were looking at. ⁓ speaker-0: That's speaker-1: We also look at peach-fronted conure at the moment and many other species. Exploded even with pointers of the number of species we look at. speaker-0: Very cool. Lots of, lots of work to do still. ⁓ Where do you hope the field stands in, in 10 years from now? What do you, what do you think will happen and what do you hope will happen ⁓ with developments in AI and deciphering ⁓ animal communication? speaker-1: Yeah, I mean, AI, it allows us to study much more sounds because it allows us to detect the sounds. Of course, you have to put that in place, but seems to, least when you have a good knowledge of the species and the sound they produce, ⁓ you can build these detectors for many species. So then you can analyze many, many more sounds. And then through classification ⁓ algorithm, you can then classify them and maybe find patterns that you wouldn't find with other. types of analysis or by year. So you can kind of get better knowledge about all the different layers of sounds that they have, like of different types of calls that they produce. yeah, is, I mean, first there is more, I mean, of course more AI use and that allows for much more data to be analyzed and patterns to be found. But also I think the field of acoustic communication has really exploded in the past years. There's been many, many more people working in acoustics and also promoting acoustic communication through books and things like that. So people are more more aware of it and many more people are working in acoustic communication. think in looking at the pace of how much publication, coming interesting publications are coming out there, I think in 10 years we'll know so much more about animals. Of course, difficult because Like people working on human psychologists, they have only one study, species to study. But us working on animals, mean, there is like millions of species out there, it's more challenging. speaker-0: Yeah, no, that makes a lot of sense. Do you hope that if we start to understand animals better, the way they communicate, the way they are feeling at specific moments, do you think that that will also help in the way we treat animals? speaker-1: Yeah, definitely. I mean, first in terms of wild species, so knowing how they communicate, it also helps to monitor them. So you can monitor how many they are and you can monitor biodiversity through sounds. That's a very good means. And in terms of domestic and pets, then using this kind of algorithm, you can know what's your emotions and motivation. And so first that will bring us closer to both wild and domestic animals. And also it will help us to be more ⁓ prone to answer their needs. Or I can have like housing systems, for example, that are better designed to feed their welfare. speaker-0: Is there anything before we wrap it up today that you would like to emphasize or something that I forgot to ask that you think is still important to discuss? speaker-1: ⁓ No, I don't think so. Yeah. We covered lots, especially I think the main point to stress is when we discuss about the fact that you cannot just throw an AI in the sea and understand the animals. You really need to see what they're doing and study them and label many, many sounds before you can actually get them to work. speaker-0: Yeah, I think that's a good takeaway for today. Thank you so much Elodie. If people want to keep up with your work, there a place they can go? Maybe they can follow you somewhere or a website or something? ⁓ speaker-1: Yeah, we have a group website. ⁓ Should I put it in the chat maybe? speaker-0: Yeah, you can send it to me and then I'll make sure to put it in the show notes. Do we have a chat? speaker-1: Use a button for chat, but it somehow doesn't seem to open. Yeah, but that's. speaker-0: Yeah, maybe if you can email it later, I'll make sure to put it in the show notes. But thank you so much for today, Ali. I had a great time. This was super, super interesting. And yeah, I'm very curious to see how this field will develop. And I'll make sure to stay up to date because I think it's really important what you're doing and it's very cool work. So thank you. speaker-1: Thanks a lot. Have a great day.