Latest / Stage Talks with Bellingcat / "Extremism in Gaming Spaces" with Rachel Kowert and Linda Schlegel
Transcript
- 0:03You're listening to a Bellingcat Discord stage
- 0:06talk about extremism on digital gaming platforms.
- 0:09This week we're joined by Rachel Coard and Linda
- 0:11Schlegel to talk about extremism in gaming communities.
- 0:15Together they published a book that brings together
- 0:17a range of insights from world -leading experts
- 0:19in the field of gaming and extremism, and during
- 0:22the talk we'll talk about everything from radicalization
- 0:24to prevention methods, and from Call of Duty
- 0:27to real -world violence. You can find all of
- 0:30the links mentioned in the talk in the description
- 0:32of this podcast. The talk was hosted by me, Medel
- 0:36Zut, on Thursday the 20th of June in the Bellingcat
- 0:39Discord server. Welcome to this week's Bellingcat
- 0:45Discord Stage Talk. This week we're joined by
- 0:47Rachel Kord and Linda Schlegel to talk about
- 0:50extremism on digital gaming platforms, such as
- 0:53Discord. I'll first quickly introduce myself.
- 0:56This is my very first time hosting a stage talk.
- 0:58My name is Merel Zoet. My colleague Charlie is
- 1:00on a well -deserved break this week, which is
- 1:02I'm taking over. Let me introduce you to our
- 1:05guest. Rachel Cord is a research psychologist
- 1:07who specializes in the uses and effects of digital
- 1:10games. She's also the founder of Psych Guys,
- 1:13a multimedia content production studio for the
- 1:15science of games and pop culture. Currently,
- 1:18she is working as one of the primary investigators
- 1:21on the first grant -funded project from the Department
- 1:24of Homeland Security about games and extremism.
- 1:28And then we have Linda Schlegel. She is a research
- 1:30fellow at the Center for Applied Research on
- 1:33De -Radicalization. She also works as a fellow
- 1:36at the Peace Research Institute in Frankfurt
- 1:38and is one of the founding members of the Extremism
- 1:41and Gaming Research Network. I'm excited to hear
- 1:43about your work in this field. I'll first give
- 1:45the stage to Rachel, who will speak about radicalization
- 1:48in the gaming spaces, and next we'll give the
- 1:50stage to Linda, who will add on to this by adding
- 1:52some prevention efforts. After the talks, there's
- 1:55time for a Q &A. You can ask your questions in
- 1:58the chat. Please point out in your question if
- 2:00you do not want me to read out your name for
- 2:02privacy reasons. All right, let's get started.
- 2:05Let's give the stage to Rachel. Thank you for
- 2:07that great introduction. Hi, everyone. I'm going
- 2:11to do my best to be brief because I find that
- 2:14the Q &A is actually the best part of these conversations.
- 2:17But I do want to create a little bit of context
- 2:20around this field that Linda and I have been
- 2:23exploring for the last several years. Games and
- 2:27extremism, it is a growing area of interest.
- 2:29It is also a wide field with a lot of different
- 2:32interests within it. We have people from psychology
- 2:36and political science and communication science
- 2:38and conflict studies, and you get a little bit
- 2:40of everyone from everywhere. And their interest
- 2:43also lies in different parts of this umbrella,
- 2:46whether it's radicalization, recruitment, propaganda
- 2:50material, which is part of recruitment, but also
- 2:53prevention, which is what Linda is going to talk
- 2:55about. So I feel like I'm a little bit different
- 2:59in this field in that I came from game studies
- 3:01and moved into studying extremism and games versus
- 3:04someone who studied extremism and terrorism and
- 3:06then and then moved into games. I read the 2019
- 3:10report from the Anti -Defamation League that
- 3:12said one in four game players are exposed to
- 3:14white supremacist ideology online. And when I
- 3:17read that, it was really shocking to me as someone
- 3:19who'd played games my whole life, even for me,
- 3:21one in four seemed like a really high proportion.
- 3:24And I called up Daniel Kelly, he wrote the report
- 3:26and I was like, you did your stats wrong. And
- 3:28he was like, I did them right. It's bad. And
- 3:30I was like, oh, that's scary. And so that's when
- 3:32I pivoted my work into this area. And as Meryl
- 3:36mentioned, I'm one of the primary investigators
- 3:38on the Department of Homeland Security funded
- 3:40work and which we were tasked with essentially
- 3:43landscaping extremism and games. Like, what does
- 3:46it look like? What is the prevalence? What is
- 3:48like the flavor of the day? And what do we know
- 3:51about the ways in which these spaces are being
- 3:53intentionally infiltrated in a way, like intentionally
- 3:57used for mobilization or communication for designated
- 4:00terror groups, but also what are the vulnerabilities
- 4:03in this space for kind of the unintentional amplification
- 4:08or normalization of extremist ideology, which
- 4:12is really where my specific area of interest
- 4:15lies is looking at kind of what is the landscape
- 4:18of games and what makes them unique in the online
- 4:21landscape in terms of being able to be used this
- 4:25way for radicalization and recruitment and mobilization.
- 4:29Now, before I turn it over to Linda, I just want
- 4:31to mention one piece of my work that I've been
- 4:33doing in the last couple years that I find interesting,
- 4:36and maybe you do too, and it's looking at this
- 4:38idea of identity fusion. So one of the really
- 4:42interesting things about games is that friendships
- 4:44form different in games than they do on other
- 4:46online platforms. Because typically, when you
- 4:49meet someone, you slowly get to know them and
- 4:51slowly learn if you can trust them. But in games,
- 4:54these relationships are essentially emotionally
- 4:55jumpstarted. You have a foundational level of
- 4:57trust that begins at the beginning. Do you help
- 4:59me kill this dragon? I like you. I know nothing
- 5:03about you, but you helped me do this task in
- 5:05this game. And now I have this foundational level
- 5:07of trust. And that can emotionally jumpstart
- 5:09the closeness that you feel with other people
- 5:11in the gaming space. And research has found that
- 5:15this is being specifically leveraged for radicalization
- 5:18purposes. But what I found really interesting
- 5:20about friendship formation in games is it also
- 5:23lends itself to fostering this gamer identity.
- 5:27We have a lot of social identities, you know,
- 5:30I'm a baker, I'm a... whatever, I like to read
- 5:33books, I like to see movies, or I'm a gamer.
- 5:36And these identities are separate from our individual
- 5:38identity. They're usually like, I am Rachel,
- 5:40I am a gamer, they're different boxes. But identity
- 5:43fusion is this psychological concept that talks
- 5:46about when these identities become fused together.
- 5:48And the best example is military identities.
- 5:51My father was a Marine, once a Marine, always
- 5:53a Marine, like there's no tearing out Marine
- 5:55identity and identity of my father as like John.
- 5:58So we did some work with the University of Texas
- 6:01and we looked at identity fusion among game players
- 6:03and we found a couple things. One, that identity
- 6:06fusion with gaming and gamer identity does exist,
- 6:09can exist and can be measured and documented.
- 6:12But the other thing is that people who have this
- 6:14fused identity reflect. the extreme cultures
- 6:19and the normalization of hate and harassment
- 6:21and misogyny and racism that we see in gaming
- 6:24spaces. So gamers with a fused identity were
- 6:27more likely to say they've had recent aggressive
- 6:29behavior. They're more likely to register on
- 6:33low measures of low empathy on racism. They're
- 6:36more likely to overtly endorse the tenants of
- 6:38white nationalism. And we didn't see that with
- 6:41non -fused. players. So this is kind of my area,
- 6:44like, okay, well, if we're seeing this reflect
- 6:46the culture, how can we then change the culture
- 6:49of gaming spaces? And what does that look like
- 6:52from a psychological perspective, but also from
- 6:55a perspective of someone who interfaces a lot?
- 6:57with the gaming industry. So it's like, what
- 6:59can we see from the industry side? And then with
- 7:02that, I'm going to turn it over to my colleague,
- 7:04Linda. Thank you, Rachel. So just like Rachel
- 7:06is sort of coming from game studies, I'm coming
- 7:08at this from a different angle. My background
- 7:11is in radicalization research and prevention
- 7:13work. So I'm mostly interested in how can we
- 7:16twist this around? We know that sort of Extremists
- 7:20are using digital gaming spaces and video games
- 7:23in various ways. They're producing their own
- 7:25games, they're modding, they're using gaming
- 7:27and game adjacent platforms. They've used visuals
- 7:32from video games like ISIS incorporated Call
- 7:35of Duty footage in their propaganda videos, that
- 7:38type of thing. So they're trying really hard
- 7:41to exploit the popularity of gaming and gaming
- 7:44spaces and everything that comes with gaming
- 7:46culture. But we also already have research on
- 7:50positive effects of video games and of gaming
- 7:54spaces, including positive effects on the social
- 7:57aspects, how you behave towards others, your
- 8:00attitudes towards outgroups, perspective taking,
- 8:04all that kind of stuff that you would like people
- 8:07to take away from these video games. We also
- 8:11have research on that. But the narrative right
- 8:13now, particularly in the media, is very negative.
- 8:17It's like, oh my god, video games are dangerous.
- 8:18And we've already had that in the 90s, right,
- 8:21after Columbine. It's like, oh my god, everyone
- 8:23who plays first -person shooters is a potential
- 8:26school shooter, which obviously isn't the case.
- 8:30And so I'm trying to work on not falling into
- 8:32that trap again and not letting the discourse
- 8:35be super negative. We need to be very aware of
- 8:38what's going on in digital gaming spaces in terms
- 8:40of radicalization. in terms of extremist content,
- 8:43Rachel did work on mobilization and recruitment.
- 8:46We need to know all this stuff, we need to be
- 8:48aware of that, but then we also need to ask what
- 8:51can we do to prevent that and how can we use
- 8:54gaming spaces in a positive way. That's my focus
- 8:57and there are different ways of going about this.
- 9:01I'm just going to quick fire some things and
- 9:04then the Q &A, we can maybe go into a little
- 9:07more depth. The one thing that we've been doing
- 9:10from the research side is obviously awareness
- 9:14raising, trying to bring all the stakeholders
- 9:16on the same page and at the same table with the
- 9:21Extremism and Gaming Research Network that Rachel
- 9:24and I are both part of. We try to facilitate
- 9:26a dialogue with international organizations,
- 9:29police forces, but also gaming platforms and
- 9:31gamers themselves to just sort of... get a common
- 9:35ground and try to kickstart some sort of cooperation
- 9:38in terms of trying to prevent this and trying
- 9:40to counteract radicalization and gaming spaces.
- 9:45Obviously, there is a policy aspect, although
- 9:48a lot of the digital policies do not explicitly
- 9:51name gaming and gaming adjacent spaces, like
- 9:55social media platforms, for instance, are regulated
- 9:57now. in the gaming sphere, it's less so. So there's
- 10:00something that we could play with on a policy
- 10:03level. We have obviously the platforms themselves,
- 10:06community guidelines, moderation, sort of reactive
- 10:10measures taken by the platforms. That differs
- 10:13a lot per platform that we're talking about,
- 10:17but generally could be improved, I would say.
- 10:21And I believe Rachel also interviewed some of
- 10:24these platform hosters who said, well, we don't
- 10:29even know where to start. We don't really know
- 10:31what to do. We don't have the tools. So that's
- 10:34something that we could be looking at in terms
- 10:35of prevention. Then we could also look at gamers
- 10:39and users themselves, obviously. bottom -up approaches
- 10:43to prevention, and that starts really with flagging
- 10:47this kind of stuff. And that's already where
- 10:48the problem starts, because rarely do people
- 10:51flag extremist content in digital gaming spaces.
- 10:55We did a study for UNOCT, interviewed over 500
- 10:58gamers, and they said, no, no, we don't flag
- 11:02stuff, we just leave. We don't even engage these
- 11:05people, we don't flag, we don't do anything,
- 11:06we just leave. So that means... We have a problem
- 11:10because we don't even know where this type of
- 11:12content is most prevalent because we cannot police
- 11:14all these different spaces. So that's something
- 11:17that we could be talking about in terms of bottom
- 11:19-up approaches. What could we do to sort of engage
- 11:23gamers, empower them to take charge of their
- 11:26own discursive spaces and do something against
- 11:29extremism? And then lastly, which is my area
- 11:34of expertise, is sort of professional prevention
- 11:37work, often implemented by civil society organizations,
- 11:41sometimes state actors, but less so. It's usually
- 11:45stuff that is aimed to bring out a positive aspect,
- 11:49to raise awareness, to increase resilience, but
- 11:52also to offer support to victims of hate. And
- 11:57there are different ways of going about this.
- 11:59We see a lot of bespoke video games being produced
- 12:03from prevention actors right now to sort of combat
- 12:07hate in a digital game that they produced for
- 12:10the specific purpose of educating audiences against
- 12:14extremists. However, not too many people play
- 12:18this in their free time, often because there
- 12:21is not enough budget to make this a really cool
- 12:23game. So it feels very educational a lot of times.
- 12:28So it's not really working the way it's intended
- 12:31to at the moment. There's also some initiatives
- 12:34sort of on platforms, like digital youth work.
- 12:38People are trying to go into these spaces, into
- 12:41the chats, into the forums and try to talk to
- 12:44potentially at risk gamers about the prevention
- 12:48space and redirect them to other sources, try
- 12:52to engage them in some sort of way. And then
- 12:55we've also seen gaming references being used
- 12:58in some video formats or in some other content.
- 13:01Just like ISIS use Call of Duty footage, prevention
- 13:05actors try to reproduce the visual aspects of
- 13:07video games to just make their content more appealing.
- 13:11Because to be honest, a lot of the content right
- 13:13now in prevention is kind of lame. So using video
- 13:17game aesthetics would be one way to make it more
- 13:19appealing and use something that's so popular.
- 13:22to bring prevention space into the forefront.
- 13:27In the future, we could also talk about modding
- 13:31that might be easier than producing your own
- 13:34games. We could talk about what else could be
- 13:37done on platforms, live streaming, influencers,
- 13:41working with e -sports teams is something that
- 13:43hasn't been explored, but that I think might
- 13:46be very cool to do and reach a lot of people.
- 13:49And then, of course, sort of cooperation with
- 13:52the platforms. Right now, there's really a disbalance.
- 13:55There's not really a connection between prevention
- 13:57actors and platform holsters. So there's kind
- 14:01of a mismatch that we need to bridge in the future.
- 14:06But generally speaking, I think I always end
- 14:09every talk with gaming is really an opportunity
- 14:12to increase our prevention work to make it more
- 14:15appealing and to use the positive aspects of
- 14:18gaming that so many of you know and like and
- 14:22love. Also to for like democratic work, let's
- 14:26say. And with that, I think I'll give back to
- 14:29Meryl. Thank you. Thank you for the great introduction
- 14:31to your work. I see a lot of questions coming
- 14:34in in the chat, so we can just start with that
- 14:36right away. Tz18f, how much does it just come
- 14:41down to unmoderated slash self -moderated spaces?
- 14:45I saw that question and I'm and I'm curious what
- 14:47do you mean how much does it come down to? Like
- 14:49what why is it so prevalent? I mean because the
- 14:53spaces that we're looking at are spaces that
- 14:55are in theory meant to be moderated in some way.
- 14:59At the very least they have terms of service
- 15:01and code of conduct, but like Linda said, a lot
- 15:03of people don't report the things that they see
- 15:06because it has become a normalized kind of part
- 15:09of the culture and not necessarily seen as something
- 15:11that's meant to be reported. Do you have something
- 15:15to add? I would just say that reactive measures
- 15:18are usually not enough when you're trying to
- 15:20prevent extremism in the long term, so that might
- 15:22help in the short term in sort of deleting violent
- 15:25content, but then we often only delete the violence,
- 15:28right? And then what we're left with is sort
- 15:31of the inspiring utopian narratives that extremist
- 15:35organizations put out there. So we also need
- 15:39proactive measures to counter the appeal of that
- 15:41and not just sort of deleting. It is helping
- 15:45extremists, I think, if there isn't enough content
- 15:47moderation, but I'm not sure that's sort of the
- 15:50only way to go about this is to increase moderation
- 15:54efforts. Yeah, actually, that reminds me. I did
- 15:57a study a couple years ago in conjunction with
- 16:00Gamersafer and Middleberry looking at private
- 16:03Minecraft servers. And we were looking at the
- 16:06prevalence of hateful kind of extremist language
- 16:08across servers. And what we found is that the
- 16:11code of conduct and the policy that set the culture
- 16:14for the space was when it was more specific and
- 16:17more actionable, that was a better predictor
- 16:19of there being kind of less levels of this toxicity
- 16:22or whatever you want to call it in the space
- 16:23than moderation efforts. So it wasn't the servers
- 16:26with more moderators that were like less extreme.
- 16:29It was the servers that had better, clearer public
- 16:32policy. So There is a lot to be said about the
- 16:35bottom -up kind of self -moderating setting the
- 16:37culture of your individual spaces in terms of
- 16:39the power there for kind of changing the norm.
- 16:42Thank you. I see some other questions coming
- 16:44in about moderation. Do you see any examples
- 16:46of what is already being done in this space?
- 16:50What progress do you see? That's a question that
- 16:52Rogue State is asking. There's a lot of progress
- 16:55that I'm seeing in the industry. One example
- 16:59I'll give is Activision. of duty, which is often
- 17:02the example. They recently changed their policy
- 17:05of their code of conduct to include the phrase
- 17:07the amplification of hateful language, which
- 17:10I think was a really big step in the right direction.
- 17:13I think a lot of times people hide under this
- 17:15guise of normality or, oh, it's funny, or I didn't
- 17:18know, and so it doesn't become action when they
- 17:20start to spread extremist ideologies. And they
- 17:23also enacted voice moderation. preventative voice
- 17:29moderation, like there's a slight delay and it
- 17:30is like muting things before their broadcast
- 17:33to everybody in this space. And I think that
- 17:36that's a really great. change that we're seeing.
- 17:39I would add Twitch to that. We see a progression
- 17:43in terms of attack live streams. The Halle attacker
- 17:47tried to stream his attack. It took 20 minutes
- 17:50for Twitch to block it. And then during the Buffalo
- 17:54shooting, it took them only two minutes to block
- 17:57the live stream. So we see a real progression
- 17:59there in terms of Extremis content. We also see
- 18:02more and more gaming platforms or gaming related
- 18:05organizations joining like the Global Internet
- 18:08Forum to counter terrorism and they have sort
- 18:11of transparency reports. They have what is called
- 18:13a hash database where they flag extremist content
- 18:17with like a code and then you can go on your
- 18:18own platform and look for this specific type
- 18:21of content that has been flagged elsewhere. So
- 18:23there are all these progress being made, but
- 18:27it's all voluntary, right? So not every platform
- 18:30is participating in that. Yeah, could you talk
- 18:33a bit more about those terrorist live streams
- 18:35and how they've been repurposed by the far right
- 18:38in games and what have been done to remove those?
- 18:42I think the question is about sort of the how
- 18:45you can reenact attacks within games, right?
- 18:48We see that a lot in terms of modding. There
- 18:52are efforts to take them down. Roblox took down
- 18:55Russian propaganda. that has sort of used the
- 18:59Ukrainian war in some Roblox games. But it's
- 19:04kind of like a needle in a haystack trying to
- 19:08moderate these types of content. So there's definitely
- 19:13progress in that area too, but more to be done,
- 19:18particularly in sort of mod forums themselves,
- 19:21who usually don't participate in anything that
- 19:23I just mentioned. And they're we see actually
- 19:27a lot of this type of content being produced
- 19:31and being played and then shared in private chat
- 19:35groups and distributed through other channels
- 19:37that we don't even see overtly. So it's very
- 19:40difficult to take a hold of that and delete all
- 19:44of this. And then we have a question that Glib
- 19:47is asking. He's long wanted to know why Xbox
- 19:50is considered more racist, possibly more radical
- 19:53than PlayStation. Does this claim have any real
- 19:56basis in your experience and what is the reason
- 19:58behind it, if so? Sounds like a question for
- 20:00Rachel. In my experience, I have not seen research
- 20:04to support that any one place is more. A good
- 20:10kind of parallel I can draw is that Roblox is
- 20:12often given as the example for where kind of
- 20:15awful things are happening, but that doesn't
- 20:16necessarily mean it's happening more in Roblox.
- 20:19We might just have more insight, like they have
- 20:21an open API, like we might just have more insight
- 20:23into what's happening. in those spaces. But in
- 20:25the work that I've done, in the work that I've
- 20:27seen, there has not been any clear kind of differentiation
- 20:30set between platforms is happening more or less.
- 20:32It's really just everywhere. Yeah, and I think
- 20:37consoles are particularly difficult to research
- 20:39as well, because you need the equipment, right?
- 20:41If you're researching Steam, you can just log
- 20:44on and look at the forums. But if you're playing
- 20:47a console, and particularly if it's voice -based
- 20:50communication, it's very difficult to research.
- 20:53And I too don't know any research on that in
- 20:57particular. The next question is from SleepyPant,
- 21:01who's asking... De -radicalization is a very
- 21:04specific field. How do we help to promote it
- 21:06and is there a way of fellow gamers can spot
- 21:09and promote de -radicalization tactics? So this
- 21:12is kind of that bottom -up approach you previously
- 21:14mentioned Linda. I think we would need initiatives
- 21:19that are coordinated as well as sort of individual
- 21:24gamers taking initiative, right? It would already
- 21:26be helpful to have some sort of counter speech,
- 21:30but if you do that, you are putting yourself...
- 21:33on the spot, right? Then you could be the target
- 21:36of hate. That's why it's always good to have
- 21:38some sort of backing. Like we had on other social
- 21:41media platforms the hashtag I am here, maybe
- 21:44you've heard about that, where people just go
- 21:46in and sort of swarms into the comment section
- 21:49and try to be more positive. So that I think
- 21:53would be really helpful. But also, like I said,
- 21:56just The basic thing, flagging stuff, helping
- 22:00us identify where this work is needed, that would
- 22:04already be a huge contribution that every gamer
- 22:06can make in this space and can help us do the
- 22:10de -radicalization work. I do agree that it's
- 22:13rather specific and I don't think sort of lay
- 22:16people have an easy time trying to deradicalize
- 22:19someone. What we would need is points of contact
- 22:23and more information on who you could contact
- 22:26if you think someone in a digital gaming space
- 22:29is potentially radicalizing or is sharing beheading
- 22:33videos or whatever it might be to not just flag
- 22:35them to the platform and have him or her deplatformed,
- 22:39but to also flag that to a prevention practitioner
- 22:41to say, here, someone might be radicalizing.
- 22:45Can you take a look? So we would need that. And
- 22:47we would need the help of the platforms to do
- 22:49that, to have some sort of area that everyone
- 22:52can access. Here's just some information on prevention
- 22:56practitioners you can contact. The social psychologist
- 23:00in me wants to say, see something, say something.
- 23:02Just reiterating what Linda said, it's important
- 23:05to report. But also from the research, we know
- 23:08that the best way to stop these kinds of actions
- 23:10from happening in the moment is someone of a
- 23:12similar social status to say something and when
- 23:15they see something. So if somebody is saying
- 23:17something really racist or really misogynistic
- 23:19or really awful and they sound like a male sounding
- 23:22voice, another male sounding voice being like,
- 23:24hey, actually, that's not cool or something to
- 23:26that effect, actually deflates. the situation
- 23:28in the moment. So I think that on a mass scale
- 23:32would start to make culture shifts about what's
- 23:35accepted and what's not in these spaces and that
- 23:38is an important every day. All right, the next
- 23:44question is by Galen who says that he would imagine
- 23:48that games with violent themes are more likely
- 23:50to grow into forums for radicalization and extremism.
- 23:53Is that the case? And are there any recurring
- 23:56features of games that often co -occur with radicalization?
- 24:01I'll let you talk about the violence aspect.
- 24:04I think you're the expert on that. Are there
- 24:06reoccurring themes? We we did this UNLCT study
- 24:12and gamers themselves reported certain characteristics
- 24:15That they said they felt made it more likely
- 24:19that a game hosts hateful content That's it had
- 24:22if it has a large player base If you have to
- 24:26play with strangers, it's player versus player
- 24:28and not just against the computer And it's voice
- 24:32based which is difficult to moderate and these
- 24:35are sort of characteristics that may make it
- 24:37more likely that hateful content occurs. Other
- 24:40than that, there is some sort of weird obsession,
- 24:43at least for right -wing extremists, with historical
- 24:46war games. So yes, that's part of it. But I wouldn't
- 24:49say that sort of the content of the game necessarily
- 24:52correlates with the radicalization potential,
- 24:55if you will, because we also see it in Roblox.
- 24:58And Roblox is like a Lego type of visual, right,
- 25:02that has nothing to do with war and violence.
- 25:06For Rachel, to you. She wants to have in a soapbox
- 25:09a little. There's no causal link between playing
- 25:14violent video games and committing real -world
- 25:16violence. We know that from decades and decades
- 25:17of research. It's a misnomer. But what I like
- 25:23to talk about in terms of content, because a
- 25:25lot of times the question comes up about content
- 25:27because the focus is on kind of military sims,
- 25:29is that the content is the mood music for the
- 25:32radicalization. It's really about the social
- 25:34environment. It's really about the people. Yes,
- 25:36the mood music can enhance the mood or make it
- 25:41more amenable to have these conversations, but
- 25:43it's not required as a space for radicalization
- 25:48to occur. And I haven't seen evidence, like hard
- 25:52objective evidence that it more likely happens
- 25:55in games with violent themes. Again, like it
- 25:59just happens everywhere. It's just a perception
- 26:01perhaps that it happens more in these spaces.
- 26:04or these spaces are already more the target of
- 26:07people's attention when it comes to other studies
- 26:10in and around games and their uses in effect.
- 26:13Yeah, and that's an important research gap, right?
- 26:16So in the future, I'm sure we're going to see
- 26:18studies looking at sort of extremist groups on
- 26:21certain gaming platforms and then sort of the
- 26:24list of games that they play and try to find
- 26:26some sort of correlation. But even then, it's
- 26:29not a causation. It's just sort of indicating
- 26:31what games are most popular in these social circles.
- 26:35E -X -E -Q -Z also asked a question about the
- 26:38genres, but also asked which people are targeted
- 26:41more, whether children or teenagers are more
- 26:44subject to these extremist ideologies on gaming
- 26:47platforms. It's a really important question,
- 26:49and I'm glad you asked it, because there's different
- 26:53ways to look at it. One, younger people may be
- 26:55more susceptible to the messaging. They may be
- 26:57more impressionable, they may be more looking
- 26:58for peer validation, right? But older people
- 27:01have means and money. Like adults, not older
- 27:03people. over 18. From the case studies that we've
- 27:06seen, I've only seen a very small amount of case
- 27:09studies. They have focused specifically on teenagers,
- 27:11but that doesn't necessarily mean that they are
- 27:13being disproportionately targeted. I would think,
- 27:16Linda, please confer more to that. I would think
- 27:19that different groups are being targeted for
- 27:21different reasons, whether it's more identity
- 27:24formation and kind of people mobilization versus
- 27:26resources and actually enacting hateful acts.
- 27:29Yeah, and in general, radicalization is a youth
- 27:32phenomenon, far and large, from the data that
- 27:34we have. It's usually young males between the
- 27:38ages of, let's say, 13, 14, and maybe their late
- 27:4220s, who are the most susceptible to radicalization
- 27:45by far, in general, regardless of whether we're
- 27:49talking gaming or offline radicalization or something
- 27:51else. And I agree that it sort of depends, because
- 27:55particularly in the gaming space, we always think
- 27:57of young people, right? And that might be true
- 27:59if we think of Roblox, which has 60 % of its
- 28:02users under the age of 17 and something like
- 28:0520 % under the age of nine. So there, if we have
- 28:09extremist content there, we can say, yes, this
- 28:11is disproportionately targeting very young kids.
- 28:15But on other gaming platforms, I mean, the average
- 28:18gamer is, what, 35 or something. So it's not
- 28:22necessarily just young people. who come into
- 28:26contact with this. And we've been talking a lot
- 28:28about racism and violence stemming from gaming.
- 28:31But Sarah is asking if there have been any effective
- 28:34efforts to counter the widespread sexism as well.
- 28:37I wish. No, I haven't seen widespread efforts
- 28:41actually to really combat any of these kind of
- 28:44flavors of what I would consider extremist language.
- 28:47I did a landscaping study a year or two ago and
- 28:50I use these graphs a lot. So it's fresh in my
- 28:52mind where it's like misogyny. 98 % of people
- 28:56have witnessed it in games. Racism is like 95.
- 28:59White nationalism is like 85. Like these things
- 29:01are all so incredibly prevalent. I have not seen
- 29:05any concerted effort or united effort to kind
- 29:10of remove this content from these spaces. Although
- 29:13I do want to make the point that it is none of
- 29:15this is unique to games, right? Like misogyny
- 29:18is everywhere. Racism is everywhere. But What
- 29:22I find from the cultural perspective about games
- 29:25is games reflect culture, but they also create
- 29:28culture. So there is something to be said about
- 29:30the normalization in these spaces. There is some
- 29:32power to it being normalized in these spaces
- 29:34specifically, but also it's not unique to games
- 29:37as compared to just like the world. But I think
- 29:41a lot of gamers, female gamers, report that type
- 29:44of hateful content. And yeah, I don't think there
- 29:50are concerted efforts to approach that. And it's
- 29:54also, it is related to radicalization, but it's
- 29:59just one of several elements and it's not necessarily
- 30:02the most important one in a lot of spaces. So
- 30:06yeah, that doesn't mean it. You know, we shouldn't
- 30:10also be working towards preventing sexism, but
- 30:14I think it's so prevalent in gaming spaces. Like
- 30:16you say, it's everywhere. The next question is
- 30:19how do we draw a distinction between people in
- 30:22game role playing versus the actual spread of
- 30:25extremist content? someone's being nice to me
- 30:39and I feel like I'm making a friend, nobody.
- 30:42And that's what makes it really difficult. It's
- 30:44hard to distinguish. And when you have kind of
- 30:48loose, open rules, like, you know, this is a
- 30:52place where you can say anything and speech is
- 30:54free and we'll play welcome. And that makes it
- 30:57exponentially more difficult to kind of tease
- 30:59that apart, which is why, you know, you can't
- 31:01rely on wordless moderation or you can't just
- 31:03rely on AI moderation because people saying I'm
- 31:06gonna kill you on Call of Duty is a totally in
- 31:09context thing that you can say that should not
- 31:11be flagged or necessarily moderated. So it's
- 31:14really challenging. Yeah, and extremists don't
- 31:16start with like the super hardcore violent extremist
- 31:19content in any case, right? We call this soft
- 31:22pill type of content where it's like maybe it's
- 31:26an anti -semitic joke. you know it's just sort
- 31:29of testing the water is someone receptive to
- 31:31that type of content and then sort of slowly
- 31:34adding on to that it's not someone going in being
- 31:37like hey do you want to join a jihadist organization
- 31:40that's not how this works so it's a process and
- 31:43it's kind of like you don't really realize maybe
- 31:45what's going on and in which direction you're
- 31:48going. We had some teenagers arrested in Germany
- 31:51and they reported that. They were like, oh, we
- 31:53were just sort of making friends and they were
- 31:56also cool. And then they asked us to do a Hitler
- 31:59salute. And we did because we wanted to fit in
- 32:02and they didn't really realize where this was
- 32:04going. So that's the other problem with that.
- 32:08Now we have a question from Chrissy. who asks
- 32:11how can we ensure that people in gaming communities
- 32:14are not being unfairly reported? Based on this
- 32:17report there have been talks on how marginalized
- 32:19game community members have been unfairly reported
- 32:22and banned. My initial response would be if flagging
- 32:28is this rare we have a different problem that
- 32:31needs to be solved in the first place and then
- 32:33we can talk about falsely flagging but first
- 32:36people need to be more aware and be more willing
- 32:38to flag in general. I don't really have a good
- 32:42answer for sort of false flagging. That's also
- 32:46an answer. We need more answers. Fair enough.
- 32:51We have a question. It's from Squirrel. And she
- 32:53asks, have you identified events which have been
- 32:56used by the far right as recruitment drives,
- 32:59like Gamergate in gaming? What the commonalities
- 33:02of such events may be? How can they be identified?
- 33:05how they can be identified. There is a clear
- 33:08link between Gamergate and how that amplified
- 33:12recruitment efforts and kind of normalized the
- 33:14playbook for recruitment efforts. There's a great
- 33:17talk. I'm going to try to find it here. I'm going
- 33:19to pass to Linda. I'm going to Google furiously.
- 33:21There's a great talk from Constance Steinkuler
- 33:23from Games for Change last year where she essentially
- 33:26laid that out. She has a really nice graphic
- 33:28about how Gamergate set the playbook for a lot
- 33:31of the extremist content radicalization that
- 33:33we're seeing in gaming spaces now. Yeah, as far
- 33:37as I'm aware, Gamergate 1 was the start of sort
- 33:41of the alt -right, the alternative right -wing
- 33:44extremism. They sort of met through Gamergate,
- 33:49for lack of a better word. So it's definitely
- 33:51related. I would say in general, right -wing
- 33:55extremists in particular try to jump on any type
- 33:58of current event that they can exploit. And whether
- 34:01that's within the gaming sphere, like it was
- 34:04in Gamergate, or whether that's a young girl
- 34:07was being raped in Germany, for instance, by
- 34:10migrants. They would try to use that as well.
- 34:13Any type of current event, they would try to
- 34:16incorporate in the propaganda and to twist the
- 34:18narrative in their favor. Can you identify this?
- 34:21I think it's very difficult. You can be very
- 34:24sure that whatever happens is going to show up
- 34:26in your data if you're monitoring right -wing
- 34:29extremist channels. If it suits them, it's definitely
- 34:32going to show up there. I don't know how to stop
- 34:37that, but it's definitely something that we see
- 34:39a lot. Ketju, maybe for the people that are not
- 34:43familiar with this, elaborate a bit on what GamerGate
- 34:46is. GamerGate was a coordinated harassment campaign
- 34:50in 2014. It was a political movement guised under
- 34:55a philosophical stance of we want better ethics
- 34:57in games journalism, which was really kind of
- 35:00a smoke screen to just really publicly and terribly
- 35:04harass a lot of prominent women in gaming spaces,
- 35:07which led to online harassment, which led to
- 35:09offline harassment. And its effects can still
- 35:12be felt today, Kazza McDonald at The Guardian,
- 35:15wrote a really great piece on it this year, kind
- 35:17of like looking back to see how things have changed
- 35:19since Gamergate of 2014, 10 years ago, and the
- 35:23spoiler is not much, if not worse. Thank you,
- 35:27and all the links and sources that are being
- 35:30mentioned in the talk will be put in the description
- 35:32of the podcast as well for the listeners that
- 35:34are not here live. Please put your questions
- 35:36in the chat. I'll continue with the one from
- 35:39Shucky Ducky who asked, to what extent do larger
- 35:42gaming communities that span across multiple
- 35:45games, such as those originating from platforms
- 35:48like Something Awful, Reddit, 4chan or Kiwi Farms,
- 35:52play a role in cultivating or spreading extremist
- 35:54ideologies? How do these cross -game communities
- 35:57compare to game -specific groups in terms of
- 35:59their potential influence on extremist attitudes
- 36:02or behaviors? Really good question that we do
- 36:06not have an answer to empirically yet. It's something
- 36:10that we definitely need to look into. I would
- 36:12say there's a lot of overlap. We did just A few
- 36:16months ago we did an exploration of like 20 platforms
- 36:19and reddit was definitely among one of the worst
- 36:23that we saw, but a lot of the hateful content
- 36:27that we saw wasn't directly related to gaming.
- 36:30So there is an overlap between reddit communities
- 36:34and gaming communities, but they don't usually
- 36:36go to the gaming reddits. to the subreddits to
- 36:41talk about extremist content because they have
- 36:43other outlets. So to sort of untangle that web
- 36:47who is part of both types of groups and how do
- 36:51they interact is something that research hasn't
- 36:54uncovered yet. And that's a very interesting
- 36:56question, a very relevant question that I don't
- 36:59have an answer to yet. It's a great question
- 37:03because it highlights the fact that it's an ecosystem.
- 37:06I think that games We're pretty clear that games
- 37:11are often a first stop to where this contact
- 37:13is made. And then they spread into these other
- 37:16places. Not to say that first contact can't happen
- 37:19other places. It certainly can. But a lot of
- 37:21first contact is happening in games under the
- 37:22guise of, hey, you helped me do this thing. Aren't
- 37:24we cool? Hey, do you know of this really niche
- 37:29Reddit board that I find awesome? And we see
- 37:32that kind of off -ramping that way. But yeah,
- 37:35that would be a really difficult empirical. question
- 37:37to tease out, but does highlight the need for
- 37:40understanding kind of the ecosystem versus these
- 37:42spaces in isolation. And then Hershey Maple asks,
- 37:50when thinking about online countering violent
- 37:53extremism, we always have to consider the online
- 37:55versus offline activity. Addressing the ideological
- 37:58radicalization is important, but without translating
- 38:01it into offline violence, it can be difficult
- 38:03to measure the efficacy of intervention methods.
- 38:06They say gaming adjacent platforms like Discord,
- 38:09facilitating that transition from online to offline
- 38:12activity. If that hypothesis is true, what can
- 38:15be done to discourage online radicalization to
- 38:17become offline real violence? Another very good
- 38:20and very complicated question. So in general,
- 38:25ideological radicalization as this person called
- 38:27it or what we say cognitive radicalization is
- 38:31much more common than violent radicalization.
- 38:34So a large part of people who radicalize will
- 38:37never go and perpetrate a violent act. It's a
- 38:40tiny fraction that will go on to actually do
- 38:43something like that. How do we reach these people?
- 38:46Well, the first question is how do we identify
- 38:48them? And in the online sphere, it can be very
- 38:51difficult to identify them. While we know that
- 38:54a lot of attackers are sort of spilling their
- 38:57plans before they are actually perpetrating an
- 39:00attack. We also see a lot of people who are just
- 39:03talking a lot of blah blah online and pretend
- 39:07they are going to do something which they would
- 39:09never do in real life. So to sort that out is
- 39:12very difficult and again it would rely on people
- 39:17flagging that to some sort of authority or the
- 39:20platform for us to intervene. So it's a very
- 39:24difficult question. I'm not sure in how far we
- 39:28could really say it's purely online content facilitating
- 39:32also offline action. That's something that in
- 39:35radicalization research we're still sort of playing
- 39:37with. We know that this hard online offline distinction
- 39:41is no longer viable in our lives because everyone
- 39:44is chronically online, we all know that, but
- 39:47sort of what does it take for someone to go from
- 39:51being in an extremist discord server, for instance,
- 39:53to then go onto the streets and beat someone
- 39:55up. That step is so individual that we're still
- 39:59trying to figure out what triggers that type
- 40:03of action. It's a very good question. I'm so
- 40:05sorry to say this again, but I don't have a good
- 40:07answer to that. It's a good question. It's the
- 40:10questions we need to still research. But when
- 40:13you mentioned that, Linda, it reminded me of
- 40:14the study of the German teenagers who were radicalized
- 40:16that you mentioned earlier. They met on Roblox.
- 40:21They met this person who helped them do something
- 40:22that thought they were cool. They said, join
- 40:24my Discord of other cool people. They joined
- 40:25the Discord. The Discord was populated with people
- 40:28who are already radicalized. And they slowly
- 40:30asked them to do more things offline. So they
- 40:32asked them to do the Hitler salute, which they
- 40:33did. Then they asked them to go offline and commit
- 40:35rural violence, which they did not. And then
- 40:37they went to the police. So for them, there was
- 40:39a line there. They did already do some offline
- 40:42behavioral actions, which in Germany, doing the
- 40:44Hitler salute is a... illegal, right? They think
- 40:46that's a significant offense. But they drew the
- 40:50line at, you know, I think it was like going
- 40:51the street and punch somebody in the face or
- 40:53something. So understanding, you know, what does
- 40:56it take to cross that line, I think is really
- 40:58important. And if we're talking about that transition
- 41:01between online extremism and offline violence,
- 41:04this might be a relevant question. Geist asked,
- 41:07has there been any research done into VR communities
- 41:10or Discord as a third place for radicalization
- 41:13specifically? Institute for Strategic Dialogue
- 41:16has some pieces, I think, do you know of any,
- 41:18Linda? We've done just a very quick peek during
- 41:23our exploration of different platforms into like
- 41:26the meta, whatever it's called, the VR meta thing.
- 41:32I don't know. Oh god, I'm gonna get roasted tomorrow
- 41:36if I still tell that my team, I didn't remember
- 41:39the name of that. But we did a very quick peek.
- 41:42into this sort of virtual reality world. And
- 41:46we saw a lot of things reflected that we also
- 41:49saw in writing on other platforms. So a lot of
- 41:52racism and targeted hatred. But because it is
- 41:55sort of also voice -based, it's very difficult
- 41:59to even take track of that, right? To even note
- 42:02that down, to make data from it. So we just took
- 42:06a very quick peek. I don't know if there's a
- 42:09lot of... research on virtual reality. Yet, there
- 42:14are some theoretical pieces on the metaverse
- 42:16and what might happen in terms of extremism.
- 42:19I can post that as well. One of our colleagues
- 42:22has pieces on that. But I think that's sort of
- 42:27something to watch for the future, the virtual
- 42:29reality space. Then our next question is, I understand
- 42:33that bottom -up and preventive tactics would
- 42:35work to change extremist audiences. Can you go
- 42:38further into digital policies that governments
- 42:41might take? Also, how do you persuade close -minded
- 42:44actors who are spreading hate? Those are two
- 42:47different questions. Big questions. I think policy
- 42:50certainly has its place. If we're going to start
- 42:51talking about digital spaces, you know, and starting
- 42:56to overcome the amount of social interaction
- 42:59we're having as compared to like traditional
- 43:00offline spaces during COVID or even, you know,
- 43:03I probably... probably I for sure spend more
- 43:06time talking to people online than I do offline
- 43:08in my daily life. We're gonna have to start to
- 43:09talk about what does policy look like and what's
- 43:11the room for that. There was a piece that was
- 43:14published by the ADL a couple years ago where
- 43:17they asked gaming companies about their policies
- 43:19on extremist content. Specifically, they reached
- 43:21out to like 12 or 14 companies and only one even
- 43:25explicitly talked about extremism in their policy.
- 43:28So I think that... at least moving forward in
- 43:31the recognition that policy is needed and important
- 43:34is the first step. The game industry is 20 years
- 43:38behind social media in talking about and understanding
- 43:40the role of extremism in their spaces. And we're
- 43:44starting to catch up. We see the US House and
- 43:47Senate starting to ask questions, like we're
- 43:48starting to catch up. But my concern with policy
- 43:51is making sure that it is tailored to games and
- 43:54not just taking what we know about social media
- 43:56and throwing it onto games because All the scary
- 43:59bits about games are also all the best bits about
- 44:01games. So we have to figure out where is the
- 44:03balance here with governmental policy and regulation
- 44:07as compared to these preventative bottom -up
- 44:10measures or these measures coming from the industry
- 44:13itself who likely have a better touch point on
- 44:17what works and what doesn't in their space than
- 44:20a governmental regulator would. Yeah, I agree.
- 44:23And we're seeing sort of more interest from policymakers
- 44:26on all different levels, definitely. You Internet
- 44:31Forum, EU Commission on the European level, they're
- 44:34very interested in sort of the gaming space and
- 44:37have talked to platforms have talked to prevention
- 44:39practitioners have talked to researchers about
- 44:42this. So I believe. There will be something coming
- 44:45in the future because the problem is being recognized,
- 44:48but I agree with Rachel. We need to be very careful
- 44:51with what we're doing to not go overboard, particularly
- 44:55because there is quite some backlash from parts
- 44:58of the gaming community. So keep politics out
- 45:02of gaming. We want nothing to do with this. Gaming
- 45:05isn't a political activity. you know, we don't
- 45:08want to be regulated, this is a free space, it's
- 45:11our leisure time, that kind of thing. And sometimes
- 45:13very aggressively. And so we need to sort of
- 45:16be very careful in how we frame things and that
- 45:18we don't portray gaming as inherently dangerous
- 45:21or gamers as inherently at risk of radicalization
- 45:25on the policy level. So yeah, lots of things
- 45:29to do in the future. But I think there are two
- 45:32lots of things that are currently happening.
- 45:34Our next question is from Shucky Ducky who asked,
- 45:37Since the October 7 attacks and subsequent military
- 45:40operation in Gaza, there's been a noticeable
- 45:42increase in both pro -Israel and pro -Palestinian
- 45:46messaging on social media platforms. Have you
- 45:48observed similar trends within gaming communities?
- 45:51If so, to what extent do these appear to be individual
- 45:54expressions versus organized efforts? How do
- 45:57researchers distinguish between spontaneous individual
- 46:00actions and coordinated campaigns in these spaces?
- 46:03Another great question. We can't really distinguish
- 46:07right now. It's very difficult to understand,
- 46:11particularly because a lot of extremism nowadays
- 46:14isn't tied to a particular organization. A lot
- 46:19of it is sort of free -floating networks of radicalized
- 46:23individuals who are not swearing allegiance to
- 46:25this group or that group in particular. So it's
- 46:28very difficult to judge what is a coordinated
- 46:31action and what is just a bunch of individuals
- 46:34expressing their opinion. We have definitely
- 46:36seen an increase in anti -Semitism and also anti
- 46:40-Israel propaganda or comments and also on the
- 46:46other side against Palestinians in the gaming
- 46:48space. I think in general, if this goes back
- 46:51to the first point that Rachel made earlier,
- 46:54that gaming spaces are discursive spaces for
- 46:58the whole of society. So everything that is being
- 47:01discussed in general is being discussed in gaming
- 47:04spaces and is being fought over in gaming spaces.
- 47:08And this is not an exception. So yeah, we definitely
- 47:11see Gaza as one of the prominent themes right
- 47:14now. The next question is from L. Werewolf who
- 47:17asked, has there been any research done on how
- 47:20to effectively reach younger slash gamer audiences?
- 47:23specifically with this messaging. Sometimes these
- 47:26kind of interventions like anti -drug campaigns
- 47:28aren't messaged properly and end up being ridiculed
- 47:31or ignored by their audience. Yes, a hundred
- 47:36times yes to that. Unfortunately, a lot of prevention
- 47:40content is quite lame, just a lot of other educational
- 47:44content and that's a real problem in the gaming
- 47:46space in particular. We're just not doing a good
- 47:50job at the moment to use the appeal of video
- 47:53games to sort of bring prevention content to
- 47:56the forefront of attention. It's definitely necessary
- 47:59and I don't think It makes a lot of sense to
- 48:02make this type of content that you're talking
- 48:03about, which is super cringy. And the kids are
- 48:07like, oh, no, let's get that away from me. I'm
- 48:11never going to listen to that. We have to tailor.
- 48:15communication efforts to our target audience.
- 48:19And so we need to change the way we do things.
- 48:21We have started using TikTok in prevention efforts
- 48:25and we also need to start talking about how to
- 48:28use gaming spaces in the way they are intended
- 48:31to be used and not just, you know, take something
- 48:34from Facebook and put a link on Discord. That's
- 48:38not the way we can go about preventing violent
- 48:42extremism. Also, I just linked in the chat, I
- 48:46did, I take this, we did some work where Nielsen
- 48:49provided us some data and we found that the younger
- 48:52players are actually less tolerant of this generally
- 48:54and tend to speak with their actions of like
- 48:57leaving gaming spaces or not playing them or
- 48:59not engaging in them. So I think that we can
- 49:01have a little bit of optimism that the younger
- 49:04people coming up maybe have some kind of resiliency
- 49:08that they're bringing to the table that we don't
- 49:11see. All right. We are in the final minutes of
- 49:14our talk today. So please, if you have some final
- 49:16questions, put them in the chat. I can make sure
- 49:18to ask them before we end. We have another question
- 49:21from Chef Raoul, who asked, one issue I see is
- 49:25the inconsistency in moderation among games.
- 49:27Should big game publishers be doing more to help
- 49:30smaller games, trust and safety teams with this?
- 49:33I think this person is my soulmate. Yeah. I have
- 49:39been pushing for many years about consistency
- 49:42across the industry standards, essentially. One
- 49:44of the real big challenges in gaming industry
- 49:47as the way the industry is just set up is that
- 49:51everyone is in their individual silos, and they
- 49:53have their proprietary data, and they have their
- 49:54own shareholders, and they do their own thing.
- 49:56I did some round tables, as Linda mentioned at
- 49:58the beginning, where we talked to people at big
- 50:00studios, at little studios. And one of the things
- 50:02that really stood out to me is that everybody
- 50:04is doing their own thing. Everyone makes their
- 50:06own word lists. Like, there's not even a consistent
- 50:08word list and I'm like, why? and some people
- 50:11have subject matter experts and some people it's
- 50:13their community manager making them and it's
- 50:15like why is there such a disparity and consistency
- 50:18across these spaces. I don't think it's necessarily
- 50:21about bigger studios helping littler studios
- 50:24although they do have more resources. I think
- 50:26it's about the industry coming together as a
- 50:28whole to create like one set of resources. I
- 50:31think that GiveCT could be a potential place
- 50:34for that to happen. Linda mentioned them earlier
- 50:37but I think that that is a huge huge piece that's
- 50:41missing and desperately needed, but it's very
- 50:44difficult to get everyone to the table when,
- 50:47you know, their businesses and they have shareholders
- 50:49and proprietary data and, you know, this, that,
- 50:51and the other thing. Yeah, I would agree. And
- 50:55there are also organizations like Tech Against
- 50:57Terrorism who do exactly that for social media
- 51:01already, right? They're helping smaller platforms
- 51:04with putting in place these sort of policies
- 51:08and these processes that you need in order to
- 51:12implement or to counter violent extremism on
- 51:16their pages. But we don't see that with gaming
- 51:19platforms and gaming studios yet. So this is
- 51:22definitely something that has already been tried
- 51:24in a different area and could definitely be also
- 51:26relevant for the gaming sphere. This is very
- 51:30relevant to a question that Rox Ox is asking.
- 51:32How have conversations been with current game
- 51:34developers and platforms? I can see from a consumption
- 51:38standpoint how they would have monetary incentives
- 51:41to not alienate core players. So this year at
- 51:44GDC, I gave a talk that was called The Return
- 51:48on Investment on Trust and Safety. And the first
- 51:50slide opened with toxic players are not your
- 51:52core demographic. There is a perception, absolutely,
- 51:56that I am a one woman show fighting against.
- 51:58I mean, I'm sure there's other people too. I
- 52:02understand the impression that you don't want
- 52:04to alienate players, but the players that you
- 52:05don't want to alienate are such a tiny percentage
- 52:08in proportion of the amount of people who are
- 52:10playing your games. And there's a lot of money
- 52:12that's actually being left on the table by allowing
- 52:15these spaces to fester with hateful extremist
- 52:20language. How do the companies receive this?
- 52:23It varies greatly. Some people are super receptive.
- 52:28Some people are in denial. So it's really a span.
- 52:34And some large companies super into it, and some
- 52:36not. Some small, some not. There's no real pattern.
- 52:39I think it just really depends on the person
- 52:42in decision -making power and what their opinion
- 52:45is. I think that the researchers here can relate
- 52:49and understand that facts don't always change
- 52:51minds. So even if I come with all the data in
- 52:54the world, like the links I posted earlier of
- 52:55the work we did in collaboration with Nielsen
- 52:58that shows like there's a lot of money being
- 53:01left on the table that doesn't necessarily change
- 53:03their impression of what they think these policies
- 53:05might do to their bottom line. I agree and not
- 53:09to sort of defend anyone but we also have an
- 53:11issue that we also have in sort of social media
- 53:14generally that it's often national legislation
- 53:17that applies very differently depending on the
- 53:20country that you're talking about in terms of
- 53:23deleting certain content. A lot of the servers
- 53:25are in the US and that's very sort of free speech.
- 53:29you can only delete violent content basically
- 53:32but not hateful content you know someone can
- 53:34say it's they're a nazi they can post a hitler
- 53:37salute or whatever because it's not illegal there
- 53:40So there's also a legal side to this. What can
- 53:42we delete? What is freedom of expression versus
- 53:45what is keeping gaming spaces safe? There's always
- 53:47this tension and it plays out differently in
- 53:50different national contexts and in our increasingly
- 53:53globalized world and particularly in the gaming
- 53:56sphere, which is super international and everyone
- 53:58speaks English, et cetera. It's so difficult
- 54:01to sort of navigate that legal field as well.
- 54:04So that's one of the issues, I think. All right,
- 54:07I think we're off to our last question, which
- 54:09will be from Therese, who asked, do you see a
- 54:12resemblance of the bunch of guys radicalization
- 54:14theory in gamers? Could this help to explain
- 54:17how seemingly lone wolf actors are radicalized
- 54:19when it comes to gaming? a bunch of guys if you
- 54:23don't know it's a theory by Sageman who talks
- 54:27about sort of groups of people coming together
- 54:30and then they sort of start a radicalization
- 54:32process and it's a group process they kind of
- 54:35play off of each other and people who don't want
- 54:37to participate they leave the group and so the
- 54:39group keeps getting more radical I think This
- 54:44is part of what we're seeing definitely in gaming
- 54:47spaces, particularly because we know people leave
- 54:50gaming spaces when they don't like the content
- 54:53being shared there. So then you have a sort of
- 54:55self -selection into certain spaces. And yes,
- 54:58these group processes can then enhance, let's
- 55:01say, certain beliefs that you already had. They
- 55:05are not sanctioned, but they are applauded in
- 55:09these type of spaces. So you feel validated,
- 55:11you feel a sense of belonging identity, which
- 55:13Rachel already touched upon. So it's definitely
- 55:16part of it. I'm not sure if it explains to the
- 55:19lone actor one that we only have like a minute
- 55:22left. I think that's a whole new talk on lone
- 55:25actors versus group processes of radicalization.
- 55:28But definitely, I think it applies in the online
- 55:30sphere as well, including in games. What she
- 55:34said. Good answer. Oh, great. Thank you so much,
- 55:38Linda and Rachel. for giving us some insight
- 55:42in your work. And thank you everyone for your
- 55:45questions. It was a lot of people, a lot of questions.
- 55:48I don't think I've been able to ask all the questions,
- 55:50but I'm sure the conversation about this will
- 55:52continue in our Discord server. Thank you very
- 55:55much. We'll be back in two weeks with a new Discord
- 55:57Stage Talks, and Charlie will be back in that
- 56:00week. Thank you very much. Thank you. Thank you
- 56:03for listening to the Stage Talk. If you'd like
- 56:06to catch a stage talk live where you can ask
- 56:08the guests questions, join the Bellingcat Discord
- 56:11server by visiting www .discord .gg slash Bellingcat.
- 56:19The music you've heard is titled Dawn by Newer
- 56:23Self and is courtesy of Artlist.