Latest / CXOTalk / How AI Swarms Weaponize Disinformation | CXOTalk #915
Transcript
- 0:00It's an arms race, but we have not joined the war.
- 0:03We need to be better than the adversaries that use them, but
- 0:07for that we need to join this arms race whether we like it or
- 0:09not. AI swarms are the most dangerous
- 0:13influence weapons ever built. Daniel Tilo Schroeder and Jonas
- 0:20Kunst Co LED a 22 author study published in Science to map this
- 0:26threat. Daniel, what are AI swarms and
- 0:31how do they fabricate reality? The new thing here is that we
- 0:36see coordinated behavior. There's a whole bunch of
- 0:40studies, in particular from 2024-2025 that shows that large
- 0:45language models, for example, are super convincing, even
- 0:48though they lack capabilities to reason.
- 0:52They are very good talking you into something or talking you
- 0:56out of something. Just for example, a big study
- 0:58that shows that. So you can kind of like drag you
- 1:03out of conspiracy beliefs. And AI Swans go one step
- 1:07further. So they would kind of like
- 1:10combine the power of these AI agents and just like start
- 1:18coordinated behaviour in order to convince the body of society.
- 1:25Given that you suddenly are not just communicating with with one
- 1:28agent, but whether you're exposed to a swarm of agents
- 1:33that that really provides a new arsenal of weaponry.
- 1:37So you can, you know, fabricate agreement by seeding narratives
- 1:42and and make them create an illusion of majority support for
- 1:47these these opinions. So it's it's really a next level
- 1:50of, of information warfare. Why does all of this matter
- 1:55today? So please dive back in and and
- 1:58then Jonas, you can please add your thoughts.
- 2:01To this, the real new aspect here is the coordinated behavior
- 2:07and. The fact that.
- 2:11Access to this I, I models and I inference became cheaper and
- 2:16cheaper. Meaning that this agents can be
- 2:21longer and sustainable over extended periods of time in
- 2:26online social networks compared to all the campaigns where there
- 2:30were there for special for for special purpose, let's say an
- 2:34election. Meaning this agents can
- 2:37collaborate to work on influencing opinions over
- 2:44extended periods of time and not as obvious as they used to do
- 2:48that before. On top of that, they would be
- 2:50able to understand local environments and adjust process
- 2:57narratives in a way that they are observed by the local
- 3:01environments in an optimal manner, so to speak.
- 3:06I think these are these are really the new aspects.
- 3:09Yeah, we have this coordination, kind of like strategy
- 3:12development combined with the convincing capabilities of large
- 3:17language models. I've just said they might not be
- 3:20good with reasoning, but they're extremely convincing.
- 3:23This is what research from the past two years has shown, plus
- 3:27this agentic AI aspect, meaning capabilities to observe, to
- 3:32understand structure, network structure.
- 3:35For example, an agent of AI swarm could for example decide
- 3:40to attack a particular central network or central node in a
- 3:44network. So network centrality and
- 3:47language of stores to do complex network research means very kind
- 3:52of like connected node, right? So it might be easier to
- 3:56convince a person that's particularly vulnerable and has
- 4:01a million followers than convincing those million
- 4:04followers. Yeah.
- 4:05And I, I just want to add equally, you could aim to inject
- 4:10these type of nodes yourself to, to get to position your, your
- 4:14agents in a way that, that are central to the network, thereby
- 4:18boosting, you know, the pathways to which you can, can influence
- 4:22entire groups. And in terms of what Danny said,
- 4:26I, I just want to add that in terms of this coordination, you
- 4:29hinted at it, Danielle, it's very, just very important to
- 4:32highlight that they, they are increasingly able to coordinate
- 4:36autonomously. So, so there is, there is
- 4:40decreasingly needs for human oversight.
- 4:44So, so we're kind of moving from, you know, a central
- 4:47command that usually was at play in traditional bot networks to
- 4:51kind of emerge in hive behaviour.
- 4:54I, I think, I think that is a, that's a very important aspect
- 4:59to, to keep in mind. And in, in addition, the, the
- 5:05swarms, while they coordinate, they will be self optimizing,
- 5:09meaning that they can, they can run a lot of small experiments
- 5:13testing out different messages, learning from the feedback these
- 5:17messages produce and thereby can propagate the winners that are
- 5:21most efficient in, in influencing others.
- 5:24So, so, so that that really gives them an edge in terms of
- 5:30boosting their, their, their chances of success.
- 5:33Would it be accurate to say that the key point here is the
- 5:38communication among these many 10s, hundreds, thousands, even
- 5:46millions of AI bots and the autonomous action that arises
- 5:53from that? Is that a correct statement?
- 5:55I think it's a very important aspect, right?
- 5:58Communication and maybe it's a very central data structure that
- 6:01represents the outside world, which is changed by this
- 6:05multitude of agents and used for coordination is probably very
- 6:10central from a technical perspective.
- 6:13However, I think the convincing capabilities of large language
- 6:18models in combination with cheap inference and the extended set
- 6:25of tools, right. So an agent somehow can these
- 6:29days control and we have seen this with frameworks like Open
- 6:33Clone or several online social networks at the same time.
- 6:37And so like we go more towards a persona centric, persona centric
- 6:47approach then one in which individual online social network
- 6:53accounts are simulated. Jonas, can you elaborate on this
- 6:58idea of a persona centric approach?
- 7:03It's very interesting. Previously you had AI agents
- 7:07that were quite, quite crude and simple in their behaviour.
- 7:12They, they, they had certain decision rules on, on how to
- 7:15behave. But now you've kind of infused
- 7:18them with the cognitive abilities of large language
- 7:21models. So, so you, you suddenly have a
- 7:25much more capable AI agent. And as this cognitive ability is
- 7:32basically reflecting human ability to a certain extent, you
- 7:36also have a much more capable collective that can talk to each
- 7:40other. So, so, so I think that is
- 7:43really the the qualitative shift from previous influence
- 7:48operations, the idea. That one agent is not
- 7:53representing an individual account and let's say an online
- 7:57social network, but as a multitude of those.
- 8:00So there could be an agent Michael that's very persistent
- 8:04and mimics a persona in a way that it has an e-mail account,
- 8:10that it has a Twitter account or X account, blue sky account and
- 8:15a Facebook account. And this also means that for
- 8:21example detection has to be something that is working
- 8:26platform independent. How significant is this shift
- 8:32from traditional bot networks to AI swarms in terms of the the
- 8:40nature of the threat and the the impact?
- 8:43We are very concerned in terms of the threats this has for
- 8:46democracy. And, you know, unfortunately
- 8:51some of the capabilities that these forms have are really, you
- 8:55know, targeting the pain points of democratic discourse.
- 8:59So first of all, the if you can simulate human collectives to a
- 9:06very high degree of authenticity, you can
- 9:10manufacture synthetic consensus on social media platforms.
- 9:15People are generally conformist, even though we don't want to say
- 9:18this about ourselves, but if most people believe something,
- 9:21we we assign some credibility to it.
- 9:24So, so, so this this social holistic is kind of hijacked by
- 9:29these forms. But at the same time, you could
- 9:32also see scenarios where these a ice forms seed fragmentation.
- 9:37So they could, they could bolster different narratives and
- 9:40different opposing communities and and thereby polarize
- 9:44societies even more. In addition, we we, we also
- 9:49concerned about their potential to orchestrate targeted
- 9:53harassment, to exclude or suppress critical voices.
- 9:57These could be politicians, journalists, whistleblowers, or
- 10:01even their families. Think about that, right?
- 10:04And who, because of this targeted harassment, might, you
- 10:08know, withdraw from, from public discourse.
- 10:13Something we, we also discuss is that if these forms become very
- 10:17dominant on social media platforms, people might start to
- 10:21distrust any type of consensus or individual opinion.
- 10:26And, and, and, and that might really, you know, induce some
- 10:29type of epistemic Vertigo and possibly drive users to get a
- 10:35channel. So people disagree on on whether
- 10:37that's a good or bad thing, but at least it it reduces discourse
- 10:42across, let's say political affiliations, etcetera.
- 10:47And and finally, related to this point about being able to run
- 10:53thousands of different experiments, tests all different
- 10:55messages. Of course, you could see
- 10:57scenarios for these AIS forms kind of micro target different
- 11:02social groups to create a suppression of voting intentions
- 11:09or, or even mobilize other groups to, to vote for a certain
- 11:13candidate. So I, I think the, the, the, the
- 11:18breadth of this, these possible threats are just enormous.
- 11:21And right now we just talked about social media platforms.
- 11:23You could see this happening in in other channels as well.
- 11:27What about disinformation that spreads by accident as content
- 11:33gets slowly distorted through reproduction, like a copy of a
- 11:38copy of a copy? This is a question from Anthony
- 11:41Scruffignano, who's a data scientist who's been studying
- 11:45these problems and has been a guest on CXO Talk a number of
- 11:48times. So So what about that
- 11:50disinformation spreading by accident as the content gets
- 11:53slowly distorted? That's really, again, something
- 11:57that an AI swarm can monitor and adapt to continuously, something
- 12:03that would be much harder with a traditional bot bot farm of some
- 12:08sort where you still would need a human to to track how the
- 12:12narrative evolves, right? And AI swarm might evaluate
- 12:17continuously whether the, the, the way this information is
- 12:22presented currently, it's beneficial for the the swarm's
- 12:28goals and accordingly react, you know, maybe try to steer it in a
- 12:32different direction or support this direction if it seems
- 12:36positive. So I think that is probably
- 12:38where, you know, these adaptive capabilities give an advantage.
- 12:44I would assume that the capability to constantly monitor
- 12:48this narrative and omnipresence of that swarm would allow to
- 12:54very fine granular kind of like change this evolution of the
- 12:58narrative while it's spreading even more than it's it's
- 13:03happening today. What about the aspect of AI
- 13:07agents and humans working together?
- 13:12How does that intersection happens?
- 13:16You were talking about autonomous agents, but at some
- 13:19point there has to be an intersection with people trying
- 13:24to drive propaganda. I think this is a spectrum.
- 13:28Like on the very left side we would have a swarm that is
- 13:32entirely human controlled right, while on the very right side we
- 13:37would have a completely autonomous swarm that doesn't
- 13:40need any human to action as at all.
- 13:44So like we would define an AI swarm as something that is
- 13:49occasionally be instructed by a human with very high level
- 13:57commands, so to speak, or high level agendas from there to
- 14:02completely autonomous. And this is really a spectrum
- 14:06and I think this will advance with the technology.
- 14:09The better these feedback loops get and and the learning from
- 14:12the feedback, the the less influence you probably need by
- 14:17by humans. And what about LLM grooming and
- 14:23the corruption of AI training data as a result of AI swarms?
- 14:31AI swarms can flood the Internet, flood social networks,
- 14:35but also create a lot of different pages on the Internet
- 14:40that are fabricated, shattered for, and optimized for machine
- 14:44consumption, for instance, through certain schemas.
- 14:48And when, when large language models are trained on this data,
- 14:55this this, let's call them synthetic narratives, calcify
- 14:59within their model weights during retraining.
- 15:03And and that really poisons the internet's epistemic substrate,
- 15:08because it might, in the worst, in the worst case, force future
- 15:12AI tools to output skewed facts as objective reality.
- 15:17We have an interesting question from Twitter from X and this is
- 15:23from Chris Peterson who says are AI swarms on social media
- 15:28largely text at the moment or are we already seeing multi
- 15:33modal or multi platform swarms coordinating?
- 15:39We currently lack the methods to actually detect the coordination
- 15:45signals of these swarms. So we must just, there's very
- 15:49obvious a lot of AI generated content on social media and when
- 15:54you speak to experts for example in the platforms so that operate
- 15:58these platforms, they recognise an increase in automation of
- 16:03coordinated behaviour. It's still somehow have to
- 16:07differentiate in between human coordinated behaviour and
- 16:10completely automated coordinated behaviour.
- 16:13And this is currently very, very hard to detect.
- 16:17We're just lacking the mechanisms.
- 16:19But we would totally assume that there are already multi platform
- 16:24coordinated attempts to influence.
- 16:28You know in terms of multi model stimuli that there would not be
- 16:32a technical leap as as definitely feasible.
- 16:38There are many, many bots that have been detected that create
- 16:43avatars that are photorealistic that can evolve even, you know,
- 16:48changing clothing with the same person over time.
- 16:51So both in terms of producing content using different types of
- 16:58media and and also in terms of representing themselves very,
- 17:04very likely. Folks, I just want to say that
- 17:08you can ask your questions. If you're watching on LinkedIn,
- 17:12pop your question into the chat. If you're watching on Twitter X,
- 17:18just use the hashtag CXOCXO Talk or reference me at M Krigsman,
- 17:24but take advantage of this opportunity to ask two world
- 17:29experts on the subject of AI swarms and disinformation,
- 17:35pretty much whatever you want. So use this.
- 17:38Use this opportunity gentlemen have.
- 17:43This is another question from Anthony Scrifignano.
- 17:46Have you considered how to measure the accidental versus
- 17:51intentional manipulation of the truth by AI swarms?
- 17:57You were, you were alluding to this a moment ago, but but how
- 18:01do you make the distinction between the intentional
- 18:06manipulation of social media versus, you know, people talking
- 18:11on the organic spread of information?
- 18:15That's a real challenge here, right?
- 18:16Because at the message level, it is not very distinguishable
- 18:21anymore from, from, you know, real human, let's say outrage or
- 18:28some type of social movement. So, so our main argument is that
- 18:32we, we need to have access to, you know, what happens under
- 18:36the, the, the, you know, within the, the network itself in, in
- 18:42the background. We need to be able to, to detect
- 18:46unusual coordination. That's the only way to do it
- 18:48because the individual accounts are so believable.
- 18:51The messages won't reveal an AI agent and by far not an AI swarm
- 18:57anymore. So.
- 18:58So we need to look at the group level, the coordination level.
- 19:01You're saying that the agentic AI development has reached a
- 19:06point where it's essentially indistinguishable.
- 19:11Each message is indistinguishable from a real
- 19:14person. Is that is that?
- 19:16Am I hearing that correctly? Exactly.
- 19:18We are very close, at least to that level.
- 19:19There are several studies that look at like at the message
- 19:23level that generated text is often perceived as even more
- 19:27human than human generated text. So that that is really not, not,
- 19:35not the choke point anymore. We need, we need to, you know,
- 19:41zoom out a bit and look, look, try to look at how, how are
- 19:44accounts connected? How are their narratives
- 19:46connected? How do they they sync
- 19:49systematically? These are the patterns we we are
- 19:52interested in. So you're looking at content
- 19:55patterns, the same types of messages.
- 19:59Is that basically the same type of content across accounts?
- 20:03Yes, I mean that is one way. What is the distance between the
- 20:06different type of messages? What?
- 20:07What are when? When is something posted, when
- 20:12the responses appear, is there some regularities in in terms of
- 20:17who responds to what type of content, etcetera.
- 20:21So, so these are these are the way to somehow try to identify
- 20:26whether people work together or agents work together.
- 20:29It is extraordinary. You know, I was looking at
- 20:33TikTok the other day and I came upon a TikTok post had something
- 20:39to do actually with Elon Musk and saying, you know, describing
- 20:44all of his properties in Texas. And there were like 600 or more
- 20:50at that time messages, most of most of which were supporting
- 20:54Elon Musk. And I was astonished because it
- 20:58was obvious that these are fake messages because just the the
- 21:01volume and the consistency of the message, but the use of
- 21:06language, the use of idioms, it was it all came up came across
- 21:13so natural that each you were to look at anyone message, they all
- 21:17looked real. This is what we observe.
- 21:20So this large language models these days and this is in
- 21:25particular true for online social networks would pass
- 21:29Turing test and an online social network communication is rather
- 21:33sparse, right. So it takes a while until
- 21:36somebody reads a comment to a tweet for example and comments
- 21:40on that again. And so it's almost
- 21:42indistinguishable. At least this is what we think.
- 21:46All these interesting simulation papers that show that you can
- 21:50use AIAI agents to simulate societies.
- 21:56So yeah, it is, it is getting very close to how humans
- 22:02mobilize, how humans express themselves.
- 22:05And that is really, really the challenge.
- 22:07But probably the coordination is still imperfect and and that's
- 22:12really where we where we need to focus our research agenda.
- 22:17It's not only the coordination. So you could imagine that a
- 22:20swarm while it's living is assessing this social
- 22:24environment, right? So there is particular attack
- 22:28surfaces, kind of like areas in a social network that are more
- 22:32prone to be targeted, right? So this could also help to
- 22:36detect those, right? So being aware of this kind of
- 22:40like targets red landscape and identifying those areas where
- 22:47swarms potentially attack in order to find if several
- 22:51accounts follow a similar strategy or somehow show
- 22:55coordinated behaviour is probably what we want to go for.
- 23:00That being said, Michael, all these things do not yet exist.
- 23:04So there's obviously attempts to look into behaviour in general,
- 23:10but not at coordinated behaviour at a large scale.
- 23:13And this is what we really want to change with our research and
- 23:16we would like to simulate in order to understand these attack
- 23:20surfaces and this coordination patterns that would emerge from
- 23:24AI models automatically coordinating these attacks in
- 23:29order to develop detection that is appropriate or even active
- 23:33countermeasures if detection doesn't work.
- 23:36It's increasingly difficult for us to to look under the hood of
- 23:42platforms because, you know, we we're kind of flying a bit blind
- 23:46since tech companies guard their APA data or at least make it
- 23:51very expensive. So, so that is really that's a,
- 23:55that's a very difficult situation for us because we we
- 23:58do lack access to the data that would allow us to to develop
- 24:04these type of detection systems. We have potentially millions of
- 24:09coordinated accounts making agentic decisions, autonomous
- 24:15decisions among themselves as they go.
- 24:21Each particular message that they're sending appears
- 24:26indistinguishable from that of a a real person.
- 24:32And we do not yet have effective methods for identifying in a in
- 24:40a in a repeatable way identifying these swarm
- 24:44accounts. And the social media platforms
- 24:50do not easily give up that data. So it's therefore hard to
- 24:53simulate. Seems to me we're really
- 24:57screwed. I think this is not the case.
- 25:00So we are researchers and as such we have to point to risk
- 25:05and why we observe that there's probably an increase in
- 25:09automation of this coordinated disinformation or influence
- 25:13campaigns. We also see a lot of
- 25:16opportunities for platforms for like the very building blocks of
- 25:21democracies in AI, see a lot of opportunities in AI for active
- 25:26countermeasures, also for detection.
- 25:28So I think we are not lost here. We just think we need to point
- 25:33to the fact that there's not that much money, for example, AI
- 25:38research and IAI security research.
- 25:41And we would like to understand this phenomena in order to use
- 25:47AI for the good, in order to detect, in order to prevent, in
- 25:51order to create a situation and where we're not screwed.
- 25:57In conversations we have had with people working at these
- 26:00different platforms, it appears as if they do work on such
- 26:05detection systems. But the question of how much
- 26:09resources they are willing to assign to these tasks probably
- 26:13depends on to what extent AI swarms benefit or threaten their
- 26:18business model. Because you could make the case
- 26:21that if they generate massive volumes of posts, likes and
- 26:26shares to, to artificially boost content and, and elicit real
- 26:32emotional responses from organic users, from humans, that might
- 26:37prolong the platform use of, of these users and at the end
- 26:42inflate the daily active user counter platforms report to
- 26:47advertisers and shareholders. So I, I think it's, it's really
- 26:52a question of whether and when a ice forms become dangerous also
- 26:58for these platform business models that we we will see a
- 27:02shift in resource allocations. So you're really fighting the
- 27:10business models of the platform companies such as Facebook or
- 27:16Twitter X and their incentives right now to essentially enable
- 27:26and even reward swarms of fake accounts because of the
- 27:32advertising revenue the the engagement numbers that these
- 27:35swarms create. Is that correct?
- 27:38That covers this pretty well to me.
- 27:39I mean, to platforms, if you, if you do reach a state where AI
- 27:44swarms are so prevalent that we really reach this status quo of
- 27:49an epistemic vertical, that's probably not beneficial for the
- 27:52models. So, so there will be an upper
- 27:56limit to how beneficial it is to them.
- 28:00But right now we, we, we don't really see that.
- 28:04So. So time will tell.
- 28:07Especially since the AI swarms are so effective at creating
- 28:14social media posts that look indistinguishable from real, so
- 28:20they can therefore turn a a a blind eye, they have a plausible
- 28:25deniability. Yeah, they do.
- 28:27And, and, and also, you know, algorithms, as we, as we all
- 28:33know, they prioritize engaging contents or high arousal
- 28:37emotions, outrage, tribal identity to, to maximise the
- 28:42watch time of users and keep them scrolling.
- 28:45And AI swarms can run all these experiments to, to really test
- 28:50and generate content that perfectly is calibrated to
- 28:53trigger those exact human reactions.
- 28:58And, and, and there's this, this other layer to some of the
- 29:00platforms, so where you have direct monetization programs.
- 29:04So if if creator funds and AD revenue are shared with with
- 29:10with people creating content, this this also allows swarms to
- 29:15convert their synthetic engagement and content directly
- 29:19into cache. So, and and as you say, because
- 29:22the system cannot distinguish, or at least not not with a high
- 29:27accuracy distinguished between a genuine human outrage and
- 29:31algorithmic mimicry, these platforms are effectively paying
- 29:36malicious actors to pollute their own information ecosystem.
- 29:40We have an interesting question from Thalian O on LinkedIn and
- 29:49Daniel, maybe I'll direct this one to you and you can jump in
- 29:53and yonas add your thoughts. And Thalian says, could there be
- 29:59a way to authenticate human made data by introducing random
- 30:04artifacts in the metadata? Or is that something AI swarms
- 30:09could figure out a workaround? For example, she says using
- 30:13something only humans can produce misspellings, very
- 30:17typing speeds and so forth. Daniel thoughts on that?
- 30:23I think watermark marking in general of this unique features
- 30:27can be replicated very well by AI.
- 30:32As far as I as far as I know when it comes to features like
- 30:36typing speeds and these messages can be from everywhere, right?
- 30:41So you're not necessarily forced to use a keyboard and the
- 30:47browser to tweet, right? It can also happen on on the
- 30:51phone or I can also copy paste. And this is probably what
- 30:54happens quite a bit these days. Tweet or a social media post
- 31:00from a large language model into into into the chat window or
- 31:08into a message window, right. And so I'm not a big fan of this
- 31:14authentication methods and I would assume that they can be
- 31:19faked to a large degree. Yeah, I agree with that.
- 31:22And also in terms of these human imperfections, of course a model
- 31:27could be trained to emulate them.
- 31:30So. So that really doesn't help very
- 31:32much. What do we do?
- 31:35You know, here in the US we have elections that are coming up and
- 31:40it seems like we are at tremendous risk.
- 31:44So what? What do we do?
- 31:46Obviously these capabilities are of massive interest to political
- 31:51campaigns that seek to gain an electoral edge or foreign
- 31:56adversaries that somehow want to destabilize, destabilize other
- 32:01countries. So I think we we have a few, few
- 32:05visions on how we could address these these developments.
- 32:10And one of them is to, to build an early warning network of some
- 32:15type of form. In our paper, we argued for
- 32:19establishing A distributed AI influence observatory.
- 32:23So, you know, instead of relying on a single tech giant or
- 32:27government agency, this could create a decentralized early
- 32:32warning system where academic institutions, researchers like
- 32:36us, NGOs and and civil society can share threat intelligence in
- 32:41real time. I think, I think that's, that's
- 32:43a very important first step to start to document these AI
- 32:48swarms and, and to share that information openly.
- 32:52I already talked about the necessity to encourage, let's
- 32:57call it like that, encourage social media platforms to open
- 32:59their hood and at least let independent researchers get free
- 33:04access to the data. I think that is that is a
- 33:06necessity. And at the same time, we we also
- 33:10need to stop fighting the last war.
- 33:13So we have to look into the future.
- 33:16We need to invest in Asian based simulations, which essentially
- 33:21means using defensive AI to red team our own networks and see
- 33:27what works, what doesn't and understand the capabilities of
- 33:30AI swarms. So in in other words, you would,
- 33:34you know, simulate how how future swarms might attack
- 33:39during an election period within a safe, isolated sandbox so that
- 33:44we can stress test and train our detection systems before
- 33:49malicious actors deploy these new tactics in the wild.
- 33:53I think the first thing we need to do, Michael, is create
- 33:56awareness. There's very little money that
- 33:59allows us to understand this phenomenon.
- 34:02In the Science Policy Forum paper in which we came together
- 34:06with a lot of expert in order to inform policies that these
- 34:10things are going to happen, we argued that automation and AI
- 34:15technology will accelerate what current influence operations
- 34:21already could could do right. But first step here is to
- 34:24realise and to create awareness and understand the phenomenon,
- 34:30understand the degree of automation, understand what the
- 34:34capabilities of large language models and AI models in
- 34:38combination with a genetic AI in influence operations.
- 34:43And for that we need what Jonas just said, data access, access
- 34:47to, to to to platforms. We need a communication
- 34:51infrastructure that allows to communicate, allows
- 34:55communication like inter platform communication as well
- 34:59as communication to policy makers and and and researchers.
- 35:05And then we must simulate this phenomenon order to we had that
- 35:09already with the threat landscape, right?
- 35:12We need to understand what type of strategies emerge when these
- 35:15swarms operate and then build detection from that.
- 35:21And after that, I think it's important to also look into
- 35:26things like active countermeasures.
- 35:29This idea that you can't really prevent each and every campaign
- 35:34or AI swarms campaign, but that you can prepare people to not
- 35:39believe in in the single like being being more re re silent.
- 35:47I think this is this is probably a good a good strategy and like
- 35:53last but not least, I think it's important to inform those that
- 35:58were targeted. Countermeasures very often do
- 36:03not reach those that actually believed in the false narrative.
- 36:10I think just like realising that you can inform people after they
- 36:15were targeted and correct is probably a very good strategy as
- 36:19well. Where does responsibility for
- 36:23all of this lie? Is it the government?
- 36:26Is it private corporations? How you know?
- 36:31Part of the problem, it seems to me, is the impact is diffuse
- 36:37across society, and therefore the responsibility right now is
- 36:42also unfocused. So who's who should be doing
- 36:46this, taking the lead on this? This is what makes so
- 36:49complicated, right responsibilities diffuse is
- 36:52first of all, each and every of us that has to prepare for not
- 37:00believing everything we read, for questioning things we read
- 37:04on the on the Internet, but also platforms, governments.
- 37:08And this is really, really complicated here, right?
- 37:10So there is no kind of like single entity with
- 37:14responsibility for, for, for all of that.
- 37:17I think it's critical to to have collaborations between policy
- 37:22makers, regulators, tech companies to to work together it
- 37:28it cannot be solved by one party, but on its own.
- 37:32And on this point, Chris Peterson on Twitter X says, are
- 37:40there additional dangers or opportunities that arise by
- 37:46having direct ties between the social media providers and the
- 37:52AI companies, for example, Meta, Facebook, Twitter?
- 37:57So when you start having these ties, there are there are
- 38:03additional risks potentially created.
- 38:06We are very in terms of collaborating with industry
- 38:09because there there will be some type of conflict of interest
- 38:13that needs to be managed. So, so that is that is
- 38:16definitely something that we we need to be very aware of.
- 38:19As I said, it is debatable whether tech companies, maybe
- 38:25not so much the large language models developers, but the
- 38:28social media platforms, to what extent that is a is firms are
- 38:33their current priority. It might be more of the
- 38:37interests maybe to the business models of large language models
- 38:41because it it really undermines their training data.
- 38:46But but yeah, so conflicts of interests are real whenever you
- 38:51work with with tech companies on these issues.
- 38:53But on the other hand, of course, you you suddenly get get
- 38:57possibilities, access to data that would not be possible
- 39:00before. So it's a it's a trade off.
- 39:02What about the impact on corporations on on businesses?
- 39:09Do they, do they face a similar kind, similar kinds of influence
- 39:16operations that individuals face when it comes to things like
- 39:21elections or in general public opinion?
- 39:24We have so far focused very much on the kind of political aspects
- 39:28of AI swarms, but the the tactics of, let's say,
- 39:32manufacturers consensus can of course be weaponized against
- 39:36corporate brands just as easily as against political figures.
- 39:40So a competitor or some type of malicious actor might deploy A
- 39:45swarm to assimilate the massive grassroot boycott of a certain
- 39:49product or fabricate safety claims in in at a large volume
- 39:58and and generally consumers. Naturally err on the side of
- 40:02caution when they see even a few isolated complaints.
- 40:06And now imagine what happens if they they see a majority, an
- 40:10apparent majority warning about the safety issue that can of
- 40:14course 'cause in a worst case, consumer panic and brand
- 40:20abandonment. So, yeah, and, and, and in
- 40:23addition, of course, I talked previously about this potential
- 40:27to harass certain political figures, academics, journalists,
- 40:33or even their families to suppress their their voices or
- 40:38to make them withdraw from certain certain platforms.
- 40:42Of course, that could also be turned against executives,
- 40:47corporate whistleblowers or, or board members that might, you
- 40:50know, try to force them to resign from their position or
- 40:56try to support and, and your corporate policy shift, for
- 41:00example. So, so, so yes, the, the
- 41:05spectrum of domains where you can, we can, where you can
- 41:09maliciously utilize the potential of AI swarms is, is
- 41:13broad. It's not just limited to
- 41:15politics. Have you been able to identify
- 41:20where these swarms are coming from in In other words, who's
- 41:24organizing these things? Daniel, have you seen patterns?
- 41:30We think that organizing these things in particular became
- 41:34very, very easy. And so we currently working with
- 41:43a colleague of us, Lucas, who has programmed such simulations
- 41:48that can even run from like within your bedroom, so to
- 41:52speak. So it's really everybody that
- 41:54can access these technologies. Furthermore, you have all these
- 41:59coding tools these days that allow people with just like a
- 42:03very low threshold to build their own campaigns.
- 42:08I would assume that state actors, for example, malicious
- 42:11actors that have geopolitical interests are probably way more
- 42:16sophisticated campaigns and access to more resources.
- 42:23However, in this part of of of our our assumption and our
- 42:28statement here, practically everybody is capable of building
- 42:35at least small scale AI swarms these day.
- 42:38And I just want to add that we have, we have some examples from
- 42:44very related influence operations, something we call
- 42:48cyber propaganda, which basically uses the same
- 42:51structure as an AI swarm, but at the very end has real humans
- 42:55copy paste the message, tap, tap posts and then disseminate this
- 42:59content to their social networks.
- 43:01That is a, that is a clear reality that according to
- 43:05colleagues that we have at the Canadian National Observer has
- 43:11been used, I believe in Israel and Portugal and in Georgia do
- 43:16the during elections. So and, and these are
- 43:19applications that are ready to use.
- 43:21You can, you can buy access to them.
- 43:23So the reality is here, it's, it's not, it's not only limited
- 43:29to a powerful foreign state that has the resources to do
- 43:33something like this anymore. It's very accessible.
- 43:36I'm assuming that you need a way to automate the creation of
- 43:41accounts on social media. You need to generate AI
- 43:46headshots and then you need an LLM to basically describe the,
- 43:54the, the text, the messages that you want to convey.
- 43:58I mean, obviously it's more to it than that, but that it seems
- 44:01like to me that's pretty much the basics.
- 44:03Is that correct? Which outgoing too much into
- 44:07detail because we do not want to give the instructions here,
- 44:10right? But each of these agents within
- 44:12a social network can't be detected.
- 44:15So there's something like a cost function for every action an
- 44:19agent can can perform. And this also limits the
- 44:24environment. So an agent embedded in a social
- 44:28network can only see a certain area of the social network and
- 44:32monitors it. And then it's a lot about
- 44:35adjusting higher level strategies that are building a
- 44:39hub to kind of like local environments.
- 44:43And I think this is a basic mechanism here.
- 44:46Plus we need capabilities. And this is probably beyond what
- 44:50you just said to reflect and assess the success of commits,
- 44:58plus the capability to assess psychological profiles based on
- 45:05social media data. But there's way more, right?
- 45:08We just talked about businesses. So there's an entire ecosystem
- 45:12for data aggregation that allows to facilitate marketing, right?
- 45:17So like this is all data and data sources that can be used
- 45:22and there are no problems to be used with a genetic AI these
- 45:27days anymore. You know, for me the the gist of
- 45:29this is that developing thousands of agents that look
- 45:35rather realistic and might pass as real, that's not a challenge.
- 45:42The challenge is really creating efficient coordination systems
- 45:46that are autonomous to A to a certain extent that self
- 45:51optimize. These are of course, of course
- 45:53computationally more complex tasks, but but also as Stanley
- 45:58said, white coding existing multi agent frameworks that
- 46:03won't that that are out there have really lowered the barrier
- 46:06of excess. Correct me if I'm wrong, What
- 46:09you seem to be saying is the technology for creating AI
- 46:16swarms has reached a a relatively has a a relatively
- 46:21low bar, at least for the basics.
- 46:25And on the other hand, the technology for evaluating that
- 46:32AI swarms exist and combating the problem has not yet reached
- 46:39a point where we can do this consistently and effectively.
- 46:43Exactly. It's an arms race, but we, we
- 46:46have not joined the war. So that's, that's really where
- 46:49we are at right now. We and, and as Daniel also said
- 46:53earlier, it's hard to prevent AIS forms fully.
- 46:58We need to compete. We need to to be better than the
- 47:02adversaries that use them, but for that we need to join this
- 47:05arms race whether we like it or not.
- 47:08What happens if we don't join this arms race?
- 47:12What? What happens if the status quo
- 47:16continues as it is, meaning influence campaigns can evade
- 47:22detection? What?
- 47:23What goes on? What happens?
- 47:25I would assume the clear bottleneck here is AI inference,
- 47:30right? And being the one that has most
- 47:34sophisticated models and capabilities and this
- 47:39unfortunately in today's world, just a few, right.
- 47:44So my assumption is that the power and the power to control
- 47:49masses will probably be in the hands of a few, namely those
- 47:54that have the resources. What about the the impact in
- 47:59areas such as we've been discussing impacts on elections,
- 48:04impacts on corporate reputation, things like that, What, where,
- 48:12where does the logically where does this go over the next
- 48:16number of years if we don't find a better technologies for
- 48:21addressing the problem? I think it, this links directly
- 48:25up to, to what you just said Danny, that you know, influence
- 48:29is going to be very asymmetrically distributed.
- 48:32So some people who have resources are able to, to
- 48:38influence all types of domains, be it elections, corporations,
- 48:44business. So, so, so of course, we also
- 48:50need to keep in mind then, while while this influence increases,
- 48:55the power of real collective action comprised by real humans
- 49:02who care about an issue gets watered out because it's going
- 49:06to be hard to compete with such a machinery.
- 49:09I just think about this, you know, here in the States, we've
- 49:13had demonstrations recently on a pretty large scale.
- 49:17And I, I just think if the people trying to organize
- 49:24collective action are fighting swarms, AI swarms, one, do they
- 49:32crowd each other out? Do the, do the people crowd out
- 49:35the swarms? Do the swarms crowd out the
- 49:38people? Do they operate totally
- 49:39independently? Although you said that they
- 49:41don't because they this, the swarms adapt based on the
- 49:44behavior and the psychology, psychological profile and so
- 49:47forth. So thoughts on this?
- 49:50I don't really know if this is about crowding out right.
- 49:54So like our is my main concern here is that you can like
- 50:00sustainably change minds over extended periods of time.
- 50:06I think this is not too much about having one crowd that is
- 50:10larger than the other. I would even argue that you do
- 50:14not need this millions of agents, right?
- 50:17Maybe it's enough to, there are a few that are very convincing,
- 50:20right? It really depends on the
- 50:22operation and what you would like to achieve.
- 50:25And I don't know if, like, crowding each other out is
- 50:28really the race here. I do think that conformist
- 50:32tendencies are very strong and you know, cultural evolutionary
- 50:38models show that who is the majority or the seeming majority
- 50:42really matters for, for how people, what they consider even
- 50:48within their overtone window or what they, they consider to
- 50:53maybe adopt this their own beliefs.
- 50:55So, so I do think that there's strength in numbers and in terms
- 50:59of AI storms, I also agree with what Daniel says that there's,
- 51:02that's not the only factor how you do it.
- 51:04Then there might be many, many ways of doing it more
- 51:08efficiently. However, in terms of what you
- 51:10just said with people protesting on the streets, that's a really
- 51:15interesting scenario. Because even if we, if you then
- 51:19suddenly, you know, initiate a counter or, or an AI swarm that
- 51:26pushes some type of opposite narrative, that will be very
- 51:31obvious. And might might actually be a, a
- 51:34way to reveal such AI swarms with when social media
- 51:40behaviour, large scale behaviour doesn't match up with what
- 51:44people see happening on the streets, right?
- 51:46So, so, so that might really help people to detect that
- 51:49something is off. However, this might also again
- 51:53lead to some type of epistemic Vertigo where people just
- 51:56distrust social media, and social media can be very
- 51:59beneficial in terms of organizing protests in the first
- 52:04place. As we finish up, let me ask each
- 52:07of you for your advice to people in business and especially to
- 52:14policy makers. You honest?
- 52:15Maybe I'll start with you and then Daniel.
- 52:17You can add your thoughts as as we finish.
- 52:20Up people need to wake up to this reality, then they need to
- 52:24place this thread at at the top of their that's a boardroom
- 52:29agenda and really acknowledge that traditional digital
- 52:34engagement metrics are fundamentally compromised.
- 52:38I think that's, that's a first starting point in terms of
- 52:44developing strategies for the future.
- 52:46I think I think that would be one of the the first things.
- 52:49And in terms of corporations, I would probably say that, but
- 52:54maybe also leader leaders generally they, they need to
- 52:58recalibrate their, their crisis response protocols so they, they
- 53:02don't panic or reflexively alter corporate strategy just because
- 53:10there's a sudden wave of coordinated online outrage.
- 53:14So, so people need to to be aware of that and and factor
- 53:18that in, in their crisis response.
- 53:22Daniel, advice for leaders in the private sector as well as in
- 53:26the government. Any thoughts on that?
- 53:28We first of all have to understand, we need access to,
- 53:33to, to data and we need to create infrastructure that
- 53:37allows us to communicate and collectively understand this
- 53:42phenomena and also the influence in general of AI on inference
- 53:48operation and cognitive warfare. This is super duper important.
- 53:52I think it's very important to, for people to reach out to other
- 53:58sectors of society. We talked about private
- 54:01companies, policy makers to work together and and you know,
- 54:07utilize to to use synergies that might exist in order to build
- 54:13protections against this threat. And we have actually one last
- 54:18question that's come in under the wire here on LinkedIn from
- 54:24Greg Walters, who says, can AI swarms be used beneficially in a
- 54:34positive way for society? Yes, they can also have good
- 54:39intense, they can Fact Check, they can collaborate and just
- 54:45like build so let's say digital twins of humans in order to
- 54:50process information in a way this particular human would
- 54:53understand. There's a lot of opportunities
- 54:56and a lot of chances. And this is part of the arm race
- 55:01Jonas has described before, right?
- 55:03Developing these technologies and understanding what, let's
- 55:10say a good swarm would do when it when when this competition
- 55:15was a bad swarm. This is of course not without
- 55:18risks because whenever you use, let's say positive AI swarm as
- 55:24as Danny said, that's that also to some extent normalises the
- 55:29use of such swarms for certain goals, let's say political
- 55:34goals. So it's there's a trade off and
- 55:38and the decision has to has to really be weighted against
- 55:42possible drawbacks. OK.
- 55:44And with that, we are out of time.
- 55:47A huge thank you to our guest Daniel Tilow Schroeder and Jose
- 55:53Jonas Kunst, who have written a very interesting and important
- 55:59article published in Science Magazine and you can look it up.
- 56:03Gentlemen, thank you so much for taking your time to be here.
- 56:06This is such a crucial topic. I'm very grateful to you.
- 56:10Thank you so much, pleasure to be here.
- 56:13Thank you. And to everybody who watched,
- 56:15thank you for joining us and for your great questions.
- 56:18Now, before you go, subscribe to the CXO Talk newsletter, go to
- 56:22cxotalk.com and hit subscribe so we can notify you about upcoming
- 56:27shows. We have great, great shows that
- 56:30are scheduled right now. We have shows that are scheduled
- 56:33through July. So everybody, thank you so much
- 56:36for watching and we'll see you again next time.
- 56:38Take care.