Latest / Practical AI / Agentic AI for Drone & Robotic Swarming

Transcript

Agentic AI for Drone & Robotic Swarming · Practical AI

• 0:04
  Jerod: Welcome to the Practical AI podcast, where we break down the real world applications of artificial intelligence and how it's shaping the way we live, work, and create. Our goal is to help make AI technology practical, productive, and accessible to everyone. Whether you're a developer, business leader, or just curious about the tech behind the buzz, you're in the right place. Be sure to connect with us on LinkedIn, X, or Blue Sky to stay up to date with episode drops, behind the scenes content, and AI insights. You can learn more at practicalai.fm.
• 0:36
  Now, onto the show.
• 0:49
  Daniel: Welcome to another fully connected episode of the Practical AI Podcast. In these episodes where we don't have a guest, it's just Chris and I, we take some time to deep dive into some interesting AI topics or explore some things in the news related to AI and hopefully give you a little bit of, learning resources or information to help you level up your machine learning or AI game. I'm Daniel Witenack. I am CEO at Prediction Guard, and I'm joined as always by my cohost, Chris Benson, who is a principal AI research engineer at Lockheed Martin. How are you doing, Chris?
• 1:27
  Chris: Hey. Doing great today, Daniel. How are you today?
• 1:29
  Daniel: I'm I'm doing well. Yeah. I just took a tried to take a run before it gets really, really hot here. So hopefully, you know, that that worked out well. It was still kind of hot.
• 1:43
  Hopefully, I don't end up with a sunburn, but
• 1:45
  Chris: Absolutely. Yeah. It's it's been hot here in The US, especially in the Southern US where we both are right now. And so yeah. You definitely don't need you don't need you collapsing from heat exhaustion right before we get into a show here.
• 2:01
  Daniel: Yeah. Yeah. For sure. I I definitely say there's fewer people out. There was not in any way a swarm of people running, anywhere that I anywhere that I saw.
• 2:12
  So, yeah, I guess that leads a little bit into what you teed me up to talk about today, Chris, which I'm excited about, which is an area of of your expertise and and interest, which is swarming, which I guess could be related you know, people might be thinking of animal swarming, robot swarming, other swarming. Maybe just starting there, what what do we mean by swarming?
• 2:39
  Chris: So before I before I get into that from my side, let me let me set that up a little bit because so because we you actually mentioned several things that knowing me and and also, you know, just the general topic as we've kinda prepped to do this a little bit. You know, you mentioned animal swarming. And so it's funny. In the AI world, we we have been talking about, you know, robotics and autonomy and, you know, unmanned aerial and ground vehicles for various things. You hear about Amazon drones and Walmart drones are out there now and various other applications.
• 3:13
  And so even though my background may be in the in kind of military applications because I'm in the defense industry, we're really talking about it in a general say. And and one thing I I should say upfront to cover myself is I'm only representing my own personal viewpoint and not that of my employer or any other organization.
• 3:35
  Daniel: Point taken.
• 3:36
  Chris: Point taken, which is kind of funny because when you talk about swarming, if you put 10 people into a room and ask them each to define swarming, you'll end up with about 17 different definitions of what swarming is. It's one of those types of terms.
• 3:51
  Daniel: Yeah. You mentioned you mentioned the delivery drones around where I'm at. I'm I live right by Purdue University and actually for quite a few years now we've had these little little box robots that will deliver you food. So they, drive along the sidewalks and, you know, go through intersections and deliver food to you, through through an app and and all that stuff. So that one definitely hits a little bit close to home.
• 4:25
  I think for the most part, I have seen them, you know, successfully navigate the terrain and and intersections and all of that. Although I will say I did see one at one point. I didn't see the the collision, but it was definitely run over on the on the road. It was in pieces at an at an intersection.
• 4:47
  Chris: Yeah. Somebody didn't get their dinner that night. That sounds
• 4:49
  Daniel: like Exactly.
• 4:50
  Chris: Somebody called up hungry. Where's my food? You know? Which, you know, and it and it was the delivery driver once upon a time. Hopefully, that person would not have gotten hit.
• 4:59
  But, yes, having having the the some sort of automated vehicle bringing the stuff. So wanted to talk a little bit about and and probably before we get into, you know, talking about what swarming is, kind of distinguishing it because it's one of those autonomy words and, you know, autonomy is another one of those words. And we have things like drone and robot and UAV, UGV, UXV, x covering kind of all the things. And so
• 5:26
  Daniel: I don't know what a lot of those things are.
• 5:28
  Chris: Okay. So so ground ground, sky. You a US phone, air. Yeah. So like like, UXV would be unmanned or uncrewed is a little bit more modern term for it.
• 5:42
  The x signifies kind of what the domain it's operating in air ground, whatever, and then vehicle. And if you've seen u x s, that would be an unmanned or uncrewed whatever system. And so those are really common lingo things, and they're not specific to military. You see those in commercial and industrial applications and stuff. And for the most part, we'll probably for simplicity's sake, mostly talk about drones to to in this conversations, we tend to stay away from jargon, and maybe robots on, you know, for ground stuff.
• 6:16
  And that way, are not trying to get through acronyms while we're talking and stuff just to simplify things a bit. But we're having the we're definitely living in this age, as you just pointed out from personal experience, where we're we're starting to see these things, which are, you know, physical embodiments of some sort of AI or or other algorithmic driven, you know, movement around the physical world for various activities and stuff. And it sounds like and I don't know the specific technology for the one that's bringing you the food on how they're how they're approaching that because there's a bunch of different approaches. But that is a either a semi autonomous presumably or a fully autonomous, you know, ground vehicle that's bringing you the stuff. Whereas going back to our topic, swarming implies numbers, first of all.
• 7:04
  And so, but it doesn't just imply numbers, it implies the way numbers are working together and collaborating. And I think that's where a lot of people get in trouble with all sorts of different definitions. And it's super popular to talk about swarming now.
• 7:19
  Daniel: Yeah. It's
• 7:20
  Chris: it's a crazy buzzword these days.
• 7:22
  Daniel: Yeah. I almost wonder, like, there could be some confusion. We hear a lot of talk these days about multi agent systems. And, of course, we've had episodes where we discuss agents specifically and what that term agents mean. But as soon as you soon as you bring in that kind of multi agent side, people might be confused.
• 7:46
  Is is that what we're talking about? I guess one question would be like swarming, does it imply like, physical, I guess, physical AI or physical autonomy in terms of things that are operating in the physical world, not the digital world? We've talked a lot about drones and robots in the beginning of this. So is that is that part of that definition or or not really? It is.
• 8:13
  Okay.
• 8:13
  Chris: It is. You know, and we've had across a lot of our episodes recently, especially our fully connected episodes like this where it's you and I discussing a topic. You know, we've talked about we're kind of, you know, we've had over the history of of the show, we've saw seen the evolution of AI, and there tend to be specific topics that get really hot. And and right now agents are really hot, but there's also the notion of agents applying themselves again, you know, in collaboration with various other AI technologies, whether they be LLMs or reinforcement learning or, you know, computer vision. And I think if you take a step farther into this world of autonomy, especially modern autonomy in 2025 agents are a big part of that, and having multiple agents collaborating, and they're using these other models to get these tasks done.
• 9:02
  So you have agents operating with LLMs for different purposes. And you know, so you're able to go and grab the right model for the right task and you're putting a bunch of tasks together to go do something which which I might call a mission, but I don't necessarily mean that in a military esque way. Could be, but it could be something that you're that is what the purpose of your company's autonomy is to do. Know? Maybe like
• 9:27
  Daniel: a a goal or a objective or outcome
• 9:31
  Chris: That's right.
• 9:31
  Daniel: Could be synonyms. Maybe I'm I'm not that great at the English language, but
• 9:36
  Chris: Those are all those are all loose synonyms that could they could apply. Yeah. And so and we're seeing this, you know, you're we're seeing these individual drones and robots that are starting to do tasks in the commercial and industrial space. Certainly, it's been that way in the industrial space and warehouses and robot. I guess the first robot I worked on was 2,000 that personally I was doing were two of them that were autonomous in 2018.
• 10:02
  So this is not new in that capacity. How it's doing it has changed over time.
• 10:07
  Daniel: Yeah. Maybe it just taking a moment kind of looking back a little bit because I remember what what year was it that the I forget the year of the Beijing Olympics when they had the It was the first one I had seen where they had a kind of, I guess we could call it a swarm, I don't know, You can correct me if I'm wrong. Swarm of of drones that were sort of in the sky in the I guess, I think it was the opening ceremony. I'm I'm trying to remember, but that was the first time I had seen that. Of course, I've seen it in other places, you know, after that.
• 10:48
  But, but yeah, what's what's kind of the The definition of that? Yeah. Well, how would you how would you represent the history of kind of coming to the point where we're at now in these, so I'm getting from you, there's these autonomous, so multiple objects in the physical world that are autonomous or multiple vehicles or robots or whatever in the physical world that are autonomous trying to accomplish a goal or mission. Now in the Beijing case, I don't know if those were autonomous or not. But, yeah, help me kinda parse through maybe a little bit of that.
• 11:27
  Chris: So I should say ahead of time that I don't have any special knowledge of that particular configuration. So I'm making educated guesses on that. Yeah. So I would I would suggest that's not what I what and I'll define a swarm in a moment as a follow-up. But I would not call that a swarm.
• 11:42
  I would call that a large number of individual aerial platforms that are operating in a in a predetermined coordinated manner.
• 11:52
  Daniel: Yeah. It's like synchronized
• 11:53
  Chris: They're synchronized from the ground control system. So the notion in in flying autonomous vehicles is that you have a ground control station, which might be like a laptop, it might be a bigger thing inside a truck or something, but it has, you know, comms and it has ways of communicating and and so for shows like that, you would use, you know, you would use things like GPS and they would have a each drone would have a three-dimensional path that it's following Mhmm. That's a little bit different from all the others. But when you put them all up there at the same time, they look like a big coordinated show, but they're not actually coordinated. It's the human on the ground programming in their path.
• 12:34
  It is
• 12:35
  Daniel: They are not determining their coordination.
• 12:39
  Chris: There you go, which is really important. So Okay. Yeah. Yeah. You went right to the heart of it.
• 12:45
  And so there's that, which is really not intelligent at all. It's pre programming these drones to do something and the visual impact is that you have this thing happening among a bunch of things. But it's, it's sort of an illusion. It's really a bunch of individual things on their path. And then there's a whole there's a whole continuum of how those kind of vehicles can operate.
• 13:07
  One kind of popular approach today that I that that a lot of people would call swarming and which I don't, and I'll define it right after I say this, is the notion of like giving plays to a group of vehicles that go out and do a task. Sort of like some people the analogy will be a football play. It's go go to this location and do this, and there might be some communication with the other platforms in your area that you're doing something with, but I also don't consider that a swarm. Because I think it comes back to the notion of what is swarming. It's it's swarming is not an autonomous thing.
• 13:48
  It's a type of behavior exhibited. And we see that in nature. And so I will I'll give the definition and then I'm actually gonna go away from technology for one second and talk about what we see in mother nature that's kind of consistent with that. So my definition sounds a little military like, but don't take it don't take it too much that way. It can be it could easily be applied to commercial or industrial.
• 14:14
  So but I think it gets the gist of how I of how I see swarming. Mine is, swarming occurs when numerous independent, fully autonomous platforms exhibit highly coordinated locomotive behaviors in any domain, be it air, ground, sea, undersea, space, functioning as a single, independent, logical, distributed, decentralized, decisioning entity for purposes of command, control, and communications with human operators on the loop to implement actions that can achieve strategic, tactical, or operational effects in the furtherance of a mission. Little military sounding, but if you translate those words into commercial and industrial, they still apply. You might use it to you might choose a synonym in some of those areas. But that is a is is a pretty high bar right there.
• 15:06
  And, and as we go forward, we can talk a little bit about kind of what in that definition, turn something from a collection of autonomous vehicles into a swarm of autonomous autonomous vehicles.
• 15:33
  Sponsor: Well, friends, if you want to build the future of multi agent software, check out Agency. That's AGNTCY. That's an open source collective building the Internet of agents. It is a global collaboration layer where AI agents can discover, connect and work across frameworks. That means better tools for you, standardized agent discovery, seamless inter agent communication and modular components to scale your multi agent workflows.
• 16:02
  And they're teaming up with Crew AI, Langchain, Cisco and many more dropping real code specs and services with no strings attached. Start building alongside engineers who care about high quality multi agent software. Learn more at agency.org. Again, that's agntcy.org. That's agency.org.
• 16:28
  Daniel: Well, Chris, I I really appreciate your definition. Maybe that's even something we can pull out and put into the show notes for people so that they can can read that, if they if they want to. I'm actually reminded as we're talking about this as an and as I'm hearing your definition of these things, you know, in the animal kingdom. Actually, I'm remembering a movie that I watched with my wife, The Murmuring. I think it was on Netflix and very interesting movie, but kind of at the core of the movie were these, I don't know if I'm using the right terminology, but murmurations of swarms of of birds.
• 17:13
  Star Lake, I'm sure
• 17:14
  Chris: famous for it.
• 17:15
  Daniel: Exactly. Yeah. So you can see in the sky, I'm sure many people have seen this swarms of birds that almost form kind of patterns and synchronizations in the sky and and move around together. Now according to your definition, each of those birds, right? Now I guess in this sense, you know, there's no connection to a, obviously a human command and control type of scenario, but certainly they are autonomous, but they're accomplishing something together, which is maybe a little bit mysterious in this case, although I I'm sure it's a topic of study.
• 18:00
  But, but, yeah, I I that's what came to my mind is is that is that movie and that kind of phenomenon.
• 18:07
  Chris: Yeah. And that murmuration, if you will, is literally it's called flocking in that case, you know, which is obviously a bird term, but that is definitely consistent with what I would consider to be a swarm. And if you if you break that down, mean, there's and there are other animals we can talk about too, they have different types of swarming behavior, but that are consistent with that. If you talk about starlings in particular, you know, they're beautiful when you see those massive, and there could be thousands of starlings moving across, and you see the waves within the system, you know, within the flock as they're doing that. And and so you kinda what's going on there?
• 18:44
  We have thousands of individual beings that are communicating and sensing each other. They have a unified purpose. They're all going someplace together, but there's no there's no one master general or CEO starling that's going, you guys are gonna go that way. Everyone do what I tell you to do. They are all subscribing to the mission, if you will, but each one has a has a a position, if you will, or a place in that mission where they are communicating and the way they do their sensing of each other and their communications helps them a not collide and and, you know, fail by by through collisions and such.
• 19:26
  It keeps them together, but they also have a way of of agreeing, if you will, to go do something. They're going to to move from one place on the earth to another place on the earth. And and that kinda comes down to the crux of it, is that swarm behavior is where you have that a bunch of fully autonomous beings in this case, they're functioning in that murmuration as a single independent logical distributed decentralized decisioning entity. It's a long mouthful that I use because all of those have a part in that describing it, but they're all working together in that way.
• 20:06
  Daniel: Do you wanna pick apart maybe each of those words? So just like why why use those multiple words kinda distributed, decentralized? Some people might kind of not know why each of those is is important.
• 20:20
  Chris: Sure. So so single implies that the joint actions together about how they're communicating and sensing give a rise to one generalized process.
• 20:34
  Daniel: Kind of an emergent or combined
• 20:36
  Chris: process. That's a great way of putting it. And that's kind of that that emergent, you know, we said mission or goal kind of thing is coming out of that. And that process is independent of any one bird. And it's and it's independent of any controlling authority that saying from afar, y'all shall go do this thing, you know.
• 20:56
  And so you have that independence, you have that single emergent characteristic, logical meaning, it's also kind of, it's kind of think about, it's arising a an emergent intelligent notion or a processing notion that's all happening by the unity. It is distributed across the entire flock or murmuration. It is decentralized. There's no one special bird or group of special birds that are the birds that are in charge of everybody else and you guys have to do what we say. And finally, it's a decisioning entity that by by this emerging thing happening, choices are made collectively for the whole thing.
• 21:38
  And yet, no one bird is saying, everyone's going to go do this thing. So it's all these kinds of characteristics about that emergent, I may add the word emergent into the definition that I have there, because I think that's really a great way of describing that. But it's that unified collection to do this. Now, that's a far cry from what we're currently doing in the autonomy space at large. I'm not saying that doesn't exist in little pockets, academic pockets and other places, labs, whether they be government labs or university labs and stuff like that.
• 22:14
  But you don't see widespread deployment of drones or robots that are acting in this manner as we sit here recording today.
• 22:24
  Daniel: Okay. That that makes sense. Yeah. Yeah. Part of my what I'm thinking and my questioning in my mind, which maybe you're going to get to as you kind of get get through some of these things is, obviously, we have a kind of inspiration or a model there from the the animal kingdom.
• 22:44
  In the case of, let's say, robots in a warehouse or a manufacturing facility or autonomous vehicles, we have in probably things that people are familiar with, the idea of self driving things or autonomy. So like self like the Waymo cars or that sort of thing. And there's certainly, obviously, AI at play there in terms of computer vision and maybe reinforcement learning and decision making. So assuming that people might know at least or be exposed to that, in terms of the connection to AI specifically, what about this kind of swarming or, you know, robots or these entities working together to do a task? What is necessary for AI to do in that swarming scenario that's not that's not necessary in just a single, like, self driving car, like a Waymo car or something like that?
• 23:50
  Chris: That's a great question. There's a lot of nuance there. So so the self driving car is having to navigate an environment in which it's both sensing and to some degree communicating potentially, although that is evolving over time, with with other actors in that environment around it. And there is emergent behavior on what to do. There's kind of the classic cases of, you know, kids playing on the sidewalk and the ball bounces into the street, know, and and reacting to various unexpected events like that.
• 24:23
  But at the end of the day, that vehicle is still, which may be fully autonomous, is still only having to decision on its own behalf. And so it has an onboard inference system, the onboard computers, the onboard models, or potentially that in combination with some remote that's doing over some sort of connection, radio connection or cellular connection, but it's still decisioning for itself. The difference according to my definition of a swarm is that the emergent quality of of the decisioning is arising from participation in many of the platforms that are participating in that unified mission or activity. But it has to decide for all of them and it may have to decide on platform A, we're looking at something, we're evaluating it, we're sensing it, we're processing based on reinforcement learning and our agents are then saying platform a, you go do this activity. Platform b, you go do that one.
• 25:26
  Platform c, you go do that one. And those are all slightly different activities as those platforms collaboratively work together based on the sensing and inferencing that's happened across the swarm to get an activity accomplished. Does that clarify a little bit?
• 25:45
  Daniel: Yeah. That that helps. And I'm trying maybe this is the decentralized part of what you're talking about, but part of maybe what is a challenge in my mind to think about is how such a thing would happen without a central kinda, so like, so to say like, oh, let's just take a very simple maybe it's simple, you know, in my mind it seems complex as well, but we have 20 robots in a manufacturing facility trying to do something. In one model, I could see where those, each of those robots detects certain things on a number of sensors, maybe cameras or temperature sensors or force sensors or something like that. All of that is communicated back to a central processing system.
• 26:41
  And it's really the single entity, the kind of master brain that makes the decision and communicates out the next step to all the robots. But in a decentralized way, you talked about how there's not this kind of central there may be a central command where maybe there's oversight or something, but things somehow happen in a decentralized way. That's what I'm trying to connect in my in my brain maybe.
• 27:07
  Chris: No. That's that's a great nuance that you're pointing out there. So in the warehouse, that's not really that that's definitely a centralized environment, and you don't even have to have onboard compute for that because your robots can have a highly reliable system of communication over WiFi, you know, with a master server that's there and you're in the distances involved or close enough to where you can have them interact with each other without having to do that. And according to the definition of the swarm that we're using here, that would not be a swarm, even though you may have many platforms working
• 27:42
  Daniel: Multiple devices.
• 27:43
  Chris: Collaboratively, they're being controlled by by a local centralized agent. And in swarming, generally, according to this definition, you're really looking at an environment where that's not possible. You're looking, you know, what we would traditionally call edge or or, you know, far edge is another term that what people have used. Basically, where you can't count on either local or cloud environments to provide the compute. And so you have to have the compute on board, including all the activities that will arise from that compute.
• 28:16
  Daniel: Yeah. I'm almost imagining moving still kind of in my mind living in the industrial world if I think more to like oil and gas and like certain of the environments that are part of part of working in that industry, whether it be deep under the ocean or in very disconnected remote places where you might not even want to ex want to or even could expose humans to those environments.
• 28:49
  Chris: That's right.
• 28:50
  Daniel: But they're also very hard in terms of connectivity and and that sort of thing. You might want to accomplish tasks in in that sort of environment. Am I getting to the right kind of scenarios?
• 29:01
  Chris: Okay. Like, one good one that is commonly used is the notion of disaster recovery, where you might go into an area that is geographic, you know, it may be geographically remote or may not be, but either way the infrastructure has likely things have happened with the infrastructure to where things have been torn down, you don't have cell towers, things like that, and you don't have the level of connectivity necessary for centralized control. And so if you're thinking about disaster recovery in a swarm context, you where you do not have that infrastructure in place to guide those, then you could put drones or robots on the ground that have their own inference capability. And if you're approaching it with that decisioning entity that is decentralized that I've described then that could occur where they're actually working collaboratively to save lives. You know if there's buildings in rubble, know one robot that has that ability as maybe pulling rubble off because it has that capability while others that specialize in getting down between the rubble and they're smaller, might be in other words, not all robots may be the same.
• 30:16
  You might have a heterogeneous mix of robots with different capabilities, but they're able to dynamically go, here's a rock and it needs to be moved while you go under the rock, you other platform go under the rock looking for a survivor there. And so you can have a collection of robots that each has an optimal function. Yeah. Where they're working together to do something and ultimately save a life and you don't have, you know, maybe you've had hurricane come through or tornadoes and you don't have the normal digital infrastructure that we're all so used to today.
• 30:52
  Daniel: Well well, Chris, that that is really helpful to break down maybe some of the use cases or scenarios that maybe compare and contrast multiple agents or robots or systems that are working in an environment that maybe are in a swarm or are not in a swarm. Maybe part of so that's helpful on the use case front. My mind is still kind of wondering on this side of the AI side, what unique AI problems are unique to the swarm environment that aren't encountered elsewhere? And how do those map to kind of maybe open challenges that are that are open or kind of maybe known models that are able to handle certain of these things?
• 31:45
  Chris: Yeah. So there's I mean, I think you just kind of alluded to the complexity involved. And I think the reason we haven't had swarms before now and that they are still something that that is emerging within what is being developed is because not all of the technologies and models that you would need have been mature enough up until this point in in time to be able to do that. So there's a lot of pieces. And some of that is on the AI side and some of that is on is operating in a physical environment.
• 32:20
  And so on the AI side, you know, over the years, we've talked about these different architectures, you know, different models with different purposes. And swarming ends up taking quite a few of those and making them available to agents so that you can within a complicated physical environment, different agents on platform are working with each other to accomplish the on platform parts, and then be able to communicate and coordinate that with other platforms that also each are multi agent multi model configurations. And so and then being able to do that.
• 32:58
  Daniel: Swarm of swarms.
• 32:59
  Chris: Yeah, sort of like if you think of all the various technologies that go into just a single platform able to do that. We're only now really getting you know, in the large getting to where these kinds of things are possible. And where the compute is powerful enough for edge devices and that the models are being reduced. We've talked a lot over the last year or two about how models are getting smaller and you get a hugging face and, you know, the vast majority of models there are very small models that are useful in a lot of specific tasks. And so you have to match up small models that can run on edge inference or or even CPUs to do various tasks for specific agents and coordinate that with sensors and comms.
• 33:43
  Yeah. To be able to accomplish stuff. So it's quite complicated. And that kind of and I use this kind of in air quotes, that kind of miniaturization of the technology. I don't mean physical miniaturization, but kind of getting everything to work and run on a remote edge device that is that is likely battery powered is is a bit of a challenge.
• 34:06
  So I think, as we look forward right now, there's, you know, we're, we're really at a inflection point in history in terms of being able to do this.
• 34:14
  Daniel: Yeah. And what just did a very practical level, I'm sure there's a great diversity here, but in some of what you're talking about, there's a unique communication element and goal setting element of these systems. And if I look at like AI models that are running on any particular platform within the swarm, what, at a very high level, what kinds of models and tasks need to be accomplished that are associated with the swarming versus just like sensing and that sort of thing? Are these models that are, you know, small SVMs or something that are making a kind of binary decision or gradient boosting machines or neural networks or gen AI models? Like what is the diversity you see and what are the kinds of tasks you just, by way of example, a couple of them that might need to be done?
• 35:14
  Chris: Yeah. I I it's a mixture of different models, some of which can be more traditional AI models like what we most often talk about here on the show. Some of them are more of the classical data science models because, you know, we've talked about this a number of times where you don't need to go to a bigger, harder, you know, expensive model to use if a smaller model will work, but it's very task specific and then and it's bound within the software systems of each platform but also the software systems which govern sensing and comms. And then so you need the ability for different nodes, if you will, of what it you know, different platforms to maybe on behalf of the swarm to take on some of the computation because you can't distribute a model inference across all of the things. But what you can do is say, if you have a swarm of a dozen, I'm just obviously making this up off the top of my head, have three of the platforms running inference on a particular model for a particular task while three others do another thing and then you can you can have things like election algorithms which decide which one you're gonna take.
• 36:22
  And so you can you can have evaluation of what who's getting the best sensing information and who has the best comms reach for the entire swarm to ensure that data gets around. So you can have some redundancy in the computation and all those things and then by election pick and election's just one mechanism. That's one common algorithmic mechanism that that one can apply. But by election within the swarm, choose who is issuing, you know, the results of the various types of inference that the agents are putting together and deciding upon a task. So that you can then go and assign specific platforms to do different parts of that task, where not every platform is doing the same thing, but they're all contributing to the larger goal.
• 37:07
  Daniel: Gotcha. Yeah. That's very helpful. Now I'm sure there's some people out there that are maybe excited by this, disturbed by this topic, maybe. Definitely.
• 37:21
  I mean, one sense, just again, thinking about things outside of the robotic world, we talked about the birds, which are beautiful and you see the patterns which they form. I also think about like in the human context, there's interesting maybe swarm behaviors that are somewhat disturbing. Like if you think about a mob mentality, you often hear about people, you know, maybe a a mob of people all accomplish a very disturbing thing, like a destructive thing. And then afterwards, you know, individuals that were in that swarm say, I don't know what, you know, the individuals might not ever have done what was done by the mob, right? But because of this emergent behavior, things happen that maybe are outside of the bounds of any individual's, you know, moral compass or that sort of thing.
• 38:21
  So if we take this then to the robotic or drone or industrial side, some people might say, well, how do we push one direction and not the other direction? And what is the oversight related to this?
• 38:37
  Chris: So it's a great question. I'm glad I'm glad you brought it up in my enthusiasm for the technology. It's very easy to lose sight of this. And we had a very recent show where we talked a little bit about, know, the anthropic, I would refer people back to that. And the fact that you had, you know, models that were coming out with what we deem to be bad outcomes.
• 38:57
  And that can certainly apply here. And so you have to have some form of guardrails. And so there's two, I want to start with, there's two phrases, one of which I mentioned before. I said human operators on the loop. And human operators on the loop versus human operators in the loop are two distinct things.
• 39:17
  Right now in the in the military space, based on current regulations here in The US and stuff, you you most often are talking about in the loop interactions between humans and stuff. But as you go forward, and this is not a military specific thing, this is something that you're gonna see in commercial and industrial as well, as you start talking about swarm capabilities with emergent properties happening, then the challenge is that there are sometimes you want a human to be able to step in the loop directly into that processing and say yes, no or make a selection or something like that. But there are other times when you're talking about a larger swarm where you need a human who where the it may be there may be too much going on and there may be too many of these platforms flying around or on the ground for one human to track everything at every moment and interact but you can have a human on the loop, can say for the overall goal or for the thing the swarm decides approve or disapprove, you know, if there an emergent behavior comes up, and the swarm makes a decision based on what it's seeing in real time, then you can say, No, that's not going to work.
• 40:33
  And that's what a human on the loop does is you can it gives you that guardrail of saying, that's not the emergent behavior I want, you know, it's time to go back or maybe do a full recall and shut down. So I think if you really want to learn more on these kinds of things you're starting to see I it's, I'm not sure that I can think of a specific swarm. I'll have to do some research and see if there's any specific swarming classes. A lot of the kind of drone and robot systems that you can get out there in open source. For instance, there's ROS and ROS two, which is the robotic operating system, first and second version.
• 41:15
  ROS two takes a slightly different approach to ROS one, so there are some people on each version. I know ROS two has some swarming capabilities in there to build on. It doesn't instantly give you swarm capability but it has some tools in there that you can start building on and that might be a good place to start to learn about the topic. But in general, kind of staying tuned into learning sources, including this podcast, where you can learn about multi agent environments and as we increasingly are talking about physical world deployments of AI technologies, those are all good places to to consume because this is a very cutting edge area to to be focused on.
• 41:58
  Daniel: Yeah. It seems like a lot of a lot of things are developing, and I personally am very excited about some of the things happening in robotics. Obviously, we've maybe some of us, the more we see chat interfaces, it's like we just wanna see some other application of of AI technology, which that is only that's such a small segment of what is possible with AI, even Gen AI, there's so many more ways that that will, I think, become embedded in our physical spaces. If you just wanna search for kind of this idea of physical AI, I know that's a big topic of kind of AI embedded in our physical spaces, maybe not through chat interfaces, but through other interfaces. And one thing that I've really enjoyed seeing is the stuff coming out of, Pollen Robotics, p o l l e n.
• 43:00
  That's a company that I believe if if I'm not mistaken was acquired by Hugging Face and they're releasing a lot of well, I don't know a lot. They're releasing robotic systems that are more geared towards experimentation, open source development, integration of open models, integration of apps and exchanging of those apps even within Hugging Face spaces but for actual physical systems. Actually, Prediction Guard, our company, we got one of the or ordered one of these, I think it's Ricci Mini, robots. Looking forward to playing around with that in our office. That should be really fun.
• 43:47
  So I think also there's an increasing number of ways that people can access even physical systems or robots in a much more accessible way than before where maybe everything was coming out of whatever it was, Boston Dynamics or or wherever very, very expensive millions of dollars systems here. There's ways to access these kind of systems, whether it's drones or robots or that sort of thing in in a in a much more accessible way, which is really encouraging.
• 44:19
  Chris: It's not expensive anymore. So in the maker space, there is a lot of resources. A lot of it is based on stuff you may already know, like Arduino or Raspberry Pi, and you know, adding a Jetson in their carrier boards where you basically get some basic stuff. So we're gonna see more and more of this going forward. And as people really kind of get even some of the more advanced topics figured out, It's gonna be in all of our lives in the years ahead.
• 44:47
  So we're gonna see a rapid introduction into daily life of these things. And so if this topic interests you, definitely, I would encourage you just as I do, is to go out and jump into the maker space and and for very little money, you can have some pretty interesting experiences building some of these things.
• 45:06
  Daniel: Yeah. And I it's occurring to me that after this, we'll we'll put up, maybe a a webinar, that where we could talk about these things a little bit more, maybe even demonstrating some things with with robotics. So don't forget to check out practicalai.fm/ webinars for, upcoming times that that will be live where discussions can happen. And, yeah, just really appreciate you helping us deep dive into these subjects, Chris. It's been it's been really good.
• 45:38
  Chris: It's a fun topic. Happy to do it. Thanks a lot, Daniel.
• 45:41
  Daniel: Alright. See you soon.
• 45:42
  Chris: See you later.
• 45:49
  Jerod: Alright. That's our show for this week. If you haven't checked out our website, head to practicalai.fm and be sure to connect with us on LinkedIn, X, or Blue Sky. You'll see us posting insights related to the latest AI developments, and we would love for you to join the conversation. Thanks to our partner Prediction Guard for providing operational support for the show.
• 46:09
  Check them out at predictionguard.com. Also, thanks to Breakmaster Cylinder for the Beats, and to you for listening. That's all for now, but you'll hear from us again next week.