Latest / The Robotics Podcast with Fexingo: Autonomous Systems, Industrial Robots, and Hardware / Why Robot Arms Still Can't Recognise a Spilled Drink
Transcript
- Lucas: Picture this: you're at a coffee shop, you knock over your latte, and a puddle spreads across the counter. For you, it's obvious — wet, shiny, a little brown. For a robot arm trying to clean it up? That puddle might as well be invisible. Luna: I've seen videos of robots confidently sweeping right over a spill like it's not there. It's almost comical. Lucas: It's one of those problems that sounds trivial until you actually try to solve it. Today we're talking about why robot arms still can't recognise a spilled drink — and it comes down to physics. Luna: Physics? Not just bad algorithms? Lucas: Partly algorithms, but the fundamental issue is that liquids create what's called a specular surface — it's mirrorlike. Most robot vision systems rely on structured light or lidar, which beams out infrared patterns and reads the reflections. When that light hits a puddle, it bounces off at an angle, just like a mirror. The sensor either gets no return or a false reading that says 'this is empty space.' Luna: So the robot literally doesn't see the puddle. It sees a hole in the world. Lucas: Exactly. There's a great paper from MIT's CSAIL back in 2023 — they tested several commercial depth cameras on transparent liquids and found that over 60 percent of the pixels on a puddle were classified as either missing or background. The robot's point cloud just has a void where the spill is. Luna: And that's not just a coffee shop problem. I'm thinking factory floors where coolant or lubricant leaks — a robot could drive right through it, slip, or fail to clean it. Lucas: Right, industrial settings too. In fact, some automotive plants have reported that their robotic floor scrubbers miss oil spills because the lidar treats the sheen as a flat surface at a different depth. So the scrubber goes around it, leaving the hazard. Luna: What are the solutions people are working on? I know there's been a push toward multispectral cameras. Lucas: That's one approach. Instead of just one wavelength — typically near-infrared — you use cameras that capture visible, near-infrared, and sometimes thermal. Liquids have a distinct thermal signature: they cool or heat differently than the surrounding surface. A warm coffee spill on a cold counter? Thermal camera sees it instantly. Luna: But thermal cameras are expensive. And they don't work well if the liquid and surface are the same temperature. Lucas: That's the catch. So some startups are combining thermal with polarisation filters. When light reflects off a wet surface, it becomes polarised — the waves align in a specific orientation. By putting a polarising filter on a standard camera and rotating it, you can detect the wet area as a change in brightness that doesn't happen on dry surfaces. Luna: That's clever. It's almost like using the physics of the spill against itself. Lucas: There's a company called SpillVision — I think they're out of Berlin — that's doing exactly that. They claim a 95 percent detection rate on water, oil, and coffee spills on common floor materials, using a cheap polarisation camera and a neural network trained on thousands of spill images. Luna: So the algorithm part is still important. It's not just the sensor. Lucas: No, you need both. The polarisation data gives you a signal, but the model has to learn that a shiny patch on tile is a spill, not a reflection from a window. That's where the AI comes in. Luna: And how far are we from a robot arm that can actually wipe up a spill? Not just detect it. Lucas: Detection is step one. Once you know there's a puddle, you still have to plan a wiping motion that doesn't just spread the liquid around. That's a whole other challenge — the robot needs to know the viscosity, the surface tension, how much force to apply. We're not there yet for unstructured homes. Luna: But in controlled environments like labs or factories, are there systems that work today? Lucas: Yes. I've seen demonstrations from a Japanese robotics lab where a dual-arm robot detects a coolant spill on a metal table, deploys a squeegee attachment, and pushes the liquid into a drain. But the table is flat, the liquid is always the same, and the lighting is controlled. Change any one variable, and it fails. Luna: So it's a reminder that these 'simple' human tasks are incredibly complex for machines. We take our ability to see a puddle for granted. Lucas: And that's exactly why we keep doing episodes like this — to appreciate what's hard. And if you enjoy these deep dives into the nitty-gritty of robotics, it's worth knowing that a couple of dollars a month is genuinely what keeps these going. You can support the show at buy me a coffee dot com slash fexingo. Luna: Yeah, it's a small thing that makes a big difference for an ad-free show like ours. Honestly, every little bit helps us keep digging into these weird, specific problems. Lucas: So back to the spill problem — I think the next frontier is bringing together polarisation, thermal, and standard RGB into a single sensor module that costs under a hundred dollars. A few startups and university labs are working on that right now. Luna: And once you have the sensor, you need a training dataset of spills — like a 'SpillNet' or something. Lucas: There actually is something called the 'Spill Dataset' from a group at Carnegie Mellon. They released it in 2024 — over 10,000 images of various liquids on different surfaces, annotated with pixel-level masks. It's been downloaded over 5,000 times. Luna: That's encouraging. So the data problem is being solved alongside the hardware. Lucas: Right. And a few companies are already commercialising spill detection for autonomous cleaners. One that comes to mind is Avidbots — they make the Neo floor-scrubbing robot used in warehouses and airports. Their latest model uses a combination of lidar and a neural network trained on the CMU dataset to avoid or clean spills. Luna: Does it actually clean them, or just avoid them? Lucas: It can do both. If the spill is small and the robot has the right attachment, it'll clean it. If it's large, it'll map the location and alert a human. That's a practical hybrid approach. Luna: So we're moving from 'the robot can't see the spill' to 'the robot can see it and decide what to do.' That's progress. Lucas: Absolutely. And it's happening faster than I expected. Two years ago, the idea of a robot reliably detecting a transparent puddle was research-only. Now it's shipping in commercial products. Luna: What about home robots? Like a future Roomba that can clean up a kitchen spill? Lucas: That's harder because homes are unpredictable — different floor types, lighting conditions, and the spills are varied: juice, milk, wine. But I think within five years we'll see a home robot that can at least detect a spill and maybe mop it up with a disposable pad. Luna: I'd pay for that. Especially after my toddler's breakfast. Lucas: Exactly. And that's the market pull — convenience. So the incentive is there. Luna: One more thing: you mentioned earlier that lidar fails on transparent surfaces. But what about using sonar or radar? Could those see through the puddle? Lucas: Sonar has been tried for liquid level sensing, but for a thin puddle on a hard floor, the echo from the floor dominates. Radar — millimeter-wave — can actually detect water because it has a high dielectric constant. But it's expensive and bulky for a small robot arm. Luna: So the polarisation-thermal combo seems like the sweet spot right now. Lucas: It is. And it's getting cheaper. Polarisation sensors that used to cost thousands are now available as off-the-shelf modules for under two hundred dollars. The Sony IMX250MZR sensor, for example, has a polarisation filter directly on the chip. Luna: That's the kind of hardware progress that makes these problems solvable. Lucas: And that's the theme of this whole series — each episode, we look at something robots can't do yet, and by the end we see the path forward. Spills are no exception. Luna: Thanks, Lucas. I'm actually going to look up that SpillVision company after this. Lucas: Do it. And if you find any good spill-detection robots in the wild, let us know.