Latest / The Robotics Podcast with Fexingo: Autonomous Systems, Industrial Robots, and Hardware / Why Robot Arms Still Can't Catch a Falling Object
Transcript
- Lucas: So a few weeks ago I was in a warehouse outside Chicago, watching a robotic arm do a pick and place task at roughly double the speed of its human counterpart. And I thought, great, we've conquered grabbing. But then I watched it fumble a dropped screwdriver that slid off a tray, and it made no attempt to catch it. It just let it fall. Luna: Right, because a screwdriver that's slipping is basically a moving target. And robots are terrible at moving targets. Lucas: Exactly. And that's the thing — we've spent a hundred and fifty episodes on gripping and feeling, but today I want to talk about a deeper problem that sits underneath all of it: catching. Not catching a ball like a human, but catching an object that's already in motion, whether it's a dropped part on a factory line or a tool that slips out of the gripper. It's a fundamental skill that robots still struggle with. Luna: So why is catching so hard? I mean, humans do it without thinking. I catch a falling phone before I even register it's moving. Lucas: Because it's a split-second loop of prediction, motion, and contact — and each of those steps is a hard problem. First, you need to sense the object's position and velocity fast enough. That means high-speed vision, often running at hundreds of frames per second, with latency in the single-digit milliseconds. And then you need to compute a trajectory in real time, which is a classic predictive control problem. But the third step is where it gets really ugly: the contact itself. Luna: The contact. Because catching isn't just about hitting the object — it's about decelerating it without bouncing it away or crushing it. Lucas: Precisely. If your gripper is too stiff, the object bounces off. Too slow, and it slips through your fingers. Too fast, and you might damage the object or the robot. The dynamics of a falling object are actually simple — it's just gravity and air resistance. But the dynamics of contact, of two surfaces meeting and transferring momentum in milliseconds, that's where the physics gets nasty. Luna: And it's not just about the fall itself. It's about predicting where the object will be when your hand gets there. And that requires a model of the object's motion, which might be spinning or tumbling. Lucas: Exactly. A dropped screwdriver doesn't fall straight down — it rotates around its center of mass. So the robot has to predict the spin, the orientation at impact, and then plan a grasp that matches that orientation. That's a lot of computation in a very short window. A human does it with a mix of reflexes and learned motor patterns. A robot has to do it with explicit models and control algorithms. Luna: So what's the state of the art? Are there any robots that can actually catch something reliably? Lucas: There are research systems that can catch a ball tossed at them — some from over a decade ago, like the DLR robot arm that caught a ball thrown by a human. But those work in controlled labs with precise motion capture. The problem is generalizing that to arbitrary objects, in unstructured environments, with noisy sensors. That's the gap. Luna: So the lab demos are impressive, but they're not ready for the factory floor. What's the biggest bottleneck? Lucas: I'd say it's the combination of sensing and actuation. Your eyes need to be fast enough to track the object, but your hand also needs to move quickly enough to intercept it. High-speed arms are expensive and often have limited payload. And then you need tactile sensing to detect the moment of contact, so you can modulate the grip force in real time. That's a lot of hardware working in concert, and the integration is hard. Luna: Speaking of tactile sensing — we've talked a lot about robot hands that can feel. How does that help with catching? Lucas: It helps a lot, actually. A hand with rich tactile feedback can detect the initial contact, the pressure distribution, and then adjust its grip accordingly — like a human catching a watermelon and softening the impact. Researchers at MIT and elsewhere have shown that adding tactile sensors to fingertips improves catching performance dramatically, because the robot can react to the actual contact instead of relying solely on predicted trajectories. Luna: And there's also the learning angle. Instead of explicitly modeling the physics, can robots learn to catch from experience, like humans do? Lucas: Absolutely. Reinforcement learning has been applied to catching tasks, where the robot trains in simulation on thousands of falls, then transfers the policy to the real world. The challenge is the sim to real gap — the physics in simulation is never perfect, so the robot might learn to catch a perfect sphere but fail on a slightly different object. But it's a promising direction because it doesn't require hand-coded models. Luna: So catching is a bit of a benchmark for dexterity. If a robot can catch a falling object, it means it can perceive, predict, and act within milliseconds. That's a huge step toward more versatile automation. Lucas: Right. And there are practical applications beyond just saving a dropped screw. Think about a robot arm that needs to hand off a tool to a human — the human might not wait until the robot is perfectly still. Catching is also relevant for sorting systems, where parts fall off a conveyor and need to be grabbed mid-air to keep the line moving. Luna: And even in space, catching a satellite or a piece of debris — that's essentially a high-speed catching problem with enormous consequences. Lucas: Exactly. So it's not a trivial problem. It's a fundamental capability that's missing from most industrial robots. And that's why the research is so exciting right now. Luna: So what's the one key takeaway you want listeners to have today? Is it that catching is the next big hurdle for robotics? Lucas: I think so. We've solved the 'gripping a static object' problem reasonably well. The next frontier is handling objects in motion, and catching is the purest form of that challenge. If we crack it, we'll see robots that are genuinely collaborative — they can work alongside humans without requiring them to hand over objects in a perfectly choreographed way. Luna: Interesting. And before we go deeper into how the leading labs are approaching this, I'm thinking about something you said earlier — about the integration of sensors and control. That kind of research isn't cheap, and it's often funded by public grants or large corporate R&D budgets. Lucas: Yeah, it's not cheap. But it's that kind of deep, foundational work that keeps this show interesting. And we're able to cover it because we're independent and ad-free, thanks to a small group of listeners who support us. If you've gotten value from our deep dives into robotics, and you want us to keep doing them, you can support the show at buy me a coffee dot com slash fexingo. It's a simple way to help us keep the lights on and the microphones recording. Luna: And every little bit genuinely helps us allocate time to research and interviews. We really appreciate it. Lucas: So, back to the labs. One approach that's gaining traction is using learned models of object dynamics, instead of rigid physics. The idea is to train a neural network to predict the motion of an object falling from a known height, based on a few camera frames. Then use that prediction to guide the robot's hand to the right intercept point. Luna: That sounds like model predictive control meets deep learning. But how do you handle the uncertainty? The object might be spinning differently than predicted. Lucas: You build uncertainty into the model. Instead of predicting a single trajectory, you predict a distribution of possible trajectories, and then you plan a grasp that maximizes the chance of success. That's robust control. It's a bit like a human playing tennis — you don't know exactly where the ball will land, but you position yourself to cover the most likely outcomes. Luna: So it's not just about being fast, it's about being smart about uncertainty. That's a nice way to frame it. Lucas: And there's another angle: soft hands. Instead of a rigid gripper that has to be precisely positioned, you can use a soft, compliant hand that conforms to the object's shape on contact. That relaxes the precision requirement a bit. There's a research group at Harvard that has a jellyfish-like gripper that can catch a live fish out of water, just by wrapping around it. That's a pretty striking example. Luna: That sounds almost organic. It's a totally different strategy than the high-speed vision approach. Lucas: Right. High-speed vision and stiff arms are one end of the spectrum. Soft, adaptive hands are the other. The future probably combines both: fast sensing and prediction, but with a compliant end effector that can handle the messiness of real-world contact. Luna: So if I'm a manufacturer looking at this technology, when can I expect my robots to catch falling parts on the line? Lucas: Probably not in the next couple of years as a plug and play feature. But I'd expect to see it integrated into specific applications, like high-speed sorting or bin picking, where the objects are relatively uniform and the environment is controlled. For truly arbitrary objects in chaotic settings, we're probably a decade away from commercial readiness. Luna: A decade is a long time, but the progress in the last five years has been pretty stunning. I mean, we're talking about robots that can learn to tie knots and now catch moving objects. Lucas: And that's why we keep doing this show — to track that progress. Catching is a great litmus test for the state of robotic dexterity. When a robot can catch a randomly tossed object in a busy warehouse, that's when we'll know we've truly moved past the era of automation that just repeats the same motion over and over. Luna: And that's a future worth watching. So thanks for joining us for episode 160. We'll be back soon with another deep dive.