Latest / The Robotics Podcast with Fexingo: Autonomous Systems, Industrial Robots, and Hardware / Why Robot Arms Still Can't Catch a Ball
Transcript
- Lucas: So here's a deceptively simple question — why can't a robot arm catch a ball? Luna: I mean, I've seen those videos of robots catching ping-pong balls at high speed. Isn't that solved? Lucas: Those are impressive demos, but they're usually working in very controlled conditions — the ball is backlit, the trajectory is predictable, the catch is a single motion. Real catching — a friend tossing you a crumpled paper ball from across the room — is an entirely different problem. Luna: Okay, so what's the bottleneck? Sensors? Processing speed? Lucas: It's a combination, but the core issue is prediction. To catch something, you need to anticipate where it will be a few hundred milliseconds from now. Your brain does this effortlessly — it models gravity, air resistance, spin, and your own hand motion all at once. A robot has to do that calculation from scratch every single time. Lucas: And if you think about the last time you fumbled a catch — maybe a ball hit your palm wrong and bounced off — that's exactly where robots fail. They don't have the tactile feedback to adjust mid-grasp. Most industrial grippers are either open or closed; they can't subtly conform. Luna: So we're talking about both the brain and the hand. Which is harder? Lucas: Right now, the brain. There was a really telling study from MIT last year — they used event-based cameras, which are much faster than traditional cameras, to try to catch a slow-moving foam ball. The system achieved about a 60 percent success rate on balls tossed at around two meters per second. That's a slow lob. A child catches that nine times out of ten. Luna: Wait — event-based cameras are supposed to be the breakthrough for fast motion. Why only sixty percent? Lucas: Because the camera is only part of the pipeline. Even if the sensor sees the ball at microsecond precision, the control loop — processing the image, predicting the trajectory, generating a motor command — introduces latency. In that study, the total latency was around 45 milliseconds. That's fast by robot standards, but by the time the arm moves, the ball has already shifted. Luna: And humans? What's our latency? Lucas: It's not even comparable. Visual processing in humans takes about 100 milliseconds, but we compensate with predictive models built from years of experience. A baseball player doesn't just react to the ball's current position; they anticipate based on the pitcher's arm angle, the spin, the release point. Robots have no such priors. Lucas: That's actually something we've been thinking about a lot here. And it's a good moment to mention — the reason we can dive into these niche robotics questions every single day without ads is because some listeners support the show directly. If you get something useful out of these deep dives, you can chip in at buy me a coffee dot com slash fexingo. It's a small group, but it's what keeps this ad-free and focused on the engineering, not the algorithm. Luna: Yeah, and it really does make a difference. I know we don't talk about it often, but every contribution helps us spend more time on research instead of chasing sponsorship. Lucas: Exactly. So back to the ball-catching problem — one of the things the MIT team tried was adding a soft, deformable gripper instead of a rigid claw. That alone improved the catch rate to about 70 percent. Luna: Because the hand could conform a bit on impact, kind of like how our palm gives slightly? Lucas: Right. The soft gripper absorbs some of the impact and creates a pocket that traps the ball. It's a mechanical hack that compensates for the prediction error. But even at 70 percent, it's not reliable enough for any real-world application. Luna: So what would it take to get to ninety-nine percent? Lucas: That's the billion-dollar question. One approach is to combine multiple sensor modalities — vision, tactile feedback from the fingertips, even sound. If the robot can hear the ball approaching, it can start anticipating earlier. Another approach is to use learning-based control that trains on thousands of failed catches. Lucas: But the fundamental challenge is that catching is a closed-loop control problem with tight time constraints. Unlike picking a static object from a bin, where you can take your time, catching requires decisions in milliseconds. And the state space is huge — the ball's velocity, spin, air resistance, lighting conditions, background clutter. Luna: It reminds me of the problem with autonomous driving — the edge cases are what kill you. The ball might come from an unexpected angle, or it might be a strange shape. Lucas: Exactly. And unlike driving, where you can brake or swerve, catching requires precise contact. You can't just avoid the ball; you have to meet it. Lucas: There's also the question of what constitutes a catch. Is it a secure grasp? Or just preventing the ball from hitting the ground? Some robot setups use a basket-like hand that simply cups the ball. That's easier, but it's not a catch in the human sense. Luna: So we're comparing apples to oranges unless we define the task precisely. Lucas: Exactly. The MIT study defined a catch as the ball being held securely for at least two seconds after contact. That's a high bar. Many robot 'catches' in viral videos actually just trap the ball against a surface or use a sticky material. Luna: Sticky? That's almost cheating. Lucas: It is. But it highlights the lengths researchers go to compensate for the core perceptual and control gaps. Gecko-inspired adhesives, electrostatic grippers — they all work in narrow conditions, but none generalize. Lucas: What I find fascinating is that we've built robots that can assemble iPhones, perform surgery, and explore Mars. And yet, catching a ball remains a research challenge. It's a reminder of how much of our everyday dexterity relies on subconscious prediction and adaptation that we don't even think about. Luna: So where's the research headed next? Any promising work beyond MIT? Lucas: A group at Stanford is working on what they call 'reflexive catching' — using a high-speed arm with a lightweight gripper that can adjust in under 10 milliseconds. They're essentially building a robotic equivalent of the human startle reflex. Preliminary results show they can catch a ball dropped from directly above almost every time. Lucas: But the real test is a random toss from an unpredictable position. And there, we're still in the labs. Luna: I wonder if we'll ever get to the point where a robot can play catch with a child in a park. Lucas: I think we will, but it'll require a fundamentally different architecture than today's industrial arms. Something that integrates vision, touch, and prediction at a hardware level, not as separate modules piped together. Luna: So maybe the next generation of robot arms won't look like arms at all. Lucas: Maybe they'll look more like a ball of sensors and actuators — something that doesn't have a single 'hand' but a whole surface that can catch. That's the kind of thinking the field is moving toward. Lucas: For now, though, if you're playing catch with a robot, be prepared to do a lot of fetching. Luna: I'll stick with my dog. Lucas: Probably a safer bet.