Latest / The Robotics Podcast with Fexingo: Autonomous Systems, Industrial Robots, and Hardware / Why Robot Arms Still Can't Grasp a Soft Ball
Transcript
- Lucas: I've been thinking about something absurdly simple: why can't a robot arm pick up a tennis ball? Luna: Like, just grab it off a table? I could do that blindfolded. Lucas: Exactly. A human hand does it without conscious thought. We sense the ball's shape, its squishiness, the friction — and we adjust our grip in milliseconds. A robot arm, even one with a state of the art gripper, fails more often than you'd think. Luna: I've seen those videos. The robot either crushes the ball or drops it. There's rarely an in-between. Lucas: Right. And this isn't some niche lab curiosity. It's a bottleneck for entire industries. Food processing — think about handling tomatoes or chicken breasts. Logistics — picking a soft package off a conveyor belt. Even medicine, where robots might one day assist in surgery on soft tissue. Luna: So what's the fundamental problem? Is it the gripper design, the sensors, or the control software? Lucas: It's all three, but let's start with the physics. A tennis ball is what roboticists call a 'deformable object.' It changes shape when you apply force. So the robot can't just use a rigid gripper with a fixed position — it has to actively control the force it applies. Too little, and the ball slips. Too much, and it deforms — or pops out. Luna: And most industrial grippers are designed for rigid parts. They're basically two parallel jaws that close to a programmed position. Lucas: Exactly. That works for a steel bracket or a plastic housing. But for a tennis ball, the gripper doesn't know exactly where the surface is. It's squishy. So the robot either overestimates the closure and squeezes too hard, or underestimates and doesn't make contact at all. Luna: So you need force sensing. But even force-torque sensors have limits, right? Lucas: They do. Most industrial robots have a force-torque sensor at the wrist. It measures the forces and torques applied at the end effector. But those sensors are designed for rigid contacts — like a peg-in-hole insertion. When you're grasping something soft, the contact is distributed over a larger area, and the sensor sees a noisy, low-magnitude signal. Luna: So it's like trying to hear a whisper in a crowded room. Lucas: Exactly. And the sampling rate matters. If the sensor updates at, say, one kilohertz, that's fast enough for most tasks. But the control loop that adjusts the gripper position might run at a lower rate, introducing lag. Meanwhile, the ball is deforming and the contact patch is changing. Luna: So you need tactile sensing, not just force sensing at the wrist. You need to know what's happening at the fingertips. Lucas: Bingo. And that's where GelSight sensors come in. Developed at MIT's CSAIL, these are essentially cameras with a soft elastomeric gel pad. When the gripper presses against an object, the gel deforms, and the camera captures the deformation pattern. You get high-resolution tactile images — like a fingerprint of the surface. Luna: I've seen those. They can detect ridges, textures, even slippage before it happens. Lucas: Yes. In a 2025 paper, MIT researchers showed that a GelSight-equipped gripper could grasp a tennis ball with over 90 percent success — but only after training on thousands of grasps. And even then, it failed on certain orientations or when the ball was slightly dusty. Luna: Dusty? That's a real-world condition. Factories aren't cleanrooms. Lucas: Right. So the sensor is promising, but it's not robust. Another approach is to use compliant grippers — soft robotic fingers that passively conform to the object's shape. Think of those three-fingered grippers that look like tentacles. Luna: Those work great for picking up eggs or tomatoes. But they struggle with heavier objects or precise placement. Lucas: Exactly. There's a trade-off: compliance gives you better grasping of soft objects, but worse precision for rigid tasks. So most robot cells today use interchangeable grippers — swap out the parallel jaws for a soft gripper when you're handling produce. But that adds cycle time and complexity. Luna: What about deep learning? Can't a neural network just learn to grasp after enough trials? Lucas: It helps, but it's not a silver bullet. Deep learning for grasping typically uses vision to predict a good grasp pose. But vision alone can't measure the force. So you might get a great approach angle, but then the gripper still over-squeezes or under-squeezes. The network has to implicitly learn the force dynamics from the image, which is hard. Luna: So you need to combine vision, tactile sensing, and force control in a closed loop. That's a lot of data and computation. Lucas: It is. And that's why even the most advanced research labs — like the ones at MIT, Stanford, and CMU — still report failure rates of 10 to 20 percent for grasping novel deformable objects. Compare that to a human, who might fail once in a thousand grasps. Luna: And if you're running a factory that processes thousands of items an hour, a 10 percent failure rate is catastrophic. Lucas: Exactly. It means you need human oversight to handle the exceptions, which defeats the purpose of automation. So the industry has largely punted on this problem. They design their processes to avoid grasping soft objects — using conveyors, chutes, or vacuum-based pick and place for flat or sealed items. Luna: But that limits what you can automate. There's a huge opportunity in food processing and e-commerce fulfillment if someone cracks this. Lucas: There is. And you know, it's conversations like this that remind me why we keep doing these deep dives. The feedback we get from listeners — especially engineers and technicians — is that these episodes give them a clearer picture of where the field really stands. Luna: And that's exactly the kind of thing that keeps a show like ours going. We don't run ads, we don't have a corporate sponsor. It's just us and our listeners. Lucas: Yeah. So if you've gotten something out of these episodes — a new way to think about a problem, or just a few minutes of genuine curiosity — a couple of dollars a month genuinely makes a difference. It's buy me a coffee dot com slash fexingo. That's it. No tiers, no perks, just knowing you're helping keep the conversation going. Luna: We appreciate every single one. Now, back to the tennis ball problem — I want to talk about a specific gripper design that's trying to bridge the gap between rigid and soft. Lucas: Oh, are you thinking about the Fin Ray effect gripper? The one with the crossed-structure fingers that passively conform? Luna: Exactly. Inspired by fish fins. When you push against it, the structure buckles inward and wraps around the object. It's purely mechanical, no sensors needed. Lucas: Right. And it works surprisingly well for a range of shapes. But it has its own limitations: it can't apply high grip forces, and the wrapping motion can be slow. For a tennis ball, it might work, but for a slippery plastic package, it might not get enough traction. Luna: So there's no universal solution yet. It's still a matter of choosing the right gripper for the task. Lucas: And that's the state of the art. We have specialized tools that work well in narrow domains, but we don't have a general-purpose robot hand that can handle a tennis ball, a screwdriver, and a raw egg with equal reliability. That's probably still a decade away. Luna: But at least now I know why my robot vacuum can't pick up a stray sock. Lucas: Exactly the same problem. Deformable objects, compliance, sensing — it all comes back to that tennis ball.