Latest / The Robotics Podcast with Fexingo: Autonomous Systems, Industrial Robots, and Hardware / Why Robot Arms Still Cant Handle a Pencil
Transcript
- Lucas: Alright, Luna, I have a new one for the 'Why Robot Arms Still Can't' hall of fame. Episode 109. Luna: Hit me. I'm ready. Is it something we all use every day? Lucas: A pencil. Just an ordinary number-two pencil, lying on a flat table. Pick it up. Luna: Oh, that's a good one. I've definitely seen robot arms fail at that. It looks so easy for us. Lucas: Deceptively easy. I spoke with Dr. Ellen Park at the MIT Manipulation Lab, and she told me the pencil is basically the perfect storm of geometric and mechanical challenges. Let's break it down. Luna: Please. Because I can grab a pencil without even looking. My hand just does it. Lucas: Exactly. And that's the benchmark. So, first problem: the diameter. A pencil is about 7 to 8 millimeters wide. Too small for most parallel-jaw grippers to get a stable hold without slipping. The contact area is tiny. Luna: Right, and it's cylindrical, so there's no flat face to grip. Lucas: That's the second problem. A cylinder has continuous curvature. When a gripper applies force, the pencil can roll right out. In robotics, they talk about 'force-closure' and 'form-closure.' Force-closure means the gripper can apply forces that resist any external disturbance. Form-closure means the geometry of the gripper physically constrains the object so it can't move. Luna: And a pencil on a table achieves neither, I'm guessing. Lucas: Not with a standard two-finger gripper. To get form-closure on a cylinder, you'd need three or four contact points arranged around it. Your hand does that naturally — your thumb opposes your index and middle fingers, creating a three-point grip. But a typical robot gripper is just two parallel pads. Luna: So why not give the robot a three-finger hand? They exist. Lucas: They do, and some labs use them. But three-finger grippers are more expensive, more complex to control, and still struggle with the third problem: orientation. A pencil on a table can be pointing any direction. The robot has to detect that orientation precisely and align its gripper accordingly. Luna: And computer vision isn't great at that, especially with a thin, shiny cylinder. Lucas: Right. A pencil is reflective, has a hexagonal or round cross-section, and often has a metal ferrule and an eraser at one end. Standard RGB cameras can pick up the shape, but depth sensors struggle with the specular highlights. Dr. Park told me their lab uses structured light and tactile sensors together, and even then, the success rate is only about 80 percent. Luna: Eighty percent on a lab bench is impressive, but in a factory or a home, that's not reliable enough. Lucas: Exactly. And when it fails, it's usually because the pencil rolls just as the gripper closes. That fraction of a millimeter shift throws off the grasp. Humans compensate with real-time tactile feedback — we feel the pencil start to roll and adjust our grip. Robots don't have that bandwidth yet. Luna: So tactile sensing is the missing piece. Lucas: It's a big part of it. Dr. Park's lab uses GelSight sensors — basically a soft gel with a camera inside that captures high-resolution tactile images. They can feel the texture and curvature. But processing that data in real time and generating a corrective action is still a research challenge. Luna: And then there's the issue of picking it up from a table, not from a gripper. Lucas: That's the fourth problem. The table surface creates a constraint, but also a friction complication. If the pencil is flush against the table, the gripper has to slide under it or push it to the edge. That requires a 'pre-grasp' manipulation — a whole other layer of planning. Luna: So what's the state of the art? Can any robot reliably pick up a pencil today? Lucas: In controlled settings, yes. Some labs report 90-plus percent success with specialized grippers and multi-modal sensing. But generalizable, robust pencil picking — the kind that works on any table, any pencil, any orientation — is still not solved. Dr. Park says it's a benchmark for dexterous manipulation, and we're not there yet. Luna: This is why your Roomba can't pick up a coin, either. Same physics. Lucas: Exactly the same. Small, thin, slippery, variable orientation. The pencil problem is a microcosm of the whole dexterity challenge. Luna: And yet, I can pick up a pencil while holding a coffee cup and looking at my phone. It's humbling. Lucas: It really is. Every time we think robots are close, a pencil reminds us how far we are. But the progress is real — ten years ago, no robot could reliably pick up a pencil at all. Now we're at 80 percent. That's a huge leap. Luna: So what's the next breakthrough needed? Better grippers, better sensing, better algorithms? Lucas: All three, but Dr. Park thinks the biggest bottleneck is tactile feedback integrated with fast control loops. If we can get that right, the pencil problem might be solved within five years. Luna: And then we move on to something even harder, like picking up a single grain of rice. Lucas: Don't give them ideas. But seriously, the pencil is a great test case because it's universal. Everyone understands it. Luna: And it's a nice metaphor for the gap between human and machine dexterity. Lucas: Before we wrap, I want to mention something. If today's tech conversation gave you something usable, a couple of dollars a month is genuinely what keeps these going. Buy me a coffee dot com slash fexingo, if you've gotten something out of them. Luna: Yeah, it's a small thing that makes a real difference. We keep it ad-free because of listeners like that. Lucas: Back to the pencil. I think the takeaway is that the hardest problems in robotics aren't the big, flashy ones — they're the small, everyday ones we don't even think about. Luna: And that's what makes this show so interesting. Every episode, I look at something I do without thinking and realize how much engineering went into it. Lucas: So next time you pick up a pencil, spare a thought for the robot that can't. It's not because robots are dumb. It's because pencils are deceptively hard. Luna: Agreed. And we'll keep tracking the progress. Thanks, Lucas. Lucas: Thanks, Luna. See you next time.