Latest / The Robotics Podcast with Fexingo: Autonomous Systems, Industrial Robots, and Hardware / Why Robot Arms Cant Handle a Single Crayon
Transcript
- Lucas: So you've got a standard industrial robot arm — six-axis, titanium gripper, repeatability down to a few microns. You ask it to pick up a steel bracket, it'll do it a million times without a miss. Then you hand it a crayon. Luna: And it just — what, crushes it? Lucas: Crush, drop, or fail to pick it up at all. The same arm that can place a bearing with sub-millimeter precision can't reliably pick a crayon off a table. It's one of those problems that sounds trivial but has kept robotics labs busy for years. Luna: We've done a whole series on things robot arms can't handle — the grape, the balloon, the wet paper towel. But a crayon feels like it ought to be easier. It's not squishy, it's not slippery, it's not super lightweight. Lucas: Right, it's stiff, it's dry, it's got a known shape. So why is it hard? I think there are three distinct challenges, and the crayon actually makes a great case study for all of them. First: grasp variability. Luna: Meaning a crayon can be oriented in basically infinite ways when it's lying on a table. Lucas: Exactly. A cylindrical object on a flat surface can be rolled, tilted, or partially wedged against something. The robot doesn't know where the center of mass is until it tries to grab it. And unlike a steel bracket, a crayon has very little internal structure — there's no flange, no hole, no feature the gripper can latch onto. Luna: So the gripper has to rely purely on friction and geometry. And if the surface is slightly dusty or the crayon is wax-coated, the coefficient of friction changes. Lucas: That's challenge number two: force control. A typical industrial gripper is either open or closed — it applies a binary force. For something like a metal part, that's fine. For a crayon, you need to apply just enough pressure to lift it without cracking the wax casing or deforming the tip. Luna: And the tip is the worst part. If the crayon is pointing toward the gripper, the robot can't grip the tip without breaking it. But if it goes for the body, it might not have enough clearance to get a reliable grasp. Lucas: That's actually challenge three: visual occlusion. Most robot arms use a camera mounted above the workspace. When the gripper approaches, its own body blocks the view of the object. So the robot is essentially grabbing blind in the last few millimeters. Luna: Which is fine for a big, rigid object. But for a small, variable one, the last millimeter is where everything goes wrong. Lucas: There was a really interesting paper from MIT's Soft Robotics Lab — published early 2025, I think — where they tried to solve this by combining a compliant gripper with real-time tactile feedback. They used a two-fingered gripper with soft silicone pads and embedded capacitive sensors. Luna: So the gripper could feel the shape of the object as it closed, rather than just relying on a pre-programmed force. Lucas: Right. And they got the success rate on crayons up from around 40 percent — for a standard parallel-jaw gripper — to about 85 percent. Which is a big jump, but still not reliable enough for a production line. Luna: So even with the best available research, a human picking up a crayon is still far more robust than any robot. That's humbling. Lucas: It is. And it's not just about crayons. The same principles apply to any object that's small, variable in orientation, and fragile. Think about pharmaceuticals — picking individual pills from a bin. Or electronics assembly — placing tiny capacitors on a board without cracking them. Luna: Or, more prosaically, sorting recycling. A mixed bin of rigid plastic bottles, crumpled cans, and broken glass is a nightmare for a robot. Lucas: Yeah, and that's actually where a lot of the commercial interest is. Companies like AMP Robotics and Covariant have been deploying vision-guided arms for recycling sorting, and they're getting better, but they still struggle with objects that don't have a predictable geometry. Luna: So a crayon is almost a perfect test case for that whole class of problems. It's small, fragile, and orientation-variable. If you can reliably pick a crayon, you can probably pick a lot of other things. Lucas: There's another angle too. The crayon problem exposes a limitation in how we think about robot learning. Most reinforcement learning for grasping is done in simulation, where objects are perfect cylinders with uniform mass. Real crayons have slightly different lengths, the paper label adds friction variation, the tip is asymmetric — the simulation never captures that. Luna: sim to real transfer is still a huge gap. And crayons are a perfect example of where it breaks down. Lucas: A group at UC Berkeley actually did a study last year where they trained a neural network on thousands of simulated grasps of cylindrical objects, then tested it on real crayons. The success rate dropped from 95 percent in simulation to about 35 percent in reality. Luna: Ouch. So the robot learned to grasp a perfect cylinder, but the real world is full of imperfect cylinders. Lucas: Exactly. And the fix isn't just more data — it's better sensors and better control policies that can adapt in real time. That's where the field is moving. Compliant grippers, tactile skins, force-torque sensors in the wrist — all of that is becoming more affordable. Luna: And those are the same technologies that would help a robot do something like fold laundry or prepare food. It all ties together. Lucas: Yeah, I think the crayon is a great microcosm of the whole 'dexterous manipulation' challenge. And it's worth noting that even with all these advances, the best robotic systems still can't match a five-year-old child at picking up a crayon. A five-year-old does it without thinking, with no training data required. Luna: That's the benchmark. And it's a humbling one. Lucas: Absolutely. But I also think these limitations are what keep the field interesting. Every time we think we've solved grasping, a crayon comes along to remind us how much we still don't understand about manipulation. Luna: And on that note — if you've gotten something out of these deep dives into what robots can and can't do, there's a way to support the show directly. A couple of dollars a month is genuinely what keeps these going — buy me a coffee dot com slash fexingo, if that feels right to you. Lucas: Yeah, it's just a small way to keep the podcast ad-free and focused on the stuff we actually find interesting. No pressure at all. Luna: And to bring it back to the crayon — I wonder what the next object in our 'robot arms can't handle' series should be. Maybe a piece of uncooked spaghetti? Lucas: Oh, that's a good one. Brittle, long, and it snaps unpredictably. I think we just found episode 90. Luna: Perfect. We'll see you then.