Latest / The Robotics Podcast with Fexingo: Autonomous Systems, Industrial Robots, and Hardware / Why Robot Arms Still Cant Button a Shirt
Transcript
- Lucas: You button a shirt in about three seconds, without looking, while your brain simultaneously plans the rest of the day. A robot arm, with sub-millimeter precision and computer vision, will take upwards of thirty seconds and still miss the hole one out of every five attempts. Luna: And that's in a lab with perfect lighting and a stiff cardboard shirt. In the real world, with rumpled fabric, it's worse. Lucas: Right. So why can't a robot do what a ten-year-old can do? That's today's thread. I want to start with a number that stuck with me. In garment manufacturing, around 90 percent of all operations are still done by hand. Luna: Ninety percent? That's enormous. I would have thought automation had eaten into that by now. Lucas: You'd think. But fabric is what roboticists call a 'deformable object'. It sags, stretches, folds in unpredictable ways. Most industrial grippers are designed for rigid parts — metal brackets, plastic housings. They fail catastrophically on a piece of cotton. Luna: So buttoning is just the hardest case of that problem? Lucas: Exactly. It combines deformable material — the shirt fabric — with a precision insertion task: pushing a small rigid button through a tight slit. You need both to sense the fabric's state and to apply exactly the right force. Too much and you jam. Too little and you drop. Luna: And the button itself is slippery, often has rounded edges, and the hole is only slightly larger than the button. There's almost no tolerance. Lucas: Let's talk about the sensing problem. Most robot arms use joint encoders and maybe a wrist force-torque sensor. That gives you force data at the macro level, but not at the fingertip. To button a shirt, you need to know if the button is aligned with the hole before you push. If it's off by even a millimeter, the fabric buckles. Luna: So you need high-resolution tactile sensing right at the point of contact. Something like a human fingertip — with its thousands of mechanoreceptors. Lucas: Researchers at MIT's Soft Robotics Lab have been working on exactly that. They built a capacitive tactile sensor that sits on the robot's gripper — it's like a thin rubber pad with electrodes that measure pressure distribution at about 100 points per square centimeter. Luna: That's dense. But still far from human skin, which has, what, two hundred and fifty thousand tactile units per square centimeter? Lucas: Yes. But even with that sensor, the robot they trained could button a shirt only about 70 percent of the time. And it was trained on thousands of buttoning attempts in simulation and real world. The failures came from edge cases: a slight fold, a seam that changed the hole's geometry, a button that rotated as it entered. Luna: It sounds like the algorithm is brittle. It learns a specific motion, but can't generalize to variations. Lucas: That's the second big barrier. Learning-based methods for manipulation tend to overfit to the training conditions. Change the fabric weight, the button size, the shirt's orientation — and the success rate plummets. There's no 'common sense' about how buttons work. Luna: And humans have that common sense. We know to hold the fabric taut, to tilt the button slightly, to use the thumb to guide it through. A robot doesn't have a concept of 'gently'. It has a trajectory. Lucas: Which brings up a deeper question about force control. In traditional industrial assembly, robots use 'search strategies' for peg-in-hole tasks — they wiggle the peg around until it drops in. But for a button, the hole is in a flexible material. That wiggle can distort the fabric and misalign everything further. Luna: So you need to control not just the robot's motion, but also the fabric's motion. That's a whole other layer of complexity. Lucas: Exactly. Some teams are trying to use two robot arms — one to stretch the fabric and one to manipulate the button. But that doubles cost and complexity. For a garment factory that might use dozens of stations, that's not economically viable. Luna: And that's the thing — this isn't just a neat academic puzzle. The textile industry is massive, and still very labor-intensive. If we could automate buttoning, it could reshore some manufacturing, reduce costs, and improve consistency. Lucas: Absolutely. There's also the potential in healthcare — think of robots helping people with limited mobility get dressed. A robot that can button a shirt could also help with other garment tasks like zipping a jacket or fastening a bra strap. Luna: But we're years away from that, aren't we? Even the best research demo has a 30 percent failure rate. Lucas: Years, maybe a decade. But the progress in tactile sensing and sim to real transfer is accelerating. Another group, at the University of Tokyo, uses magnetic-based tactile sensors that are even more sensitive. They reported 85 percent success on a single shirt type. But again, transfer to unseens is poor. Luna: It's fascinating how much we take for granted. I mean, I button my shirt while thinking about the day's podcast topics — I never once think about the physics of it. Lucas: And that's exactly the kind of conversation we love bringing you on this show — diving into the details that get overlooked. Which is also why we deliberately don't run ads on these episodes. If you want to support that choice, the link is buy me a coffee dot com slash fexingo. Luna: Yeah, it keeps us independent, focused on the technology rather than selling something. Appreciate everyone who chips in. Lucas: Now, back to buttons. There's another angle I find really interesting, and it's about the material of the button itself. Most buttons are made of plastic, sometimes with a glossy surface. That gloss creates two problems for vision systems: specular reflections that make it hard to detect the button's exact pose, and the fact that the hole often looks like a dark circle against a dark background. Luna: So even before the robot touches the button, it might not know exactly where it is or how it's oriented. Lucas: Right. And depth cameras often struggle with small, thin objects. A typical Intel RealSense camera has a depth resolution of about one millimeter at best. For a button that's maybe eight millimeters wide, that millimeter of uncertainty is huge. Luna: So you'd need a higher-resolution perception system, like a structured-light scanner or even a specialized camera. That adds cost and complexity. Lucas: Especially in a factory environment with dust and vibration. The trade-off between robustness and cost is why you still see people doing button attaching in apparel assembly lines. Luna: And there's also the question of feedback during insertion. Humans get continuous tactile feedback — we can feel if the button is catching on the fabric, and we adjust. Robots typically only get force feedback at the joint, which is filtered and delayed. Lucas: That's where the research frontier is. High-bandwidth, local tactile feedback that can be used in a control loop. Some labs are even doing what's called 'haptic rendering' — simulating the feel of the fabric to train the controller. Luna: So you're saying we might not solve this with traditional control. We need a fundamentally different approach to perception and actuation at the fingertip. Lucas: That's the bet of soft robotics. By using compliant materials, you allow the robot to conform to the environment rather than precisely planning every trajectory. But soft robots have their own issues — they are hard to control precisely, and they wear out faster. Luna: It's a balancing act. We want the adaptivity of soft, but the precision and durability of rigid. Lucas: And that's essentially the holy grail of dexterous manipulation. Buttoning a shirt is just one specific barometer of how close we are. And right now, we're not very close. Luna: But it's worth noting that we don't need perfect automation. Even a 90 percent reliable robot buttoner could be useful in a factory setting, with a human handling the failures. That's the model in many industrial applications now. Lucas: That's a good point. And actually, some start-ups are already commercializing fabric-handling robots for tasks like folding towels or sorting garments. Buttoning might come later, but the pieces are falling into place. Luna: So where do you see the next breakthrough coming? Better sensors, better algorithms, or better hardware? Lucas: I think all three, but the sensor piece is most critical. Without high-resolution tactile feedback at a reasonable cost, algorithms are flying blind. Hardware like the BioTac sensor from SynTouch is promising, but still too expensive for wide deployment. If someone can mass-produce a durable, sensitive fingertip sensor for under 50 dollars, it would change the game. Luna: That's a challenge to the hardware engineers listening. We'll keep watching that space. Lucas: Indeed. And as always, thanks for thinking alongside us about the real frontier in robotics.