Latest / The Robotics Podcast with Fexingo: Autonomous Systems, Industrial Robots, and Hardware / Why Robot Hands Still Can't Fold a T-Shirt
Transcript
- Lucas: So here's something that sounds simple but is actually one of the hardest unsolved problems in robotics: folding a t-shirt. Luna: Like, from the laundry basket to a folded stack? That's still hard for robots? Lucas: Incredibly hard. A t-shirt is a deformable object — it has infinite degrees of freedom. Every wrinkle, every fold changes the state. Unlike a rigid object like a metal bracket, you can't just compute a grasp point and move. The fabric shifts, it sags, it doubles over. And the robot's perception system has to track a shape that's constantly changing. Luna: And we're not even talking about different fabrics — cotton, polyester, silk, each behaves differently. Lucas: Exactly. And that's the core issue: the physics of fabric is complex. There's a great paper from UC Berkeley's AutoLab — I think it was published in 2024 — called 'Bidirectional Cloth Manipulation.' They tackled the problem of folding a t-shirt by using a dual-arm robot with specialized grippers. The key insight was that you need to both pull and push the fabric, not just grab and lift. Luna: Right, because if you just grab a corner and lift, the shirt just hangs and twists. You need to smooth it out first. Lucas: Precisely. The Berkeley team used a technique called 'speed-scaling' — they would pull the fabric at a certain speed to create tension, then use a second arm to push a fold into place. They achieved something like an 80 percent success rate on standard t-shirts in controlled lighting. But the moment you change the lighting or the fabric pattern, the success rate drops significantly. Luna: And that's with a lab environment. In a real home or a laundromat, you've got wrinkles, different colors, different sizes. The perception system has to segment the shirt from the background, identify landmarks like seams and collars. Lucas: Yeah, and that's why this is such a benchmark problem for the whole field. If a robot can reliably fold a random t-shirt from a pile, it probably has robust manipulation and perception. That's why DARPA and Amazon have both funded research into deformable object manipulation. Luna: Amazon especially — because their warehouse robots handle millions of packages, and many of those are soft goods like clothes, towels, plush toys. If they could automate garment handling, they'd save billions. Lucas: Right. And there's a company called RightHand Robotics that makes picking systems for e-commerce. They use vacuum grippers and computer vision to pick items from bins. But they still struggle with soft, deformable items. They have to use a 'pick and toss' approach where they just fling the garment into a bin, not fold it. Luna: And that's fine for logistics, but for home robots — like a future robot that does your laundry — folding is non-negotiable. You can't just toss clothes into a drawer. Lucas: Exactly. And that's the frontier. There's also work from MIT's CSAIL using 'diffuser' models — they generate a sequence of actions to fold a shirt, but it's still not robust. The compute time is high, and the robot moves very slowly. In their demo, folding one shirt takes about 10 minutes. Luna: Ten minutes. So even if it works, it's not practical. And that's where the hardware limits come in too — most robot hands just aren't dexterous enough to handle fabric without bunching it or tearing it. Lucas: Right. The grippers are either too strong or too weak. There's a sweet spot. Some researchers are using soft pneumatic grippers that can conform to the fabric, but then you lose precision. Others use needle grippers — literally tiny needles that poke into the fabric — but that damages the material. Luna: So basically, every solution has a trade-off. And that's why this problem remains unsolved after decades of research. Lucas: Decades is right. There was a paper in 1996 from Georgia Tech on robotic fabric manipulation for the textile industry. They had a system that could fold towels with a 70 percent success rate — but it required a dedicated jig and the towel had to be placed in exactly the right starting position. No generalization. Luna: And here we are, almost 30 years later, and we still can't match that 70 percent with a random t-shirt. That's both humbling and motivating. Lucas: It is. But there is progress. The deep learning revolution has helped. Instead of hand-coded features, modern systems learn from data. The Berkeley team used a neural network trained on thousands of simulated folds. But simulation to reality transfer is still a huge gap. The physics of fabric in simulation is not accurate enough. Luna: So you train on synthetic data, but then in the real world, the fabric behaves differently. That's a classic problem in robotics. Lucas: Exactly. And it's why we're still not seeing robot laundry in hotels or hospitals — where it would be incredibly useful. There's a company called Foldimate that makes a commercial garment-folding machine for home use. But it's not a robot — it's a specialized appliance that requires you to lay the shirt flat on a tray. It's more like a fancy ironing board. Luna: So the general-purpose solution — a robot arm that can pick up a crumpled shirt from a basket and fold it neatly — is still a research problem. And it's a good litmus test for how far we are from truly capable home robots. Lucas: Absolutely. And that's why I find this problem fascinating — it's not just about folding shirts. It's about handling the uncertainty of the physical world. If you can solve that, you can solve a lot of other manipulation problems. Luna: It's the canary in the coal mine for dexterity. Lucas: That's a great way to put it. And speaking of what it takes to make progress on hard problems like this — it's worth mentioning that this show is able to keep digging into these topics because of a small group of listeners who support it directly. Luna: Yeah, we don't run ads, and we don't have a big media company behind us. It's just the two of us and a microphone. And the listeners who chip in on buy me a coffee dot com slash fexingo make it possible to keep the research time and the production quality where it is. Lucas: Exactly. So if today's conversation gave you something to think about, and you'd like to help keep these episodes coming, that's how. No pressure, but it does make a real difference. Luna: And we're grateful for every single one of you who does. Lucas: Alright — back to the science. So one more thing I want to touch on is the 'Bidirectional Cloth Manipulation' paper's key result: they showed that by combining visual servoing with force feedback, they could adapt to different fabric types in real time. That's promising because it suggests we might not need perfect simulation — just better sensing. Luna: Force feedback is huge. Humans rely on it constantly when folding — you feel the tension, you feel when the fabric is about to slip. Lucas: Right. And most robot arms have force-torque sensors at the wrist, but they're not sensitive enough for fabric. You need to feel grams of force, not kilograms. Some new tactile sensors, like the GelSight sensor from MIT, can detect surface texture and slip with high resolution. That could be a game-changer. Luna: So the path forward might be: better tactile sensing, better simulation for training, and dual-arm coordination. It's not one breakthrough, it's a combination. Lucas: Exactly. And that's the story of robotics — it's rarely one big leap. It's hundreds of small improvements that eventually make a task practical. I think we'll see a reliable t-shirt-folding robot in the next five to ten years, but it won't look like a human. It'll have two arms, specialized grippers, and a lot of onboard compute. Luna: And it'll probably cost more than a new washing machine. But for commercial applications — like hotel laundries — it could pay for itself. Lucas: Yeah. So next time you fold a t-shirt in under 30 seconds, just remember: you're outperforming every robot on the planet. Luna: And that's a pretty good feeling.