Latest / The Robotics Podcast with Fexingo: Autonomous Systems, Industrial Robots, and Hardware / Why Robot Arms Cant Tie a Knot
Transcript
- Lucas: So, you've probably tied your shoes this morning without thinking about it. But for a robot arm, tying a single knot is one of the hardest unsolved problems in robotics. Luna: I've seen videos of robots folding laundry or assembling IKEA furniture — but a knot? That seems almost trivial by comparison. Lucas: It's deceptively hard. And it's not just about shoelaces. Surgeons tie knots with sutures every day, and if a robot could reliably do that, it would change operating rooms. But the physics of a loose string is incredibly complex. Luna: Before we dive too deep — I just want to say, if you're getting something out of these deep dives, a couple of dollars a month is genuinely what keeps these going. Buy me a coffee dot com slash fexingo, if you've gotten something useful out of them. Really helps us stay ad-free and focused on the real engineering. Lucas: Yeah, it's a small gesture that makes a big difference. So, back to knots. Why is it so hard? First, a string is a continuous, flexible object with infinite degrees of freedom. You can't just pick it up like a rigid block. Luna: And unlike a rigid object, the string changes shape as you touch it. The robot has to constantly sense and adapt. Lucas: Exactly. Plus, you need to manage friction — the string can slip, tangle, or tighten unpredictably. And to tie a knot, you have to loop one part around another, which requires coordinated movements of both arms or fingers, all while maintaining tension. Luna: So what's the state of the art? I know there was a big paper from MIT a couple years ago about a robot that could tie a surgical suture knot. Lucas: Right, the MIT group led by Daniela Rus published work on a robot that could tie a 'surgeon's knot' — that's a specific knot used in suturing. They used two Panda arms from Franka Emika, each with two-fingered grippers, and a camera system to track the thread. Luna: But that's still very controlled. The robot knew the exact position of the thread because they used a dark thread on a light background. Real-world isn't that clean. Lucas: That's the key limitation. The MIT system was essentially doing visual servoing with a well-contrasted thread. In a real operating room, the thread might be wet, bloody, and partially occluded by tissue. So it's a proof of concept, not a clinical tool. Luna: What about tactile sensing? Some groups are working on that. Lucas: There's promising work at the University of Tokyo using GelSight sensors — those are soft, camera-based tactile sensors that can detect texture and slip. They've shown a robot hand that can tie a simple overhand knot by feeling the tension in the string rather than relying on vision. Luna: So is the approach moving away from vision toward more touch-based control? Lucas: Both are needed, but I think the real breakthrough will come from combining them with reinforcement learning. In simulation, you can train a neural network to manipulate a virtual string, then transfer that policy to a real robot. Luna: sim to real transfer for deformable objects is notoriously hard. The simulation has to model friction, string stiffness, and dynamics accurately. Lucas: Right, and that's where groups like Google's robotics team and OpenAI have invested. They built a simulation called DManus that can simulate strings and cloth with pretty high fidelity. But even then, the policy often fails when the real string behaves slightly differently. Luna: So we're still a ways off from a robot that can tie your shoelaces reliably? Lucas: For a single specific knot in a controlled environment, we're getting close. But the general skill — picking up any string and tying any knot in any configuration — that's still a grand challenge. The current record for a robot tying a shoelace is something like a minute or two, and it only works with a thick, stiff lace on a flat surface. Luna: And a human can do it in a few seconds, blindfolded. Lucas: Exactly. That gap shows how much we take for granted in dexterous manipulation. But the applications are huge. In manufacturing, think about tying cables or wires in a harness. Right now that's done by hand. In surgery, autonomous knot-tying could reduce variability and fatigue. Luna: What about in space? NASA has been interested in robots that can tie tethers or secure payloads. Lucas: Great point. In zero-G, the dynamics are even more complex because there's no gravity to keep the string taut. There's research from the European Space Agency on that. So it's not just a niche problem — it's a bottleneck across multiple industries. Luna: So if someone listening is an engineer or researcher, where should they focus? Vision, tactile, simulation? Lucas: I think the most promising direction is learning-based tactile control. Vision gives you a coarse plan, but tactile feedback lets you correct in real-time when the string slips. And combining that with a good simulation for pre-training could accelerate progress. Luna: Have there been any recent breakthroughs in the last year or so? Lucas: In early 2025, a team from Stanford and Berkeley published a paper on 'KnotNet' — a deep learning model that takes a depth image of a string and predicts the sequence of gripper movements to tie a specific knot. They achieved a 70 percent success rate on a single overhand knot, which is pretty good for research. Luna: Seventy percent in a lab. Real-world would probably be lower. Lucas: Definitely. But it's a start. They also showed that the model could generalize to different string thicknesses and materials, which is encouraging. The next step would be to handle knots that require multiple loops, like a bowline or a reef knot. Luna: What about using a soft robot arm instead of rigid joints? Would that help? Lucas: Some groups think so. Soft grippers can conform to the string and distribute force better. But they also have less precision. It's a trade-off. I think the answer is hybrid designs — rigid for gross motion, soft for fine manipulation. Luna: So where do you see this in five years? Lucas: I'd guess we'll see the first commercial surgical robot that can tie a simple suture knot autonomously, but only in a specific, controlled step of a procedure. For general-purpose knot tying, like a household robot tying your shoes, probably ten years out. But the progress is real, and each small step unlocks new capability. Luna: It's fascinating how something so simple for us reveals the complexity of the physical world. Lucas: That's why I love this stuff. Every time we think a problem is solved, the robot reminds us how much we take for granted. And that's the frontier.