Latest / The Robotics Podcast with Fexingo: Autonomous Systems, Industrial Robots, and Hardware / Robot Hands Are Learning to Judge Weight
Transcript
- Lucas: You know, we've spent a lot of episodes on what robot hands can and can't feel. Texture, shape, whether they're about to drop a screwdriver. Luna: Right, and we keep coming back to the same theme. The hand is incredible at some things, and then it hits a wall on something a five-year-old does without thinking. Lucas: Exactly, and today I want to hit another one of those walls. Something that sounds trivial but is actually a nightmare for roboticists. Judging weight. Not measuring it with a scale, but feeling it. The difference between picking up a full bottle of water and an empty one. Luna: Ah, that's a good one. Because a robot arm can be programmed to exert a certain force, but if it doesn't know the mass, it's either going to crush the thing or drop it. Lucas: Right, and that's exactly the problem. Most industrial robots today work in highly controlled environments. They know exactly what they're picking up because it's the same box, the same pallet, day in and day out. But the moment you put them in a less structured setting, like a warehouse or a lab where objects vary, they get thrown. Luna: Is it purely a sensor problem, or is it a software problem? Lucas: It's both, and that's what makes it interesting. You can buy off-the-shelf force-torque sensors that measure the forces at the wrist really precisely. But weighing an object dynamically, as you're lifting it, involves separating the gravitational force from the inertial forces. And that's a perception problem, not just a measurement one. Luna: So the robot feels the gravitational pull, but it also feels the acceleration of its own arm, and it has to separate those out. Lucas: Exactly, and that's harder than it sounds because the robot's own arm is moving. The forces the sensor sees are a mix of the object's weight, the arm's own dynamics, and the inertial effects. You need a really good model of your own arm to subtract out its contribution. And if that model isn't perfect, you get errors. Luna: So the robot's perception of its own body is a bottleneck. That's kind of humbling. Lucas: It is, and honestly, it's a nice segue to something I want to mention. We keep this show ad-free, and it's listener support that makes that possible. If today's conversation gives you something usable, there's a link at buy me a coffee dot com slash fexingo. Tiny ask, but it genuinely keeps the lights on. Luna: Yeah, and we don't do sponsors or mid-rolls, so every little bit helps. Now, back to the weight problem. How are roboticists actually approaching this? Lucas: So there are a few approaches. The classic one is to use a force-torque sensor in the wrist and just infer the weight from the steady-state reading after the robot has stopped moving. That's reliable, but it requires stopping, or moving very slowly, so it's not great for fast picking. Luna: Right, and in a fast-moving e-commerce warehouse, you don't want to pause at every pick. Lucas: Exactly, so the more interesting work is in estimating weight during the motion itself. There's a technique called inertial parameter identification, where you excite the arm with specific movements and use the force sensor readings to solve for the mass and the center of mass of the payload. It's like the robot is doing a little dance to weigh the object. Luna: A dance to weigh it, I like that. But that's still a fairly slow approach, right? Lucas: It is, but there are also learning-based approaches now. You can train a neural network on a dataset of force sensor signals and known masses, and then it can estimate the weight in a fraction of a second. And those models can also incorporate tactile sensors on the fingertips, which give you information about the pressure distribution. Luna: So the robot is basically using the same cues that we do. We feel the pressure on our fingertips, and we have a sense of the inertia as we move the object. Lucas: Exactly, and it's interesting because humans are actually really good at this. You can pick up a cardboard box and immediately know if it's heavy or light, even before you've fully lifted it. And we do that by combining visual cues, like the size of the box, with tactile cues, like the initial pressure. Luna: So the visual context matters too. If a robot sees a large box, it might expect it to be heavy, but it's actually full of packing peanuts. Lucas: Yes, and that's where some of the most recent research is going. There's a paper from a few months ago where they used a dataset of household objects and trained a model to predict weight from both vision and touch. And they found that combining the two modalities significantly improved accuracy over using either one alone. Luna: That makes sense, because vision gives you a prior and touch gives you the actual feedback. Lucas: Right, and the key metric is that the robot can estimate weight within a few percent error, which is good enough for many tasks. But it's still not at human level. Humans can estimate weight within a few percent as well, but they do it almost instantaneously, without any conscious effort. Luna: So what's the practical bottleneck? Is it the sensors, the algorithms, or the computing power? Lucas: I'd say it's the combination of all three, but the biggest issue is the lack of good tactile data. Force-torque sensors are relatively cheap and reliable, but they only measure the net forces at the wrist. They don't give you the high-resolution, spatially distributed pressure maps that your skin provides. Luna: Right, and the tactile sensors that can do that are still quite expensive and fragile, with limited durability. Lucas: Exactly, and that's why you see many research labs building their own custom tactile arrays, because the commercial ones don't meet their needs. But I think we're at an inflection point. There are startups now that are developing more robust, lower-cost tactile sensors, and that could make weight estimation much easier. Luna: So what would you say the current state of the art is, in terms of a robot that can reliably weigh an object while picking it up? Lucas: I'd say we're at the point where a robot can estimate the weight of a common object with a median error of around five to ten percent, but only if the objects are in a training distribution. Once you put something completely novel in front of it, that error can jump significantly. Luna: So it's still a generalization problem, just like many other areas in robotics. Lucas: Yeah, it's a recurring theme. But the interesting thing is that weight estimation isn't just about not dropping things. It also enables other capabilities. For example, if you know the weight, you can adjust the grip force to be just enough to hold the object without deforming it. That's critical for food handling or fragile items. Luna: That's a great point, because we've talked about how robot hands can't handle raw chicken or crack an egg. Weight estimation is part of that equation. Lucas: Exactly, and it also enables more natural human-robot collaboration. If a robot hands you a tool, it should hand it to you in a way that's comfortable and safe, which depends on the weight. A heavy wrench and a light scalpel require very different handover dynamics. Luna: I hadn't thought of that, but it's a really nice application. So the problem is more than just a curiosity, it's a building block for a lot of other robot skills. Lucas: Absolutely, and I think the next big step will be integrating weight estimation into the control loop, not just as a post-hoc calculation. Imagine a robot that adjusts its motion on the fly based on the weight it feels. That would be a huge leap forward. Luna: And do you see this happening in the near term, or is it still years away? Lucas: I think we'll see it in controlled environments within a couple of years, especially in logistics and manufacturing. The technology is almost there, and the demand is high. But for fully unstructured environments, like a cluttered kitchen, it's going to take a bit longer. Luna: So the next time we're amazed that a robot can fold a towel, we should also be amazed that it can't tell if the towel is wet or dry. Lucas: Ha, exactly, and that's the kind of thing that keeps roboticists up at night. But it's also what makes this field so exciting. Every time we solve one problem, we uncover a new one that's even more subtle.