Latest / The Robotics Podcast with Fexingo: Autonomous Systems, Industrial Robots, and Hardware / Why Robot Arms Still Cant Pick Up a Lego Brick
Transcript
- Lucas: So, I spent part of the weekend trying to get my two-year-old nephew to pick up a Lego brick from the carpet. Took him about half a second. And then I watched a video of a million-dollar robot arm trying the same thing — and failing, again and again. Luna: Wait — a robot arm can't pick up a Lego brick? Even with all the vision systems and grippers we have today? Lucas: Nope. And this isn't just a fun party trick — it's actually a really hard research benchmark. There's a 2025 study out of the University of Tokyo where they tested several commercial robot arms on picking up small, rigid plastic objects — like Lego bricks — and the failure rate was over thirty percent. Luna: Thirty percent failure? That's terrible. I mean, I get that a brick is small, but it's not like it's a soft grape or something. Lucas: Right, and that's the weird part. Soft objects are hard in their own way. But rigid small objects — think electronic components, screws, buttons — they expose a different kind of gap. The problem is that when a gripper closes on a rigid object, any tiny misalignment in position or force causes the object to slip or launch. It's actually harder than gripping a soft ball. Luna: Launch — as in the brick flies out of the gripper? Lucas: Exactly. The gripper jaws come together, but because the brick is hard and has low friction, the force vector pushes it sideways and it skitters away. Humans have incredibly sensitive force feedback in their fingertips — we adjust grip pressure unconsciously. Robots have force-torque sensors, but they're not nearly as fast or as nuanced. Luna: So it's a sensing problem, not a strength problem. Lucas: Partly. But also a control problem. The University of Tokyo study used a standard industrial arm with a two-finger parallel gripper — pretty typical for assembly lines. The robot had a high-res camera and a force sensor. But when it tried to pick a single two-by-four Lego brick from a flat surface, the success rate was under seventy percent. If the brick was slightly tilted, it dropped to fifty. Luna: That's wild. I feel like we hear about robots doing surgery or assembling cars, and then something as mundane as a toy brick trips them up. Lucas: Yeah. And that's the thing about robotics — the stuff that's easy for humans is often the hardest for machines. Folding laundry, tying a knot, picking up a Lego brick. Those tasks involve contact-rich manipulation with very low tolerance for error. Luna: Has anyone tried a different gripper design? Like a suction cup or a soft gripper? Lucas: Great question. Suction works well on flat, smooth surfaces, but Lego bricks have those little ridges and the studs — they break the seal. Soft grippers that use flexible fingers can conform to shape, but they often lack the precision to grab a tiny object without squeezing it out. The team tested a soft gripper and it actually performed worse — the brick would stick to one finger and then get stuck. Luna: So we're stuck with this failure mode. What's the fix? More sensors? Better algorithms? Lucas: A lot of researchers are working on something called 'tactile sensing' — basically giving robot fingers a sense of touch that's closer to human skin. There are these new sensors that measure pressure distribution across the fingertip in real time, not just a single force vector. If a robot can feel that the brick is starting to slip, it can adjust grip mid-lift. Luna: Sounds expensive. Lucas: It is. The high-resolution tactile sensors can cost thousands of dollars per finger. But the bigger challenge is the control loop — processing that tactile data fast enough to react. Humans do it subconsciously in milliseconds. Robots have to run a control algorithm, compute adjustments, and send motor commands. There's latency. Luna: But some of your listeners might be wondering — why should I care about robot arms picking up Lego bricks? What's the real-world implication? Lucas: Great question. The core problem here is small parts handling. Think about electronics manufacturing — placing resistors, capacitors, connectors onto circuit boards. Those are tiny rigid objects. Any failure in picking them up means a mis-placed component, a damaged part, or a jammed machine. Even a one percent failure rate in a high-volume line can cost millions in downtime and scrap. Luna: So solving the Lego brick problem is actually a step toward better electronics assembly. Lucas: Exactly. And it's not just electronics. Packaging — think of picking up a single candy from a tray. Pharmaceutical pills. Screws and washers in automotive assembly. Anywhere you have to pick a small rigid object reliably, this challenge shows up. Luna: Interesting. So what's the state of the art trying to solve this? Is there any promising approach? Lucas: One approach is 'data-driven grasping' — training neural networks on thousands of grasp attempts so the robot learns the right strategy. The University of Tokyo team used a reinforcement learning approach where the robot practiced for hours in simulation before trying real bricks. In simulation, they got success rates up to ninety-five percent. But in the real world, it dropped again because of physics mismatches — friction, lighting, tiny imperfections in the brick surface. Luna: Sim to real gap. Classic. Lucas: Classic. There's also work on using 'compliant grippers' — grippers that have a bit of give, like spring-loaded fingers, so they can conform to the object without needing perfect sensing. Those help, but they reduce speed and repeatability. Luna: So it's a trade-off. And speaking of trade-offs — it's interesting that you mentioned a million-dollar robot arm failing at something a toddler can do. It makes you think about the economics of automation. Lucas: That's a good point. You know, this kind of conversation — where we get into the nitty-gritty of what robots can and can't do — it's exactly why we're able to keep making this show ad-free. A small but dedicated group of listeners helps us out through buy me a coffee dot com slash fexingo, and that support means we don't have to run ads or chase sponsors. It lets us focus on the details. Luna: Yeah, and we really appreciate it. It's a pretty small thing for each person, but it adds up to let us keep diving into topics like this — where the answer isn't a simple yes or no. Lucas: Exactly. And speaking of that — let's go back to the Lego brick. One thing I didn't mention is that the challenge isn't just picking it up — it's also placing it accurately. In assembly, you have to put the brick into a specific position with tight tolerance. That adds another layer of difficulty. Luna: Right. So the pick is hard, but the place is also hard. What about the orientation? Picking up a brick that's randomly oriented is different from one that's perfectly aligned. Lucas: Huge difference. Most industrial systems rely on precisely oriented parts — they come in trays or feeders. But in many real-world scenarios, parts are jumbled in a bin. That's called 'bin picking' — and it's one of the hardest open problems. If you have a bin full of Lego bricks in random orientations, the robot has to identify each one's pose, plan a grasp that won't collide with other bricks, and then execute it. Success rates for bin picking of small parts are often below fifty percent. Luna: That's where the term 'jam' comes from. If a robot fails to pick, it can disturb the rest of the bin and cause a cascade of failures. Lucas: Exactly. That's why many factories still use humans for small parts assembly. We're just too good at it. But the labor shortage in manufacturing is pushing companies to invest in better robotics. So the incentive is there. Luna: Are there any startups or labs that are making notable progress on this? Lucas: There's a startup called 'RightHand Robotics' that focuses on piece-picking for logistics — they use a combination of suction and a 'soft gripper' approach that's pretty effective for things like packaged goods. But for small rigid parts, there's a company called 'Grasping Robotics' out of MIT that's working on a tactile-sensing gripper with a fast control loop. They claim near hundred percent success on objects like Lego bricks in controlled settings. But we haven't seen it in the wild yet. Luna: So it's a lab to market gap as well. Lucas: Always is. And that's the fascinating thing about robotics — the easy problems get solved fast, but the hard ones, like picking up a Lego brick, reveal how far we still have to go. It's humbling. Luna: Humbling, but also exciting. Because every time we crack one of these small challenges, we unlock a new set of applications. Lucas: Absolutely. And maybe someday soon, a robot will be able to pick up that Lego brick my nephew played with — and then maybe we can finally get robots to assemble IKEA furniture without swearing. Luna: That's the real benchmark.