Latest / The Robotics Podcast with Fexingo: Autonomous Systems, Industrial Robots, and Hardware / Why Robot Vision Still Fails in Bright Light
Transcript
- Lucas: A few months ago, a German automotive tier-one supplier lost an entire eight-hour shift on a transmission assembly line. The culprit wasn't a mechanical failure or a software bug. It was glare. Luna: Glare. Like — sunlight hitting a phone screen kind of glare? Lucas: Exactly that. Except instead of squinting at a text message, a robot arm was trying to find a clutch housing on a conveyor belt. The ceiling lights were reflecting off a freshly machined surface, the camera saw a white blob where the part should be, and the robot just stopped. It wouldn't pick. Luna: So it wasn't a pick and place error — it was a vision-input error. The robot literally couldn't see the part. Lucas: Right. And this isn't a one-off. It's a class of problem that's been hanging over industrial vision for years. Machine vision systems have gotten dramatically better — higher resolution, faster frame rates, cheaper sensors — but they still get confused by the same physics that makes your car's lane-keeping fail in heavy rain. Luna: I want to understand the actual bottleneck here. Is it the sensor chip itself, or the processing software? Lucas: Both, but the sensor is the harder constraint. The key spec is something called dynamic range — measured in decibels. A typical consumer camera sensor has a dynamic range around 60 to 70 dB. That means it can capture a certain ratio between the brightest and darkest parts of a scene. On a sunny day with bright white metal parts, the difference between a glare reflection and a shadow under the part can exceed 100 dB. The sensor just clips. Luna: Clip, as in, the bright parts all become pure white and the dark parts become pure black — you lose the detail. Lucas: Exactly. So the robot sees a uniform white rectangle where there's actually a complex metal shape. And since its grip is programmed based on precise geometry, it can't safely grab a white blob. So it halts. Luna: And that's why the supplier lost a shift. But what did they actually do to fix it? Did they swap out the cameras? Lucas: They tried a few things. First, they installed polarized lighting filters on the overhead LEDs. That cut the specular reflections significantly — like putting polarized sunglasses on the robot. But that only helped in certain orientations. If the part rotated slightly on the belt, the polarization angle mismatched and the glare came back. Luna: So not a silver bullet. Did they move to higher dynamic range cameras? Lucas: They tested some. The current high-end industrial cameras — think 125 dB dynamic range — are about three times the cost of the standard 70 dB units they were using. And they're bigger. But the bigger issue is that even a 125 dB camera can't always resolve the geometry if the light source is too directional. The fix ended up being a combination: higher dynamic range cameras plus a dome-shaped diffuser around the workstation that scatters light evenly from all angles. Luna: So it's a lighting redesign, not just a sensor upgrade. That makes sense — if you can't change the sensor's physical limits, change the light entering it. Lucas: Exactly. And that's the pattern across a lot of industrial vision failures: it's rarely the algorithm that's the bottleneck, it's the raw signal to noise ratio at the input. There's a famous 2019 paper from MIT that showed a state of the art object detection network failing 40 percent of the time on parts with even moderate glare, even though it scored 99 percent on clean datasets. Luna: Forty percent — that's huge. And that was seven years ago. Has the gap narrowed since? Lucas: It has, but not as much as you'd think. The big advance has been in synthetic data — training neural networks on millions of simulated glare conditions so they learn to infer the missing geometry. A 2024 paper from Fraunhofer showed that a network trained on synthetic glare could predict the correct part pose from a partially clipped image about 80 percent of the time. That's much better than zero, but it's still not production-ready for safety-critical lines. Luna: Right, because 80 percent means one in five picks is still a guess. In an automotive plant, that's a crash or a scrapped part every few minutes. Lucas: Exactly. So most production engineers still rely on the lighting approach. And it works, but it's expensive and fragile. A dome diffuser array for a single workstation can cost fifteen to twenty thousand dollars. Multiply that across a factory with three hundred stations and you're looking at millions. Luna: That's wild. But I guess that's why the problem hasn't been solved with software alone. The physical world is just harder to model than a virtual one. Lucas: And that brings us to a question a lot of listeners might be thinking: is this a problem that's going to be solved by better sensors, better algorithms, or better lighting? I'd argue it's all three, but the biggest unlock is actually in new sensor architectures — specifically, sensors that capture multiple exposures simultaneously. Luna: Like HDR photography on a phone, where it takes several images at different exposures and combines them? Lucas: Exactly. But in a phone, you can take three frames sequentially. In a factory, the part is moving at two meters per second, so sequential exposures blur. The new approach uses a sensor with multiple charge wells per pixel — essentially capturing high and low exposures at the same instant. Sony released a sensor like that in 2023, and it's now being integrated into some industrial cameras. Early tests show it can handle up to 145 dB without any external lighting changes. Luna: That's a game changer if it holds up. But I'm guessing it's expensive and not widely deployed yet. Lucas: Right. The cost is about four times a standard industrial camera. But for high-value lines — say, aerospace engine components where a single scrapped part costs fifty thousand dollars — the ROI is immediate. On a lower-margin line like consumer electronics assembly, it's harder to justify. Luna: So there's a segmentation emerging: high-end lines get the fancy multi-exposure sensors, while the rest make do with lighting workarounds and synthetic-data algorithms. Lucas: That's exactly the picture. And that segmentation is likely to persist for at least the next three to five years, until the sensor cost comes down and the algorithms get robust enough to handle even the messy cases. Luna: I want to circle back to something you mentioned earlier — the Fraunhofer synthetic data work. Are there any production lines actually running with that, or is it still lab-stage? Lucas: A few pilot lines, mostly in Germany and Japan. One BMW plant near Munich has been testing a system that pairs a standard 70 dB camera with a neural network trained on synthetic glare. Early results show it can handle about 90 percent of glare conditions — but the remaining 10 percent still cause false picks, which means a human has to supervise. So it's not fully autonomous. Luna: So we're in that awkward middle ground where automation works most of the time, but not all of the time. That's actually harder to manage than a fully manual line, because you have to keep a human standing by even though they're mostly idle. Lucas: Exactly. And that's the hidden cost. It's not just the hardware — it's the operational overhead of babysitting a system that's almost reliable. The industry term is 'the 90 percent trap.' Luna: I've heard that phrase. It applies to a lot of AI systems too. They hit 90 percent accuracy fast, and then the last 10 percent takes ten years. Lucas: Right. And in robotics, that last 10 percent often involves edge cases in lighting that are hard to anticipate. Like a worker walking by and casting a moving shadow, or a fingerprint smudge on the part creating a weird reflection. The real world is messy. Luna: It is. And this episode is a good reminder that a lot of the exciting robotics breakthroughs we hear about — the demos on YouTube — are in controlled lab lighting. The factory floor is a different animal. Lucas: That's actually a perfect way to frame it. The gap between lab and production is often just a matter of lighting. And that's not a sexy headline, but it's the truth. Luna: If today's episode was useful to you and you want it to stay ad-free and accessible, listener support is what makes that possible. There's a link — buy me a coffee dot com slash fexingo — if you want to chip in. It's a small thing that keeps these conversations going. Lucas: Yeah, and we mean that. It's not a big ask. The show stays free either way, but the support helps us put time into episodes like this one. Luna: All right, back to the factory floor. So the tier-one supplier I mentioned — the one that lost a shift — what was the final fix they settled on? Lucas: They went with a combination of polarized diffusers and a software update that added a 'glare detection' module. The module doesn't try to see through the glare — it just detects when the camera is getting a bad signal and triggers a re-illumination sequence where the robot moves to a different angle and takes a second image. That way, they don't need the expensive multi-exposure sensor. The fix cost about eight thousand dollars per workstation and reduced glare-related downtime by 95 percent. Luna: So a pragmatic, hardware plus software solution. That's often what works in industry — not the flashiest tech, but the one that fits the constraints. Lucas: Exactly. And that's the lesson I want listeners to take away: robot vision is not a solved problem, but it's a solvable one if you're willing to engineer around the physics. And the physics, in this case, is just light reflecting off a surface. Luna: It's also a reminder that the most expensive robotics failures are often the ones you don't see coming — like a reflection at the wrong angle. I'd love to hear from listeners who have dealt with similar issues. Our DMs are open. Lucas: Absolutely. And in the next episode, we're going to look at a related challenge: how robots handle changing light conditions across a full workday, from morning to evening. It's a problem that sounds trivial until you try to solve it. Luna: Can't wait. Thanks for listening. Lucas: See you next time.