Latest / Embodied AI 101 / One Shot, Twenty Screws: Robot Vision for the Dirty Reality of Appliance Recycling

Transcript

One Shot, Twenty Screws: Robot Vision for the Dirty Reality of Appliance Recycling · Embodied AI 101

• One Shot, Twenty Screws: Robot Vision for the Dirty Reality of Appliance Recycling. Industrial-Grade Robust Robot Vision for Screw Detection and Removal under Uneven Conditions • Tomoki Ishikura, Genichiro Matsuda, Takuya Kiyokawa, and Kensuke Harada • Panasonic Holdings Corporation, The University of Osaka, and AIST • Advanced Robotics / arXiv • 2026. The problem begins where conventional factory automation ends. Industrial robots are extremely good at repeating operations in controlled environments. Give them rigid fixtures, clean parts, stable illumination, known product geometry, and millimeter-accurate taught points, and they can run for years. End-of-life product disassembly removes almost every one of those assumptions. A used air-conditioner outdoor unit may have spent more than a decade in rain, dust, salt, insects, and repeated thermal cycles. Its panels may be bent. Its screws may be rusted, painted over, packed with mud, or hidden by spider webs. Two units sharing the same nominal design may arrive at the recycling plant with meaningfully different geometry. The robot therefore cannot simply replay a trajectory derived from CAD or from a pristine reference product. This is the setting addressed by Tomoki Ishikura and colleagues. Their paper presents an integrated system for removing screws and exterior panels from used air-conditioner outdoor units. Its two central technical components are a task-specific, two-stage vision pipeline for finding degraded screws and a dense, lattice-based calibration procedure for mapping wide-field RGB-D observations into robot coordinates with submillimeter accuracy. The authors validate the resulting cell on 120 units obtained from a real recycling plant. The paper’s headline results are a 99.8% screw-detection recall in a preliminary evaluation, a specified manipulation accuracy of plus or minus 0.75 millimeters, a 78.3% disassembly completion rate on workpieces deemed eligible for the process, and a reported average cycle time of 193 seconds. Those figures matter, but the more interesting contribution is how the authors connect them. This is not primarily a new detector architecture. It is a study in allocating uncertainty across vision, calibration, tooling, mechanics, and plant operations. That systems perspective is what makes the work relevant beyond screw removal. Deriving robot requirements from the recycling line. One unusually strong aspect of the paper is that its performance targets do not come from a generic robotics benchmark. They are derived from the throughput and staffing requirements of the Panasonic Eco Technology Center. The plant reportedly processes about 142,000 air conditioners per year. Assuming 240 operating days, this corresponds to roughly 600 units per day. The authors envision five parallel disassembly cells, each handling 120 units during an eight-hour shift. After accounting for 90% equipment utilization, they derive a maximum acceptable cycle time of 216 seconds per unit. They also seek to reduce labor for this portion of the process from two operators to half an operator, meaning one person would supervise several automated cells. Under that staffing model, the system must complete more than 75% of eligible units without intervention. This immediately changes how detector performance should be interpreted. A typical object-detection paper might regard 95% recall as respectable. Here, a housing can contain approximately 20 screws, and missing only one may prevent the panel from being removed. If each screw is detected independently with 95% probability, the chance of detecting all 20 is only about 36%. To achieve a 90% probability of finding every screw on such a unit, the per-screw detection probability must exceed roughly 99.5%. At exactly 99.5%, the 20-screw batch probability is about 90.5%. At the reported 99.8% recall, the corresponding probability would be approximately 96.1%, assuming independent errors. The independence assumption is optimistic. Dirt, poor illumination, unusual paint, or a previously unseen model can cause correlated failures across several screws. Nevertheless, the calculation captures an important lesson: in long-horizon manipulation, apparently small component-level error rates compound rapidly. The authors similarly decompose the positioning requirement. From the wide-field image, the vision system must estimate the center of the cross recess to within two pixels, corresponding to about 0.4 millimeters. Camera-to-robot mapping and mechanical positioning are allocated another 0.35 millimeters. Together, these define the overall plus-or-minus 0.75-millimeter operating tolerance. The screw occupies only about 34 by 34 pixels in the full-resolution view, so the system is attempting subpixel-to-few-pixel metrology over a workspace measuring 900 by 400 by 750 millimeters. This is the paper’s core engineering trade: eliminate slow close-up sensing, but accept a much harder wide-field calibration and small-object localization problem. The cell: replacing camera motion with calibration. The physical system contains two Yaskawa MOTOMAN-GP25 six-axis arms, a magnetic mounting table, a model-number reading device, and a fixed, high-resolution RGB-D sensing assembly. The color channel uses a 4104-by-3006-pixel CMOS camera with a 16-millimeter lens. Depth comes from an Ensenso X36 stereo camera with 2448-by-2048 resolution and a specified depth accuracy of 0.281 millimeters at the system’s approximate 1.5-meter working distance. The complete installation occupies roughly five by 2.5 by 2.5 meters. Earlier versions of the approach used a wide camera for coarse localization followed by an end-effector-mounted camera for close inspection. That is a natural robotic solution: move close to each candidate, acquire a high-resolution observation, refine the pose, and then engage the screw. But it also adds a camera motion and sensing cycle for every fastener. The authors report that their earlier system required approximately 15 seconds per screw, pushing a 20-screw unit beyond 300 seconds. The new system instead takes one wide-view RGB-D acquisition per side and localizes all visible screws from it. Once the screw coordinates have been estimated, the robot visits them continuously. Depending on the side and screw count, the system can remove between four and 20 screws in 15 to 40 seconds, which the authors describe as a four- to 7.5-fold throughput improvement over their prior configuration. This is a classic industrial-automation exchange. Active perception makes each local measurement easier but consumes cycle time. Fixed sensing reduces motion overhead but requires better optics, denser calibration, stronger error modeling, and more forgiving tooling. It is also worth clarifying what “without pre-programmed coordinates” means here. The precise screw and cover positions are detected online, so the system is not replaying fixed Cartesian points for each product model. But the overall process remains structured. It follows a predefined workflow covering front, left, right, top, inner-front, and cover-removal operations, and it assumes that incoming units share a broadly consistent panel configuration and fit inside a specified spatial envelope. This is adaptive geometric execution inside a manually designed task template, not unrestricted autonomous disassembly. Two-stage screw detection: recall first, precision second. The vision pipeline is deliberately asymmetric. The first stage is allowed to be generous because its job is to avoid missed screws. The second stage is conservative because its job is to reject the rust spots, holes, shadows, edges, and surface defects that resemble screws. Both stages use fully convolutional networks that produce pixelwise probability maps for screw and non-screw regions. The training set contains approximately 39,000 screw images and 43,000 non-screw images spanning multiple screw appearances, backgrounds, and illumination conditions. The networks are trained with cross-entropy loss using Adam, a batch size of 128, and 30 epochs. The initial learning rate is 0.01 and is reduced by a factor of four every ten epochs. In the first stage, the high-recall model operates on the full RGB image and produces a coarse probability map. Thresholding and an area filter remove small responses, after which the system extracts image patches around the remaining connected regions. The objective is not yet to declare that every patch contains a screw. It is to construct a candidate set with as few false negatives as possible. Each candidate then enters the precision-oriented second stage. The patch first undergoes gamma correction to reduce illumination variation. Three independently trained high-precision models process the corrected patch, and their probability maps are averaged. A second threshold converts the ensemble output into a fine mask used to accept or reject the candidate. This cascade is more appropriate for the task than forcing a single detector to operate at one confidence threshold. In ordinary image retrieval, a false positive may be a minor inconvenience. In robotic disassembly, it commands physical motion toward an incorrect point. That costs cycle time and can create a collision or tool-engagement risk. A false negative is equally costly because one retained screw can block panel removal. The two stages allow these two error classes to be treated separately. The final screw coordinate does not come directly from the neural network. Once a candidate has been verified, the patch is converted to grayscale and histogram-equalized. The system then extracts two geometric cues: the outer circular boundary of the screw head and the edges and intensity structure of the cross-shaped recess. These estimates are fused to produce the local engagement center, mapped back to the full image, and paired with the corresponding depth value to produce a 3D point. This separation between recognition and metrology is an important design choice. The networks answer the semantic question—“Is this actually a screw?”—while conventional image processing answers the geometric question—“Where should the driver bit enter?” A bounding-box center would be too sensitive to loose annotations, asymmetric corrosion, partial occlusion, and variation in the visible screw head. The explicit cross-recess and circular-head geometry is a useful task prior. The paper is less complete about the learning component than many ML readers would prefer. It identifies the models only as FCNs and does not provide a detailed layer specification, input-patch size, augmentation policy, train-validation-test split, threshold values, gamma-selection procedure, or explanation of how the three precision models were diversified. It also does not present an ablation comparing one-stage detection, two-stage detection without an ensemble, or learned versus geometric center estimation. The result is a convincing system recipe but an incomplete account of which detection choices generate the reported gains. Calibration as a spatial error field. Even perfect 2D screw localization would not solve the manipulation problem. At a 1.5-meter camera distance and over a large robot workspace, small residual errors from lens distortion, stereo depth, camera mounting, robot compliance, and kinematics can add up to more than the driver can tolerate. Standard hand–eye calibration estimates one rigid transform between camera and robot coordinates. That is appropriate when the residual error is largely global and rigid. The authors argue that their dominant errors are spatially varying and nonlinear. Lens residuals change across the image. Robot-link deflection depends on pose and payload. Motion near less favorable kinematic configurations can amplify errors. One homogeneous transform cannot model these effects. Their alternative is essentially an empirically measured 3D correction field. A calibration jig is attached to the arm. It carries a circular marker, and its mass and horizontal offset are designed to approximate the real driver hand so that the calibration samples include similar structural deflection. Across the 900-by-400-by-750-millimeter volume, the system defines a three-dimensional lattice with 50-millimeter spacing. This produces 2,736 calibration points. The robot automatically visits each point, the RGB-D camera observes the marker, and the system stores a correspondence between the measured camera coordinate and the commanded robot coordinate. Data collection takes about 4.5 hours per robot and is intended as a one-time setup operation. At runtime, a global linear mapping first produces an approximate robot position for the detected screw. That rough result identifies the relevant local region of the lattice. The system selects a three-by-three-by-three neighborhood—27 calibration samples—and performs second-order local interpolation to obtain the refined robot coordinate. Conceptually, this is closer to calibrating the full embodied sensing-and-actuation chain than merely calibrating a camera. The measured field implicitly absorbs imperfections that would be awkward to model separately: camera distortion, depth bias, mounting offsets, arm sag, and repeatable kinematic error. The authors verify the mapping using graph paper and a needle-shaped tool tip. A target is selected in the RGB-D image, the arm moves to the corresponding interpolated coordinate, and the needle marks its reached position. Where necessary, additional correction values are inserted using nearest-neighbor or interpolated values. After this verification-and-correction process, the system reportedly satisfies the required plus-or-minus 0.35-millimeter mechanical positioning accuracy. That last manual or semi-manual correction step deserves attention. The final accuracy is not shown as the output of the lattice interpolation alone. It follows interpolation, physical verification, and additional correction. The paper does not report the baseline error from a conventional rigid transform, the error before these extra corrections, or the final mean, percentile, and worst-case distributions. It therefore demonstrates that the calibration procedure can be made to meet the specification, but it does not isolate how much each stage contributes. The manipulation hardware provides a final tolerance layer. Springs allow passive self-alignment, a conical guide funnels the screw head toward the bit, and a pneumatic impact driver helps with resistant fasteners. The robot does not rely on vision and calibration to eliminate all error; it deliberately leaves the final fraction of a millimeter to mechanical compliance. That is arguably the most industrially mature idea in the paper. Robustness comes from arranging several imperfect subsystems so that their failure modes do not coincide. What the experiments actually establish. The evaluation has two levels: a controlled screw-detection test and an integrated disassembly trial. For the preliminary perception test, the authors build a Test Evaluation Gauge with dimensions comparable to an outdoor air-conditioning unit. It permits controlled in-plane rotations of plus or minus four degrees and tilts of plus or minus six degrees. Across 3,070 screw instances, the system records 3,064 true positives, six false negatives, and no false positives, yielding 99.8% recall and 100% precision. The recall result clears the derived 99.5% per-screw target. Under an independence approximation, 99.8% per-screw recall implies that all 20 screws would be detected on roughly 96% of units. The precision result should be interpreted more cautiously. The authors state that true negatives are not applicable because the evaluation set consists exclusively of positive samples. Although false detections could still arise from the background inside those images, this is not a broad negative-scene study containing arbitrary holes, labels, grille features, reflections, corrosion spots, and other plant clutter. It is principally evidence for high recall over a controlled range of target poses, not a complete characterization of false-alarm behavior in open-world recycling imagery. The integrated evaluation uses 120 outdoor units collected from a recycling plant. Refrigerant recovery and power-wire disconnection have already been performed, so the experiment covers housing disassembly rather than the complete end-to-end treatment of an untouched appliance. Cycle timing begins at the loading switch and ends when the unit is discharged. The units are divided into three categories:. OK: successfully disassembled;. NG: processed but not completely disassembled;. Excluded: judged outside the system’s workable conditions. There are 72 OK units, 20 NG units, and 28 exclusions. Removing the exclusions from the denominator gives 72 successful units out of 92 attempted eligible units, or 78.3%. This exceeds the paper’s 75% target. The conditional denominator is operationally defensible if a reliable upstream inspection system can identify unsuitable units and route them elsewhere. But it is important not to confuse conditional completion with total intake yield. Relative to all 120 arriving units, the robot fully disassembled 60%. The excluded group represents 23.3% of the total sample. Those excluded units include screws hidden by mud or spider webs, severely seized fasteners, deformed panels whose surface normals fall outside the operating range, and cracked or otherwise unstable housings. These are not minor statistical outliers. They represent a major operating regime for end-of-life products and reveal the boundary between improved perception and physically unobservable or mechanically infeasible tasks. The reported overall cycle time is 193 seconds, below the 216-second requirement. The paper separately reports averages of 197 seconds for successful units and 191 seconds for NG units. Its approximate process breakdown is 45 seconds for material handling, 80 seconds for screw removal, 45 seconds for cover removal, and 30 seconds for inspection. Screw removal therefore consumes a little over 40% of the total cycle. There is a small reporting ambiguity here. The rounded OK and NG subgroup averages do not reproduce an overall mean of 193 seconds when weighted by their reported counts, and the stage-level breakdown sums to roughly 200 seconds. The discrepancy may reflect rounding, the inclusion of excluded units in one aggregation, or a different averaging procedure, but the paper does not specify which. Even with that ambiguity, the throughput advance is substantial. Prior close-up approaches discussed by the authors take approximately 30 to 55 seconds per screw, while the new system spends around 80 seconds on the entire screw-removal phase of a unit. More importantly, the experiment is not a few repeated removals from one clean laboratory panel. It covers model variation and naturally degraded products from a real recycling stream. How this differs from earlier screw-removal systems. Screw detection itself is not new. Earlier dismantling systems often used hierarchical perception: detect the large structure, move an eye-in-hand camera close to likely fastener locations, and then apply template matching or a classifier. A 2011 building-dismantling study, for example, combined large-structure pose estimation, stereo imaging, template matching, an SVM, and temporal integration, obtaining roughly two-millimeter screw-position accuracy. More recent WEEE approaches have used deep detectors. One 2021 system for LCD-monitor disassembly combined gamma correction, a learned screw detector, reasoning about expected screw arrangements, and retraining on missed examples. A 2025 product-specific framework used YOLOv8 and very small per-product datasets, exploiting known fastener locations and screw types rather than trying to build one broad detector for arbitrary products. The Ishikura paper describes laptop-oriented work reaching around 95% detection in relatively clean conditions. It also contrasts its system with Clark and Jouaneh’s robotic fastener extractor, which reportedly reaches plus-or-minus one-millimeter localization and a 100% extraction rate within two attempts, but requires 30 to 55 seconds per screw because of overview imaging, close-up imaging, and sequential engagement checks. The present paper’s distinction is therefore not a radically new neural representation. It is the operating point:. one wide-field acquisition per side rather than active close-ups for every screw;. recall-oriented proposal generation followed by precision-oriented verification;. explicit cross-recess metrology after semantic recognition;. dense workspace calibration rather than only a global hand–eye transform;. passive mechanical alignment and impact tooling;. and evaluation against plant-derived cycle-time and staffing requirements. In other words, the contribution is a production architecture for uncertainty. The strongest contributions—and the missing evidence. The first major contribution is the translation from plant economics to module-level specifications. The authors begin with annual volume, cell count, equipment utilization, and labor reduction, then derive cycle time, unit-level completion, per-screw recall, and positioning tolerances. That chain is far more informative for industrial adoption than reporting mean average precision in isolation. The second contribution is the hybrid perception pipeline. The work uses learning where appearance variability dominates and geometry where the task supplies a reliable prior. Rust can make “screw versus background” difficult, but once a screw has been confirmed, its circular head and cross recess remain useful cues for driver placement. This semantic-versus-metric decomposition should transfer to other insertion, fastening, and tool-engagement tasks. The third is the decision to calibrate spatially varying embodied error. Large fixed cameras and articulated robots commonly exhibit repeatable residual fields after nominal calibration. Rather than continuously increasing model complexity, collecting a dense set of physical correspondences and interpolating locally can be an effective industrial solution—especially when installation time is less important than stable production accuracy. The fourth is the scale and realism of the integrated experiment. A trial on 120 naturally degraded products is meaningful by robotics standards, and the explicit accounting of excluded cases is valuable. The result supports the claim that the approach is feasible as a partial-automation cell. It does not yet prove unattended production readiness. The paper does not provide shift-long or month-long operation, calibration-drift measurements, mean time between failures, tool-wear analysis, safety-system behavior, maintenance burden, or economic payback. A one-time 4.5-hour calibration is reasonable only if the resulting error field remains stable after camera vibration, end-effector service, temperature changes, and robot collisions. The evaluation also lacks a detailed failure decomposition for the 20 NG units. We are not told how many failures came from missed screws, incorrect 3D localization, bit-engagement errors, seized screws, incomplete extraction, panel manipulation, or process-planning assumptions. Without that breakdown, it is difficult to determine whether the next ten percentage points should come from better vision, compliance, tooling, or exception handling. Generalization is another open question. The training corpus is large, but the paper does not describe held-out product families or whether train and test images were separated by appliance model, physical unit, recycling batch, or acquisition date. Randomly separating patches from the same product population would be much easier than deploying to an unseen manufacturer or surface finish. The geometric center estimator also appears tailored to cross-recessed screw heads; extensions to hex, Torx, slotted, damaged, or missing-head fasteners would require additional models and tooling logic. Finally, reproducibility is limited. The arXiv record does not expose a paper-specific public code or dataset link, and important architecture and threshold details are omitted. None of these issues invalidate the system results. They define what the next evaluation should measure. What roboticists should take from this work. The first lesson is to optimize for task-level reliability, not detector-level averages. If a manipulation episode contains many required detections, compute the batch success probability before selecting a perception architecture. A 99% module can still be inadequate in a 20-step process. The second is to separate candidate generation, semantic verification, and geometric localization. These stages have different losses and asymmetric costs. A recall-heavy full-image model, a precision-heavy patch classifier, and a task-specific metrology routine can be more controllable than one end-to-end network. The third is to treat calibration as part of embodiment. If the robot repeatedly exhibits location-dependent error, model the residual field directly under the real payload and tool configuration. A mathematically elegant rigid transform is not necessarily the most accurate description of the installed machine. The fourth is to spend mechanical design on the final uncertainty. Springs, funnels, guards, and compliant guides can be more dependable than demanding another fraction of a millimeter from perception. The system succeeds because software accuracy and physical tolerance are co-designed. The fifth is to make exception routing a first-class subsystem. The 28 excluded units are not merely failures to remove from a benchmark. They are a separate product population requiring detection, classification, and routing. A practical plant architecture might send ordinary units to reversible robotic disassembly, visually ambiguous units to human-assisted operation, and severely damaged units to controlled destructive processing. That is also where the authors point next. They propose robot-friendly product design, automated generation and scheduling of disassembly plans, human-in-the-loop correction that turns interventions into new training data, and drilling or milling for units that cannot support precise visual screw removal. They emphasize the trade-off between destructive processing for material recovery and reversible disassembly for component reuse. This agenda is consistent with related work from the same research group on adaptive vacuum fixturing and on identifying fasteners whose removal at the product-design stage would simplify later robotic disassembly. The broader project is not merely to build a better screw detector. It is to redesign the relationship among products, recycling processes, and robots. Bottom line. “Industrial-grade” is sometimes used as a synonym for high accuracy. This paper gives the term a more useful meaning. Industrial performance emerges from requirements traced to plant throughput, sensing designed around cycle time, calibration measured over the real workspace, tooling that absorbs residual error, and an explicit policy for products the robot cannot handle. The system is specialized, its evaluation denominator requires careful reading, and several ML and calibration details remain underreported. Yet it demonstrates something that many more sophisticated robot-learning papers do not: a coherent path from perception output to physical operation on a heterogeneous stream of real end-of-life products. Its most important message is that robust robot vision is not located entirely inside the vision model. It is distributed across the detector, the camera, the calibration field, the robot structure, the screwdriver, the fixture, the workflow, and the exception policy. For industrial disassembly, that distribution of intelligence may matter more than the choice of neural backbone.