Latest / Embodied AI 101 / Pi0.5 + SLAM: What 97% Success Means for Clothing Manipulation
Transcript
- Pi0.5 + SLAM: What 97% Success Means for Clothing Manipulation. π0.5: a Vision-Language-Action Model with Open-World Generalization • Kevin Black et al. • Physical Intelligence • arXiv • 2025. Cold Open. Picture a sweatshirt on the bedroom floor. You ask a robot to put it away. The instruction is tiny. The job spans the entire machine. The robot has to find the garment, approach it, position the gripper, establish contact, lift, transport, and release. Imagine changing the sweatshirt to a crumpled pair of trousers. You want the same instruction to work, without teaching a new trajectory. Where should that adaptability live?. One answer is a policy that controls everything, from the wheels to the fingertips. Another is a system that combines a learned manipulation policy with mapping, visual checks, and reusable motions. That choice is the tension behind this clothing-cleanup demonstration. The interesting question is not simply whether a vision-language-action model can grasp fabric. It is how a model trained for broad robot control becomes a component in an autonomous household workflow—and what evidence would establish that the resulting system generalizes. The Idea. There are two artifacts here. Physical Intelligence’s π0.5 paper appeared on April 22, 2025. The linked November 6, 2026 post, retrieved through a text mirror, is Chris Paxton commenting on a later demonstration by Antoine, posting as @antoinemarcel. PI developed the underlying model; Antoine describes the application stack. That stack has an explicit division of labor. SLAM and a component named JEV locate the clothes. Astra, π0.5, and a collision detector support grasping. SLAM and motion replay handle putting the garment away. This is a hybrid mobile-manipulation system, with the learned policy occupying a particular part of the workflow. A useful technical detail comes from Antoine’s reply to a question about Astra: the system sends it a gripper image to check alignment before descending. That is a targeted visual judgment at a transition between behaviors. My reading is that this interface deserves attention: a compact check can decide whether the next motion should begin, while the manipulation controller handles the motion itself. PI’s original research system gives the policy broader control. It commands the arms, grippers, torso lift, and mobile base through simple tracking controllers, without an additional trajectory planner or collision detector. Navigation and manipulation are learned together. The learning recipe is heterogeneous co-training. π0.5 combines mobile-robot demonstrations, stationary robots working in other environments, cross-embodiment laboratory data, web vision-language tasks, and semantic subtask labels. Human verbal instructions provide additional supervision. At inference, the same model can select a subtask in language and then generate the actions that execute it. The training signal therefore covers both what to do next and how to move. For continuous control, the π0 family uses a smaller action expert conditioned on the vision-language backbone. Flow matching transforms a noisy candidate action sequence into an observation-conditioned motion sequence. The output is an action chunk rather than a single isolated motor command. This provides a representation suited to coordinated, continuous manipulation. The original π0.5 recipe first trains with compressed FAST action tokens, then introduces the continuous action expert during post-training. PI’s follow-up, Knowledge Insulating Vision-Language-Action Models, formalizes and extends that separation. The useful principle is straightforward: the action representation that makes learning efficient can differ from the representation used to execute precise movements. You can preserve a language-model-style training interface while generating continuous actions at deployment. This architecture has a clear systems lineage. OK-Robot, coauthored by Paxton, combined open-vocabulary perception, navigation, and grasping primitives on a Stretch robot for real-home pick-and-drop tasks. It treated integration as part of the research contribution. Here, a generalist VLA becomes a candidate manipulation component inside that same broad style of modular system. The design question is where to put the boundary. A garment location is useful for approaching the work area. A contact policy needs the current local observation. Those are different interfaces, and a builder can choose to learn one while engineering the other. The Result. The headline is 97% reported success on unseen garments, averaging two minutes per item end to end. Antoine describes this as full-pipeline success for autonomous pickup and putting away, rather than a grasp-only score. That choice of endpoint matters. For a household chore, I want to optimize the completed transaction: the garment starts somewhere inconvenient and finishes where it belongs. A successful pinch is an intermediate event. Transport and release belong in the objective because they are part of the work the user actually delegated. PI’s paper measures a separate experiment: household tasks in held-out real homes and physical mock-home setups. Tasks include laundry collection, dishes, and bed making. Long-task evaluation credits completed steps through a task-progress rubric. The controlled mock rooms support comparisons and ablations; these are physical robot experiments. In those comparisons, π0.5 outperforms π0 and a FAST-plus-flow variant using the same cross-embodiment action dataset and comparable training budgets. The additional ingredients include semantic subtask supervision and web-data co-training, connecting the improvement to the broader training recipe. The ablations provide the more useful engineering lesson. Data from other robot embodiments and environments improves performance across the evaluated conditions. Web data particularly helps with unfamiliar object categories and language grounding. The different supervision sources contribute complementary capabilities rather than behaving like interchangeable additions to a larger demonstration pile. For a dataset designer, that changes the collection question. Instead of asking only how many more trajectories to record, ask what new variation the next collection session introduces. A different room, another embodiment, or an intermediate semantic label can teach something that repeating an already familiar interaction does not. That is the investment hypothesis I would take from these ablations. The systems perspective also connects to OK-Robot’s evaluation. Its home experiments followed the chain from semantic object retrieval through navigation, grasping, and dropping. That framing makes a complete garment cycle a meaningful unit of progress: it evaluates the connections between capabilities as well as the capabilities themselves. Put the two results together and the story becomes concrete. The research develops transferable control through diverse supervision. The application organizes a household task around a learned manipulation component. For practitioners, that is an actionable connection between foundation-model training and robot-system design: learn adaptable interaction, then decide where it belongs in the execution sequence. The Biggest Weakness. The strongest objection is that the public clothing claim does not define its generalization test. The retrieved thread gives no trial count, held-out-home protocol, or rule for accounting for retries and interventions. “Unseen garments” is not defined relative to the system’s training or adaptation data. These are facets of one problem: we do not know what population the success percentage represents. Imagine two evaluations. In the first, someone introduces new clothing items into a familiar mapped room, using an established destination and a repeatable approach. In the second, the robot enters unfamiliar homes, encounters garments placed by residents, and receives no task-specific adaptation. Both evaluations can involve unseen garments. They test substantially different kinds of generalization. A high score in the first cannot, by itself, establish performance in the second. This distinction already has a vocabulary in mobile-manipulation research. HomeRobot explicitly separates new object instances from unseen object categories and defines the surrounding navigation-and-placement task. Those distinctions make a generalization result interpretable. The decisive next experiment would preserve that clarity. Freeze the policy and its adaptation data, specify what mapping and teaching are permitted, and evaluate garment novelty separately from home novelty. Count every assigned garment from the initial command to accepted placement, including recovery attempts and human assistance. Publish the complete run ledger so the denominator is visible. That experiment would also distinguish a useful personalized household routine from a system that transfers between households. Both are valuable targets; they need different claims. The objection is not that a hybrid stack somehow disqualifies autonomy. It is that an unspecified test distribution cannot establish the breadth of generalization claimed in the summary. The available evidence supports a builder-reported clothing-relocation result. Establishing a generalization rate requires the evaluation boundary around that result. Verdict. Mobile-manipulation teams should pay attention to this work because it offers a productive way to think about the role of foundation policies: as adaptable components with deliberately chosen responsibilities. For a deployment engineer, the central decision is the interface around the learned skill. What observation starts the skill? What condition ends it? What state does the next behavior receive? I would make that interface a first-class design artifact. It turns “run the model” into an operation the rest of the robot can reason about. There is a concrete starting point for that approach. Physical Intelligence’s openpi repository provides π0.5 checkpoints, fine-tuning examples, dataset-conversion guidance, and policy-serving tools, including remote inference. A team can use that tooling to develop its own manipulation component and connect it to an existing robot stack. The engineering opportunity is reuse at two levels. Reuse the pretrained model’s experience when adapting the contact behavior. Reuse the surrounding system’s spatial knowledge and task structure when composing the chore. I would pursue that combination because it gives each development effort a clear purpose, rather than making every change depend on retraining the entire workflow. For foundation-model researchers, the exciting question is where learned adaptability creates the most leverage. A generalist policy can be studied as a whole-robot controller, a manipulation skill, or a component selected by another controller. Those are useful experimental roles. Choosing between them should follow the task and the deployment objective. Paxton’s reaction focused on the practical value of even a robot that collects toys and puts them in a box. That is a sensible unit of usefulness: a recurring chore removed from someone’s attention. Takeaway: use the VLA for adaptable interaction, and make the surrounding autonomous workflow an explicit engineering contribution. Outro. Return to the sweatshirt on the floor. The user asks for an outcome. As designers, we get to decide which parts require learned flexibility, which parts benefit from persistent spatial state, and which motions are worth reusing. The elegance is in making those choices work together so the user can delegate the entire chore. Watch for pretrained manipulation policies becoming reusable building blocks inside increasingly capable household systems. The interesting frontier is the connection between a flexible skill and a completed task. That is the thread to follow—from a model that can move a robot to a robot that can finish the job.