Latest / Embodied AI 101 / SLA: Can Spatial Pretraining Improve VLA Robot Manipulation?
Transcript
- SLA: Can Spatial Pretraining Improve VLA Robot Manipulation?. SLA: Scaling Spatial-Language-Action Model via Continual Pre-training • Fuhao Li, Wenxuan Song et al. • HKUST (Guangzhou), OLA Dimensions, BAAI, Tsinghua University • arXiv • 2026. Cold Open. Imagine a robot reaching for fruit. A person slides it sideways just before the gripper closes. The instruction has not changed. Neither has the object’s identity. What changed is the spatial relationship that determines whether the action succeeds. Now replace the fruit with a watch, and raise the table underneath it. Recognizing “watch” gets you only partway there. The policy has to update its reach while the world moves. Both kinds of disturbances appear in SLA’s physical experiments. They turn an abstract argument about representations into a practical question: what has the policy learned to pay attention to?. Should spatial competence emerge from predicting enough robot actions? Or should we explicitly teach the visual representation about geometry before adapting the robot to a particular job?. That is the tension behind this episode of Embodied AI 101: whether a geometric teacher can change what a robot policy learns from experience—and then leave before deployment. The Idea. SLA starts from pretrained π0.5, pronounced pi-zero-point-five. Its implementation retains the familiar PaliGemma vision-language backbone and flow-matching action expert. Images, instructions, and robot state condition predictions of continuous action chunks. The intervention is concentrated in how the model learns its visual representations. The direct ancestor is the authors’ earlier Spatial Forcing. That work aligned intermediate VLA features with representations from the geometric foundation model VGGT during downstream training, including implementations for OpenVLA and π0. Spatial Forcing established the basic recipe: supervise the policy’s internal visual features with a model trained to understand scene geometry. SLA moves that recipe into continual pretraining across heterogeneous robot experience, then retains spatial supervision during task adaptation. This is the central change in emphasis: geometry becomes something the reusable policy learns before it encounters a specific downstream benchmark. The geometric teacher is OmniVGGT. Its distinctive capability is accepting different combinations of RGB images, depth, and camera parameters. Built on VGGT, it combines information within images and across views. Its adapters consume available geometric measurements, and its training exposes it to incomplete combinations of those measurements. Consequently, it can operate with RGB alone or exploit additional geometric information when supplied. That makes it a useful teacher for a mixed robot-data collection. The alignment itself is straightforward. SLA extracts visual-token representations from the policy, projects them into a compatible feature space, and compares them with teacher features at corresponding image locations. The comparison rewards agreement in feature direction. Ordinary action prediction remains part of the same training objective. The teacher supplies a target representation; robot demonstrations still supply the desired actions. Think of those as complementary requirements. “Produce the demonstrated action” constrains behavior. “Preserve features useful for geometry” constrains what information the policy carries into that decision. The design aims to make spatial structure available to the controller across many different examples. SLA also learns how to combine representations from different transformer layers. Feature-dependent gates weight the projected features before aggregation. Instead of committing to one alignment layer for every scene, the training branch can draw differently on multiple depths according to the observed content. These gates serve the auxiliary alignment objective. Another ingredient is probabilistic state dropout: training sometimes omits the proprioceptive state tokens while keeping the visual observations and instruction. This connects to research such as Do You Need Proprioceptive States in Visuomotor Policies?, which found that robot-state inputs can encourage trajectory shortcuts. SLA uses intermittent omission to encourage visual dependence while preserving normal state-conditioned operation at deployment. For scale, the authors curate from a source pool totaling approximately 1.7 million episodes and 14,330 hours before selection and processing. The mixture includes real demonstrations and simulated experience, with sources such as DROID, AgiBotWorld, and RoboCasa365. A shared processing pipeline reconciles state and action conventions, including end-effector-relative action targets. The resulting design separates training resources from deployment inputs. A rich geometric teacher can shape the policy during learning without becoming a required depth-estimation stage in the action-generation path. The Result. The public headline is 33.9% task success on RoboDojo-Real. The official leaderboard’s November 3, 2026 update lists the submission as SLA-Preview, first overall, ahead of InternW0-Delta and substantially above the listed π0.5 baseline. That ranking is an actual benchmark entry, not just a comparison assembled for the project page. RoboDojo-Real evaluates manipulation across ARX X5, Piper, and Piper X. Its tasks include contact-rich interactions, precise placement, multi-object operations, and extended sequences. The benchmark standardizes physical configurations and initial layouts, and reports both partial-progress scores and complete-task success. The headline here is the completion metric. That external benchmark matters to me because it supplies a shared physical test environment. It gives the result a different evidential role from an author-selected demonstration: a policy has to work within another evaluation framework’s task definitions and scoring procedure. The number that matters most for the paper’s central training argument comes from RoboTwin Clean2Random. RoboTwin is a simulated bimanual manipulation benchmark with structured variation in lighting, clutter, backgrounds, table height, and language. Clean2Random trains on clean demonstrations and evaluates under randomized conditions. It asks whether a learned behavior survives changes around the task. At the full selected pretraining mixture, enabling spatial alignment during both training stages adds 24.1 percentage points of success over disabling it in both. The experiment toggles alignment independently during continual pretraining and downstream adaptation, across several data scales. Using it in both stages wins throughout. This is the comparison I would put at the center of the paper. It asks what the spatial objective adds to additional robot experience. A practitioner considering the method needs precisely that distinction: whether to collect more of the same supervision, or change the supervision applied to the data already available. The project also reports results on LIBERO-Plus perturbation robustness and RoboCasa-Panda kitchen manipulation. Its physical demonstrations test recovery after object displacement, moving grasp targets, and changing table height. Together, these evaluations connect the representation-learning proposal to several recognizable control problems. My take from the experimental design is that the scaling ablation carries the scientific story. The leaderboard establishes the performance of the submitted system. The controlled alignment switches explain why this particular training direction deserves attention. The Biggest Weakness. The headline real-world comparison does not isolate spatial alignment from added visual memory. For RoboDojo-Real, SLA adds historical main-camera images during post-training and inference. The context includes the episode’s initial frame and a sequence of recent frames. Those images pass through the policy’s existing visual backbone alongside current observations. They remain available when the deployed robot chooses actions. The reported real-world comparison does not include a matched π0.5 baseline equipped with the same memory mechanism. That changes the causal question. A policy with history has information that a current-observation policy does not. RoboDojo deliberately includes requirements involving partial observability, memory, and progress through extended tasks. Remembering an earlier scene can help identify what has already happened and what should happen next. That benefit is distinct from representing the geometry of the current scene more accurately. Consider a hypothetical placement task where an object disappears behind a container. Better current-frame geometry and remembering the previous view are two different ways to help the controller. If the improved system receives both advantages, its higher success rate cannot tell us how much came from either one. The distinction also sharpens the deployment claim. Removing the geometric teacher eliminates that auxiliary computation. It does not remove the historical images used by the leaderboard configuration. “The spatial branch is training-only” and “the complete evaluated system has unchanged inputs” are different statements. The resolving experiment is straightforward: hold the robot data and adaptation budget fixed, give both policies identical history, and toggle spatial alignment. A crossed comparison with memory enabled and disabled would make the separation even clearer. The current evidence supports a strong combined system. The unresolved issue is how much of its real-world advantage belongs specifically to the paper’s spatial-pretraining thesis. Verdict. Foundation-model teams should pay attention to SLA because it proposes an attractive division of labor: use a geometry specialist to teach the policy, then let the policy perform control with the learned representation. For a team already building on a pretrained VLA, that is a concrete direction for improving training rather than redesigning the robot’s entire perception stack. The important intellectual move is placing geometric supervision earlier. Spatial Forcing treated alignment as a way to improve downstream learning. SLA treats it as part of constructing the transferable starting point. I see that as the paper’s most useful contribution: a proposal about what a robot foundation model should learn before specialization. For representation-learning researchers, the attractive question is what makes a useful control feature. Semantic recognition tells you which object the instruction refers to. Geometric supervision asks the representation to preserve information about the scene’s physical structure. I would build on this work by studying how those two demands cooperate inside the policy, especially as the diversity of robot experience increases. For practitioners, there is a tangible entry point. The official repository provides SLA training and serving instructions, benchmark configurations, and download references for base and post-trained checkpoints. Its documented workflow includes action normalization and downstream adaptation, so teams can investigate the method through an existing policy-development process. The sensible experiment is to start from the supplied foundation checkpoint and adapt it to a manipulation problem where visual feedback clearly determines the next action. Object relocation and changes in working height are especially informative questions to ask. They connect the proposed representation improvement to something a roboticist can observe directly. My verdict: SLA makes spatially supervised continual pretraining worth pursuing. Its strongest contribution is a reusable training recipe for making robot experience more informative. Outro. Return to the moving fruit. The useful question is whether the robot has learned a representation that helps it change the reach when the scene changes. SLA puts that question inside foundation-model training, where the answer could transfer across many downstream jobs. What I would watch next is how this recipe behaves as camera arrangements, embodiments, and task families become more diverse—not simply as the episode count increases. The lasting idea is that scaling robot learning involves choosing what each demonstration teaches. An action label teaches what happened. A geometric teacher can help specify what the policy should preserve while learning to act.