Latest / Embodied AI 101 / VAMOS: Can One Navigation VLA Work for Both Wheels and Legs?
Transcript
- VAMOS: Can One Navigation VLA Work for Both Wheels and Legs?. VAMOS: A Hierarchical Vision-Language-Action Model for Capability-Modulated and Steerable Navigation • Mateo Guaman Castro et al. • University of Washington and DEVCOM Army Research Laboratory • IEEE RA-L, 2026; technical analysis based on the public arXiv manuscript of October 23, 2025, and released artifacts. Cold Open. Imagine two robots looking at a destination above a staircase. One has legs; the other has wheels. Their cameras reveal the same useful clues: a landing, the stairs, and a nearby ramp. They should share an understanding of the scene. They should not necessarily share a route. That is the problem VAMOS tackles with Boston Dynamics Spot and the UW Hound rover. A shared planner proposes routes; a robot-specific model evaluates which routes fit the machine’s capabilities. Now think about the training data. A successful demonstration of climbing stairs teaches something useful about where those stairs lead. Whether another robot should imitate that traversal is a different question. Pooling experience therefore creates two learning problems: understanding possible routes and determining which ones a particular body can execute. Where should that distinction live? Inside the parameters of a giant navigation model? Or at an explicit interface between planning and control?. VAMOS chooses the interface. The Idea. The high-level planner is a three-billion-parameter PaliGemma 2 model adapted with LoRA. It consumes an RGB image and a goal encoded as text, then generates five image-space waypoints. Those coordinates emerge as location tokens. The output describes a route through the picture, rather than wheel velocities or joint commands. The released dataset makes that training contract concrete. Examples contain images, camera parameters, corresponding image-space and three-dimensional trajectories, and goal descriptions. A wheeled demonstration and a legged demonstration can therefore use the same prediction format. This separates the representation of a route from the commands required to execute it. Training combines SCAND, TartanDrive 2, CODa, and a small collection of Spot demonstrations, drawing on roughly thirty hours of recordings. Data processing mixes trajectory horizons and favors informative curved paths while filtering extreme outliers. The engineering intuition is straightforward: teach the model route shape and turns, rather than allowing abundant straight-ahead motion to dominate supervision. Language steerability comes from additional supervision. GPT-5-mini annotates augmented navigation examples, while COCO-QA and Localized Narratives supply auxiliary language-grounding data. Preferences about which side to pass on or which surface to follow become part of the planner’s input. Language changes the proposed route itself. The embodiment-specific component answers a different question. A small multilayer perceptron predicts traversal success from elevation, position, and heading. It learns success-or-failure labels from short simulated rollouts over procedural terrain in Isaac Lab. For Spot, an RL locomotion policy provides the simulated experience. Think of this as learning the operational meaning of terrain for a controller. Recognizing a staircase says what the structure is. Predicting whether this robot can traverse it says what the structure permits. That second judgment belongs to the robot-and-controller combination, rather than to scene semantics alone. At deployment, sampled image-space routes are converted into physical coordinates and evaluated against a traversability map. Selection follows a worst-segment principle: a favorable stretch cannot compensate for an impassable section. The implementation exposes both candidate routes and their scores, making the selection process inspectable. The surrounding navigation system handles world-frame goals, camera calibration, depth, coordinate transforms, and execution through a downstream controller. A state machine manages planning, waypoint following, and turning to bring a goal into view. This is a learned planner and capability evaluator integrated into a metric navigation stack. That division gives the architecture its character. The VLM supplies alternatives informed by visual experience and instructions. The affordance model changes which alternative gets executed. Embodiment influences the high-level decision through selection, while the existing controller remains responsible for movement. The useful abstraction is therefore a shared proposal format coupled to a specialized feasibility judgment. The Result. The headline is 90% average success, compared with 67% for the strongest baseline, ViPlanner. The benchmark comprises six physical navigation courses, covering indoor and outdoor settings. Each course receives five trials; completion is the success metric, with interventions during successful runs recorded separately. The environments include hallways, an atrium, a laboratory, a campus route, forest terrain, and a descending ramp. This gives the comparison a useful spread: constrained indoor geometry alongside outdoor terrain and vegetation. The project reports results against a conventional modular stack, ViPlanner, NoMaD, and NaVILA. ViPlanner is an important comparator because it already combines semantics with geometry. It learns local paths from depth and semantic information, using simulation and a differentiable semantic costmap. Beating that system is a more informative result than beating a planner that treats every elevation change as an obstacle. The relevant competition already understands that terrain categories affect traversability. NaVILA supplies another useful reference point. It already separates a high-level VLA from locomotion, communicating spatial movement instructions to a visual RL policy. Hierarchy is therefore established territory. VAMOS’s distinctive architectural choice is the combination of pixel-path proposals and explicit, embodiment-dependent candidate selection. The ablations connect that architectural choice to behavior. Enabling affordance modulation improves single-robot obstacle rejection. Separately, pooled-data training reduces offline path-prediction error relative to dataset-specific specialists. These experiments address complementary parts of the design. The offline comparison asks whether heterogeneous demonstrations improve the proposer. The modulation comparison asks whether feasibility evaluation improves the decisions made from those proposals. Together, they provide a reason to invest in both components, rather than treating the specialist as incidental deployment machinery. The two-platform demonstration makes the intended behavior easy to understand: changing the capability model directs Hound toward the ramp, while Spot can exploit stairs. The shared representation supports different execution choices for different machines. The release also distinguishes navigation-only and steerable checkpoints, identifying the navigation-only model as the one used for the navigation experiments. Language steering is evaluated through path visualizations and a VLM-judged path-choice test. Those are separate experimental outputs: physical navigation performance and instruction-conditioned route prediction. The Biggest Weakness. The cross-embodiment experiment uses a scene-adapted planner, so it does not establish transfer of an unchanged generalist to a new robot in a new place. The public manuscript discloses collecting fifty static images from each robot in the stair-and-ramp test environment, labeling both routes, augmenting those sketches, and fine-tuning the shared planner. That setup supports a narrower conclusion than the broad transfer story: once the proposer has learned to offer the relevant alternatives at this site, an embodiment-specific evaluator can improve route selection. Here is the experimental logic. A proposal-and-ranking system needs two things to work. First, its candidate set must contain a useful, feasible route. Second, the evaluator must recognize and select that route. The scene-specific fine-tuning directly strengthens the first condition. The modulation comparison then tests the second. A ranker cannot select an alternative that was never proposed. This follows directly from VAMOS’s candidate-selection architecture. Consequently, proposal coverage is part of the cross-embodiment problem, not merely a convenience preceding it. The same adapted planner really is shared across the robots, and holding that proposer fixed preserves the value of the affordance comparison. The unresolved question is what transfers before adaptation. The planner could be learning broadly reusable route reasoning, local route templates, or some combination. This experiment does not distinguish those explanations. That matters because the architecture’s attractive scaling story is to share expensive semantic learning while specializing physical feasibility cheaply. If a new deployment also needs target-viewpoint images and route annotations, the adaptation burden extends beyond the capability module. The decisive experiment would freeze the proposer before introducing new robots and sites, then change only the capability model and controller integration. Measuring both feasible-candidate coverage and closed-loop completion would reveal whether the shared planner supplies useful alternatives as reliably as the specialist chooses among them. Verdict. VAMOS should interest teams that already have competent locomotion controllers and want to exploit broader navigation data. It offers a concrete way to divide the learning problem: learn route proposals from heterogeneous demonstrations, and learn feasibility from the behavior of a particular platform. Its intellectual lineage helps identify the contribution. SayCan combined a language model’s assessment of useful actions with learned affordances for executable skills. That work established the value of grounding high-level suggestions in what a robot can actually do. HAMSTER made image-space paths an interface between a high-level VLM and a three-dimensional manipulation policy. It showed how a coarse visual plan can carry task intent while a separate controller handles execution. VAMOS brings those ideas together for navigation: visual route proposals provide a common language, and capability estimates participate explicitly in choosing the route. Its strongest engineering lesson is the placement of that decision boundary. Sharing knowledge and specializing execution become distinct operations. For practitioners, this also suggests a productive debugging workflow. Examine the candidate paths first. Then examine their capability scores. Then examine tracking. That separation gives a team interpretable objects to investigate rather than a single action stream whose errors could originate anywhere. There are concrete research assets to work with. The authors provide inference and deployment code, ROS integration, planner checkpoints, Spot and Hound affordance checkpoints, and training datasets. The release supports inspecting route generation and the surrounding navigation interfaces directly. The research opportunity is to improve those components independently and study their interaction. Better demonstrations can improve proposals. Better capability models can improve selection. Better controllers can change which proposals should be considered executable. The takeaway: share the route representation, specialize the feasibility decision, and make their interaction observable. That is a useful architectural principle for embodied AI well beyond this particular staircase. Outro. A navigation system has to answer several different questions: Where should I go? What routes look promising? Which route can this machine execute?. The development to watch is whether capability models become a standard interface through which improvements in locomotion inform high-level planning. That would give advances in semantic understanding and advances in physical control a clear place to meet. For Embodied AI 101, the lasting question is simple: once a model has imagined several ways forward, what gets to decide which one your robot should try?.