Latest / Embodied AI 101 / VLS: Steering Frozen Robot Policies Without Retraining

Transcript

VLS: Steering Frozen Robot Policies Without Retraining · Embodied AI 101

• VLS: Steering Frozen Robot Policies Without Retraining. VLS: Steering Pretrained Robot Policies via Vision-Language Models • Shuo Liu, Ishneet Sukhvinder Singh, Yiqing Xu, Jiafei Duan, and Ranjay Krishna • University of Washington, University of Oxford, National University of Singapore, and Allen Institute for AI • arXiv; project page lists CoRL 2026 • 2026. The real promise: a new constraint, not a new skill. Imagine a robot that already knows how to pick up an object and place it on a plate. Now move the plate, substitute a different object, or put an obstacle across the familiar approach path. There are at least two possible reasons the robot might fail. It may genuinely lack the required motor competence. Alternatively, it may possess useful manipulation behavior but fail to deploy it in the right place, toward the right object, or under the right constraint. These are different problems. More demonstrations can help both, but they are not necessarily the most economical solution to the second. Vision-Language Steering, or VLS, targets that second problem. It leaves a pretrained generative policy’s parameters unchanged and intervenes in action generation. An external vision-language model helps construct a task-specific objective; an inference-time steering mechanism then changes which action trajectories the frozen policy produces. The released framework supports both diffusion policies and a flow-matching π0.5 policy. The crucial distinction is that VLS does not merely ask a VLM to improve the robot’s instruction. Nor does it ask the VLM to judge every finished trajectory. It turns visual and linguistic understanding into executable reward programs, then uses those programs to influence partially generated actions. That makes the work interesting beyond its benchmark numbers. It proposes an interface between semantic reasoning and continuous control: the VLM specifies what should change, while the learned policy continues to supply the motion prior. A scope clarification matters at the outset. The project includes a demonstration of steering around a wall, but I did not find a controlled lighting-robustness evaluation in the paper. Lighting adaptation should therefore not be presented as an established quantitative result of this work. Throughout this episode, I distinguish the authors’ reported experiments from observations about their public implementation. The implementation observations come from static inspection, not an independent reproduction. A VLM that writes the objective. The most useful way to understand VLS is as a compiler from a scene and instruction into a differentiable objective. The first step is geometric grounding. The described pipeline combines segmentation, DINO visual features, and depth to identify task-relevant spatial keypoints. These points provide an explicit geometric representation on which the guidance functions can operate. In the released keypoint detector, masked visual features are reduced and combined with normalized three-dimensional coordinates before clustering. Candidate points are subsequently filtered and merged. The implementation supports both DINOv2 and DINOv3 feature extractors, although the paper’s method description centers on DINOv2. Why introduce keypoints instead of feeding the whole image into a reward network?. Consider a placement instruction. “Put the orange on the green plate” contains semantic decisions—identify the orange, distinguish the green plate—but the controller eventually needs spatial relationships. The target must become a location, region, or relationship in the robot’s coordinate system. Keypoints form a deliberately narrow interface between those two levels. Once the relevant geometry is identified, evaluating whether a proposed trajectory approaches the target no longer requires repeating broad visual-language reasoning. Next comes reward generation. The released VLM agent receives an annotated image, the instruction, and metadata describing the keypoints. It generates functions organized by task stage, which the system parses and saves for subsequent use. The agent explicitly derives from the constraint-generation machinery of ReKep. For an illustrative pick-and-place task, one stage might favor approaching the correct object, while another favors transporting it toward the destination. A reward can express these preferences using distances and other differentiable geometric operations. The exact program matters: a penalty on the trajectory endpoint is not equivalent to a penalty applied along the entire path. This leads to the central computational separation. The VLM writes a function; automatic differentiation operates on that function’s execution. The steering implementation evaluates the generated functions on trajectory tensors and differentiates the resulting reward with respect to the action proposal. It does not need gradients through the VLM’s generated text. That is a practical design choice. A language model can provide a relatively expressive task interpretation without sitting inside every gradient calculation. Once the program exists, the inner loop works with tensor operations rather than repeated natural-language judgments. But it also changes where errors can occur. A fluent explanation does not ensure a correct reward. The VLM might select the wrong object, choose an unsuitable keypoint, omit an important constraint, or produce an objective that rewards a geometric shortcut. Differentiability only means the optimizer can follow the objective. It does not mean the objective faithfully captures the task. My interpretation is that reward-program quality should be treated as a first-class component, separate from both perception quality and policy quality. Those three components can fail independently, and a successful deployment needs to diagnose which one is responsible. How the gradient reaches the action. Let us now move inside the sampler. A useful conceptual model is to start with the action distribution supplied by the frozen policy and preferentially weight trajectories that satisfy the new objective. In words: retain the policy’s preferences, but give additional probability to actions with higher reward. Under an exponential reward weighting, the corresponding log-density gradient separates into two contributions: the original policy score and a reward-gradient correction. This is the familiar logic behind classifier-style diffusion guidance, whose foundational formulation uses an external classifier gradient to alter generation. For VLS, however, the reward is a programmatic surrogate. It should not be mistaken for a calibrated probability that the physical task will succeed. That distinction is important. The probabilistic interpretation explains why guidance is sensible, but it does not establish that arbitrary generated rewards, evaluated on intermediate noisy actions, produce exact samples from an ideal constrained distribution. Diffusion: modify the noise prediction. The released diffusion wrapper obtains the base model’s noise prediction and subtracts a scaled reward-gradient correction from its translational components. The scale includes a factor derived from the diffusion noise schedule. In the subsequent reverse-diffusion update, this biases the action proposal toward higher reward. This is why “add a gradient” can be an imprecise implementation description. One must distinguish the score, the predicted noise, and the actual sample update. Their signs are not interchangeable. The network parameters remain fixed. What changes is the evolving action tensor. Flow matching: modify the velocity field. For the flow-matching policy, the intervention occurs in the predicted velocity field. The released π0.5 wrapper integrates from noise time one toward clean-action time zero. It subtracts the reward-gradient correction from the velocity, then takes an Euler step with a negative time increment. The resulting action displacement therefore follows the reward-improving direction. This is a small detail with substantial engineering consequences. When porting guidance between flow implementations, copying a plus or minus sign without checking the time convention can reverse the intended behavior. The general recipe is portable; the numerical convention is not. The bridge from action coordinates to geometry. A geometric reward cannot operate meaningfully on arbitrarily normalized action coordinates. The action proposal first needs a differentiable conversion into the representation used by the reward. In the CALVIN adapter, relative translational actions are scaled and cumulatively summed from the current end-effector position. This produces a three-dimensional end-effector trajectory using differentiable tensor operations. Think through what that means for the gradient. An early displacement affects not only the next waypoint but also later positions in the accumulated trajectory. Consequently, a reward associated with a later target can influence earlier action increments. This provides temporal structure without requiring the VLM to reason separately about every low-level command. It is nevertheless a kinematic construction, not a learned prediction of contact dynamics. A proposed end-effector path does not tell us whether the object will slip, whether a grasp will establish force closure, or whether an articulated mechanism will move as expected. Those outcomes remain contingent on physical execution and the competence already present in the policy. The current wrappers directly inject guidance into the translational action dimensions. That is a narrower mechanism than unconstrained optimization over every aspect of manipulation. For adaptation, I would therefore ask two questions before tuning anything: does the reward describe the failure that actually matters, and does the steered action representation contain enough control authority to correct it?. A placement-location error is a natural fit. A failure dominated by subtle gripper timing or contact mechanics may not be. Why one trajectory is not enough. Local gradients are useful, but manipulation objectives can be multimodal. An obstacle might admit two reasonable routes. A graspable object might admit several approaches. If every candidate starts near the same unhelpful mode, improving each candidate locally may still leave the system with no viable alternative. VLS addresses this using multiple action proposals and two complementary mechanisms: diversity promotion and reward-weighted resampling. These are also identified as separate components in the released framework. The diversity mechanism discourages proposals from becoming too similar early in generation. Although the authors call it RBF diversity, the released implementation uses an inverse-distance potential between flattened trajectories, rather than a conventional Gaussian kernel. The distinction is worth noticing when reproducing the method, but the operational intent is straightforward: avoid spending the entire batch on nearly identical guesses. The second mechanism is Feynman–Kac steering. Here, action proposals are treated as particles. Their rewards determine positive weights, and resampling replicates preferred particles while removing less promising ones. The released implementation supports several potential constructions, configurable resampling intervals, and an effective-sample-size criterion for adaptive resampling. The important contrast is with ordinary best-of-many selection. VLS can redistribute the population during generation, while the surviving proposals still have denoising steps left. It is not restricted to choosing among completed trajectories. My reading is that the two mechanisms solve different search problems. Gradient guidance improves an individual proposal. Resampling changes how much of the available computation is allocated to different proposals. Diversity helps keep that allocation from collapsing prematurely. These mechanisms can also work against one another if configured poorly. Excessive resampling can erase alternatives; excessive repulsion can spread proposals away from useful behavior. Their value is therefore an empirical question, not something guaranteed by combining their names. The paper’s pseudocode additionally specifies MCMC-style inner refinement: four inner updates for diffusion and one for flow matching. That detail reinforces a broader point: “training-free” describes the absence of parameter learning, not the absence of optimization or additional computation. Closing the loop around the sampler. A trajectory can score well and still fail physically. The controller therefore needs to revisit both its objective and its confidence in the current stage. The released wrappers adapt guidance strength using reward progress relative to the beginning of a stage. A sigmoid schedule reduces intervention as normalized progress increases, allowing the base policy to contribute more near the target. Intuitively, steering should help with coarse correction without continuously overpowering useful learned behavior during the final approach. Stage management is a separate problem. The public execution loop uses upper and lower reward thresholds to trigger reassessment, along with gripper-state changes. When triggered, a VLM-based stage recognizer can select a different stage or change whether guidance is needed. Using different thresholds for rising and falling progress introduces hysteresis. Rather than reacting to every small fluctuation near one boundary, the system waits for a more meaningful change before reconsidering its state. For a roboticist, this is an important part of the contribution. The objective appropriate before grasping an object is not necessarily appropriate after grasping it. A controller that optimizes the wrong stage extremely well can still fail the overall task. I would also separate “the reward predicts progress” from “the world confirms progress.” A high trajectory score is evidence about a proposal. It is not proof that the preceding manipulation succeeded. Event-triggered visual reassessment helps bridge that distinction, but it does not eliminate it. Where VLS sits relative to other approaches. VLS is best understood as a synthesis rather than the invention of gradient-guided robot sampling. ReKep supplies an important conceptual precursor: express manipulation through relational keypoint constraints generated with vision-language reasoning, then use those constraints in a control or optimization process. The released VLS agent’s explicit adaptation of ReKep code makes that relationship particularly concrete. The interesting architectural choice is where the resulting objective acts. Instead of treating geometric optimization as the sole motion generator, VLS uses the pretrained generative policy as a continuing source of candidate behavior. DynaGuide is the most important comparison for the guidance mechanism. Its original method uses an external latent dynamics model to predict outcomes of proposed actions. Distances to representations of desired and undesired visual outcomes provide differentiable guidance during diffusion. It is therefore more than a static DINO-feature similarity heuristic. This gives the two methods different strengths and dependencies. A dynamics model can represent effects that are difficult to encode as immediate geometry, but it introduces model-learning requirements and prediction error. Programmatic geometric rewards avoid that particular dependency, while placing greater responsibility on grounding and reward specification. Inference-Time Policy Steering, or ITPS, is another relevant predecessor. It uses human interactions—including point, sketch, and physical-correction inputs—to bias generative sampling. Its original evaluation compares multiple sampling strategies and emphasizes the balance between following the interaction signal and avoiding harmful distribution shift. VLS’s distinction is thus not that previous methods lacked continuous guidance. It automates the construction of deployment-specific guidance from visual observations and language, then combines that guidance with particle management and stage feedback. For practitioners, these approaches suggest a useful decision tree. If the deployment objective is naturally specified by a person’s trajectory correction, ITPS-style interfaces may be appropriate. If success depends on predicting visual consequences, dynamics-guided steering is attractive. If the mismatch can be expressed through grounded spatial relationships, VLS offers a relatively direct route from instruction to action-space correction. What the experiments actually establish. CALVIN: strong steering, but interpret the task correctly. The CALVIN comparison is striking. Figure 3 reports approximately the following group averages:. | Task group | Frozen diffusion policy | DynaGuide | VLS | |---|---:|---:|---:| | Movable objects | 13% | 26% | 94% | | Articulated parts | 9% | 69% | 87% |. ITPS reaches approximately 39% on the articulated group. The figure caption reports 600 episodes per task. The different group-level gaps matter more than a single headline improvement. On movable objects, the gain over DynaGuide is much larger than on articulated parts. A plausible interpretation is that explicitly grounding the current target geometry is particularly useful when object locations vary. That is an interpretation of the pattern, not a controlled causal result isolating the grounding module. There is another important qualification. These are steering evaluations over specified behaviors. They should not be read as the standard CALVIN five-instruction-chain result. The distinction is not semantic bookkeeping. A multitask policy can have useful behavior in its distribution while often selecting something other than the requested behavior. Indeed, DynaGuide’s own presentation frames its CALVIN experiments as steering a diverse base policy toward particular behaviors. Consequently, the very low unsteered scores do not establish that the backbone possesses almost no manipulation competence. They establish that the complete unsteered system performs poorly on the requested behavior under this protocol. LIBERO-PRO: the cleanest paired adaptation result. LIBERO-PRO was developed to expose generalization failures that ordinary LIBERO evaluation can obscure, including sensitivity to changed initial states, instructions, objects, and environments. VLS evaluates task and position perturbations across four ten-task suites, with twenty episodes per task. The most informative comparison keeps the LeRobot π0.5 policy fixed:. | Evaluation slice | Frozen π0.5 | π0.5 + VLS | Absolute gain | |---|---:|---:|---:| | Task perturbations | 23.13% | 38.50% | 15.37 points | | Position perturbations | 24.25% | 35.13% | 10.88 points | | Overall | 23.69% | 36.81% | 13.12 points |. OpenVLA and π0 also appear as comparison baselines, but the table does not show VLS-wrapped versions of those models. The headline “13% gain” is therefore best expressed as roughly thirteen percentage points in this paired comparison. Relative to the baseline, the increase is about 55%. Both descriptions are mathematically valid, but they communicate different things. The remaining failure rate is equally important. Improving overall success to roughly 37% is meaningful evidence of adaptability; it is not evidence that these perturbations are solved. Nor does this experiment prove that inference-time steering is universally necessary. It shows that this steering system helps these frozen checkpoints under the evaluated shifts. Different training distributions or architectures might also address those failures. Ablations: gradients do most of the work. The released plotting data reports 88% success for full VLS, compared with 17.3% without gradient guidance, 85.3% without Feynman–Kac resampling, and 86.0% without diversity. This is a clear hierarchy. The gradient contribution is large. The incremental contributions of the other two components are much smaller on this evaluation. I would describe the evidence as supporting gradient guidance as the main driver, with particle mechanisms providing additional improvements—not as showing that every component is equally indispensable. The same plotting data exposes the compute trade-off. Increasing the sample count from one to ten changes success from 88% to 90%, reduces average episode length from 73.0 to 55.9 steps, and increases reported inference time from 665 to 1,239 milliseconds. Full VLS’s separate ablation timing is approximately 1,189 milliseconds. Those numbers deserve attention. Fewer environment steps do not automatically mean faster wall-clock completion if planning becomes more expensive. More generally, a deployment decision should compare useful task progress per second, not success rate or simulator step count alone. Action chunking can amortize inference, but the appropriate chunk length depends on how quickly the environment can change. Real hardware: encouraging, with a crucial metric caveat. The real-robot plotting data shows an average score increase from 50% to 69% in distribution, and from 42% to 69% across the evaluated out-of-distribution conditions. The novel-mug condition rises from zero to 40%. One in-distribution condition, however, declines from 90% to 84%. That last result is useful: steering helps on average, but it is not automatically harmless. More importantly, the real-world metric awards half credit for grasping the correct object and full credit for completing the task; each task uses twenty trials. Therefore, a reported 69% is not necessarily a 69% full-task-completion rate. The mug result illustrates the ambiguity. A normalized score of 40% could arise from eight complete successes, sixteen correct-grasp-only outcomes, or a mixture. Without the underlying outcome breakdown, those possibilities cannot be distinguished. The hardware evidence still supports a meaningful conclusion: the method improves object-directed manipulation under deployment variation. But it should not be converted into a stronger binary-completion claim than the metric supports. What I would check before putting it on a robot. The authors release code and a CALVIN diffusion-policy checkpoint, making the work more actionable than a paper-only proposal. The checkpoint is explicitly identified as the frozen base policy used by VLS. Still, I would approach adoption as systems integration rather than a one-line guidance toggle. First, establish the action-to-geometry contract. Check units, normalization, coordinate frames, action horizon, and whether commands are relative or absolute. The CALVIN adapter’s cumulative-displacement construction is appropriate for its stated action convention; a different controller may require a different differentiable mapping. A useful initial test would avoid language models entirely: provide a simple target-distance reward and verify that an infinitesimal guidance update moves the predicted end-effector trajectory in the expected direction. This separates sampler bugs from semantic-grounding failures. Second, audit the source of geometry. The released keypoint tracker attaches points to object frames and updates them using object poses supplied by the environment adapter. The CALVIN adapter can obtain those poses directly from PyBullet. That is valuable for testing steering mechanics, but it is not equivalent to solving perception and tracking from noisy real-world images. I would therefore evaluate oracle geometry and estimated geometry separately. Otherwise, a strong simulation result can conceal the very error source that dominates deployment. Third, treat generated reward code as an artifact to test. Check that gradients exist, remain finite, have sensible scale, and point in sensible directions. Test nearby counterexamples: the wrong object, the wrong side of an obstacle, or an endpoint that is correct despite an unsafe intermediate path. These are engineering recommendations, not additional capabilities established by VLS. Their purpose is to prevent the optimizer from efficiently exploiting an incomplete specification. Fourth, keep steering separate from hard safety enforcement. A soft reward for clearance is not a proof of collision avoidance. I would retain independent limits on workspace, velocity, acceleration, and collision risk, particularly when guidance strength is large. Finally, pin the implementation and evaluation configuration. The accessible eleven-page paper repeatedly refers to an appendix that is not included in that file, leaving some promised implementation detail unavailable there. For reproduction, I would record the policy checkpoint, perception models, reward-generation prompts, generated programs, sampler settings, and stage decisions. I would also report end-to-end latency separately from policy inference: the operational budget includes perception, reward generation, reassessment, and command execution. Most importantly, I would test preservation of already successful behavior. An adaptation layer should be evaluated not only on how many failures it repairs, but also on how often it damages a rollout that the base policy would have completed. The foundation-model lesson. The strongest lesson of VLS is not that retraining has become unnecessary. It is that deployment adaptability can have its own architecture, rather than being treated exclusively as a property of the pretrained weights. The resulting division of labor is appealing: use a learned policy for reusable motion structure, semantic reasoning for interpreting the new request, differentiable geometry for directing local correction, and feedback for deciding whether the current plan still makes sense. That modularity also gives researchers clearer questions to ask. Is the failure semantic, geometric, motor, or temporal? Does improvement require a better policy, or merely a better way to condition its sampling? Which constraints can be specified cheaply, and which demand a dynamics model?. My assessment is that VLS provides persuasive evidence for this direction, especially through the paired LIBERO-PRO improvement and the gradient ablation. It does not establish universal out-of-distribution robustness, servo-rate efficiency, or guaranteed preservation of base-policy competence. For a robot whose underlying skill is adequate but whose deployment conditions have changed, however, the proposal is compelling: before collecting another dataset, try making the action-generation process more controllable.