Latest / Embodied AI 101 / Flow Reversal Steering: Guiding Diffusion-Based Robot Policies with High-Level Reasoning
Transcript
- Flow Reversal Steering: Guiding Diffusion-Based Robot Policies with High-Level Reasoning. Imagine a generalist robot policy trained on massive multi-task datasets – for example, a vision-language-action (VLA) model that has observed thousands of demonstrations across many tasks. These diffusion-based policies can sample rich, low-level control sequences conditioned on visual inputs and even language instructions. In practice, though, directly commanding such a generalist policy can fail when handed a new, difficult task, especially if its raw output isn’t quite what we want. How can we steer these powerful but opaque policies towards a desired behavior using high-level guidance like a human or vision-language model (VLM)?. Recent work by Andy Tang, William Chen, Andrew Wagenmaker, Chelsea Finn, Sergey Levine and co-authors introduces Flow Reversal Steering (FRS) – a method that leverages the invertibility of flow-matching policies to bridge the gap. The key idea is this: if you have a coarse, plausible action (perhaps suboptimal) that a human or VLM believes the robot should take, you can run that action backwards through the policy’s flow. This finds the latent noise vector that would generate that action under the model. Then you sample forward again (the “denoising” step) to get an in-distribution action that is nearby in latent space. In short, FRS transforms a rough high-level instruction into a refined, high-quality action drawn from the policy’s learned behavior prior. Concretely, the authors demonstrate working with state-of-the-art diffusion VLAs (specifically, the π0.5 vision-language-action model) in both simulated and real robot tasks. They show three ways to use FRS: (1) Zero-shot steering, where you simply take a coarse guidance command (from a human or VLM) and apply FRS each step to get a refined action; (2) Distillation via Behavioral Cloning (DSBC), where you collect many good latent-noise vectors (via FRS) and train a noise-policy to imitate them; and (3) Latent-space RL with FRS, where you bootstrap a reinforcement learning process by planting those seed trajectories and adding a behavioral cloning loss on successful noise samples. Across a suite of manipulation tasks (simulated LIBERO tasks and real DROID tasks), Flow Reversal Steering dramatically improves performance: it can convert coarse semantic goals into effective control, yield nearly 95% higher success on some tasks after just a minute of training, and rescue RL on tasks where standard reinforcement both fails and struggles. Below we unpack the ingredients of Flow Reversal Steering, how it works, and why it matters. We’ll start with some background on diffusion and flow-based robot policies, then describe the FRS method and its three modes of use, and finally dive into the experimental results and comparisons to alternatives. Background: Diffusion and Flow-Based Robot Policies. Before diving into flow reversal, it helps to understand the nature of the underlying policies. Modern generalist robot controllers often use diffusion or flow-based generative models to produce actions. For instance, diffusion policies or Vision-Language-Action (VLA) models treat an action trajectory (or next action) as a sequence to be sampled from a learned prior conditioned on the current image (and possibly language). These models have gained popularity because they can capture multi-modal behaviors and be trained from raw trajectory data with simple imitation objectives. The basic idea in a diffusion policy is: sample a noise vector and run it through a stochastic reverse diffusion process (or use a denoising neural network) to produce a target action. In more recent variants, however, researchers have looked at “flow matching” as a deterministically invertible alternative to diffusion. In a flow-matching policy, one learns a continuous velocity field $v_\theta(a_t, t\mid o)$ over “time” $t\in[0,1]$ such that if you start at random Gaussian noise $a_{t=0}\sim \mathcal{N}(0,I)$ and integrate forward using $\dot a = v_\theta(a_t,t\mid o)$ up to $t=1$, you recover a data action $a_{t=1}$. In essence, the network is trained so that this time evolution from $t=0$ to $t=1$ carries simple noise samples to real actions from the data (for a given observation $o$). The key consequence is that this flow process is strictly invertible: running the same ODE backwards from $t=1$ to $t=0$ will return you to the original noise. By contrast, classical diffusion processes usually add random noise at each (forward) timestep and then require denoising networks to “undo” that. Diffusion methods can match the same marginal distributions as flows, but they do not preserve a one-to-one correspondence between a specific data sample and a specific noise vector. The viewer image from the FRS paper highlights this: with flow, each action has a unique noise, so reversing yields the exact original data; with standard diffusion, no such exact inversion guarantee holds. In short, flow-based policies allow exact “thinking in reverse” about actions, an opportunity that FRS exploits. This invertibility – often phrased as learning a deterministic velocity field – is described in the FRS paper along the lines of: “Flow Reversal maps data to noise by integrating the flow field backward in time. As flow matching learns a deterministic velocity field, there is a fixed correspondence between noise and data, so the resulting noises denoise back to the original data points”. Because of this fixed mapping, if you have any proposed action $a_1$, you can integrate backwards through the learned velocity field to find the unique initial noise $a_0$ that would generate $a_1$. Taken literally, that means one could compel the policy to produce a particular action by injecting the right noise. In practice, FRS takes advantage of this: instead of forcing an out-of-distribution action, it uses the inversion to land in a similar, high-density region. The “coarse reference action” might be off by a bit (since it came from a human reasoning or an imperfect language model), but when it’s run through the flow policy backwards, we get a noise vector that lives in the model’s prior. Then running the noise forward (the normal denoising) yields a polished action that is close in spirit to the reference, but guaranteed to be a valid, high-probability action under the policy. In robotics terms, the policy’s prior over actions encodes all the behaviors it learned from data. FRS is a way to search in that space: take a goal-directed guess, and map it into the nearest point in the learned prior. The paper succinctly defines the method as:. “Flow Reversal Steering (FRS): a method for guiding generalist flow policies’ action sampling by finding underlying noises that map to semantically-reasonable behaviors. By passing a coarse reference action – which captures roughly how the robot should move – through the flow policy in reverse, FRS finds the noise which approximately maps to the action. When subsequently denoised, FRS effectively finds a nearby good behavioral mode from the generalist’s prior that is similar to the reference.”. This allows one to infuse high-level semantic guidance into the action sampling. It is crucial that the underlying policy is a flow policy (or equivalently a score-based policy cast as an ODE) so this backward integration works exactly. If you only had a standard diffusion policy, you could not generally invert an action exactly. With flows, it’s literally solving an ODE backwards. Flow Reversal Steering (FRS) Method. Let’s unpack the FRS pipeline step by step. The zero-shot use case is easiest to imagine: at each timestep, we want to pick an action that roughly aligns with some high-level goal. Get Coarse Guidance: First, we obtain a reference action – a rough, “suboptimal but reasonable” movement. This could come from a human (e.g. a teleoperated demonstration or a spoken direction) or from a vision-language model. For instance, given an image of a shelf with objects, a VLM might output instructions like “move the gray cube to the left side of the box.” In the FRS experiments, the authors use DeepMind’s multi-modal Gemini-ER-1.6 model to produce such directional commands for each situation. In practice, one might input the current scene and ask the VLM “What joint action should the robot take now?” or otherwise query a learned reasoner. The key is that this reference action encodes high-level intent (pick-and-place object, open drawer, etc.) but is not guaranteed to be precise or even physically executable by the robot on its own. Flow Reversal (Noising): Taking the reference action, we run the learned flow model in reverse. Since the policy provides a velocity field $v_\theta(a,t|o)$, we integrate backward from $t=1$ to $t=0$. Intuitively, we ask: what noise vector $z$ would the policy have started with so that, after denoising, it would produce something like our reference? By stepping the ODE $\dot a = v_\theta(a,t|o)$ backwards in time, the reference action $a_{t=1}$ is carried to some initial $a_{t=0}$ which we treat as latent noise. Because the flow is deterministic, this back-propagation is well-defined. The resulting $a_{t=0}$ is the latent noise $z$ corresponding to the coarse command. Forward Sampling (Denoising): Now we run the model forward as usual, starting from the recovered noise $z$. We sample or deterministically integrate forward from $t=0$ up to $t=1$ to generate a final action $a'$. By design, $a'$ will be close to the reference if the model’s velocity field perfectly matched, but more importantly, $a'$ is guaranteed to lie in the support of the policy’s learned distribution. In other words, we move from a semantically-driven guess to a precise, in-distribution action. This final action $a'$ is what we execute on the robot. The FRS process thus “steers” the sampling of the policy. Instead of simply drawing random noise and hoping to land in a good mode, we choose noise that is guided by the reference. One can view it as a form of conditional sampling, where the condition is a coarse hint rather than a fixed state transition. Figure 8 in the paper (textually described on the website) illustrates this contrast between flow-based inversion and ordinary diffusion. With flow reversal, the noising process exactly inverts the matching denoising, so we get back exactly original points. With forward diffusion (e.g. adding Gaussian noise), there is a blurring of correspondence and we cannot invert to retrieve the original. In practice, this means standard backward diffusion tricks wouldn’t work as well for steering. Flow matching’s exact invertibility is the key enabler for FRS. In code or pseudocode, the FRS loop at each timestep looks something like:. This is in fact exactly how the authors describe the “zero-shot FRS inference loop”: query a reasoner for a coarse reference action, pass it through flow reversal to find the corresponding noise, and denoise this noise to get the final action. Because steps (2) and (3) use the flow model’s learned dynamics, the final action is “nearby” the reference but respects the robot’s data-driven priors. A critical insight is that FRS is low-cost at inference time. Inverse integration is just running the network backward, which costs roughly the same as a forward pass. It does not require additional learning or gradients through the model. So zero-shot FRS can be applied on the fly, with no extra training of the big model. We will see that even this straightforward mode yields large gains on many tasks – essentially converting semantic hints into better low-level actions. With the method itself now laid out, let us explore how the authors use FRS in various contexts and what they find. Zero-Shot Steering with High-Level Guidance. The simplest use of FRS is truly “zero-shot”: no further training, just hooking up a reasoner (like a person or VLM) to provide guidance on the fly. In each environment step, one asks the reasoner for the next move – for instance, “pick up that block and move it to the left shelf” – obtains a rough target action, and then applies FRS to refine it. Essentially, the robot is being given high-level commands that the flow prior fleshes out. In the experiments, the authors test this mode in the simulated LIBERO benchmark. LIBERO tasks consist of various multi-step household manipulation jobs (like sorting objects, opening drawers, stacking) in a simulated environment. They pair their FRS method with a state-of-the-art VLA policy (the π_{0.5} model) and with a powerful VLM (Gemini-ER-1.6) that is used as the “reasoner.” The Gemini model is prompted to output a rough motion plan in natural language, which is then translated into a coarse action vector for the robot. (For example, providing Gemini the image of a shelf might yield “Put the black bowl on the plate in front,” which the authors interpret as a relative motion vector for the arm.). What happens is that with FRS, those coarse semantic outputs from Gemini can indeed be turned into effective robotic motions. As the authors say, “Zero-shot FRS allows a VLM to guide the generalist policy based on its semantic reasonings, raising performance across LIBERO”. In practice, this means that tasks which the base policy struggled with often become solvable once guided by FRS. The paper reports that for the hardest LIBERO tasks (those where the original policy’s success was ≤2%), zero-shot FRS boosts success by at least 10 percentage points on 11 out of 42 tasks, whereas other steering methods only managed 3 or 4 such improvements. In other words, in scenarios where the policy was nearly useless on its own, simply telling it “move like this” and applying FRS yielded a big jump in solving the task. Moreover, FRS even outperforms simply executing the VLM’s commanded action directly. This is crucial: if the VLM direction were already perfect, one could just tell the robot to do exactly that movement. The results show that letting FRS refine it yields better outcomes than naively doing what Gemini said. This underlines the fact that many high-level suggestions need polishing. FRS effectively “takes a noisy instruction and finds the nearest good solution in the robot’s repertoire.” As the authors note, “FRS also outperforms directly executing VLM actions, showing that flow reversal refines the coarse reference actions, rather than simply reconstructing them.”. When tallying up results over all LIBERO tasks, zero-shot FRS achieves higher success rates across the board compared to the base un-steered policy. A summary figure (not reproduced here) shows FRS providing consistent lifts in success, especially on those challenging tasks that originally had near-zero performance. This demonstrates the power of combining semantic reasoning with the policy’s low-level knowledge: the VLM or human can propose “what to do,” and FRS figures out “how to do it well” by exploiting the flow prior. In practical terms, zero-shot FRS means the difference between a robot randomly failing and a robot purposefully succeeding – all without any extra policy training for those tasks. It turns out that even a small amount of semantic clue can steer the diffusion policy to a meaningful mode. Distilling Flows with Behavioral Cloning (DSBC). Zero-shot FRS is flexible, but it still relies on an external reasoner at run-time for guidance. What if you want your robot to accomplish these better behaviors without querying Gemini or a human each time? The authors propose a clever distillation strategy: Diffusion Steering via Behavioral Cloning (DSBC). The idea is to use FRS itself as a data generator, then train a compact “noise policy” that imitates FRS’s outputs. In DSBC, one collects example rollouts where FRS was used. Each such trajectory provides pairs of (observation → latent noise) that led to good actions. Specifically, while running zero-shot FRS, we record the noise vectors $z_t$ that were found by inverting each reference action. These $(o_t, z_t)$ pairs form a supervised dataset, because we know that feeding $z_t$ into the full model yields a high-quality action at $o_t$. We then train a small parametric policy (a neural network usually) that, given an observation $o$, directly predicts a noise vector. At training time, this noise policy tries to reproduce the FRS noise $z$ under each state. In effect, we are performing imitation learning in the latent (noise) space instead of on the raw actions. Why is this appealing? Because once the noise policy is trained, we no longer need to run Gemini or do any flow reversal at inference. The robot can simply sample noise directly from this learned noise policy to decide on actions. It distills all the semantic knowledge and FRS magic into a compact function. And since it is supervised learning (behavioral cloning in noise space), it is typically much faster and easier to train than full RL or finetuning the big model. The paper shows that DSBC effectively recovers almost all the gains of zero-shot FRS. On 15 LIBERO tasks they tested, the noise policies trained via DSBC achieved performance matching the original FRS-based steering – and significantly outperforming a standard BC policy that was trained to imitate actions without using the flow prior. In fact, DSBC wins out because learning noise-to-action through the original model preserves the complex maneuver structure that plain BC can lose. The authors observe: “DSBC on latent noise actions matches zero-shot FRS and beats standard BC on robot actions”. Furthermore, DSBC is extremely efficient to train. The authors mention that each noise policy took under one minute of GPU training time (and about 1 GB of memory), all without ever loading the full VLM or VLA model weights. This is because the auxiliary noise policy is typically very small, and the targets (noise vectors) come from running FRS once per example. In short, DSBC provides a quick route to embed FRS-derived behaviors into a lightweight policy. DSBC in Simulation and Reality. The benefits of DSBC are confirmed in the simulator and in the real world. In the LIBERO experiments, the distilled noise policies led to the same high success rates as zero-shot steering. The authors emphasize that with just a few demonstrations steered by FRS, DSBC catsched up with the performance of having a reasoner in the loop. There’s no drawback: one just carries the upside of FRS (semantic insight) but gets a fast, standalone controller at run time. Perhaps more impressively, DSBC shines in real robot experiments on the DROID benchmark. DROID is a collection of challenging one-step manipulation tasks performed with a worker robot – things like precise tape-hanging, opening variety of containers, etc. Normally, the pre-trained π0.5 VLA struggle on some of these fine or novel tasks, and training new policies in the real world is costly. Instead, the authors collected only 10 successful FRS-guided trajectories per task (think: a human or the system using FRS took the robot through the solution path 10 times). From these, they computed the corresponding noise labels via FRS flow inversion. Then they trained the DSBC noise policy. The result: the new policy solved most of these tasks with approximately 80% success, whereas the base VLA only managed around 20% on the same tasks. By contrast, a “standard BC” policy trained on those same 10 trajectories – trying to predict the actual action vectors – failed entirely. In some tasks, like the precise tape-hanging shown in their figure, the base VLA and standard BC could not hang the tape at all, but DSBC succeeded reliably. This real-world evaluation underscores one of FRS’s big advantages: it leverages the generalist model’s existing knowledge. By inversely mapping a handful of good demonstrations into noise space, we can bootstrap a policy that outperforms the generic model in these new tasks. The results are striking: the DSBC-distilled policies “significantly outperform the base π0.5 VLA on six DROID tasks,” resolving difficult behaviors that the base model missed. In summary, DSBC takes FRS from an on-the-fly steering trick to an offline learning routine. It yields a stable, small policy that encapsulates the best of FRS. And crucially, it achieves this with minimal data – just a few FRS examples per task. The running time and resource savings are also huge, since we avoid heavy RL or requiring large demonstration datasets. Reinforcement Learning with FRS (DSRL + FRS). The third usage of flow reversal steering is more subtle but powerful: combining it with reinforcement learning in the latent noise space. The idea is to treat steering as an RL problem – much like the “Diffusion Steering via RL (DSRL)” method introduced by Wagenmaker et al. – but to accelerate it using FRS trajectories. In pure DSRL (Diffusion Steering via RL), one would train a policy $\pi^{\mathcal{W}}(w|o)$ in the latent space, where actions are noise vectors $w$, and the environment reward is based on the resulting denoised action. Essentially, the RL agent learns to pick better noise vectors to achieve high reward. This approach has the advantage (as [61] puts it) that it is “highly sample-efficient” and does not require fine-tuning the whole base policy. Instead, one only trains a small latent-action policy, treating the original diffusion policy as a black box. However, learning from scratch via RL can still be slow if the task is hard. What FRS offers is a way to give that RL a smart head start. The authors introduce two simple augmentations to DSRL:. Pre-fill the replay buffer with FRS rollouts. Before any RL training, collect all the trajectories from zero-shot FRS (or human-steered FRS). These are proven-to-be-good (or at least better) trajectories. They are stored in the replay buffer as if they were successful experiences. This way, the RL agent immediately has some successful episodes to learn from, instead of starting from scratch. Auxiliary BC loss on successful FRS noise actions. When an episode yields high reward (e.g. it accomplishes the task), the latent noise actions from that episode are used as demonstration to further guide the learning. Concretely, the agent’s policy is also trained to imitate those particular noise vectors on the corresponding states (in addition to the RL objective). This encourages the agent to repeat what worked. Combining these with latent-space RL is called “DSRL + FRS”. The experiments show that this combination far outperforms vanilla DSRL. In a set of 25 LIBERO tasks (15 “moderately hard” and 10 very hard ones), DSRL + FRS consistently learns more quickly and achieves higher final success rates than standard DSRL or a baseline residual method (PLD-style RL). Crucially, on the hardest tasks – those where the base VLA essentially gets 0% and zero-shot FRS only achieves a small fraction of success (around 8%) – even a single successful FRS trajectory is enough to kick-start learning. The paper highlights that “even a single successful steered trajectory enables DSRL to learn substantially faster and better”. In one example, tasks where standard RL got stuck at ~0% success, DSRL + FRS was able to climb markedly upward. The intuition is clear: by pointing the RL agent toward semantically meaningful behavior at the outset, FRS helps it find the sparse reward in the first place. Figure 9 in the paper (described in text) shows curves of success vs. training steps for these variants. In the “easy-ish” group, DSRL + FRS curves shoot up faster than DSRL, rapidly reaching high success. In the “hard” group, we see that standard DSRL flounders, while DSRL + FRS jumps up once the buffer has that seed success, learning on that momentum. The authors summarize: “By directing the learner toward semantically-meaningful behaviors early in training, DSRL + FRS improves performance in the sparse-reward regime where standard generalist RL struggles.”. As a concrete example, consider a task where the robot must precisely place an object in a small target zone. The base policy never randomly finds that tiny region, so standard RL explores for a long time in vain. But if a VLM or human can give one hint and FRS refines it to one successful noise, then DSRL + FRS can immediately exploit that knowledge. The VLA’s strong prior and the latent policy now have a foothold. Importantly, this approach still works entirely in the latent noise space. The RL agent never needs to produce low-level torque or joint commands directly; it only picks noise for the fixed diffusion model. This has the benefit of low-dimensional actions (if the noise space is smaller) and no need to learn the full dynamics. [32] emphasized that steering RL in latent space avoids the challenges of finetuning a large model, and only requires training a small policy to select noise. FRS-enhanced DSRL continues this theme by making the latent search smarter. Overall, DSRL + FRS demonstrates that FRS isn’t just a zero-shot or imitation trick – it can coherently integrate into an RL training process, fusing semantic knowledge with trial-and-error. For many tasks, it was the difference between success and perpetual struggle for the agent. Why Flow Reversal Steering Matters. Flow Reversal Steering opens up several new possibilities for embodied AI:. Harnessing Language at Low Level. One of the holy grails is letting high-level reasoning (often captured in language) inform low-level control. FRS does exactly this by translating high-level instructions into action distributions. It acts as a kind of “semantic adapter” that converts the VLM’s coarse output into a precise motor command via the existing model. Crucially, it doesn’t require retraining the big diffusion model to understand those instructions – it can channel them post hoc. Sample-Efficient Adaptation. Bootstrapping policies for new tasks usually needs lots of data or RL samples. FRS + DSBC learns to solve tasks with as few as 10 demonstrations. Even RL is sped up dramatically. This kind of efficiency is valuable in robotics, where physical trials are expensive. By leveraging the learned prior and a semantic hint, we make the problem much easier. Generalist Priors for New Goals. The method shows how to exploit the implicit knowledge in multi-task policies. A generalist policy is like a big pool of motor skills. FRS gives us a controlled way to fish for just the skill we need, guided by a hint. This is better than naively fine-tuning or hoping new data covers the case. Real-Time Guidance. Since FRS only adds a bit of ODE inversion to inference, it can be done on the fly. A user could even interactively give the robot a gesture or voice hint at each step and have it respond in real-time. This contrasts with offline methods like fine-tuning a policy via RL after data collection. Modularity. FRS splits reasoning and execution. The reasoner (human or VLM) stays generic, and the flow model stays fixed. We’re not baking new language capabilities into the diffusion model; instead we interface with it through mathematical operations. Of course, there are limitations and caveats. Flow Reversal relies on having an invertible policy, so it’s not directly applicable to all diffusion models. It also assumes the reasoner can produce a useful coarse action; a completely wrong reference won’t be magically fixed. And the success still depends on the quality of the underlying VLA. If that prior is very weak for a domain, FRS can’t create skill out of nothing. But for the class of strong generalists (like π0.5), it’s remarkably powerful. Another consideration is safety and predictability. Because FRS might move the robot in unexpected ways if the reference is ambiguous, one should ensure that human or VLM hints are constrained properly. However, in experiments the refined actions were generally sensible. One advantage is that FRS tends to produce outputs similar to its reference but “snapped” to the nearest valid mode – so egregious mistakes are somewhat smoothed out. The analysis shows FRS rarely degenerate into bizarre moves (if the coarse hint is reasonable). Its greatest weaknesses would likely be if the hint is actively misleading, but in practice VLM instruction tended to be helpful on target tasks. Comparison to Other Methods. It’s worth comparing FRS to other recent approaches for steering or improving robot policies:. Latent-space RL (DSRL). The work we mentioned by Wagenmaker et al. on Diffusion Steering via RL already showed the promise of learning noise policies to adapt a diffusion controller. FRS+DSRL can be seen as building on that: FRS provides a way to inject semantically-guided demonstrations into that RL process. The FRS paper shows that this hybrid accelerates learning compared to DSRL alone. In essence, FRS can be thought of as a dynamic guidance function that DSRL can exploit. Whereas DSRL relied purely on exploration, FRS uses reasoning to cut through the search. Explicit Steering Commands. The “Steerable Policies” line of work (by William Chen, Danny Driess, etc.) proposes to train policies that can accept fine-grained commands (like subtask specifications or pixel coordinates). That approach fundamentally changes how the policy is trained, by including richer control inputs. In contrast, FRS doesn’t require retraining the generalist; it treats the model as fixed and attaches guidance externally. Steerable Policies train a policy to obey commands, which is powerful but requires annotated data. FRS takes an existing policy and makes it obey commands roughly, with no new annotations. Residual Finetuning. Some works do task-specific finetuning of policies or reserve some capacity for adaptation. FRS avoids modifying the model at all. This has advantages in safety (the base model stays intact) and deployment (you can apply FRS to any frozen model). It’s orthogonal to doing some finetuning – one could even use FRS-generated rollouts to supervise such finetuning if desired. Direct Optimization. One could imagine directly optimizing latent noise (via gradients) to maximize some success metric (like classifier-free guidance in generation). The FRS approach has a flavor of that, but it uses external semantics rather than gradients of reward. It would be noisy and slow to rely solely on gradient-based steering for task success; FRS is a much more interpretable and guidance-driven method. In short, Flow Reversal Steering is unique in what it uses (it exploits invertibility of a flow policy) and how it uses high-level guidance (turning an action estimate into a valid action). It complements these other paradigms by staying at inference-time and serving as a plug-in steering mechanism. Conclusion. Flow Reversal Steering addresses a key challenge in embodied AI: bridging the gap between high-level reasoning (often in natural language) and precise robot motion. By inverting a diffusion/flow policy at inference time, it allows us to take a rough, semantic action suggestion and turn it into a good control signal. The strategy works out-of-the-box on a pre-trained generalist policy, improving zero-shot performance, enabling rapid imitation learning (via DSBC), and kickstarting reinforcement learning (via DSRL+FRS). The empirical results – large success boosts on simulated tasks and the ability to solve hard real tasks with just handfuls of demonstrations – speak for themselves. For practitioners, FRS offers a toolkit: if you have a powerful diffusion-based robot policy, you can now augment it with a VLM, human operator, or any other high-level “reasoner,” and get the best of both worlds. It also suggests an interesting research direction: what other inference-time transformations of generative policies could unlock new capabilities? As generalist robot models continue to improve, methods like FRS that merge them with symbolic or semantic guidance will likely become ever more valuable. References: FRS is introduced in by Tang et al. The concept builds on diffusion-policy adaptation methods like DSRL (Wagenmaker et al., “Steering Your Diffusion Policy with Latent Space Reinforcement Learning”) and complements recent work on steerable vision-language-action policies. Its core innovation—using flow model invertibility for control—derives from the idea of flow-based generative models. The experimental results cited above come from Tang et al.’s evaluation on LIBERO and DROID tasks.