Latest / Embodied AI 101 / HiFlow: Tokenization-Free Scale-Wise Autoregressive Policy Learning via Flow Matching

Transcript

HiFlow: Tokenization-Free Scale-Wise Autoregressive Policy Learning via Flow Matching · Embodied AI 101

• HiFlow: Tokenization-Free Scale-Wise Autoregressive Policy Learning via Flow Matching. Robotic manipulation policies can be trained by thinking of them like language models: instead of predicting one motor command at a time, we can model entire action sequences. Traditionally, autoregressive (AR) policies generate actions step-by-step (like words in a sentence), which makes inference very fast but sometimes loses sight of the whole plan. By contrast, diffusion-based policies (recently popularized for robotics) generate actions by gradually denoising a random sequence into a coherent trajectory, yielding smooth long-horizon plans but at the cost of many iterative sampling steps (and therefore slower inference). A promising compromise is coarse-to-fine generation: first predict a rough global outline of the trajectory, then refine it at higher resolution. In the image domain, this idea goes by “snake” or “scale-wise” autoregression (e.g. VAR, FlowAR) and has been adapted to robot actions (e.g. CARP). However, prior work in this vein relied on an intermediate tokenization step: the continuous actions were squashed into discrete codes using something like a VQ-VAE, and a transformer was trained to predict those codes. HiFlow, introduced by Yashima et al. (2026), cuts out that middleman. Instead of converting continuous actions into discrete tokens (which can introduce quantization error and require a separate training stage), HiFlow operates directly in the continuous action space. It does this by building a multi-scale representation of the action sequence through simple temporal pooling, and by using flow matching to generate real-valued actions at each scale. In short, HiFlow is a hierarchical flow-based policy: a scale-wise autoregressive transformer provides coarse-to-fine context, and a shared flow network outputs continuous actions at each level. The entire model is trained end-to-end, with one loss, and requires no special tokenizer. In experiments on both simulated and real robot tasks, HiFlow consistently outperforms previous methods (both diffusion models and discrete-token autoregressors) on task success rate, while retaining fast inference. Below we unpack how HiFlow works and why it matters. We first recall the motivation for coarse-to-fine policies and the issues with discrete tokenization. Then we describe HiFlow’s approach in intuitive terms: how it builds coarse summaries of an action chunk, how the transformer and flow network work together, and how the model is trained. Finally, we review the experimental results that validate the approach. Why Coarse-to-Fine? The Tradeoffs in Action Generation. When teaching a robot from demonstrations (imitation learning), we must choose how to turn a dataset of trajectories into a policy. A simple approach is behavior cloning with a neural network that predicts the next action given the current observation and some history. But straightforward one-step prediction can fail to grasp long-term structure: it may do fine in the short term but wander off over a longer horizon. This insight led people to treat action prediction as a sequence modeling problem, borrowing ideas from language models. In this view, a policy is something like a GPT that generates an action sequence token by token, conditioned on visual and proprioceptive context. Transformer-based models like the Decision Transformer (2021) and Behavior Transformer (2022) adopted this view. While these models can capture multimodal action distributions better, they still predict actions one-at-a-time in a fixed sequence order, which can miss global coherence. Diffusion-based policies (Diffusion Policy by Chen et al., 2023, and follow-ups) approach this differently. They view action generation as a gradual denoising process: start from a random sequence of “noisy” actions and iteratively refine it until it matches the data distribution. Diffusion policies have shown very good performance on many tasks because the iterative denoising tends to enforce global structure. But the downside is speed: each new action sequence requires dozens of denoising steps, which can be slow for real-time control. Coarse-to-fine generation attempts to capture the best of both worlds. The idea, inspired by recent work in image synthesis, is to generate a trajectory in multiple passes. On the first pass, the model predicts a very coarse summary of the future actions (say, a handful of numbers capturing the global direction). On subsequent passes, it fills in more and more detail. This maintains autoregressive efficiency (you still generate in one shot per pass) while enforcing consistency across the whole chunk of actions. In effect, the model decides on a rough plan first, then drills down. An example of this idea is the CARP method (Gong et al., 2024, “Coarse-to-Fine AutoRegressive Policy”). CARP defines several temporal scales (e.g. 1, 2, 4, 8 timesteps). First it encodes whole 8-step actions into just 1 code (a bottleneck) via a VQ-VAE, then predicts that code; then it goes to 2 codes, etc. Each scale’s output conditions the next. CARP achieved strong performance, but at the cost of learning a separate vector-quantized autoencoder (VQ-VAE) for actions, for every embodiment or task regime. This pipeline is two-stage: train the action autoencoder, then train the autoregressive model on discrete codes. Moreover, mapping a fine-grained continuous motion into a limited codebook inevitably introduces quantization error—small nuances of the action can be lost. Luckily, robot actions are usually quite low-dimensional (for a 6-DoF arm you might have 6 or 7 numbers per timestep, maybe a bit more), not the millions of pixels of an image. There’s no obvious need to squeeze them through a codebook. The HiFlow insight is to bypass discrete tokens altogether and operate purely in continuous space. Doing so preserves all the fine details of motions and simplifies training. Instead of learning a complicated VQ-VAE, HiFlow simply averages the actions over time windows to get coarser representations. And instead of using cross-entropy on tokens, it uses flow matching to directly model the continuous distribution of actions at each scale. Put simply, HiFlow asks: why not smooth the trajectory out at multiple resolutions and teach a continuous model to generate it? The answer, it turns out, is yes—that yields better control. HiFlow Overview: Building a Hierarchy of Actions. Here is a step-by-step picture of what HiFlow does. Suppose our policy at test time will output a short chunk of actions (say 8 timesteps long) given some observation (images, joint states, task ID, etc.). Instead of generating those 8 actions one by one, HiFlow does it in stages. Multi-Scale Action Targets: We start with an 8-step ground-truth action sequence (from the demo dataset). HiFlow creates several downsampled versions of this sequence by simple averaging. For example, scale 1 might be just 1 action that is the average of all 8 (a global “centroid” of motion). Scale 2 has 2 actions (each averaging 4 timesteps), scale 4 has 4 actions (averaging 2 timesteps each), and scale 8 has the full 8 actions (which is just the identity). More generally, the method partitions the 8 timesteps into i equal groups and averages each group to get i coarse actions, for i = 1,2,4,8. This is a parameter-free temporal pooling that yields a smooth hierarchy of action sequences. The authors note that because this stays in the native continuous action space, it keeps the Euclidean geometry intact and avoids any quantization loss. Scale-Wise AR Transformer (ScaleAR): Next we need a way to predict these multi-scale actions in order. HiFlow uses a transformer that is masked so that coarser scales come first. Concretely, the transformer takes as input a special task token (for scale 1) and the upsampled actions from the previous scale for all finer scales. For example, for scale 2, it sees the single average from scale 1 (upsampled to 2 timesteps); for scale 4 it sees the 2-step sequence from scale 2 (upsampled to 4); and for scale 8 it sees the 4-step sequence from scale 4. At training time those “previous scale” sequences are the ground-truth averages; at inference time they are the model’s own predictions. Critically, attention is only allowed from each scale to coarser scales (never to finer). Thus the output of the transformer has separate tokens corresponding to each scale’s positions. In effect, this transformer produces a set of conditioning features for each scale that encode all coarser context. Mathematically, it respects an autoregressive factorization of the joint path: (p(a^{(1)},a^{(2)},\ldots,a^{(8)}) = \prod_{i \in {1,2,4,8}} p(a^{(i)} \mid \text{features from scale }i)). ActionFlowNet (Flow Generator): Now for the heart of the continuous prediction. For each scale (i), HiFlow needs to generate (i) continuous actions from the conditioning features. Rather than predicting them directly in one shot or sampling from a learned Gaussian, HiFlow uses flow matching: it trains a network to transform a random Gaussian sample into a desired action sequence. The network (called ActionFlowNet) is itself a small transformer with adaptive layer-norm conditioning. At scale (i), we randomly sample a noise vector (\epsilon) of size (i \times D) (same shape as the action sequence we want). We then consider a continuous interpolation (x_\tau = (1-\tau),a^{(i)} + \tau,\epsilon) for (\tau) in [0,1], where (a^{(i)}) is the true (i)-step action from the dataset. At (\tau=0) this is (a^{(i)}), and at (\tau=1) this is (\epsilon). The target velocity field along this straight line is simply (\epsilon - a^{(i)}) (the difference between noise and the true action). The ActionFlowNet is trained to predict this velocity given the noised input (x_\tau), the time (\tau), and the transformer features for scale (i). In practice the flow network outputs a matrix (\hat{v}\theta(x\tau,\tau)), and we minimize the mean-squared error (|\hat{v} - (\epsilon - a^{(i)})|^2). In other words, HiFlow’s flow network learns how much “push” is needed at each time-step and interpolation point to move from a pure Gaussian into the target actions. Because every scale’s network is the same neural net (shared weights) and sees different sequence lengths, it naturally handles multiple resolutions. The conditioning features from the transformer are fed in via per-step adaptive layer norm, so the flow at each timestep knows the context. Training End-to-End: Unlike past coarse-to-fine methods, HiFlow doesn’t involve any separate pretraining. The vision encoder (for images), the ScaleAR transformer, and the ActionFlowNet are all optimized together with one unified loss. The loss is simply the average of the flow-matching MSE’s across all scales (weighted so that finer scales get a bit more emphasis). Crucially, during training each scale’s “previous scale” input is the ground truth (teacher forcing), so every stage sees perfect information. This makes training stable and fully end-to-end. According to the authors, this also means the scale-wise transformer can tailor its features to the flow objective directly, instead of having to rely on a possibly imperfect tokenizer as an intermediary. Autoregressive Inference: At test time, HiFlow generates actions in a scale-wise loop. First, the transformer (using just the task token for scale 1) produces a feature vector for scale 1. Then we sample a standard normal noise (x_1^{(1)}) and run a short integration of the learned velocity field from (\tau=1) down to (\tau=0). This is done with a small fixed number of Euler steps (the paper uses (N=25) steps) according to (x_{\tau-\Delta} = x_\tau - \Delta , f_\theta(x_\tau,\tau)). The result (x_0^{(1)}) approximates the true 1-step action (a^{(1)}). Now that we have scale-1 actions, we upsample to 2 timesteps and feed that (plus the task token) back into the transformer to get scale-2 features. Repeat: sample new noise of shape (2\times D), integrate those 25 steps to produce the 2 actions. Continue scale 4, then scale 8 (the full 8-step chunk). In summary, each scale requires 1 forward pass through the transformer and (N) steps through the flow network. In practice, the total forward calls is (|\mathcal{S}|*(1 + N_{\text{steps}})), which for 4 scales and 25 steps is 104 passes. This may sound like a lot, but it is comparable to or even less than the hundreds of diffusion steps some policies require, and much less than 1000! Also note that the policy still runs “in one shot” per chunk, not step-by-step in a loop. The end result is a full 8-step action sequence ready to execute. Figure 1 (below) sketches the difference between conventional tokenized AR and HiFlow. Previous “next-scale” AR policies would quantize each chunk at each scale (via VQ-VAE) and predict the discrete codes. HiFlow instead deterministically averages actions to form scales and flow-matches to generate the actual continuous actions (no cross-entropy needed). The pipeline is simpler (no codebook to train) and keeps all computations in continuous space. Why This Matters: No Quantization, One-Shot Training. The main selling points of HiFlow are straightforward but powerful:. No discrete tokenization. The authors emphasize that for robotics actions, VQ-VAE based tokenizers were an unnecessary relic from image modeling. Actions are already low-dimensional vectors; one can simply group and average them. This removes two burdens: (a) you don’t have to engineer or train a VQ-VAE (or any latent codebook) for your robot and tasks, and (b) you eliminate the quantization error that comes from replacing an action with the nearest codebook entry. In practice, this means HiFlow’s outputs can capture subtle continuous movements that discrete baselines might blur over. Single-stage training. Without tokenizers, there’s no need for a separate pretraining stage. HiFlow’s vision encoder, transformer, and flow network are all learned jointly from the raw action data. This end-to-end simplicity can ease the engineering work and hyperparameter search. It also lets the entire pipeline be optimized for the final task, rather than splitting it into a “VQ-VAE loss” and a “policy loss”. Preserving smoothness and geometry. A continuous model can naturally respect the smoothness of actions. Because HiFlow’s flow matching operates in Euclidean space and learns continuous velocity fields, it can produce trajectories that are smoother and more precise. The authors visualize this in Figure 6 of the paper: the coarse scale outputs a big straight vector (one long arrow), and as the scales refine this vector breaks into smaller segments that closely follow the true path. Each refinement is a subtle warping of the previous scale’s motions, rather than jumping between quantized codes. Efficiency with coherence. HiFlow retains the fast inference of autoregressive models (just one or a few passes through a transformer) but gains the global consistency that comes from planning with flows. In effect it merges two paradigms: like a GPT it conditions on prior context and generates in one pass, and like a diffusion it ensures the plan is globally sensible. The coarse-to-fine AR structure provides global anchoring (via the single vector at scale 1), while each flow step zeroes in on detail. This balances efficiency and performance. Flexibility. Because everything is continuous, HiFlow can handle any robot embodiment or action space without redesigning a tokenizer. If you change the robot or the action dimension, you don’t need a new VQ-VAE; just adjust the network sizes. The temporal averaging trick works for any chunk length. Overall, the authors argue that HiFlow turns the coarse-to-fine AR idea into something more natural for robots: remove arbitrary discretization and work directly with motions. They show (via experiments) that this pays off: the simplified model actually does better. Experiments: Tests in Simulation and Reality. To back up their claims, the authors performed extensive experiments comparing HiFlow to state-of-the-art baselines. The tests covered diverse scenarios: single-arm multi-task, bimanual manipulation, and even a real mobile robot. Across the board, HiFlow was at least as good or better than all contenders, often by a healthy margin. Single-Arm Multi-Task (MimicGen): This benchmark uses the MimicGen dataset (a large human-demo collection) of 8 tabletop tasks: things like cleaning up coffee mugs, hammer cleanup, nut assembly, threading a needle, etc. The robot has a 6-DoF arm (so 7 dimensions with the gripper) and is given images from a head camera and a wrist camera. Models are trained jointly on all tasks with a task ID to switch context. In computing success rates, a trial only counts if the exact task goal is accomplished. Table II (from the paper) reports success rates for HiFlow vs. three baselines:. TCD (Task-Conditioned Diffusion): a recent multi-task diffusion policy (similar to SkillDiffuser, 2024). SDP (Sparse Diffusion Policy): a transformer-based diffusion that uses mixtures of experts. CARP: the coarse-to-fine autoregressor with tokenization (Gong et al., 2024). HiFlow achieved an 88% average success, beating CARP’s 85% and far ahead of the diffusion models (TCD ~68%, SDP ~78%). Notably, HiFlow got perfect scores on two tasks (Coffee and Hammer clean-up), significantly outdoing CARP. On the tricky Threading task (needle insertion), HiFlow scored 90% while CARP only managed 70%. This suggests HiFlow’s fine-grained control really helped in the precision task. Across all 8 tasks, HiFlow was the top performer, confirming that removing the tokencode bottleneck didn’t hurt anything—in fact, it helped. Bimanual Manipulation (RoboTwin 2.0): Here the robot has two 6-DoF arms (14 DoF total) and the tasks require coordination, like “click an alarm clock” (which may involve threaded sequences), moving a pot, and placing a can in a basket using both hands. The policy sees images from one overhead camera plus wrist cameras on both arms. For training, only 50 demonstrations were provided per task—a rather small dataset. Baselines were:. ACT (Action-Chunking Transformer): from Zhao et al. (2023), a vision-based imitation that predicts short action sequences. DP (Diffusion Policy): the original diffusion model from 2023. On these three RoboTwin tasks (averaging 100 trials each), HiFlow again led the pack. For example, on “place basket” HiFlow achieved 39% success, compared to 18% for diffusion and almost 0% for ACT. (Note: this task is quite hard with few demos, so all numbers are modest; HiFlow roughly doubled the diffusion baseline.) On the simpler alarm clock and pot tasks, HiFlow scored 69% and 42%, where diffusion scored 61% and 32%. ACT struggled on all, apparently because it wasn’t designed for vision-based multi-arm actions. These results show that HiFlow’s global reasoning (via scale 1 context) and precise action outputs dominated the others, even with limited training data. Real-World Mobile Manipulation: As a final stress-test, the team took HiFlow onto a real robot: Toyota’s Human Support Robot (HSR). This is an 11-DoF mobile manipulator with a torso, 6-DoF arm, parallel jaw hand, and cameras on the head and hand. They devised 5 real tasks of increasing complexity:. Object Grasping: Pick up a random object (Apple). Object Relocation: Grasp and move an object to a target spot (Mustard bottle and Coke bottle tasks). Precise Placement: Grasp and precisely place an object on a container (Ball → Dish, Orange → Plate). For each task, 50 demonstrations were collected via a leader-follower (teleoperation) setup. At test time the object and robot start pose were varied. The models competed on 12 trials per task. Baselines were Diffusion Policy (DP, same as above), ACT (chunking transformer), and CARP (coarse token AR). Table IV shows the success rates. HiFlow outperformed all competitors, sometimes by a large margin. For example:. On the Mustard relocation task, HiFlow got 83% success, versus 58% for diffusion and 58% for CARP. On Coke relocation, HiFlow 75% vs diffusion 66% vs CARP 58%. On the ball-to-dish placement, HiFlow achieved 58%, whereas diffusion was only 25% and CARP 33%. On the Orange→Plate placement, HiFlow 42% vs diffusion 25% vs CARP 25%. Even on the simplest Apple grasp, HiFlow led at 75% vs 66% (DP) and 66% (CARP). (ACT was much lower on all tasks, presumably because its teleoperation-based dataset was for fine hand manipulation, not generic objects.). These real-robot results underscore that HiFlow is not just a simulation toy. The model trained entirely in simulation was made to run on HSR (apparently via domain randomization or some bridging) and succeeded robustly on real tasks. The qualitative rollout images (Figure 5 in the paper) show that the robot could reliably pick up the target and place it. The inspectors even note “the third-person view are not included in the observation”—so the policy only sees from robot cameras, making it a fair on-robot test. Figure 4 and other parts of the paper provide visual examples of HiFlow’s trajectories in action. For instance, in one MimicGen Threading trial, HiFlow smoothly aligned and inserted a needle into a delicate slot, demonstrating the precise motion needed. In RoboTwin’s Place can basket, HiFlow had the left arm grasp the can and the right arm deftly sling the basket under it, never knocking anything over (a subtle sequence of moves). Qualitative Analysis and Ablations. Beyond raw success numbers, the authors offer some insight via ablations and visualizations. A key question is: how many scales are really needed for best performance? HiFlow’s default was ({1,2,4,8}). In a sensitivity test on the MimicGen tasks, they tried fewer and more scales. Using only ({1,8}) (one coarse global action and then full-detail) gave 84% average success, slightly below the 88% obtained with ({1,2,4,8}). Adding just one intermediate ({1,4,8}) gave 86%. Clearly, having a gradual ladder of resolutions helps. With four scales they got 88%. Going beyond (5 scales: {1,2,4,8,16}, even though the chunk length was only 8, so 16 is extrapolating) actually slightly hurt them (85%), probably because the coarsest scale of 16 is too coarse (averaging over the entire chunk multiple times) and understates nuance. The takeaway is that a smooth doubling schedule was sufficient: stepping by factor 2 at each hop yields the best refinement ladder. All configurations were fairly robust (all were in the 84–88% range), which means HiFlow does not rely on a very finicky scale choice. Figure 6 in the paper provides a nice illustration of the progressive refinement. It shows the predicted end-effector trajectory for a MimicGen task at each scale. The scale 1 panel is just a single straight arrow from start to finish (the average displacement). Scale 2 splits it into two segments. Scale 4 splits into four segments, bending them a bit, and scale 8 (the final) is the full detailed curve (fruitfully matching the true path). One can see that each extra scale adds some curvature and detail. The dotted markers on the figure show the “token boundaries” (where each scale’s outputs end). This visualization makes it clear: the model first commits to a rough line, then iteratively tweaks that line to wiggle around obstacles or align precisely with targets. In contrast, a fully tokenized approach would jump between fixed codewords at each refinement, which could never produce a new small bend not in the codebook. The continuous refinement of HiFlow neatly sidesteps that issue. Discussion: Strengths, Trade-offs, and Future Directions. HiFlow represents a neat melding of ideas from generative modeling and robotics. Its main strength is pragmatism: by noting that robot actions are already compact, the authors simplify the pipeline and get better results. Removing quantization means no “mosaic artifacts” in actions; all corrections come through as real-valued adjustments. The single-stage training is also appealing to practitioners. And quantitatively, HiFlow hit state-of-the-art on several benchmarks that mix vision and control. One might ask about the computational cost. HiFlow does require a transformer forward pass and several flow-net steps per scale. In practice, the paper used 25 Euler steps per scale, for 4 scales: that’s 104 network evaluations. If each evaluation is reasonably fast (they used fairly modern GPUs), the authors imply this is an acceptable cost—much less than a 100-step diffusion. It is slower than a vanilla autoregressive policy that would just do 8 single-step predictions, of course; so there is a speed-accuracy trade-off. For real-time robots, one would have to benchmark actual inference latency. The authors do mention using a single 4090 GPU in their tests. In many robotics setups, inference must run on either an onboard embedded GPU or offloaded with low-latency. We might guess that HiFlow could likely reach real-time speeds on such hardware (especially since the flows are small networks), but that deserves exploration. Another point is horizon. The paper’s experiments all use short action chunks (T=8 timesteps). These cover about a second of action or less (depending on the frequency). For longer tasks, one could either loop the policy or design bigger chunks. In principle, performing next-scale AR on longer horizons (T=16 or 32) is straightforward; it just adds more scales or bigger transformer. But the balance of N_steps, model size, and memory will shift. If one wanted truly long-horizon planning, one might stack the policy multiple times or incorporate higher-level skill search. The paper doesn’t test very long tasks, but given the success on short chunks, it seems a promising base for sequence chaining. One of HiFlow’s conceptual niceties is that it naturally handles rich multi-modal action distributions. Because it’s not committing to one mode (as a deterministic planner would), it can produce different plausible trajectories by rerunning with different noise. This is not explicitly highlighted in the paper, but flow models in general have that flexibility. In tasks where multiple disjoint strategies work equally well, a policy like HiFlow could by sampling yield variety. That could be useful for planning under uncertainty or for generating a small set of candidate motions. It would be interesting in future work to see if HiFlow was used in a stochastic setting (maybe to propose alternatives and refine). How does HiFlow compare to alternatives? We mentioned CARP closely. The main difference: CARP (and similar) answer “how do we get from continuous to discrete tokens?” whereas HiFlow says “let’s not do that at all.” Instead of a codebook, we use averages. Instead of classifying indices, we fit flows. This is reminiscent of FlowAR in image generation, which removed the need for a spotlessly aligned VQGAN and replaced it with a simple downscaling + flow approach. Just as FlowAR found that continuous VAEs and flow matching could simplify high-res image generation, HiFlow is the analogous insight in action space. Compared to pure diffusion policies, HiFlow has a very different flavor: an autoregressive backbone, but with a single “denoising” step (or a few) per chunk. It also shows that we can get diffusion-like coherence without 50 or 100 steps. Some may wonder: could we further reduce N_steps or make a one-step flow? Possibly, using more advanced integration or training tricks. The consistency models and learned samplers in diffusion research (consistency distillation) hint that similar speedups might be possible here too. Overall, HiFlow’s success highlights that matching the model’s design to the nature of the data pays off. Roboticists have long struggled with “how to represent actions” (should it be velocities, torques, motions in some latent space?). HiFlow suggests a practical path: treat them as what they are (numbers on a timeline) and model them directly with a continuous generative twist. Conclusion. The “Hierarchical Flow Policy” (HiFlow) demonstrates a clever unification of recent ideas into a working robot policy. By combining a coarse-to-fine autoregressive schedule with a deterministic continuous flow model, the approach sidesteps cumbersome quantization and yields a single-stage training process. In multiple benchmarks, including real robot tasks, HiFlow outperformed both diffusion-based policies and codebook-based autoregressive policies. Notable advantages include preserving the continuous geometry of actions and naturally capturing multi-modal distributions. From a practitioner’s point of view, the takeaway is clear: if you’re building a visuomotor policy that benefits from global context and fine detail, consider modeling actions as real-valued flows at multiple resolutions rather than as tokens. HiFlow shows that this not only simplifies the system but can also unlock better performance. As robots increasingly learn from data, we may see more hybrid generative policies like this – ones that build coarse plans and then refine them, all within a streamlined continuous framework.