Latest / Embodied AI 101 / SkipVLA: Let the Planner Move, Let the VLA Manipulate

Transcript

SkipVLA: Let the Planner Move, Let the VLA Manipulate · Embodied AI 101

• SkipVLA: Let the Planner Move, Let the VLA Manipulate. SkipVLA: Skipping VLA Steps with Classical Planning for Fast Robot Manipulation • Kaivalya Agrawal, Md Ashiqur Rahman, Raymond A. Yeh, Zachary Kingston • Purdue University, Department of Computer Science • arXiv preprint • 2026. The expensive part is not always the difficult part. Imagine watching a robot pick up a block and put it in a box. There are moments when you genuinely want an expressive learned controller: approaching an imperfectly localized object, adjusting the fingers, establishing a grasp, and releasing the object without upsetting its surroundings. Then there is the journey between those moments. If the destination is already known, should a large vision-language-action model keep generating every increment of that journey? Or should the robot hand the problem to a motion planner, execute a geometrically valid trajectory, and bring the learned policy back when interaction becomes interesting?. That is the central proposition of SkipVLA. The Purdue team combines a pretrained VLA with classical motion planning rather than asking either component to solve the entire manipulation problem. Its evaluation spans three VLA backbones, thirteen simulated LIBERO tasks, and three physical pick-and-place tasks on a six-degree-of-freedom YAM arm. The laboratory currently lists the work as under review. The distinction matters because this is not simply another way to compress a neural network. It is a proposal about where neural action generation belongs in the execution stack. For this episode, we will separate the reported mechanism and measurements from the engineering interpretation. The latter is where the work becomes especially useful: what should the interface between a planner and a VLA look like, what can invalidate that interface, and how would we establish that a hybrid actually improves a deployed robot?. We also need to disambiguate the headline immediately. The reported 59.6% reduction concerns task wall-clock completion time, not individual neural-network forward-pass latency. This discussion uses the September 18, 2026 revision. That is still a meaningful result. A robot operator cares about completed work, not just kernel benchmarks. But it changes the scientific question. We are investigating a different allocation of computation and control—not a claim that the underlying VLA itself suddenly runs two and a half times faster. What actually sits between language and motion. The reported execution mechanism is compact. SkipVLA ranks 3D object boxes using cross-attention to frozen VLA tokens, then assigns a top-down hover pose. FastSAM and DBSCAN supply boxes; PyRoki supports the VAMP branch. VAMP or cuRoboV2 executes transit. Arrival activates the VLA; gripper-state changes restore planning. The scorer runs once per transit phase. A useful way to understand that arrangement is to separate three questions that an end-to-end action interface can otherwise conceal. First: which part of the scene matters next?. Second: what robot configuration would make the next interaction possible?. Third: which controller should currently have authority?. These are not interchangeable questions. Correctly identifying a receptacle does not establish that an approach pose is reachable. Finding a reachable pose does not establish that the manipulation policy can succeed from it. And producing a good action does not establish that this is the right moment to stop executing another controller. This separation gives us a practical debugging vocabulary. Instead of treating every failure as “the policy failed,” we can ask whether the referent was wrong, the geometric goal was unsuitable, or the transition between controllers happened at the wrong time. Consider the perception side first. FastSAM is itself an efficiency-oriented segmentation approach: its authors reformulate segment-anything behavior around a convolutional instance-segmentation model and a subsequent selection stage. That is useful context because a fast manipulation architecture can still contain learned perception; replacing learned motion generation does not imply replacing all learned computation. For a hypothetical deployment, imagine two touching objects becoming one candidate region. The downstream selector might confidently choose that region while still receiving an unusable geometric description. Alternatively, a thin handle might be omitted even though the object identity is correct. Neither error is naturally diagnosed by inspecting action-model loss. I would therefore treat candidate quality as a separate measurable variable. Can the system represent the relevant object at all? Does its geometry support the intended approach? Does the candidate remain consistent when the arm occludes part of the scene?. The kinematic interface deserves equally explicit treatment. PyRoki provides composable tools for inverse kinematics, trajectory optimization, and related kinematic problems. Its role illustrates the distinction between specifying a Cartesian objective and finding robot configurations that realize it. Suppose two joint configurations reach the same end-effector pose. One leaves the elbow comfortably clear of the table; the other makes the next descent awkward. For a hybrid controller, those are not necessarily equivalent endpoints. The planner’s output should be evaluated as an initialization for the next policy, not merely as a successful pose solve. That is the first substantial lesson: the interface needs more meaning than “here is a target.”. Why the planner can change the compute budget. The classical component is worth taking seriously on its own terms. VAMP’s underlying research accelerates sampling-based planning through fine-grained parallelism in forward kinematics and collision checking. Its official repository reports a 35-microsecond median for a particular Panda benchmark on a consumer desktop CPU core. That is a planner benchmark—not a measured SkipVLA end-to-end planning overhead. The qualification is essential. Perception, coordinate transformations, inverse kinematics, trajectory preparation, and communication can all sit outside the search routine we benchmark. A remarkably fast planner does not automatically make an entire planning service remarkably fast. The alternative planner represents a different systems trade-off. cuRoboV2 brings dynamics-aware motion generation together with depth-fused distance fields and optimization machinery for high-degree-of-freedom robots. Its formulation explicitly includes considerations beyond finding a collision-free geometric path, including robot dynamics. For an implementation, I would compare the complete pipelines rather than declare one algorithmic family the winner. A CPU planner might leave accelerator capacity available for perception. A GPU-based optimizer might better match the constraints or scene representation already used by the robot. Those are hypotheses to profile on the target machine. Here is a deliberately simplified timing example. Suppose a task takes ten seconds: six seconds of physical movement and four seconds of exposed inference waiting. Imagine that a hybrid removes three of those four waiting seconds while leaving the motion unchanged. Completion time becomes seven seconds, not two and a half seconds. Eliminating most inference waiting does not eliminate the physical task. Now suppose the replacement trajectory also takes a different route, or uses a different velocity profile. The time improvement has two causes: fewer waits and changed motion. That might be entirely desirable, but we should identify both. Conversely, suppose inference was already overlapping execution. Removing model calls could reduce compute consumption without proportionally reducing elapsed time. Real-time chunking research exists precisely because the execution schedule determines how much model latency becomes a visible pause. This suggests three clocks for any serious evaluation: the clock measuring network computation, the clock measuring the controller’s physical execution, and the clock measuring task completion. I would log all three. There is also a fourth quantity hiding behind those clocks: observation age. A robot can move continuously while acting on increasingly old information. Avoiding pauses is valuable, but continuous movement and responsive feedback are not synonymous. The right optimization objective is therefore not simply fewer VLA calls. It is useful work completed under acceptable sensing, control, and resource constraints. The handoff is part of the policy. The most consequential engineering question is often not how to plan. It is when to stop planning—or when to stop letting the learned policy act. Take a hypothetical grasp. The gripper closes, but the object slips between the fingers. A controller that interprets closure as successful acquisition may immediately begin transport. The motor event happened; the intended world-state transition did not. Or consider placement. Opening the gripper might release the object successfully, leave it precariously balanced, or fail to disengage it. The same actuator transition can correspond to very different task outcomes. My proposed design rule would be to distinguish an event that permits checking completion from evidence that completion occurred. That distinction does not require a large semantic model at every control cycle. It requires deciding which signals justify a more consequential transition. MPVI provides a relevant comparison. Its original paper combines classical navigation with VLA manipulation for long-horizon mobile tasks. It uses proprioceptive triggers to initiate VLM-based completion checks. Importantly, MPVI routes base navigation through the planner as a normal execution mode; it should not be characterized as merely an emergency recovery mechanism. That comparison suggests a useful experiment for an arm-only hybrid: retain cheap physical triggers, but test whether validating selected transitions reduces downstream failures enough to justify its cost. A confirmation step after every trivial release might be wasteful. A confirmation before carrying an uncertain grasp over a fragile assembly might be worthwhile. LiLo-VLA addresses another side of the boundary problem. It separates reaching from object-centric interaction, trains with perturbed approach states, and includes recovery that can return execution to the reaching module. Its emphasis is on reusable local skills and compositional long-horizon behavior. The particularly useful idea here is to evaluate a region of acceptable handoff states rather than celebrate one successful nominal pose. For our own adaptation, I would sample small changes in translation, orientation, joint configuration, and scene visibility around each proposed handoff. Then I would measure the probability that the unchanged manipulation policy completes its local job. That experiment would tell us whether the planner is delivering the robot into a robust operating region or balancing on a narrow point. It would also expose an important asymmetry. A planner can accurately reach a bad initiation state. Better geometric tracking does not fix an interface that asks the learned policy to begin somewhere it cannot handle. A more general switching mechanism could incorporate progress estimates, tactile evidence, or task-conditioned completion prediction. But I would add those only after measuring the simple system. Otherwise, it is easy to replace a transparent heuristic with a learned module whose errors are harder to diagnose. The boundary is not plumbing around the policy. In a hybrid robot, the boundary is part of the policy. What “no additional demonstrations” should mean. SkipVLA’s training procedure labels each preceding transit segment using the box center nearest the end effector at its demonstration gripper event. Only scorer components learn, through cross-entropy. The physical VLA baselines were fine-tuned using fifty demonstrations per task. This is a useful distinction between reusing data and requiring no training. The laboratory’s description emphasizes that the target predictor can be learned from demonstrations already available to the system rather than requiring a new collection effort. For a research team, that can be a very attractive bargain. Existing trajectories may contain supervision for decisions that were never explicitly annotated. But automatically extracting a label does not make that label infallible. Imagine a demonstration in which a grasped object is released near the edge of a large container. Depending on the representation, the container’s center might be farther from the end effector than another nearby candidate. Or imagine a corrective regrasp halfway through a trajectory. Treating every gripper event as an equivalent semantic boundary could assign inconsistent targets. These are hypothetical label-audit cases, not reported SkipVLA failure rates. Before changing the learning architecture, I would visualize a sample of automatically labeled transitions. That inexpensive inspection might reveal that the dominant problem is candidate association rather than representation capacity. I would also split validation by complete demonstration, not by randomly sampled adjacent frames. Otherwise, a target-selection dataset can look much larger and more diverse than its independent experiences really are. There is a broader imitation-learning issue underneath this design. Ross and Bagnell’s analysis explains why supervised action prediction does not receive independently sampled test inputs during execution: the learner’s actions influence the states it subsequently encounters, allowing errors to compound over a trajectory. A hybrid offers an interesting experimental opportunity here. We can deliberately alter how a manipulation policy arrives at an interaction without changing the policy’s weights. But I would resist the shortcut from “the planner reaches a nominal pose” to “the policy is now guaranteed to be in distribution.” A useful test would compare the observations and configurations at planner-generated handoffs with those preceding successful demonstrated interactions. More importantly, we should test downstream success directly. This makes the learning objective operational: not merely selecting the right object, but enabling a controller transition that remains reliable under the variations the deployment will encounter. Reading the results without importing the headline. The strongest physical speed example is specific. MolmoAct2’s one-block mean drops from 46.56 to 18.82 seconds, with 18/20 versus 19/20 successes. Time and energy statistics condition on success. Across the three physical tasks, average time reductions are 30.6% for π0.5 and 46.0% for MolmoAct2; corresponding Jetson Thor energy reductions are 29.1% and 52.4%. Dividing the two one-block times gives approximately 2.47, conventionally rounded to the advertised 2.5-times speedup. The 59.6% reduction and the 2.5-times figure describe essentially the same comparison, not two independent improvements. The encouraging interpretation is that a change in execution architecture can improve useful throughput without requiring a faster version of the same action generator. The cautious interpretation begins with the denominator. Successful-trial timing answers: how long did successful executions take? It does not, by itself, answer: how much robot time is required per completed job, including failures, resets, and retries?. Consider an invented comparison. Controller A completes ten of twenty attempts in ten seconds each. Controller B completes eighteen attempts in fifteen seconds each. A is faster conditional on success. Whether it is more productive depends on how long failures take and what recovery costs. Neither conditional mean contains the whole answer. For a follow-up evaluation, I would report successful completions per wall-clock budget, with failed attempts included in the accounting. I would also retain the conditional timing distribution because it helps diagnose execution behavior. These measurements complement one another rather than compete. The simulation evidence adds an important counterweight to the headline. The tested set contains ten LIBERO-Object tasks and three LIBERO-10 tasks. SmolVLA’s Object success rises from 39% to 82%. Yet π0.5 with cuRoboV2 takes 17.84 seconds against 15.69 for its baseline on the long-horizon subset. That slower configuration is valuable information. It argues against turning the architectural idea into a universal performance guarantee. A replaceable component is not necessarily a performance-neutral component. LIBERO itself was designed to study knowledge transfer across manipulation tasks and includes multiple suites with controlled differences in objects, spatial relationships, and goals. A selected subset should therefore be interpreted as a test of a particular execution regime, not as complete coverage of the benchmark’s intended variations. For my own experiments, I would stratify results by transit distance, clutter, number of interactions, and sensitivity to approach orientation. Those categories might explain where the hybrid helps more clearly than one aggregate percentage. The energy result needs a similarly careful interpretation. A compute-module measurement is not a measurement of the entire robot’s energy consumption. Before making a whole-system claim, I would add actuator and peripheral accounting. I would also separate energy from average power. A task can consume less energy simply because it finishes sooner, even if its average power barely changes. That is still useful—especially for throughput and thermal budgeting—but it is a different engineering conclusion from claiming that every computational operation became cheaper. Finally, a small physical evaluation should motivate replication rather than certainty about rare failures. A handful of changed outcomes can materially move the reported success percentage. The right response is neither to dismiss the experiment nor to treat the estimated rate as a deployment guarantee. The measurements make the architecture worth investigating. They do not remove the need to characterize the regime in which it works. Related approaches solve different versions of the problem. There are several ways to reduce the cost of a learned robot controller, and they should not be collapsed into one category. SmolVLA pursues a compact VLA architecture and also introduces an asynchronous inference stack separating action prediction from execution. Its original work explicitly treats model efficiency and execution scheduling as complementary design choices. The implication for a practitioner is straightforward: choosing a smaller backbone and choosing a hybrid controller are separate decisions. I would test them independently. A smaller model might make the planner’s overhead relatively more important; alternatively, replacing difficult transport could make a small local controller more useful. Neither outcome should be assumed. Real-time chunking attacks another problem: how to generate a new action chunk while the previous one is still executing without introducing discontinuous behavior. Black, Galliker, and Levine use an inpainting-style procedure that preserves actions already committed to execution and conditions the remaining prediction accordingly. The method can be applied without retraining the base diffusion or flow policy. It would therefore be unfair to summarize asynchronous execution as simply running stale actions and hoping for the best. The research explicitly addresses continuity across chunk boundaries. For a hybrid, this creates an obvious combination to investigate. Classical planning might handle transport while an asynchronous executor handles the learned interaction intervals. Whether that combination wins would depend on the remaining inference bottleneck and the complexity of switching between execution modes. ReKep represents a different division of labor. It uses foundation models to produce relational constraints over 3D keypoints, then solves for robot motion through hierarchical optimization. Its demonstrations include multi-stage and reactive manipulation without task-specific training data. That is not equivalent to using a pretrained action policy near contact. The key difference is what knowledge crosses the interface: a structured constraint description in one case, versus access to a learned interaction controller in the other. I would not infer from that distinction that optimization is inherently incapable of contact-rich behavior, or that a VLA necessarily handles contact well. Those are empirical questions about the task, representation, sensing, and controller. Together with LiLo-VLA and MPVI, these works suggest a more productive comparison than “classical versus learned.” Ask where each system places the boundary between geometric execution, semantic interpretation, and interaction behavior. Then ask what supervision that boundary requires and what happens when either side is wrong. For researchers, the interesting contribution of a hybrid is often not the existence of two modules. It is the evidence that a particular interface between them preserves the capabilities we care about while reducing the costs we can measure. How I would adapt the idea on a real robot. I would start with profiling, not integration. The first artifact should be a timeline showing sensor acquisition, preprocessing, backbone computation, action generation, controller execution, and idle intervals. If the current system is already movement-limited, removing neural calls may produce a smaller throughput gain than expected. If it repeatedly stops to infer during long approaches, the opportunity is much clearer. Next, I would define a narrow initial task family. An accessible tabletop workspace with repeatable object interactions is a better first test than simultaneously adding unfamiliar grasps, moving obstacles, articulated objects, and long-horizon reasoning. The purpose is to isolate the controller split before expanding its responsibilities. Then I would run a matched execution comparison. The useful matrix contains the original policy with synchronous execution, the original policy with asynchronous execution, the hybrid with synchronous local interaction, and the hybrid with asynchronous local interaction. LeRobot provides an RTC implementation that could serve as one implementation reference rather than requiring a new executor from scratch. Across those comparisons, I would hold robot speed limits, acceleration limits, success definitions, and observation handling fixed wherever possible. If one configuration receives a more permissive motion profile, I would report that separately. The next priority would be geometry. Before celebrating semantic target accuracy, I would check camera-to-robot calibration, robot collision geometry, joint conventions, and treatment of grasped objects. A planned motion for an empty gripper is not necessarily an acceptable motion when that gripper carries a wide object. This is also where I would make safety claims deliberately conditional. A collision check establishes something about represented geometry and the trajectory being checked. If an obstacle is missing, the scene has changed, or execution deviates from that trajectory, the original check does not settle the new situation. An illustrative failure is carrying a pan through a passage that comfortably fits the wrist but not the pan. The planner could be internally correct about the wrong collision model. Calling the result “classically planned” would not rescue the experiment. Controller authority would receive its own tests. On every transition, I would invalidate obsolete command buffers and tag newly generated actions with the mode that requested them. Otherwise, an inference call launched before a handoff could finish afterward and accidentally submit actions into the wrong phase. Those are proposed integration precautions, not claims about the authors’ implementation. For perception and learning, I would begin with two diagnostic substitutes: oracle target identity and a manually verified handoff state. If neither improves results, the bottleneck probably lies elsewhere. If either produces a large improvement, it tells us where to spend the next engineering week. I would also include simple motion alternatives. In an uncluttered task, an interpolated or locally optimized approach may be a strong baseline. It is useful to establish whether the gain comes primarily from bypassing action inference, from collision-aware search, or from a better initiation state. Finally, I would define failure behavior before increasing autonomy. What happens if no target is credible? If inverse kinematics fails? If planning times out? If the object moves during transit? Depending on the setting, the appropriate response might be another observation, a slower attempt, a controlled stop, or a return to learned control. It should be an explicit decision. For reproducibility, the Purdue listing links a paper and demonstration video. As of September 27, 2026, I could not verify a public SkipVLA implementation from the authors’ pages checked for this episode. An adaptation should therefore be described as a reimplementation unless a corresponding release is obtained. The real takeaway: choose where learning is necessary. My reading of SkipVLA is not that classical planning has defeated end-to-end learning. It is that an action policy’s deployment boundary deserves as much attention as its architecture and training recipe. A powerful model can remain valuable even when it does not control every interval. Conversely, a very fast planner is useful only when the system can provide it with an appropriate goal, a sufficiently accurate world representation, and a sensible moment to relinquish authority. For an engineering team, the practical question is therefore not “Can we insert a planner?” It is: Can we replace a substantial portion of learned execution while preserving reliable transitions into and out of the remaining learned behavior?. That question leads to measurable work: profiling exposed latency, testing handoff regions, auditing target labels, accounting for failures, and separating compute energy from whole-robot consumption. It also suggests a research direction that scales beyond this particular implementation. Instead of asking a generalist policy to be uniformly responsible, we can investigate which decisions genuinely benefit from its learned knowledge and which are better expressed as explicit constraints. The attractive future is not a robot that reasons less because it is less capable. It is a robot that spends its expensive learned computation where that computation actually changes the outcome.