Latest / Embodied AI 101 / VLAs in the Wild: What 142 Papers Really Say About Generalist Robot Intelligence
Transcript
- VLAs in the Wild: What 142 Papers Really Say About Generalist Robot Intelligence. Systematic Literature Review of Vision-Language-Action Models for Generalist Robots • Umair Cheema, Youakim Badr, Thao Minh Le, and Katie Fitzsimons • Pennsylvania State University • Robotics (MDPI) • 2026. The Field Needed an Atlas, Not Another Leaderboard. Vision-language-action models have become the default answer to one of robotics’ oldest questions: how do we build a machine that can understand an open-ended instruction, perceive an unfamiliar scene, and produce useful physical behavior without training a new policy for every task?. The attraction is obvious. A pretrained vision-language model already contains broad semantic priors about objects, relationships, tasks, and human language. If that model can be connected to a robot’s state and action space, perhaps web-scale knowledge can compensate for the relative scarcity of robot experience. Instead of teaching a manipulator every possible cup, drawer, cleaning tool, or linguistic paraphrase, we adapt a multimodal foundation model and let transfer do some of the work. But the label “VLA” now covers a remarkably heterogeneous collection of systems. Some generate quantized joint deltas as language-like tokens. Others attach a diffusion or flow-matching controller to a vision-language backbone. Some are monolithic end-to-end policies; others are modular stacks containing object detectors, segmenters, planners, grasp generators, and classical controllers. Some act at tens of hertz, while others call a large model only when a high-level plan must be revised. Cheema and colleagues attempt to impose structure on this landscape. Their review, published on August 17, 2026, applies the PRISMA methodology to 142 studies selected from literature published between January 2020 and August 2025. The authors present it as the first PRISMA-compliant systematic review dedicated specifically to VLAs, while also surveying adjacent methods for constructing generalist robot policies. The paper’s most useful contribution is not a declaration of which VLA wins. The evidence is far too heterogeneous for that. Its value is as an atlas: a map of architectural choices, training recipes, action representations, optimization methods, embodiments, benchmarks, and unresolved engineering constraints. And the dominant message of that map is that “the VLA architecture” does not exist. VLA is a design space. What Counts as a Vision-Language-Action Model?. The review adopts a broad functional definition: a VLA receives multimodal inputs that include vision and language and produces robot actions. Those actions may be low-level motor commands, temporally extended action chunks, symbolic skills, waypoints, trajectories, or high-level task plans. The policy may additionally consume proprioception, action history, depth, point clouds, force signals, tactile observations, or audio. In ordinary robot-learning language, the policy conditions an action on some combination of the current visual observation, the language instruction, the robot state, and possibly a history of previous states and observations. The important point is that the word “action” is underspecified. Selecting the skill grasp the red mug and predicting the next 50 joint commands both satisfy the review’s definition, even though they pose very different learning and control problems. It is therefore helpful to think about VLA systems along three largely independent axes. The first is semantic abstraction. At one extreme, the model emits task descriptions or subgoals and delegates execution to existing skills. At the other, it directly predicts end-effector or joint-level commands. Between those extremes are systems that produce poses, object-centric waypoints, short trajectories, or skill tokens. The second is temporal abstraction. A model may predict one action at a time, a fixed-length action chunk, an entire motion segment, or a long-horizon sequence of subtasks. This choice affects latency, feedback, smoothness, compounding error, and the model’s ability to respond to disturbances. The third is system integration. A monolithic model attempts to learn most of the perception-to-action mapping jointly. A modular architecture uses separate components for semantic reasoning, geometric perception, planning, and reactive control. The review also identifies dual-system designs in which a computationally expensive model performs deliberative reasoning while a smaller policy handles fast low-level behavior. This broad definition gives the paper substantial coverage, but it also creates a comparison problem. A modular surgical-instrument handover pipeline built from speech recognition, detection, segmentation, and an action predictor is classified alongside end-to-end foundation policies. Autonomous-driving and UAV-planning systems appear in the same taxonomy as tabletop manipulation models, even though the review excludes vision-language navigation because its stated focus is manipulation. That boundary is pragmatic, but conceptually porous. The practical lesson is that “VLA” should be treated as a family resemblance, not a single model class. When a paper claims improvement over another VLA, the first question should be whether the two systems solve the same control problem at the same abstraction level. How Systematic Is the Systematic Review?. The authors began with five research questions. They asked what policy-learning methods exist alongside VLAs, what terminology the field uses, what the state of the art looks like in architecture and training, how VLA inference is being optimized, and which open challenges remain. Their search identified 1,722 records. After removing 465 duplicates, 1,257 titles and abstracts were screened. The authors examined 256 studies more closely for eligibility and ultimately included 142. For each included study, they attempted to extract architecture, model size, inference speed, robot platform, performance on seen and unseen tasks, evaluation metrics, datasets, benchmarks, optimization techniques, challenges, and future directions. Missing values were left blank rather than inferred. This is a meaningful improvement over a conventional narrative survey. The search strategy, inclusion criteria, screening flow, and extracted variables are documented. The supplementary material includes the search strategies, included-study metadata, code repositories, quality criteria, and per-study quality scores. The review also explicitly considers publication and availability bias, noting that positive results, high-profile preprints, and systems with public models are more likely to be visible. There are, however, important caveats. Initial title and abstract screening, full-text eligibility assessment, data extraction, and the post hoc quality assessment were performed by one reviewer. The other authors subsequently reviewed the evidence tables, synthesis, and conclusions through iterative discussion, but there was no independent duplicate screening and no inter-rater agreement statistic. The quality assessment was also added after the original review plan rather than specified prospectively. The resulting quality statistics are among the paper’s most revealing findings. About 92% of studies adequately described their architecture, and 75% provided sufficient training details. Only 60% adequately reported evidence about generalization. Just over half received a high reproducibility assessment, even though 64% released code. Robustness was adequately evaluated in 47%, and only 48% clearly documented limitations or failure cases. In other words, the field is good at explaining what was built and substantially weaker at proving where it works, where it fails, and whether another laboratory can reproduce the result. That should shape how we interpret every architectural trend in the remainder of the review. There is also a significant time boundary. The database search began in March 2025 and was updated through a free search on August 13, 2025. The review explicitly excludes publications from the remainder of 2025 onward, even though the final paper appeared on August 17, 2026. In a field moving at the pace of foundation-model robotics, that roughly one-year publication lag is not a footnote; it is part of the result. The Architecture Space Is Really Four Design Decisions. Where the semantic prior comes from. The review reduces VLA training to three broad approaches: train a policy from scratch on robot data, fine-tune a pretrained VLM using robot trajectories, or co-train on robot data and general vision-language data. Training from scratch offers architectural freedom and may avoid dragging irrelevant linguistic capacity through a control loop. But robot-only training cannot easily reproduce the semantic coverage available in web-scale pretrained models. Pure fine-tuning is more efficient, yet risks narrowing or forgetting the original model’s broader capabilities. Co-training is the central VLA proposition. RT-2 made this recipe concrete by mixing robot trajectories with internet-scale vision-language tasks and representing quantized robot actions in the model’s token vocabulary. The same autoregressive machinery could therefore answer a visual question or emit an action sequence. In 6,000 evaluation trials, RT-2 demonstrated improved object and instruction generalization as well as simple forms of semantic reasoning not explicitly represented in its robot demonstrations. The review consequently identifies co-fine-tuning as more promising for generalization than robot-only fine-tuning. It also emphasizes that data diversity can matter more than raw trajectory count and that broad cross-embodiment pretraining is most effective when followed by post-training on the target domain. This is not evidence that arbitrary web data automatically improves control. The transfer works when semantic priors are connected to action-relevant visual concepts and when the training mixture preserves both capabilities. The research problem is increasingly one of data-mixture design: how much semantic data, how much robot data, which embodiments, what action normalization, and which auxiliary targets?. Open X-Embodiment changed that conversation by standardizing datasets contributed by 21 institutions and spanning 22 robot embodiments. Its RT-X experiments showed that experience gathered on one platform could provide positive transfer to others. OpenVLA then trained a seven-billion-parameter open model on approximately 970,000 Open X-Embodiment episodes, using fused DINOv2 and SigLIP visual features with a Llama-family language backbone. Octo provides a different open baseline. It is a transformer-based diffusion policy pretrained on 800,000 Open X-Embodiment episodes, accepts language or goal-image conditioning, and was designed to accommodate new observations, action spaces, and embodiments through relatively lightweight fine-tuning. Its evaluations covered nine platforms, including force-torque inputs and bimanual settings. Together, Open X-Embodiment, Octo, and OpenVLA shifted the question from whether cross-robot pretraining is possible to how heterogeneous robot experience should be represented and balanced. Whether actions should be language-like tokens. Discrete action tokenization is attractive because it turns control into a familiar next-token prediction problem. Continuous action dimensions can be normalized, divided into bins, and mapped to reserved vocabulary symbols. The model can then use one training objective and one decoder for text and action. That conceptual simplicity has costs. Multi-dimensional actions require several output tokens per control step. Autoregressive generation adds latency, and independent quantization can introduce discontinuities. Predicting one step at a time also asks the large semantic backbone to run at the control frequency, even when consecutive frames and actions are nearly identical. The alternative is to treat action generation as continuous conditional modeling. Diffusion policies begin with noisy action trajectories and iteratively denoise them. Flow-matching models learn a continuous transformation from a simple source distribution into a distribution over useful action chunks. Both naturally represent multimodal trajectories and can generate temporally coherent sequences rather than isolated commands. The review sees diffusion and flow-matching heads as especially promising, while carefully grounding that conclusion in particular evaluation settings rather than a universal controlled comparison. It also finds that historical visual and proprioceptive context, continuous actions, and chunked prediction generally outperform purely reactive one-step discrete policies in the studies that test those choices. Physical Intelligence’s π₀ is the clearest expression of this direction. It combines a pretrained VLM with a flow-matching action expert and trains across single-arm, dual-arm, and mobile-manipulator datasets. The model targets dexterous behaviors such as laundry folding, table cleaning, and box assembly. π₀.₅ extends the recipe with heterogeneous co-training over robot data, web data, object detections, high-level semantic predictions, and low-level actions, demonstrating long-horizon manipulation in previously unseen homes. The deeper architectural shift is from actions as another language to language-conditioned continuous trajectory generation. The VLM remains valuable for semantics, but low-level control is delegated to a head with a stronger inductive bias for smooth, correlated motion. Where planning ends and control begins. Long-horizon manipulation forces the system to operate at multiple timescales. Semantic reasoning may need to run only when the task state changes. Contact-rich execution may require feedback at tens or hundreds of hertz. Trying to satisfy both requirements with one autoregressive decoder is computationally awkward and operationally brittle. This motivates the review’s monolithic-versus-modular and System-1-versus-System-2 distinctions. A deliberative model can decompose “clean the kitchen” into object-centered subtasks, while a fast visuomotor controller executes each segment. The planner can be called again when the environment changes, uncertainty grows, or execution fails. Gemini Robotics illustrates the large-model end of this spectrum: Google DeepMind describes it as a VLA built on Gemini 2.0 with physical actions added as an output modality. GR00T N1 and N1.5 illustrate an explicitly split architecture for humanoids, where an Eagle VLM encodes language and visual observations and a diffusion transformer processes state and noisy action trajectories. Hierarchies do introduce interface errors. A high-level plan may be semantically sensible but geometrically infeasible. A low-level controller may finish its chunk in a state that violates the planner’s assumptions. The critical research problem is therefore not merely decomposing the system, but designing representations, confidence signals, termination conditions, and recovery mechanisms that cross the planning-control boundary. How much of the physical world reaches the model. Most VLAs remain visually dominated. The review finds extensive use of RGB images and proprioception but much less work on speech, force, tactile sensing, depth, and point clouds. It catalogues emerging systems that incorporate direct speech input, six-axis force feedback, tactile observations, RGB-D geometry, and point-cloud-derived spatial features. This matters because many manipulation failures are not semantic failures. A model may correctly identify the mug and understand the instruction but still lack the depth accuracy, contact estimate, friction information, or state history needed to grasp it reliably. The paper highlights improved spatial performance when OpenVLA combines DINOv2 and SigLIP features, as well as benefits from fine-tuning rather than freezing the vision encoder. It also reports that depth and point-cloud information can improve robustness to altered layouts and unseen tasks. For precision manipulation, “vision-language-action” may ultimately be too narrow a label. The useful agent is closer to a vision-language-state-touch-force-action model, with the fusion mechanism chosen according to control timescale and sensor reliability. Evaluation Is the Field’s Largest Unpaid Debt. The review identifies LIBERO, SIMPLER, and CALVIN as the most frequently used VLA benchmarks. LIBERO separates spatial, object, goal, and broader lifelong-learning task suites. SIMPLER provides simulated environments intended to track real-robot manipulation performance. CALVIN emphasizes language-conditioned, long-horizon task sequences. These benchmarks are useful, but they do not create a common experimental regime. Papers use different subsets, demonstration budgets, visual inputs, action spaces, control frequencies, reset procedures, random seeds, and definitions of success. Real-robot experiments vary even more dramatically. The review lists task success, plan success, completion time, inference latency, control frequency, memory consumption, and computational cost as relevant metrics. Yet success rate dominates actual reporting, while deployment metrics are often absent. A single success number compresses away the information a roboticist most needs. It does not say whether the model failed semantically, selected the wrong object, collided, lost the grasp, timed out, produced unstable commands, or entered an unrecoverable state. Nor does it distinguish autonomous success from success after retries, interventions, or planner resets. Long-horizon tasks make this especially severe. If every primitive succeeds 95% of the time, a sequence requiring 20 flawless primitives has only about a 36% chance of completing without failure. Generalist performance therefore depends as much on detection and recovery as on average single-skill accuracy. The term “generalization” also requires decomposition. At minimum, evaluations should distinguish novel objects, novel visual scenes, linguistic paraphrases, new task compositions, altered geometry, changed dynamics, different embodiments, and longer horizons. A model that transfers between backgrounds is not necessarily one that transfers between grippers, and neither result implies robust task composition. The hardware distribution reinforces this problem. Most reviewed experiments use single-arm or bimanual research platforms. Quadrupeds and humanoids are rare, and the review found little work on inexpensive hardware beyond SmolVLA experiments with the SO-100 and SO-101 arms. Consequently, the literature currently measures benchmark-relative breadth more reliably than real-world generality. Deployment Is Now a First-Class Learning Problem. A robot policy operates inside a deadline. Its useful compute budget is determined not only by mean inference time but by worst-case latency, jitter, sensor synchronization, actuation frequency, power, thermal constraints, and the behavior of the system when a deadline is missed. The review devotes an entire section to this issue. It catalogs quantization, pruning, knowledge distillation, early exit, dynamic layer activation, action chunking, token compression, caching, parallel decoding, small backbones, and non-transformer alternatives. Some methods reduce the size of the semantic backbone. Quantization stores weights and activations at lower precision. Structured pruning removes consistently unimportant computation. Distillation transfers behavior from a large teacher into a smaller policy. Early-exit systems stop processing once an intermediate representation is judged sufficient, while mixture-of-layers approaches dynamically select only part of the network. Other methods reduce how often the backbone must run or how much it must emit. Action chunking predicts multiple future commands in one pass. Parallel decoding avoids token-by-token generation. Compression schemes represent an entire chunk with fewer tokens. Visual-token caches exploit the fact that most of a robot’s scene remains unchanged between consecutive frames. Dual-system architectures assign fast, intuitive behavior to a lightweight controller and reserve the VLA for difficult or ambiguous decisions. These are not merely inference tricks. Each changes the closed-loop policy. Longer chunks improve throughput but reduce opportunities for feedback. Caching assumes limited scene change. Early exit requires a calibrated termination criterion. Aggressive quantization may preserve benchmark success while degrading rare contact-sensitive behaviors. The correct deployment target is therefore not “smallest model” or “highest average frequency.” It is the smallest system that preserves acceptable closed-loop behavior under disturbances, uncertainty, and hard real-time constraints. The review also notes that most optimization work remains transformer-centric, leaving structured state-space models and other recurrent alternatives relatively underexplored. The Open Problems Are Increasingly Systems Problems. The first unresolved problem is still robot data. Web-scale semantics are abundant, but action data are expensive, embodiment-specific, temporally correlated, and dependent on camera calibration, control mode, morphology, and operator quality. Pooling trajectories does not automatically produce useful transfer. The field needs better representations of embodiment, action semantics, sensor configuration, task phase, and demonstration quality. Second is physical robustness. The review’s quality assessment found that fewer than half of the included studies adequately tested robustness. Background changes, distractor objects, illumination, camera viewpoint, clutter, occlusion, and altered layouts remain underreported. Third is long-horizon autonomy. Planning a plausible sequence is not enough. A useful generalist robot must detect that a drawer did not open, infer why, reposition, retry with a different strategy, and know when to ask for help. Failure recovery needs to become part of the learned policy and part of the benchmark, rather than a manually scripted exception. Fourth is contact-rich multimodality. Speech is useful for natural interaction, but audio must handle accents, noise, interruptions, and ambiguous references. Tactile and force sensing can reveal slip and contact state, yet they operate at higher bandwidths than most VLM backbones. The likely solution is not to append every raw modality to one token sequence. It is to design modality-specific encoders and controllers whose information is exchanged at appropriate timescales. Fifth is safety and security. The review finds these topics largely absent from experimental protocols, despite the possibility that vulnerabilities in a VLM or other upstream component can affect physical action. For deployed systems, a meaningful threat model must extend beyond adversarial images. It should include visual prompt injection, malicious language instructions, compromised demonstrations, corrupted sensor streams, unsafe tool-use requests, cloud connectivity failures, and planner-controller disagreement. Safety evaluation should report constraint violations, collision severity, stop distance, force limits, uncertainty-triggered abstention, and recovery behavior—not just successful task completion. Sixth is accessible embodiment coverage. A policy evaluated only on expensive research hardware may depend on sensing accuracy, calibration, stiffness, and compute unavailable to consumer robots. Running on lower-cost systems is not a simple compression problem; it introduces noisier state estimates, cheaper cameras, more backlash, less repeatable actuation, and tighter power budgets. Finally, the field lacks a shared definition of generalist. A model might be broad across tasks but narrow across robots, broad across objects but narrow across contact modes, or broad across language while remaining fragile to camera shift. Generality should be reported as a profile, not awarded as a binary label. What This Review Gets Right—and What It Cannot Establish. The paper is strongest when treated as a structured inventory. Its tables make visible just how many design dimensions are being explored: discrete and continuous actions, 2D and 3D perception, single-arm and bimanual systems, task planning and low-level control, imitation and reinforcement learning, monolithic and modular architectures, and a rapidly growing catalogue of deployment optimizations. Its emphasis on reporting quality is equally valuable. The central scientific obstacle is not a lack of impressive demonstrations. It is that the demonstrations are difficult to compare and often inadequately characterized. The statistics on code release, robustness testing, generalization evidence, and documented limitations turn that concern into a concrete result. The review cannot, however, support a clean ranking of architectures. Claims that flow matching, diffusion heads, co-training, depth, or action history improve performance are plausible and often well supported within individual studies. But the underlying comparisons usually involve different datasets, parameter counts, demonstrations, robots, action spaces, and evaluation protocols. This is descriptive synthesis rather than a meta-analysis of controlled effect sizes. The breadth of the VLA definition also blurs technically meaningful distinctions. A system using an LLM to choose among verified skills has different failure modes from a model directly emitting joint commands. A modular stack with external segmentation and grasp estimation has different data requirements from an end-to-end policy. Grouping them is useful for discovery but less useful for causal conclusions. The review’s publication timeline introduces another unavoidable limitation. Its corpus ends on August 13, 2025, while the paper was published on August 17, 2026. The authors explicitly acknowledge that later work is absent. Readers should therefore use it as a systematic account of the field up to its search cutoff, not as a complete catalogue of the models available at publication. Even with these caveats, this is a useful reference precisely because it does not reduce VLA research to a handful of flagship models. It captures the surrounding engineering work—caching, compression, multimodal sensing, clarification, federated learning, safety constraints, and embodiment adaptation—that will determine whether foundation policies leave the laboratory. A Practical Blueprint for Building or Adapting a VLA. For a research team deciding how to use this literature, the review suggests a pragmatic sequence of decisions. Define the intended generalization profile. Specify whether the policy must transfer across objects, scenes, instructions, task compositions, embodiments, or dynamics. Also specify the required horizon, control frequency, and recovery behavior. “General-purpose manipulation” is not an actionable evaluation target. Choose the planning-control boundary before choosing the backbone. If the task is slow and semantic, direct token generation may be sufficient. If it is contact-rich or high-frequency, use the VLM for conditioning or planning and attach a continuous action expert or dedicated reactive controller. Do not force a language model to occupy a control timescale it cannot reliably meet. Preserve semantic priors deliberately. Mixed co-training can retain web-derived knowledge better than narrow robot-only fine-tuning, but the mixture should reflect the intended capability. Spatial language, object relationships, affordances, and task decomposition are more relevant than generic language modeling alone. Pretrain broadly, then post-train narrowly. Cross-embodiment data can provide useful initialization, while target-robot demonstrations calibrate camera geometry, action conventions, dynamics, and task precision. Keep explicit metadata for embodiment and control mode instead of assuming one normalization erases those differences. Evaluate a matrix, not a headline score. Report seen and unseen tasks, object and scene shifts, paraphrased instructions, distractors, lighting and viewpoint changes, latency distributions, memory, control frequency, interventions, retries, constraint violations, and categorized failure modes. Include at least one test that was not used repeatedly during development. Co-design deployment and safety. Quantization, chunk length, asynchronous inference, runtime monitoring, and fallback control should be selected together. A deployable VLA should expose uncertainty, permit interruption, validate high-level plans against geometric and safety constraints, and degrade into a known safe mode when the learned policy becomes unreliable. This blueprint points toward a hybrid architecture: a semantically capable pretrained model; a temporally appropriate action generator; explicit state, geometry, and contact information; target-domain post-training; and a safety-aware runtime around the learned components. That may be less aesthetically pure than one transformer from pixels to torques, but it is more aligned with the evidence summarized by the review. The Bottom Line. Cheema and colleagues show that VLA research is converging on several recurring ideas: use pretrained vision-language knowledge, train across diverse robot experience, preserve history, predict temporally coherent action chunks, separate deliberation from high-frequency control when necessary, and optimize the whole system for on-robot execution. What has not converged is the evidence. Benchmarks remain fragmented, generalization is inconsistently defined, robustness and safety are underreported, and most experiments still occupy a narrow band of research-grade manipulation hardware. So the review’s most important conclusion is not that VLAs have solved generalist robotics. It is that they have given the field a common interface through which semantics, perception, control, data scaling, and embodiment can finally be studied together. The generalist robot will not emerge from a language model alone. It will emerge when semantic priors, physical sensing, temporal abstraction, data mixtures, control bandwidth, recovery, and safety are designed as one system. This review gives us a valuable map of that system—and an equally valuable record of how much of it remains unfinished.