Latest / Embodied AI 101 / Qwen-VLA: A Generalist Vision–Language–Action Robot Model

Transcript

Qwen-VLA: A Generalist Vision–Language–Action Robot Model · Embodied AI 101

• Qwen-VLA: A Generalist Vision–Language–Action Robot Model. Modern robotics and embodied AI are moving toward generalist models that can handle many tasks with one network. Traditionally, robots have separate controllers for manipulation, navigation, or prediction, each trained on narrow data. This fragmentation limits how well a robot can generalize. Qwen-VLA, a new model from Alibaba’s Tongyi Lab, embraces a different philosophy: it unifies manipulation, navigation, and trajectory-prediction tasks in a single architecture. Built on the Qwen3.5 vision-language model and augmented with a diffusion-based action generator, Qwen-VLA can control many different robots (“embodiments”) by using text prompts that describe each robot. According to the authors, this unified model matches or beats specialist policies on diverse benchmarks – all without any per-task fine-tuning (that is, the same pre-trained model is applied to every task). In this article we dive deep into Qwen-VLA’s design and results. We’ll explain how the model works, what data and tasks it sees, and why its “embodiment-aware prompting” enables it to run the same policy on eleven different robots. We’ll describe the multi-stage training recipe that brings vision, language, and control together, and review its performance on standard robot benchmarks. Along the way, we’ll compare Qwen-VLA to related work like Stanford’s OpenVLA or Toyota’s VLA Foundry. By the end, you’ll understand how Qwen-VLA pushes the idea of a foundational robot policy further than ever before. Background: Vision–Language–Action (VLA) Models. The term Vision–Language–Action (VLA) model refers to an agent that takes visual observations and textual instructions (vision-language inputs) and produces actions (e.g. robot controls). This is a natural extension of vision–language models like CLIP or LLaVA into the realm of control: instead of outputting text, a VLA outputs motor commands or trajectories. In recent years, researchers have built several partial solutions toward generalist VLAs. For example, Stanford’s DUET/OFF-THE-SHELF projects (RT-1, RDT-1) showed that large generalized policies can perform many manipulation tasks by fine-tuning on mixed data. Toyota’s “VLA Foundry” collection trained a 2.1B model (Qwen3-VL-2B visual backbone + diffusion action head) on large simulated/real grasping data. Stanford’s OpenVLA (7B parameters) was trained on nearly a million robot episodes and demonstrated strong zero-shot skill across multiple robotic platforms. At the same time, academic work like π₀ (pronounced “pi-zero”) designed a “flow-matching” transformer policy that can zero-shot follow language instructions after massive pretraining. In π₀’s approach, a pretrained vision–language model (VLM) is combined with a diffusion/flow-based decoder for actions. Other research (e.g. RynnVLA) even integrates a world model to predict future observations as well as actions. The community is converging on two ideas: (1) leverage Internet-scale VLMs to add semantic knowledge, and (2) use generative decoders (diffusion/flow) to output continuous multi-step actions. Qwen-VLA follows this philosophy but scales it up: it uses a natively multimodal large model (Qwen3.5) as the vision-language core and adds a 1.15B-parameter diffusion (“DiT”) decoder to produce actions. Crucially, it is trained once on a mixture of many task domains and robot types, so the single model can handle all of them. In their own words, the authors emphasize that Qwen-VLA aims to break down the “fragmented” approach of one-model-per-task. Instead, they study “whether heterogeneous embodied decision-making problems can be unified within a single vision–language–action model.” The result is a unified embodied foundation model that extends Qwen’s vision-language stack with continuous action generation. In practice this means Qwen-VLA accepts images and text instructions, and emits either continuous robot controls or predicted trajectories, depending on the task. Let’s look closer at how it does this. Model Architecture: Qwen3.5 Backbone + DiT Action Decoder. At the core of Qwen-VLA is Qwen3.5-4B, a 4-billion-parameter multimodal large language model from Alibaba. Qwen3.5 is part of the Qwen family, an open-source series of models that natively handle text and images (and even code/audio). It uses an efficient mixture-of-experts design for large contexts, but from the perspective of a robotics researcher all you need to know is: Qwen3.5 can take an image (or a sequence of images) plus text instructions and produce text tokens or embeddings that capture the scene and command. To turn these embeddings into actions, Qwen-VLA attaches a specialized action decoder. This action decoder is 1.15 billion parameters and uses a diffusion/flow-based transformer architecture (often called a “DiT” or Diffusion Transformer head). The paper does not explicitly explain the DiT acronym, but it refers to prior work on “Scalable Diffusion Models with Transformers” (Facebook’s DiT). In this context, it means the model is trained to generate sequences of continuous actions or waypoints in one shot via a diffusion-style process. Concretely, during training the decoder learns to map random noise to the ground-truth action sequence that accomplished a task, given the vision-language context. At inference time, the model can sample from this learned flow distribution to produce an action plan. Why use diffusion/flow rather than a simple autoregressive or direct regression decoder? Two reasons: First, it lets the model generate an entire action sequence at once (e.g. a 10-step trajectory) instead of step-by-step. This can be more efficient and can capture dependencies across the sequence. Second, diffusion/flow training tends to produce sharp trajectories and stable generation, as we have seen in prior works like π₀. In fact, the Qwen-VLA action head is conceptually similar to π₀ and the Toyota Foundry models, which also used flow-matching decoders on top of vision-language backbones. Putting it together, Qwen-VLA’s pipeline works roughly like this: an input image (or video frames) is processed by Qwen3.5’s vision encoder to get embeddings. The text instruction (what to do) is also tokenized and passed through Qwen3.5’s language archive. These are fused (as in any vision-language model). Then, instead of a language token output, we attach the DiT decoder. The DiT head receives the Qwen embedding (plus a noise vector) and gradually “denoises” it into a full sequence of action vectors. Each action vector could encode, for example, XYZ velocity, orientation change, and gripper command. The output is a continuous multi-step control signal that the robot can execute. Importantly, this architecture is task-agnostic: the same pipeline can produce a pick-and-place arm motion, or a mobile navigation path, depending on the instruction. One subtlety is how the model “knows” what kind of robot it’s controlling. Here Qwen-VLA’s key innovation is embodiment-aware prompting. Instead of having a separate network for each robot, the model is conditioned on a brief text description of the robot’s body and control scheme. For example, a prompt might include “You are controlling a Franka Panda manipulator with 7-DoF, equipped with an eye camera and wrist camera” or “You are controlling the ALOHA mobile robot with a gripper, dual-arm and base”. This prompt is prepended (or appended) to the instruction so that Qwen3.5’s attention can tailor its outputs. Effectively, the language model treats information like “6-DoF arm on a wheeled base”, “friction zones”, or “camera on gripper” just as it would any other world description. In training, each demonstration or simulated episode includes the appropriate embodiment text. At test time, specifying the robot in text tells the model how to interpret the scene and how to format actions. The authors call this embodiment-aware prompt conditioning. It lets one network flip between multiple bodies. Figure 1 (conceptual) illustrates this: a single Qwen-VLA model is shown with multiple inputs like desk scenes or maze views, each paired with an instruction like “move to the kitchen” or “put the red block in the bowl”. In addition, each example includes a textual hint like “Robot: WidowX robot, 6-DOF arm”. Because this text travels through the same transformer as the images and commands, the model adjusts its policy accordingly. Later we’ll see that Qwen-VLA supports 11 different embodiments (various arm robots and mobile bases) using this trick. In summary, Qwen-VLA’s architecture is essentially a multi-modal LLM (Qwen3.5) augmented with a continuous action generator, plus a clever prompting scheme. This design unifies perception, language understanding, and control in one stack. Next, let’s discuss how it is trained to tie all these pieces together. Training Pipeline and Data. Training Qwen-VLA is a multi-stage, multitask endeavor. The authors describe a four-stage training recipe that gradually builds the model’s capabilities. While the paper’s details are complex, the overall strategy can be summarized as follows:. Language-Vision Pretraining (Qwen-base): The Qwen3.5 model was already pretrained on massive text and image data (comparable to GPT-4-Vision or similar), so it starts with strong general perceptual and language skills. This step is done prior to Qwen-VLA (Qwen3.5 was open-sourced in early 2026). It learns to caption images, answer questions, and so on, but no actions yet. Multitask Joint Pretraining: In this stage, the model is trained on a diverse mixture of embodied tasks. The key idea is to present many different scenarios and let the single model learn them all at once. The authors call it “joint pretraining over diverse data sources”. Concretely, the training data includes:. Robot Manipulation Trajectories: Recorded trajectories from physical or simulated robot arms completing pick/place, rearrangement, and other standard tasks. This likely includes datasets where a robot arm was teleoperated or learned any number of tasks in cluttered scenes. Human Egocentric Demonstrations: Videos of humans performing tasks from a first-person view (like cooking or assembly). These videos are annotated and converted into action sequences relative to a robot or person. Including human demos helps the model see a wide variety of tasks and objects, albeit with a different morphology. Synthetic Simulation Data: Large-scale simulated episodes where either the robot learns through reinforcement or scripted policies operate in various virtual worlds. These allow scaling to millions of samples cheaply, covering things like motion planning in new environments or interaction with many object shapes. Vision-and-Language Navigation (VLN) Data: Datasets like Room-to-Room (R2R) and its multilingual version RxR, where the task is to navigate in a 3D environment given a language instruction (“Go to the kitchen and then turn left”). These train the model to produce navigation trajectories from language. Trajectory-centric Supervision: Possibly pre-collected datasets that focus on predicting object or agent trajectories (e.g. forecasting where a person or ball will move). This helps the model learn the notion of future motion given current observations. Auxiliary Vision-Language Data: General image–text data (like captioning or VQA) is also included to keep the model’s visual understanding strong. By training on all these sources together, Qwen-VLA learns to handle multiple modalities and tasks in one go. The training is multi-task and continual, meaning the model sees a randomly mixed batch of these different tasks and simply minimizes the respective losses. The shared network thus accumulates knowledge that can transfer across tasks (“transferable visual grounding and spatial reasoning”). Embodiment-Conditioning Fine-Tuning (Prompt Calibration): Once the core multitask model is trained, the authors fine-tune it specifically to interpret the textual embodiment prompts correctly. In practice, this might mean collecting or synthesizing data where the same visual observation and instruction is paired with different robot descriptions. For example, showing how the effect of a particular command differs if the robot has longer arms or cameras placed differently. Fine-tuning ensures that giving a prompt like “robot type A” or “robot type B” reliably changes the output action conventions. This stage solidifies the “embodiment-aware” conditioning. (The paper lumps all this under their joint recipe, but it likely involves stages focused on multi-robot data.). Instruction Tuning (Instruct Phase): The final stage is akin to how language models are tuned to follow instructions. The authors create a variant called Qwen-VLA-Instruct, where the model is further trained on data that emphasize following specific natural language commands. This might involve converting raw trajectory data into “conversations” format with system/user messages, or generating imagined dialog instructions. The output is the same – actions – but the training leverages techniques from LLM instruction tuning to make the policy more responsive to arbitrary phrasing. The reported performance numbers for Qwen-VLA are all from this Instruct-tuned version. It’s not explicitly stated how different the base model is, but this final tuning puts the model in “ready to follow user instructions” mode. Overall, this four-stage pipeline ensures Qwen-VLA has broad coverage. It starts from vision-language abilities, augments them with action supervision from many domains, clarifies how to read robot descriptions, and finally polishes the language following. It’s important to note that none of these stages involve training separate networks per task. The same model weights gradually learn all tasks. This is in contrast to traditional pipelines where one might train a vision encoder first, then a task-specific policy on top. By the end of training, Qwen-VLA has effectively seen a very large, mixed dataset: millions of manipulation trajectories (both real and simulated), thousands of navigation episodes, and so on. Experiences from one domain can help another: for example, learning to pick objects may improve understanding of spatial language that also helps navigation. The authors describe it as yielding “transferable visual grounding, spatial reasoning, and continuous action generation” across scenarios. Multi-Robot Embodiment Conditioning. A standout feature of Qwen-VLA is how it handles multiple robot embodiments. In our context, an “embodiment” means the robot’s physical body and sensing modalities – e.g., a 6-DOF table-top arm versus a mobile base with two arms and an overhead camera. Typically, one would need separate policies for each setup. Qwen-VLA sidesteps this by using textual prompts to specify the embodiment. Concretely, at train time each demonstration or episode is tagged with a brief description of the robot. For example: “Robot: WidowX 6-DOF arm with gripper; Camera: wrist-mounted; Control: Cartesian delta commands.” This textual token is fed into Qwen3.5 along with the usual language instruction. The model learns to associate that description with how it should interpret the scene and format its action outputs. At test time, if you want to run Qwen-VLA on a particular robot, you simply include the matching prompt. This works even if the model has never seen that exact robot in training, as long as the prompt is descriptive enough. By using words, Qwen-VLA reuses the same weights for all robots. The authors report supporting 11 different embodiments in their experiments (the exact list isn’t in the summary, but likely includes common platforms like WidowX, Franka Panda, UR5, Spot (mobile base), ALOHA (mobile-bimanual), etc.). For each robot, the control conventions (space of actions, frame of reference) are encoded into the prompt so that the decoder outputs a compatible action format. For instance, the model might output 7 joint angles for one robot or a 3D velocity and turn rate for a mobile robot, depending on the prompt. This prompt-conditioning approach has a powerful advantage: if a new robot is introduced, one only needs to write a sensible description of it and the general model can often adapt immediately. There’s no need to retrain. Moreover, sharing data across embodiments improves performance. For example, the way a 7-DOF arm grasps cups is similar whether mounted on a stationary base or on a mobile robot; Qwen-VLA can learn that common strategy and apply it in both scenarios. The paper likens this to “multi-robot collaboration” learning (see related work on embodiment transfer learning) but here it’s achieved through language conditioning. In practice, embodiment-aware prompting means Qwen-VLA can do things like: “As a two-arm mobile robot, arrange these objects on the shelf” or “As a slow-reacting arm, move carefully to avoid collisions.” The text hint steers the internal representation about reachability, sensor view, and so on. The results (as we’ll see) confirm that the model indeed generalizes across robots. Without such prompts, a single model would be “confused” about which action space to use. With them, all 11 robots use the same policy backbone with just a different head task. Unified Task Framework. One of Qwen-VLA’s key claims is that manipulation, navigation, and trajectory prediction are all cast under one framework. What does that mean? Essentially, every task is treated as “given an image (or sequence of images) and a text instruction, output a sequence of actions or waypoints.” The same network architectures and losses apply, regardless of the task domain. Manipulation tasks are the classic robotics problems: e.g. “pick up the red block and place it in the green bowl.” The input might include a static or time-series camera view of a table. The instruction is a free-form text command. The output is a time sequence of arm motions. Qwen-VLA handles these like an imitation learning policy. Navigation tasks (VLN) are when the agent must move through an environment to a goal. Here, the “actions” might be base movements or turning angles rather than arm joint angles. Nevertheless, Qwen-VLA outputs them the same way. For example, for Room-to-Room navigation (R2R) or its multilingual variant RxR, the model produces a forward velocity and turn angle, step-by-step, conditioned on the instruction like “go down the hallway and enter the second door on the left.”. Trajectory prediction tasks are slightly different: instead of controlling the robot, the model predicts what will happen to some object or agent. For instance, “given this image of a ball rolling, predict where it will be in the next 5 seconds.” This is essentially a forecasting problem (often framed as “action+image -> future image” tasks in related works). Qwen-VLA can do this by treating the trajectory as if it were a control signal: it generates the future path as a sequence of “pseudo-actions”. (Alternatively, some frames in training might have been labeled as “predict this object’s next pose”, with the model outputting a predicted motion sequence.). By aligning all these under “action sequence prediction”, the model shares architecture and parameters. The authors explicitly mention they “cast manipulation, navigation, and trajectory prediction into a unified action-and-trajectory framework”. In practice, this means they use a single loss function (flow matching on the action sequence) and the same DiT decoder for all tasks. The difference shows up in the training data: each sample in the dataset has a type tag (manipulation, navigation, or prediction) and the format of the output sequence is adjusted accordingly. But to the model, it all looks just like “text+image in, action-sequence out.”. This unification encourages positive transfer. For example, learning to navigate through cluttered spaces might help the model reason about spatial layouts that also arise when reaching for objects. Learning to predict how a thrown object flies might help it plan a catching motion with a robotic arm. The authors emphasize that this enables “transferable visual grounding” and “spatial reasoning” across tasks. Importantly, Qwen-VLA does not dedicate any sub-networks exclusively to one domain. There’s no switch that says “now do navigation” vs “now do manipulation.” The only signaling is in the input: the instruction itself differs. (For instance, navigation instructions often mention landmarks and movement verbs, whereas manipulation instructions mention objects and actions like “pick” or “push.” The language model part of Qwen can pick up on those cues.) Thus the same weights do all the work. This is in contrast to some prior works that trained separate policies or heads per task. For example, one could imagine a multi-task network with distinct output heads for navigation vs. manipulation. But Qwen-VLA avoids that: there is effectively one head (the DiT decoder) whose output semantics come entirely from context. The shared core network is more powerful because it can pool knowledge from every task. Experimental Results. Qwen-VLA is evaluated on a wide array of benchmarks covering all mentioned tasks. The headline numbers are impressive and consistent across tasks, suggesting the unified training really worked. We summarize the key results (all for the instruct-tuned version, “Qwen-VLA-Instruct”) below:. LIBERO (Manipulation) – Qwen-VLA achieves 97.9% success. LIBERO is a benchmark suite of diverse language-conditioned manipulation tasks (including object rearrangement, tool use, etc.). Scoring 97.9% means it solves almost every task in this suite. This essentially matches or beats the best prior numbers for generalist policies (LIBERO’s state-of-the-art was in the high 90s with huge models or multi-modal training). In practical terms, Qwen-VLA can follow free-form instructions like “open the drawer named ___ and take the blue cup” nearly perfectly. Simpler-WidowX (Manipulation) – 73.7% success. This benchmark involves an actual WidowX robot (a simple 6-DOF arm) performing tasks under varied conditions. Qwen-VLA’s 73.7% means it handles most tasks correctly. For comparison, specialized policies on WidowX in the literature often score in this range (60–80%) depending on conditions. RoboTwin (Sim Manipulation) – 86.1% / 87.2% success in Easy / Hard split. “RoboTwin” is a simulated benchmark (part of StarVLA benchmarks) for vision-language tasks. Getting in the high 80s indicates strong performance even on more challenging unseen scenarios. R2R (Vision-Language Navigation) – 69.0% Oracle Success Rate (OSR). R2R is the classic Room-to-Room dataset: the robot must follow an English navigation instruction through a simulated indoor environment. “Oracle success” means it succeeds if it stops at any point along the correct path. 69.0% is quite high for a zero-shot policy (no environment-specific fine-tuning). For reference, specialized VLN models trained on R2R sometimes get around this mark. RxR (Multilingual VLN) – 59.6% Success Rate. RxR is a trilingual extension of R2R with longer instructions. Nearly 60% success in this much tougher setting (and likely zero fine-tune) is very solid. It shows Qwen-VLA has strong language grounding. Real-World ALOHA (Bimanual Mobile Manipulation) – 76.9% average OOD success rate. ALOHA is a real bimanual mobile robot (with a Duckiebot base plus two arms) doing complex tasks like folding or scooping, often under language commands. “OOD” means they tested in out-of-distribution conditions (new object colors, clutter, etc.). Averaging 76.9% across these tasks indicates the model generalizes well from its training. This is especially notable because ALOHA was not explicitly in the training mix (it’s a low-cost platform used in Stanford’s recent work). The Qwen-VLA policy works out of the box on it via prompting. DOMINO (Dynamic Manipulation) – 26.6% zero-shot success. DOMINO is a benchmark involving fast-moving objects (e.g. swinging a tool) where the robot must react in real time. Getting nonzero performance here is actually remarkable, because the model had seen no such data and the dynamics are tricky. A 26.6% success shows that sometimes the diffusion model can predict how to intercept or track a moving target. It is still much lower than static tasks, highlighting the challenge of dynamics. (Training data for dynamic scenes was limited in Qwen-VLA’s schedule.). Across all these benchmarks, Qwen-VLA consistently excels. The paper notes that the model demonstrates “consistent multi-task performance and out-of-distribution generalization”. That means it handles variations in object layouts, lighting, backgrounds, and even different robot bodies, without breaking a sweat. The fact that nearly the same architecture scored ~98% on LIBERO and ~60–70% on R2R/RxR is evidence that the multi-domain training paid off. In many cases, Qwen-VLA’s one-size-fits-all policy even outperforms task-specific specialists. For example, in their ALOHA experiments they compare to other policies (like π₀, RDT-1B, and pure imitation learners) and find Qwen-VLA is either on par or better in out-of-distribution tests. Earlier work has shown that pretrained VLA models beat from-scratch RL or imitation baselines on tasks like ALOHA, and Qwen-VLA continues that trend with an even larger, multi-task model. Notably, Qwen-VLA did this without any domain-specific fine-tuning. That is, they used the same pretrained model and just switched prompts. In other robot benchmarks like Simpler-WidowX or RoboTwin, they likewise report using Qwen-VLA “out-of-the-box”. No new training was needed for each robot or task family – just evaluate the pretrained model. A few more observations about the experiments:. Visual and Environmental Generalization: The model was tested under variations unseen during training – different object placements, novel backgrounds, altered lighting, etc. It still performed well, suggesting the vision encoder and multimodal reasoning are robust. This matches the claim that it generalizes “under variations in scene layout, background, lighting, [and] object configuration”. Spatial and Language Reasoning: Tasks often require inferring implicit goals from language (“the cup on the left side of the white plate”). Qwen-VLA’s high success on RxR/R2R and Libero tasks indicates it can ground such language effectively. This is a benefit of using a strong language model core. Continuous Action Quality: Because of the diffusion decoder, the generated trajectories tend to be smooth and physically realistic. The authors note that Qwen-VLA can properly orient the robot’s end-effector before grasping and even recover from initial misalignments (as seen in their supplementary videos). This suggests the DiT head produces sensible motion plans, not just jittery or erratic outputs. Overall, the empirical results show that a single 5.15B parameter model (Qwen3.5 4B + DiT 1.15B) can be a universal robot policy across a wide spectrum of tasks and platforms. This is a milestone in embodied AI: rather than dozens of specialist networks, one foundation model covers it all. Discussion and Outlook. Qwen-VLA represents a significant step towards generalist robotic intelligence. By unifying multiple embodied tasks, it simplifies what would otherwise be a huge engineering effort. As one broad summary, the authors report their model can do “visual grounding, spatial reasoning, and continuous action generation” regardless of robot or task. In practical terms, this means you could deploy Qwen-VLA on a new robot by simply telling it (in text) what that robot is and asking it to accomplish any task it knows. How does Qwen-VLA compare to contemporaries? It stands shoulder-to-shoulder with the best vision-language-action systems. For example, Stanford’s OpenVLA set a high bar by showing that a 7B policy could beat many 50B models in zero-shot manipulation tasks. Qwen-VLA with 5B and multi-task training surpasses that level on almost all common benchmarks. Toyota’s VLA Foundry (2.1B) proved that even smaller VLAs can learn diverse grasping tasks. Qwen-VLA takes the same architecture style but scales it and extends it far beyond just grasping. Alibaba Labs’ own prior work (RynnVLA, etc.) also aimed for unification, but Qwen-VLA seems to be the most comprehensive effort yet, focusing purely on VLA (no world model) and achieving excellent results. Of course, some challenges remain. The zero-shot performance drops on highly dynamic tasks shows that more work is needed for fast-moving scenes (like in DOMINO). And while 76.9% success on ALOHA is excellent for a single model, there may still be edge cases where a bespoke policy could outperform it. The model also requires a lot of data and compute to train (the paper implies large-scale training on many GPUs over weeks). Because it’s so large, running inference in real-time on a robot might be non-trivial, though robot speeds for manipulation are often slow enough that high throughput may not be needed. Another question is “what’s next after Qwen-VLA?” The techniques it uses are modular, suggesting future work could extend it by adding more tasks (e.g. aerial drone control, deformable-object manipulation) or more modalities (haptic feedback, more camera views). One could also imagine continuous lifelog data collected by robots fine-tuning such a model online using its architecture. The concept of embedding robot descriptions in language opens the door to in-context learning: perhaps you could teach the model about a new robot by showing it a manual or demonstration and then it could adapt instantaneously. Finally, the success of Qwen-VLA highlights the power of vision-language foundation models in robotics. This approach – pretrain on diverse data then apply to control – mirrors trends in language (GPT) and vision (ViT/GPT-4V): large networks absorb world knowledge and can generalize widely. For robotics, it means hoping to eventually replace painstaking, task-by-task engineering with a single capable model. Qwen-VLA is a bold step in that direction. References: The Qwen-VLA results and descriptions cited here come from the authors’ arXiv report. We also referenced π₀’s flow-matching VLA approach and Stanford’s OpenVLA benchmark results for context.