Latest / Embodied AI 101 / The 100,000-Hour Bet: How Xiaomi-Robotics-1 Tries to Make Robot Scaling Real

Transcript

The 100,000-Hour Bet: How Xiaomi-Robotics-1 Tries to Make Robot Scaling Real · Embodied AI 101

• The 100,000-Hour Bet: How Xiaomi-Robotics-1 Tries to Make Robot Scaling Real. Xiaomi-Robotics-1: Scaling Vision-Language-Action Models with over 100K Hours of Real-World Trajectories • Xiaomi Robotics Team, Jun Guo et al. • Xiaomi Robotics • arXiv preprint 2607.15330 • 2026. Why this paper matters. The headline number in Xiaomi-Robotics-1 is 100,000 hours. That is deliberately meant to evoke the scale story behind language and vision foundation models: collect a sufficiently broad corpus, train a sufficiently expressive model, and capabilities should improve predictably rather than arriving one task at a time. But the most important thing to understand is that Xiaomi did not collect 100,000 hours of conventional robot teleoperation. Most of the pretraining corpus consists of real-world demonstrations recorded with Universal Manipulation Interface, or UMI, devices—portable grippers that humans move directly through the environment. The actual robot-alignment stage uses a smaller, though still enormous by academic standards, mixture of roughly 10,000 hours, including more than 7,200 hours collected with Xiaomi’s mobile manipulators and dual-arm robots. That distinction is the paper’s central idea. Xiaomi-Robotics-1 is less a story about scaling robot operation than about decoupling the acquisition of physical experience from the robot itself. A portable demonstration device supplies breadth. Automatic language annotation turns that experience into conditional policy supervision. A separate post-training phase then maps the resulting physical prior onto actual robot bodies and the imperative commands people naturally issue. The report first appeared on July 16, 2026, and its current arXiv version was posted on July 22. On August 3, Xiaomi released the 5B checkpoint, post-training and deployment code, example data, and several benchmark-specific checkpoints under Apache 2.0. As of August 10, this is therefore more than a paper announcement, although the public release does not include the 100,000-hour corpus or the complete pretraining pipeline. The paper’s strongest claim is not simply that a large VLA performs well. It is that better pretraining—in the form of more UMI data and larger models—produces better real-robot behavior after embodiment and instruction alignment. If that relationship continues to hold, robotics has something resembling a data flywheel. The 100,000-hour headline, decoded. Robot learning has traditionally tied data collection to a particular machine. A teleoperator controls a specific arm, gripper, camera stack, and control interface. Collection is expensive, hardware utilization is poor, and the resulting distribution often contains many repetitions of a small number of tasks in a small number of spaces. UMI changes the economics. Its handheld grippers combine human dexterity with robot-like observations and relative end-effector trajectories. The original UMI work showed that policies trained from these demonstrations could transfer to multiple robot platforms while handling dynamic, bimanual, precise, and long-horizon tasks. Because the demonstrator can simply carry the interface into a kitchen, office, store, or workshop, collecting environmental diversity no longer requires moving and configuring a robot in every location. Xiaomi uses that property at unprecedented reported scale. Its pretraining set contains more than 100,000 hours of UMI trajectories spanning over 1,700 scenarios in homes, offices, commercial premises, industrial sites, and outdoor environments. The observations come from egocentric cameras attached to the handheld devices, while the actions describe the motion of the grippers. The term “embodiment-free” deserves a qualification. UMI data are not literally without an embodiment: there is still a gripper, camera geometry, coordinate frame, and action representation. What they avoid is dependence on the target robot embodiment. A more precise description would be portable demonstrator embodiment. The useful property is that a single collection interface can produce data intended for many eventual robots. Scale introduces a labeling problem. A conventional imitation-learning episode is usually segmented according to task semantics and assigned an instruction such as “put the cup in the sink.” Manually segmenting and describing 100,000 hours would be impractical. Xiaomi instead divides trajectories into equal-length clips and asks a vision-language model—identified in the report as Qwen3.5-27B—to describe the state transition involving the grippers and manipulated objects. This produces labels closer to hindsight descriptions than ordinary user commands. A clip might be characterized by what physically changed: a gripper approached a handle, the handle rotated, and a drawer moved from closed to open. The policy is then trained to produce the actions that transform the current scene into the language-described state. Xiaomi reports that a producer–consumer annotation system, with CPU workers preparing clips and hundreds of captioning requests kept in flight, labeled the entire corpus in about two weeks. Conceptually, this is natural-language goal relabeling. Rather than requiring the collector to announce an intention before acting, the system observes the outcome and writes a description afterward. The advantage is not just cheaper annotation. State-transition language can be more local and less ambiguous than a high-level task label. “Move the left gripper below the paper stack while the right gripper stabilizes the edge” gives denser guidance than “refill the printer.”. The trade-off is that fixed-length segmentation can cut across semantic boundaries. A clip may contain the end of one subskill and the beginning of another, or little meaningful motion at all. Captioning errors can also become action-conditioning errors. The paper does not report a systematic human audit of label quality, compare different segment lengths, or ablate automatic captions against manual instructions. At 100,000 hours, however, Xiaomi is betting that coverage and redundancy can outweigh some amount of annotation noise. From descriptive pretraining to imperative post-training. Pretraining teaches Xiaomi-Robotics-1 to answer a descriptive question: given the current observation and a description of the desired state transition, what motion would produce that change?. That is not yet how users prompt robots. A user says “tidy the sofa,” not “move the left gripper toward the blue cushion while raising its near edge.” The pretraining action space also belongs to UMI grippers, not a mobile manipulator. Xiaomi therefore treats deployment as a two-dimensional alignment problem:. Embodiment alignment maps portable-gripper action knowledge onto real robot bodies. Instruction alignment maps descriptions of observed state changes onto imperative task commands. The post-training mixture contains about 10,000 hours. More than 7,200 hours come from Xiaomi mobile manipulators and dual-arm robots operating across household environments. More than 1,000 hours are high-quality UMI demonstrations manually segmented and labeled with task instructions. The remainder includes filtered data from BridgeData V2, RT-1, and DROID, with idle trajectory regions removed. The in-house data dominate the sampling mixture. Cross-embodiment alignment requires more than concatenating datasets. Xiaomi converts arm commands into relative end-effector transformations measured from the current pose. It unifies end-effector frame orientations so that, for example, “move forward relative to the gripper” has consistent numerical meaning across hardware. Mobile-base commands are represented as velocities, waist motion as a relative displacement, and missing dimensions are masked during the loss computation. This is a mundane-looking but consequential part of the recipe. A foundation model cannot learn transferable control if identical action values mean incompatible motions on different platforms. The unified action vector is effectively an interface contract between data sources. It preserves slots for dual arms, grippers, waist, and mobile base even when a particular robot lacks some of those components. The public code makes that contract concrete. Its default representation predicts a 30-step action chunk with 60 dimensions per step. Designated dimensions encode left- and right-end-effector translation, axis-angle rotation, gripper state, waist motion, and base velocity; the remaining slots are reserved. The released data format uses three synchronized videos—an ego view and two wrist views—plus proprioception, actions, and a language instruction stored in per-episode JSON. Inside the policy: a VLM and an action transformer. Xiaomi-Robotics-1 uses what the authors call a Mixture-of-Transformers architecture. This should not be confused with a sparsely routed mixture-of-experts language model. Here, the “mixture” is a coupled pair of transformer streams: a pretrained Qwen3-VL vision-language model and a smaller diffusion transformer dedicated to continuous actions. The VLM processes the camera observations and language prompt. The action transformer receives robot state and attends to the cached visual-language representations. It then produces an action chunk through flow matching. In plain terms, inference begins with a randomly sampled action trajectory and repeatedly refines it toward a coherent motor command. Xiaomi uses five Euler integration steps, balancing the expressiveness of a generative action distribution against control latency. There are three nominal model scales:. The 2B model contains about 2.6 billion total parameters: a 2.1B VLM and a 470M action transformer. The 5B model contains about 5.1 billion parameters: a 4.4B VLM and a 604M action transformer. The 10B model contains about 10.5 billion parameters: an 8.8B VLM and a 1.5B action transformer. The action transformer has the same number of layers as its paired VLM but a narrower hidden dimension. That preserves a layer-by-layer coupling structure while concentrating most of the expensive semantic computation in the pretrained backbone. The architecture contains an interesting auxiliary training path. In addition to conditioning the diffusion transformer, the VLM itself predicts several candidate action chunks and estimates an error score for each. Only the candidate closest to the demonstrated action receives the direct action regression loss, while the score heads learn to predict each candidate’s distance from the target. This “choice policy” objective gives the visual-language representation a direct incentive to preserve action-relevant information. It also acknowledges that manipulation is multimodal: from one observation, several grasps or approach directions may all be plausible. But Xiaomi discovered a shortcut. If the diffusion transformer could attend to the VLM’s action-candidate tokens, it tended to copy those proposals rather than grounding its own generation in the visual and language context. The final design therefore hides the action-related tokens from the diffusion transformer. It sees only the original observation and instruction representations. Pretraining combines three losses: flow matching for the diffusion policy, candidate-action regression for the VLM, and ordinary next-token prediction on vision-language examples. The vision-language data are mixed with UMI trajectories at a ratio of one to nine, with a smaller weight on the language-modeling term. This is intended to prevent the backbone from forgetting the semantic capabilities that made it useful in the first place. From a systems perspective, Xiaomi amortizes the expensive VLM pass by sampling four flow-matching noise levels for every visual-language representation. The four action-transformer inputs are packed into one pass. These details matter because scaling embodied models is partly a throughput problem: every unnecessary VLM evaluation multiplies across millions of video clips. What the scaling experiments actually show. The title emphasizes 100,000 hours, but the controlled scaling experiments use a subset of approximately 20,000 hours because of compute limits. This is an important distinction. The final system is reported as pretrained on the full corpus, while the clean ablations compare 12.5%, 25%, 50%, and 100% of the 20,000-hour experimental subset. In approximate terms, those conditions correspond to 2,500, 5,000, 10,000, and 20,000 hours. Using the 5B model, validation action error decreases as the dataset grows. The two smallest conditions eventually overfit: validation error initially improves and then rises. The 10,000- and 20,000-hour runs continue improving, and the largest dataset descends fastest. Model scaling is also positive. When the 2B, 5B, and 10B variants train on the same 20,000 hours, larger models predict actions more accurately. But the differences between model sizes are smaller than the differences between data scales. Xiaomi interprets this as evidence that billion-parameter capacity is already sufficient to model much of the present distribution and that diversity of experience is the tighter bottleneck. This is evidence of monotonic scaling, but not yet a robotics scaling law in the stronger language-model sense. The report does not fit a power law, study compute-optimal allocation, or show how loss changes when data, parameters, and training compute are increased jointly. “Scaling behavior” is the more defensible phrase. The stronger result comes after post-training. Xiaomi initializes otherwise comparable 5B models from checkpoints trained with different quantities of UMI data, then applies a shared robot-alignment recipe. A model starting from Qwen3-VL without action pretraining succeeds on 26% of the real-robot trials. Pretraining on only 12.5% of the 20,000-hour subset raises that to 53%. Using the full 20,000 hours reaches 75%. Doubling the corpus from 50% to 100% still adds six percentage points, which motivates the authors’ claim that the tested regime has not saturated. Larger pretrained models transfer too. After equivalent post-training, overall success rises from 61% for the 2B variant to 75% for the 5B model and 79% for the 10B model. On shoe storage, the improvement is more dramatic: 58%, 75%, and 92%. That is the central experimental contribution. An offline action-prediction improvement could be dismissed as fitting the UMI validation distribution more closely. Here, stronger UMI pretraining survives a change in embodiment, language style, environment, and object instances, and appears as higher physical success. What “out of the box” means here. The out-of-the-box real-robot evaluation covers shoe storage, bag packing, table organization, and sofa tidying. The evaluation environments and object instances were held out, and the models received no task- or environment-specific fine-tuning after the shared post-training phase. However, the four task categories were present in the post-training data. The experiment measures environmental and instance generalization, not zero-shot acquisition of unseen task semantics. That is still valuable—home manipulation often fails because layouts and objects differ—but it is a narrower definition than “perform any novel task from a prompt.”. Unseen tasks are addressed in a separate adaptation experiment. Xiaomi holds out phone packing, printer refilling, laundry loading, and language-conditioned box packing from the in-house robot corpus. These tasks stress bimanual coordination, deformable paper handling, mobile manipulation, multi-stage execution, and object-level language grounding. The low-data condition uses 36 hours across all four tasks, or fewer than ten hours per task on average. The high-data condition uses 144 hours. Xiaomi fine-tunes one model jointly on all four tasks using the asynchronous recipe introduced in Xiaomi-Robotics-0 and compares it with π0.5 and Xiaomi-Robotics-0. Each method receives only ten physical trials per task. With the low-data budget, Xiaomi-Robotics-1 averages 75% success and 90% milestone progress. π0.5 reaches 40% success and 66% progress. XR-1 succeeds on 70% of phone-packing trials, 70% of printer-refilling trials, 80% of laundry-loading trials, and 80% of box-packing trials. With the larger dataset, overall success increases to 85%, compared with 53% for π0.5. The result suggests that broad physical pretraining supplies useful primitives even when the final task is absent. But the small evaluation count matters. A single trial changes a per-task score by ten percentage points. Printer refilling actually drops from 70% in the low-data condition to 60% in the high-data condition, despite the overall benefit from more data. That could reflect optimization variance, task variance, or simple binomial noise. Larger studies with confidence intervals would make the data-efficiency claim substantially stronger. Simulation results—and their fine print. Xiaomi also adapts the model separately to RoboCasa, RoboCasa365, VLABench, and RoboDojo. These are not zero-shot evaluations of the generic post-trained checkpoint. Each benchmark uses its official training demonstrations, and VLABench additionally uses chain-of-thought labels and an auxiliary text objective on half the examples. On the paper’s reported protocols:. RoboCasa: 74.5% average success, versus 72.6% for World2Act. RoboCasa365: 57.4% average success, versus 46.6% for ABot-M0.6. VLABench: 59.1% average success and 70.3% progress, versus 53.2% and 65.9% for ERVLA. RoboDojo: an average score of 20.07 and success rate of 13.93%, versus 13.07 and 8.80% for Hy-Embodied-0.5-VLA. RoboCasa365 provides the most interesting breakdown. XR-1 scores 80.2% on atomic tasks, 57.1% on seen task compositions, and 32.1% on unseen compositions. The previous best average model scores only 7.9% on that last split. This points toward recombination of familiar primitives rather than mere memorization of benchmark instructions. RoboDojo exposes a clear weakness. XR-1 leads in generalization, precision, long-horizon tasks, and the open category, but trails the memory-oriented Hy-Embodied model on the memory dimension. Xiaomi explicitly evaluates without observation history, so the policy must infer state from the current frame and proprioception. A recurrent state, persistent visual memory, or history tokens are obvious next additions. A little numerical hygiene is also needed. One passage in the paper says 57.6% for RoboCasa365, while the version-two abstract and main table report 57.4%. The project page lists 13.93 for RoboDojo under a table described as success rates, whereas the paper clarifies that 13.93 is success rate and 20.07 is the benchmark’s average score. The released RoboCasa evaluation logs reproduce 74.2% rather than 74.5%, with the maintainers noting machine-dependent variation. These are modest discrepancies, but the main tables are the safest source for comparison. Where Xiaomi-Robotics-1 sits in the VLA landscape. Octo and OpenVLA established the open generalist-policy template using large cross-robot datasets. Octo trained on 800,000 Open X-Embodiment trajectories and emphasized adaptation to new sensors and action spaces. OpenVLA combined internet-pretrained visual-language representations with 970,000 real-robot demonstrations in a 7B model, while making parameter-efficient fine-tuning and quantized serving accessible. π0 and π0.5 shifted toward expressive continuous action generation and highly heterogeneous co-training. π0.5 combines data from multiple robots with web data, object detection, high-level subtask prediction, language, and low-level actions. It demonstrated 10-to-15-minute cleaning behaviors in entirely new homes. Xiaomi uses π0.5 as its principal adaptation baseline, but its own data strategy places much greater emphasis on portable manipulation demonstrations and explicit UMI-to-robot alignment. RDT2 is the closest conceptual neighbor. It reports more than 10,000 hours of UMI demonstrations, builds around a 7B VLM, and targets zero-shot generalization to unseen objects, instructions, scenes, and robot embodiments. Xiaomi-Robotics-1 pushes the claimed UMI scale by another order of magnitude, but takes a more conservative transfer route: broad UMI pretraining followed by approximately 10,000 hours of robot-heavy post-training. RDT2 emphasizes zero-shot embodiment transfer; Xiaomi emphasizes predictable scaling followed by alignment. Xiaomi-Robotics-0 is the immediate engineering predecessor. It focused on asynchronous action-chunk execution, latency alignment, and smooth real-time control on consumer-class GPUs. XR-1 retains that deployment philosophy while replacing robot-only foundation pretraining with the much larger UMI-first stage. The distinction is therefore not primarily architectural. Flow matching, VLM backbones, cross-embodiment mixtures, and action chunking are now common ingredients. XR-1’s differentiator is the proposed data abstraction layer:. portable action capture, automatic state-transition language, geometric action normalization, and robot-specific alignment. That stack is what allows physical experience to be gathered without committing every hour to a particular production robot. What is genuinely new—and what remains unproven. The first meaningful contribution is the scale of non-robot physical data. Even if “embodiment-free” is overstated, 100,000 hours of action-aligned real-world video is qualitatively different from web video alone. The trajectories contain explicit gripper motion and contact-driven scene changes. The second contribution is the separation between descriptive physical pretraining and imperative instruction alignment. This resembles the division between base-model pretraining and instruction tuning in language models. The pretraining stage learns how scenes can be transformed. Post-training teaches the model how human requests select those transformations. Third, the paper provides a relatively clean transfer experiment. The jump from 26% without action pretraining to 75% with the largest controlled pretraining condition is more informative than a leaderboard win. It isolates initialization while holding the robot-alignment phase broadly constant. But several open questions limit how strongly we should interpret the result. Most importantly, the controlled scaling curve stops at 20,000 hours. The report does not show a direct 20,000-versus-100,000-hour real-robot ablation, a fitted scaling exponent, training compute, or a compute-optimal frontier. The 100,000-hour final model may be much stronger, but that conclusion is not isolated experimentally. The automatic annotation pipeline is also under-ablated. We do not know how often captions hallucinate, whether object-state descriptions matter more than gripper descriptions, how fixed segment duration affects learning, or whether the language labels outperform simpler goal-image or temporal-offset conditioning. Choice Policies, vision-language co-training, token masking, and cross-embodiment frame alignment are all plausible contributors, but the paper does not quantify them individually. The physical generalization study contains four seen task categories and relatively few trials. The adaptation study contains only 40 trials per model condition. There are no confidence intervals, formal failure taxonomy, intervention tests, or systematic safety analysis. Data composition remains opaque. We know broad environment categories and total hours, but not the long-tail distribution of skills, the number and expertise of collectors, duplicate rates, demographic or geographic coverage, camera failures, trajectory filtering, or how much low-value motion remains. Data governance is especially relevant when thousands of hours are captured in real homes. Finally, the public release is useful but incomplete. Xiaomi provides the approximately 10.2GB 5B checkpoint, benchmark checkpoints, post-training and deployment code, and a 336MB demonstration-format dataset. It does not publicly provide the 100,000-hour UMI corpus, the 7,200-hour in-house robot corpus, or a turnkey reproduction of the large-scale captioning and pretraining run. How a robotics lab could use the release. For most teams, XR-1 is currently more useful as a policy initialization than as a fully out-of-the-box controller. The released stack requires Python 3.9 or newer, CUDA, FlashAttention, and exactly version 4.57.1 of Hugging Face Transformers. The Qwen3-VL-4B-Instruct backbone must also be accessible. Training and deployment use separate configuration files and a client–server runtime; asynchronous training is enabled by default. A sensible adaptation workflow would look like this:. Define the embodiment mapping. Express arm motion as relative end-effector deltas and align end-effector coordinate conventions with XR-1’s representation. Map unsupported joints or effectors into masked dimensions rather than inventing incompatible semantics. Match the observation interface. The provided loader expects an ego camera and two wrist cameras. A single-arm platform can adapt the schema, but should preserve ordering, crop conventions, synchronization, and the visual perspective expected by the checkpoint. Collect a compact, diverse task dataset. The paper’s adaptation results argue for breadth within the target task: vary objects, initial conditions, language, failure recoveries, and layouts rather than recording one nominal trajectory repeatedly. Recompute all normalization statistics. The public configuration expects per-step means and standard deviations for the 30-by-60 action chunks, plus low and high quantiles for robot state. These are not incidental preprocessing constants. A mismatch directly changes the physical meaning of generated actions. Validate through staged deployment. Start with open-loop action inspection, then simulation or a digital twin, then low-speed execution with workspace, velocity, force, and collision constraints. A large pretrained policy does not remove the need for conventional robot safety layers. The client takes three images, an instruction, and current robot state, then returns raw actions, decoded action components, and target poses. For asynchronous execution it can also accept an unnormalized action prefix, allowing the new prediction to account for commands already scheduled while inference was running. The public checkpoint’s Apache license and standard Transformers-based implementation lower the software barrier. The hard part remains embodiment integration: coordinate frames, camera calibration, control frequency, action delay, gripper conventions, and data normalization. Foundation models move effort away from learning every skill from scratch, but they do not make those interfaces disappear. Final take. Xiaomi-Robotics-1 is compelling because it identifies a plausible route around robotics’ most stubborn scaling constraint. Instead of demanding 100,000 hours from expensive target robots, it gathers broad physical experience through portable interfaces, describes the resulting state changes automatically, and pays the embodiment-specific cost later. The paper does not yet establish a universal robotics scaling law. Its controlled study covers only 20,000 hours, the strongest physical evaluations remain relatively small, and the data behind the headline result are private. “Out of the box” also means unseen rooms and objects within previously aligned task categories—not arbitrary new tasks or hardware. Still, the transfer result is difficult to ignore. More portable-manipulation pretraining leads to better physical performance after robot alignment, and more model capacity provides an additional gain. The effect is strongest on contact-rich behavior, precisely where semantic web pretraining alone is least likely to be sufficient. The lasting idea may therefore be larger than Xiaomi-Robotics-1 itself: robot foundation models need not obtain all of their physical knowledge from robots. If portable action capture, hindsight language labeling, and embodiment alignment continue to scale together, the field’s data bottleneck may shift from scarce robot hours to a much more tractable problem—collecting, curating, and translating human-guided physical experience.