Latest / Embodied AI 101 / Dyna-2's Million-Hour Bet: When Human Video Starts Improving Robots
Transcript
- Dyna-2’s Million-Hour Bet: When Human Video Starts Improving Robots. Dyna-2: A 1-Million-Hour Scaling Law for World-Action Models • Dyna Robotics • Dyna Robotics, Inc. • Company technical report • 2026. The central claim. Most robotics scaling programs begin with robot data: more teleoperation, more embodiments, more environments, and eventually more autonomous experience. Dyna-2 begins with an inversion of that strategy. If general-purpose robots are ultimately meant to perform work now done by humans, perhaps the largest relevant pretraining corpus is not robot experience at all. Perhaps it is video of humans doing the work. Dyna Robotics reports pretraining Dyna-2 on more than one million hours of primarily egocentric human-manipulation video—roughly 170 years of waking experience. More important than the headline number is the experiment built around it. Dyna trains models using nested data budgets of 1,000, 10,000, 100,000, and 1,000,000 hours, then asks whether increasing human experience predictably improves not only held-out human action prediction, but also prediction on robot trajectories excluded from pretraining. The reported answer is yes. Performance improves monotonically on held-out human data, on zero-shot offline robot data, and—after identical robot-data post-training—on physical-robot evaluations. The company argues that this is the first measured human-to-robot transfer scaling law: more human video alone produces better predictions for a robot embodiment the model has not previously encountered. There is an equally important qualification. Dyna-2 is not shown executing the full physical benchmark zero-shot from human pretraining. “Zero-shot” in the main scaling experiment refers to offline prediction on unseen robot trajectories. The on-robot results come after post-training on task-specific robot demonstrations. Elsewhere, zero-shot customer deployment means transferring a post-trained task policy to a new site. These are all meaningful forms of generalization, but they should not be conflated. One numerical clarification is also worthwhile. The experiment uses four logarithmically spaced data budgets, from (10^3) to (10^6) hours. That is four rungs spanning a thousandfold range, or three orders of magnitude—not four orders of magnitude in the usual sense. The data wager. The economic argument behind Dyna-2 is straightforward. Every hour of conventional robot demonstration must be deliberately produced using a specific robot, sensing stack, control interface, and workcell. Even increasingly scalable systems such as UMI or low-cost bimanual teleoperation remain data-collection operations. Human activity video, by contrast, can be gathered without tying collection to a target robot’s morphology. Dyna’s corpus is described as mostly head-mounted, first-person footage of people cooking, tidying, folding, assembling, cleaning, and performing similar everyday manipulation. The data comes from internal collection and external partners. A processing pipeline filters clips, extracts three-dimensional hand tracks, and converts those tracks into pseudo-actions. Wrist motion becomes end-effector supervision, while the aperture between the thumb and index finger supplies a continuous grasp signal. Dyna says it does not apply a special visual transformation or embodiment-specific alignment intended to make the human footage resemble downstream robot observations. This representation is deliberately simpler than reconstructing every human finger joint. It captures the task-space motion of each wrist and a low-dimensional grasp state. That is naturally closer to a dual-arm robot with parallel-jaw grippers than raw human joint angles would be. It also gives the model a common action vocabulary across people: move the hand here, orient it this way, and open or close the grasp. The million-hour corpus should not necessarily be interpreted as one million hours of equally reliable action labels. Dyna distinguishes between video that passes its hand-pose quality threshold and video usable only for future-frame prediction. That distinction becomes central to the ablation results. Instead of throwing away clips whose hand tracking is insufficient for action supervision, Dyna can retain them as video-only world-modeling data. For the primary scaling experiment, the subsets are nested. The 10,000-hour set contains the 1,000-hour set; the 100,000-hour set adds data without exchanging the earlier examples; and the million-hour set extends the same hierarchy. Dyna preserves the proportion contributed by each source and evaluates all models on a fixed, disjoint 100-hour human validation set. Model and evaluation configurations are held constant, while compute scaling and model-size scaling are explicitly left for future work. That design is one of the report’s strongest features. Robotics scaling comparisons are often confounded by changing task mixtures, improved data pipelines, larger networks, or more extensive hyperparameter tuning. Nested datasets do not eliminate every confound, but they make the question cleaner: what happens when the principal changing variable is the diversity and quantity of human experience?. What Dyna-2 actually is. Dyna-2 belongs to the emerging category of world-action models, or WAMs. A conventional vision-language-action model usually begins with a vision-language model whose pretraining teaches semantic recognition and linguistic reasoning. RT-2 represented robot actions as text tokens and co-trained them with web-scale vision-language tasks. OpenVLA built a seven-billion-parameter open model from 970,000 robot demonstrations. Physical Intelligence’s π0 combined a pretrained vision-language backbone with a continuous flow-matching action expert. All three approaches aim to transfer Internet-scale semantic knowledge into control. A WAM changes the pretraining prior. Instead of beginning primarily with a model of what images and words mean, it begins with—or adds—a model of how visual scenes evolve through time. The hope is that video prediction carries information about contact, object permanence, deformation, tool use, and the relationship between an agent’s motion and the resulting state of the world. Dyna-2 uses a video-diffusion backbone with separate transformer streams for video and action. Each modality is tokenized independently and receives its own diffusion-transformer layers, while attention operations permit information exchange. Video tokens are causally masked. Action tokens use bidirectional attention over the action chunk and attend to observed visual context. Proprioception enters the action stream directly. Text conditions the video tokens rather than connecting directly to the action tokens, so language reaches control indirectly through the visually grounded representation. The model is trained with flow matching. In plain language, both future video latents and future action chunks are progressively corrupted toward noise. The network learns the direction that transports a noisy sample back toward a clean future. This is similar in spirit to diffusion policy, but it is applied to both visual futures and continuous actions inside a shared model. The most consequential architectural detail is easy to miss: the scaling-law variant does not need to generate a future video during control. Its video and action objectives share representations, but the action predictor does not consume the noisy or generated future-video sample. At inference time, the policy remains reactive, predicting action chunks from observed frames, proprioception, and the representation shaped during video pretraining. That makes Dyna-2 different from an imagination-in-the-loop controller. It is not rolling out several visual futures and selecting an action through model-predictive control. It is closer to a multitask generative model in which future prediction acts as a physically grounded representation-learning objective. This distinction matters when interpreting the results: if video co-training improves robot control, the improvement cannot simply be attributed to explicit planning through generated futures. Dyna also reports that much of a video diffusion transformer’s useful temporal reasoning is concentrated in earlier layers. The action transformer is therefore shallower and connects to the video stream mainly in those early layers. The goal is to keep the action path fast enough for reactive deployment while retaining the representations acquired through large-scale video learning. Scaling on held-out human behavior. Dyna evaluates predicted action chunks using mean-squared error, L1 error, and two thresholded accuracy measures. The looser threshold is intended to measure broad motion intent; the tighter one emphasizes precision. Reporting both continuous and thresholded metrics is a sensible response to concerns about apparent “emergence”: a threshold can make a gradual improvement look discontinuous if many predictions cross the boundary at roughly the same scale. Across all four metrics, performance reportedly improves monotonically from 1,000 to one million hours and is well fit by a power law in dataset size. The tight-threshold accuracy improves by 51 percent across the ladder, compared with a 12 percent improvement in mean-squared error. Dyna evaluates ten checkpoints from a late training window and reports their mean and standard deviation, reducing dependence on selecting one especially favorable checkpoint. This establishes the most conservative result: Dyna-2 can absorb increasingly large quantities of human-manipulation data without saturating by the million-hour point. The model becomes better at predicting held-out human wrist and grasp trajectories as the training corpus grows. It is still a preliminary scaling law rather than a complete one. There are only four data points, and the public report does not foreground fitted exponents, goodness-of-fit comparisons against alternative curves, or extrapolation tests. Averaging nearby checkpoints is also not equivalent to training multiple random seeds. The evidence supports a smooth scaling trend through the measured range, but it does not yet tell us the compute-optimal model size, the expected gain at ten million hours, or where the curve will bend. The cross-embodiment result. The more interesting experiment evaluates the same checkpoints on a held-out robot dataset without robot pretraining or adaptation. The evaluation contains 39 tasks across two stationary bimanual YAM platforms. Twelve tasks are internal, while 27 come from the externally released ABC behavior-cloning dataset. The tasks include cloth handling, knot tying, cleaning, packing, assembly, and food-service manipulation. ABC itself contains approximately 3,500 hours and 130,000 episodes across 195 tasks, so using part of it gives Dyna at least some evaluation data outside its internal task-design process. All four offline metrics reportedly rank the models in the same order as the human pretraining budgets. The one-million-hour model predicts unseen robot actions more accurately than the 100,000-hour model, which outperforms the 10,000-hour and 1,000-hour models. Dyna observes its largest qualitative change between 10,000 and 100,000 hours and describes this as a possible emergence threshold for cross-embodiment transfer. Why might this work? Human wrists and robot end effectors may look different, but the object-level transformations are shared. Opening a drawer, pulling cloth taut, inserting a part, or turning a key all involve visual trajectories and contact patterns that survive a change in morphology. A model trained only to imitate human pseudo-actions can overfit to human appearance or annotation artifacts. A model forced to predict what happens to the scene has an incentive to represent the task’s physical structure. There are limits to what the offline test proves. Action prediction error is not closed-loop success, and manipulation is multimodal: several different motions can complete the same task, while an offline metric penalizes any valid action that differs from the recorded demonstrator. The report also does not fully spell out how the human pseudo-action coordinates are mapped into the robot evaluation action space. A shared wrist-pose and grasp representation is the likely bridge, but that is an inference from the described data pipeline rather than a documented implementation detail. The zero-shot robot benchmark also uses bimanual YAM systems with parallel-jaw-style control. The much larger embodiment changes—to dexterous hands and a semi-humanoid platform—appear in the physical experiments only after robot-data post-training. So the result is genuinely cross-embodiment, but the largest morphology gaps are not crossed by the offline zero-shot evaluation alone. Does the trend survive contact with the robot?. To test whether offline scaling predicts real behavior, Dyna post-trains equivalent checkpoints from each data rung on identical robot datasets. The evaluation covers 14 tasks, with no task using more than ten hours of robot data. Eleven tasks run on dual six-degree-of-freedom YAM arms with parallel-jaw grippers. Two use the same arms fitted with WUJI-2 five-fingered hands, and one language-conditioned retrieval task runs on an early semi-humanoid prototype. The task suite is unusually broad for a company report. It includes rope tying, food scooping, key turning, tube insertion, tote construction, mug unboxing, object kitting, deformable-object handling, and targeted drink retrieval. Dyna uses ten trials per task, or twelve for language retrieval, with evaluators who were not involved in model development. Because the tasks have different native metrics, Dyna normalizes each score by its maximum possible value and averages across the suite. The aggregate score rises from 20 percent at 1,000 pretraining hours to 28 percent at 10,000, 45 percent at 100,000, and 53 percent at one million. The million-hour checkpoint is the strongest on nine of the 14 tasks. Some individual results look threshold-like. No model through 100,000 hours succeeds at turning the lockbox key, while the million-hour model reportedly succeeds in nine of ten trials. Bottle-cap untwisting uses only about ten minutes of teleoperated robot data yet rises to 50 percent success at the largest pretraining scale. Targeted drink retrieval improves from 58 to 83 percent. With ten trials, any single task’s apparent phase transition should be treated cautiously. A result of nine successes rather than zero is certainly meaningful, but the precise underlying success probability remains uncertain. The aggregate monotonic trend across 14 heterogeneous tasks is more persuasive than any individual bar. The major takeaway is therefore not that human video eliminates robot data. It is that human pretraining appears to amplify small robot datasets. The post-training recipe is unchanged, yet the final robot policies improve as the amount of preceding human experience grows. Why the video objective matters. The report’s most scientifically interesting section asks whether scale alone is sufficient. Dyna compares three training recipes at 5,000, 50,000, and 100,000 hours of action-labeled human data: action prediction alone; joint action and future-video prediction on the same clips; and the joint recipe augmented with additional video-only human data. Joint training beats action-only training on all 39 robot tasks at every tested scale. But merely adding the video objective to the action-labeled clips does not produce a clean scaling curve. The curve emerges when the model is additionally trained on substantial quantities of video-only data. Action-only models show unstable overfitting, whereas the video-co-trained models improve as the data budget grows. Dyna then fixes action-labeled data at 50,000 hours and increases only the video-only allocation from zero to 1,000, 10,000, and 50,000 hours. It repeats the experiment in a larger regime, with 250,000 action-labeled hours and up to 750,000 video-only hours. In both regimes, robot-domain generalization improves monotonically as video-only data increases. Curiously, held-out human action prediction remains flat or becomes slightly worse. That asymmetry is revealing. Video co-training is not simply improving every metric through generic regularization. Its benefit is concentrated where morphology and visual distribution change. A plausible interpretation is that action-only supervision encourages the representation to specialize in the human pose-estimation pipeline, while future prediction pushes it toward object motion, contact consequences, and scene dynamics that remain stable across embodiments. This mechanism is not directly proven, but it is consistent with the ablations. It also reframes what a “world model” contributes. In Dyna-2, the benefit does not require consulting a generated future during execution. The future-prediction loss changes the features from which the reactive policy acts. World modeling here is as much a pretraining principle as a planning mechanism. How Dyna-2 fits into the field. Unified World Models from the University of Washington and Toyota Research Institute provided an earlier clean formulation for coupling video and action diffusion. Its independent noise levels allow one network to function as a policy, forward model, inverse model, or video generator, and it can learn from action-free video. Dyna-2 shares the basic thesis that video prediction and action prediction should inhabit one model, but moves the data scale from conventional robot datasets toward million-hour human video. NVIDIA’s DreamZero is the closest architectural comparison. DreamZero is a 14-billion-parameter WAM built on a pretrained video-diffusion backbone that jointly denoises videos and actions at inference. It reports real-time control at roughly seven hertz after extensive model and systems optimization, along with transfer from short human or cross-robot video demonstrations. Unlike Dyna-2’s scaling variant, DreamZero explicitly generates the visual future in the control loop and studies model-size scaling, modality alignment, and autoregressive execution. EgoScale is the closest data-scaling comparison. It trains a VLA on 20,854 hours of action-labeled egocentric human video and finds a log-linear relationship between human data and validation loss. It then uses aligned human-robot mid-training to transfer a high-degree-of-freedom human hand prior into dexterous robot policies. Dyna-2 extends the human-video scale by roughly two orders of magnitude and removes alignment from its principal scaling study, although its pseudo-action representation is considerably lower-dimensional. EgoMimic likewise extracts three-dimensional hand motion from egocentric video, but combines explicit human-robot alignment, co-training, and hardware chosen to reduce the kinematic gap. RDT2 and LAP attack zero-shot embodiment transfer from different directions: RDT2 scales embodiment-agnostic UMI demonstrations beyond 10,000 hours, while LAP expresses low-level actions in natural language and reports substantial transfer to unseen robots. Those systems establish that zero-shot embodiment transfer is possible at a chosen scale. Dyna-2’s distinct claim is a measured transfer curve arising from purely human pretraining. The field should therefore resist reducing the comparison to WAM versus VLA. The real design space includes semantic pretraining, video dynamics, explicit action labels, shared task-space representations, embodiment alignment, and post-training. Dyna-2 suggests that video prediction becomes especially valuable when crossing morphology or visual-domain boundaries; it does not show that language or robot demonstrations have become unnecessary. Beyond the scaling curves. Dyna also compares an early Dyna-2 checkpoint with its production VLA, Dyna-1, under matched pretraining and post-training conditions. Across seven internal robot tasks, the early WAM reaches 1.55 times the VLA’s pooled success rate and 1.12 times its quality grade. In matched task-and-checkpoint comparisons, Dyna-2 wins 65 percent, Dyna-1 wins 29 percent, and six percent are ties. This comparison needs careful interpretation. The early Dyna-2 model did not use the million-hour corpus and was trained with an action-only loss, while the overall pipeline had been tuned for Dyna-1. It therefore supports the value of the video-model-derived architecture or initialization, but it does not isolate the final report’s video co-training objective. At unseen customer sites, both models reportedly pass nearly all in-house tests, but Dyna-1 passes 46 percent of customer acceptance evaluations while Dyna-2 passes 87 percent. The task datasets and post-training budgets are matched, and neither policy has seen data from the deployment sites. This is encouraging evidence for environment generalization, though the report does not provide the number of sites, task-level breakdowns, or confidence intervals. Language following also improves with both world modeling and scale. On counterfactual instruction tests involving object selection, directional pushing and pulling, stacking, and napkin manipulation, Dyna reports scores of 35 percent for action-only pretraining, 67 percent after video co-training on the early corpus, and 96 percent after video co-training on the full Dyna-2 corpus. The action stream does not receive text directly, so this result suggests that language grounding can be mediated through a video representation of the instructed physical outcome. Finally, Dyna presents a one-step video distillation method. Rather than ask a student to leap immediately toward a distant teacher distribution, the method moves the target gradually, advancing it only when the student has caught up. On an H100, Dyna reports reducing a three-second, three-view video generation from roughly 10.2 seconds and 100 network evaluations to 110 milliseconds and one evaluation. The one-step student reaches an FVD of 121, versus 80 for the full teacher, while retaining 75 percent of real-video motion and low flicker. This distilled generator is separate from the reactive scaling-law policy, but it points toward future systems in which video imagination could be cheap enough for online planning, evaluation, or synthetic data generation. Reading the evidence critically. Dyna-2 includes several pieces of good experimental hygiene: nested datasets, a fixed validation set, multiple continuous and thresholded metrics, an external robot dataset, identical post-training recipes, blind physical evaluation, and controlled objective ablations. These choices make the central conclusion considerably more credible than a collection of selected robot demos. At the same time, this is a company research release rather than a peer-reviewed conference paper. The page attributes authorship collectively to Dyna Robotics and does not link a conventional PDF, public implementation, model checkpoint, or dataset release. Important details—including parameter count, base video model, frame rate, context length, action horizon, image resolution, optimizer, training compute, data deduplication, and exact action-space conversion—are not specified in the public report. The scale of the proprietary corpus creates additional questions. We do not know the geographic and occupational coverage, the balance between repeated and unique tasks, how language instructions are produced, how near-duplicate recordings are handled, or whether held-out tasks have close analogues in pretraining. Those details determine whether the curve reflects broad physical knowledge, increasingly complete task coverage, or some combination of both. The phrase “power law” is also stronger than “monotonic improvement.” Four points can support a plausible power-law fit, but they cannot tightly identify its asymptote or predict distant scales. The apparent 10,000-to-100,000-hour inflection on robot data could represent a genuine coverage threshold, a smooth curve viewed through noisy metrics, or properties of this particular model and action representation. Offline action accuracy deserves its own caution. A stronger model may predict the recorded demonstration more closely while still failing in closed loop, and a successful alternative strategy may receive a poor offline score. Dyna’s physical experiments substantially reduce this concern because the aggregate trend survives post-training, but the mechanism ablations—especially the comparison among action-only, joint, and video-co-trained models—are reported primarily on the offline robot benchmark rather than across the full physical suite. The aggregate physical score also gives equal weight to heterogeneous normalized task metrics. That is defensible, but it can conceal whether gains come from broad moderate improvement or a handful of newly solvable tasks. With ten trials per task, the individual success estimates are relatively coarse. None of these concerns invalidate the result. They define what an independent replication should test. The strongest future confirmation would combine public nested data subsets, several random seeds, disclosed compute, absolute robot metrics, and an on-robot objective ablation. Until then, Dyna’s “first” claim should be viewed as a compelling and technically plausible company result awaiting broader verification. What practitioners can take from it. The most immediately actionable lesson is to stop treating failed action annotation as failed data collection. A scalable human-data pipeline should preserve multiple quality tiers. High-confidence wrist and hand tracks can supervise actions; lower-confidence clips can still train future prediction. Dyna’s results suggest that the latter may be disproportionately useful when the downstream embodiment differs from the data-collection embodiment. The second lesson is architectural. A robot policy can benefit from world modeling without paying the cost of video generation at runtime. Sharing early representations between a video predictor and a lightweight action transformer offers a practical compromise: dense temporal supervision during pretraining, reactive action chunks during deployment. Third, action representation remains the bridge. Relative end-effector motion, wrist orientation, and a morphology-neutral grasp variable are easier to transfer than raw human or robot joint angles. More dexterous targets may need a richer learned hand latent or an alignment stage, as EgoScale demonstrates, but the task-space core can remain shared. Finally, scaling should be measured before committing to million-hour collection. Construct nested subsets, keep source proportions constant, establish a held-out robot dataset, and periodically test whether human-domain improvement correlates with robot-domain metrics. If action loss improves while robot transfer remains flat, the missing ingredient may not be more pseudo-labels. It may be a stronger predictive objective or more unlabeled video diversity. The bottom line. Dyna-2’s most important result is not the million-hour headline or the polished manipulation videos. It is the interaction between data scale, objective, and embodiment gap. More human action data improves human prediction, but cross-embodiment scaling appears only when the model is also trained to predict how the visual world changes—and video without usable action labels continues to move that robot-transfer curve. That is a narrower claim than “human video solves robot learning,” but a more valuable one. It identifies a potentially scalable pretraining axis and shows how to use it without turning runtime control into expensive video planning. If the result reproduces across organizations, model families, and robot morphologies, embodied AI’s limiting resource may shift from robot-hours alone toward something much larger: sensorized records of human physical work, paired with objectives that force models to learn what those actions do to the world.