Latest / Embodied AI 101 / ENPIRE: Physical AutoResearch with a Fleet of 8 Robots
Transcript
- ENPIRE: Physical AutoResearch with a Fleet of 8 Robots. Imagine a robotics lab where humans no longer babysit every experiment. Eight robot arms sit poised at benches in NVIDIA’s GEAR Lab, each hooked up to its own computer. As night falls, no engineer is at hand. Instead, a team of AI “coding agents” – essentially large language models (LLMs) that write and run code – take over completely. Their mission: teach these robots new skills as fast as possible without any human in the loop. This is ENPIRE (Agentic Robot Policy Self-Improvement in the Real World), a new NVIDIA-led framework that orchestrates an entire autonomous research loop on real hardware. Given emboldened robots and virtual token budgets, the coding agents relentlessly iterate roles – resetting the world, reading logs and vision, writing new control code, testing it on the arms, and refining again – until the robots master tasks like inserting GPUs into motherboards, threading tiny pins, or cutting zip ties. The results are striking: across a suite of dexterous tasks, the system hit about 99% success under an “up to 8 tries per task” metric. In essence, for the first time ever, AI agents have run the full experimental loop on physical robots, enabling what NVIDIA calls “AutoResearch in the physical world”. The Bottleneck in Physical Robot Learning. Why does this matter? In robot learning, the biggest bottleneck has always been the need for human intervention between trials. A typical robotics experiment goes like this: the robot runs a trial, inevitably fails, and a human assistant trudges in to clean up, reset the scene, and label whether it succeeded. Even a lab full of engineers can only reset and evaluate perhaps tens of trials per day at best. In contrast, RL algorithms could run millions of tries in simulation in the same time. This gap – the so-called sim-to-real gap – has long driven robotics researchers to build ever-more realistic simulators so policies can be trained before they touch real hardware. ENPIRE’s bold claim is that this constraint is now addressable. By automating the reset-and-evaluate loop itself, ENPIRE turns physical robot learning into a process as repeatable and fast as software testing. In Jim Fan’s words, the GEAR Lab simply told the agents “solve the task as quickly as possible, keep the robots busy” and then stepped back. The agents were given 8 robots, plenty of GPU time, and generous token budgets. Over days, the agents brought the robots to life: they read papers, searched past logs, tweaked controllers, and even argued among themselves about options – all without a human watching. It is, in effect, continuous integration for robotics, running on real physical arms. The ENPIRE Architecture: Environment, Policy, Rollout, Evolution. Under the hood, ENPIRE is structured around four core modules – nicknamed E, P, R, and E (Environment, Policy Improvement, Rollout, Evolution) – that together create a closed feedback loop on the hardware. These modules embody the classic experimental cycle: reset the situation, run a trial, evaluate success, learn, and repeat. Environment (EN) – This module handles everything to do with the physical setting. It automatically resets the scene and verifies outcomes, including built-in safety checks. For example, in the GPU insertion task, the robot may knock the graphics card out of the slot; the environment module ensures the arm picks the card up again, returns it to a pre-insertion pose, and unplugs it from the board, all autonomously. The module provides a unified interface (via software APIs) so the agents can command a complete reset and then forget about it. Crucially, the system also auto-evaluates each trial: it uses vision pipelines to score whether the task succeeded without any human eyeballing. In one demo, multiple cameras per arm watch the scene; if a zip-tie strap is routed correctly through its head, the system detects it and assigns a positive reward. Policy Improvement (PI) – Here the LLM agents work their magic. Given the trial results, the agents generate, refine, and test control code for the robot behaviors. They have access to rich information – video recordings of each trial, full state-action traces, reward signals, and failure logs – and they can even consult online resources. In practice, a coding agent might search recent research submissions or programming forums, then write Python code to implement a new strategy. The agent can switch between approaches: it might try a heuristic rule (say, aligning the card at a slightly different angle), use offline imitation from logged successes, or launch an on-robot RL loop. This code is then deployed back into the system to be tried on hardware. Rollout (R) – Once a new policy is ready, the Rollout module executes it on the physical robots. Each robot station has local compute to run the controller, and trials are run under the watch of the environment. The Rollout system records everything: the full state of the robot, every action command, all sensor readings and camera frames, and the final outcome. It adheres to specified budgets (time, a number of trials, etc.) so the agents don’t over-spend compute or robot time, but otherwise it simply runs the code to see what happens. This data is then fed back to the agents. In fact, ENPIRE was built for parallelism: in the demo there were eight identical robot+computer stations, so up to eight experiments could be run simultaneously. Evolution (E) – After many trials branch out across the fleet, the Evolution module ensures the fleet shares knowledge. All code and logs are stored in a common Git repository. When one robot’s agent discovers a successful strategy (say, a robust way to pick up the GPU card), that branch is merged and pushed to all the others. Ineffective or doomed strategies get pruned. In this way, the fleet collectively “evolves” the training recipes. The Git-based approach means there’s no central controller dictating progress – instead, the breakthrough ideas propagate organically across the fleet. Together, these four components turn real-world robot learning into “a controllable optimization procedure that agents can manage,” as NVIDIA aptly states. In short, ENPIRE replicates the human research loop with code-writing agents: hypothesize (Policy), experiment (Rollout), observe (Environment), and refine (Evolution), all done automatically. Setting the Stage: Demonstrations and Frozen Reward Models. One key question is how the agents even know what a “correct” result looks like. ENPIRE solves this by having a brief human-guided setup phase at the start of each task. A person need only demonstrate a few handfuls of examples: a few minutes of video showing the robot succeeding and failing at the task, and – crucially – teach the system how to reset itself. From this input, two critical tools are built once and then frozen. First, the reset procedure is coded (often by the agents themselves with a bit of initial direction) so that the robot can put the workspace back to a known starting state. For example, in the GPU insertion scenario, this involves programming the arm to retrieve a dislodged graphics card, realign it, and park it ready for the next insertion. Interestingly, the ENPIRE team observed that writing a reset policy is often easier than the task itself: resets generally reduce to pick-and-place chores that can sometimes be solved by existing modules (the write-up mentions using a tool-using agent like CaP-X for simple resets). Once the reset script works reliably, it is run automatically after each trial. Second, a reward function is created. Instead of hand-coding a reward or labeling every outcome by hand, the system turns the demonstration data into a vision-based classifier. This “referee” watches camera feeds and outputs a binary success/fail signal. Crucially, after the initial setup the reward model is frozen – it never updates – so it serves as a consistent truth detector throughout the run. For instance, on the zip-tie task the team used a pretrained segmentation model (SAM-3) along with object detectors to identify the zip-tie’s head and strap in each frame. Each of two camera views independently checks if the strap has fully passed through the head, and only then is the binary success reward given. In the pin insertion experiments, one agent even devised a hybrid check combining the gripper’s height, the visual alignment to the hole, and an estimated contact force to decide success. These evaluation pipelines require no human judgment after the initial examples – they simply watch robot attempts and score them. With reset and reward in place, the agents have what they need to run experiments unattended. The “frozen” nature of the reward model is important: it prevents the agents from gaming the scoring by tweaking the classifier itself. Instead, the obstacles become the physical tasks and dynamics. The agents cannot see inside the robot’s code – they only read the fixed success signal. In essence, after a few setup steps the lab becomes completely self-verifying. Agents in the Loop: Codex, Claude, Kimi and All. Who are these “coding agents”? In the ENPIRE experiments, the team tested some of the latest LLM-based code assistants. These included OpenAI’s Codex (running on what they call GPT-5.5), Anthropic’s Claude Code (Opus 4.7), and Moonshot’s Kimi Code (K2.6). Each of these is a powerful large language model fine-tuned for writing and reasoning about code. The NVIDIA team set loose multiple agents of each kind across the fleet. Conceptually, you can think of each agent as a software engineer with access to the Internet – it can search papers, browse forums, and even consult online documentation while deciding what to do. In practice, each agent runs in a loop: it looks at the logged data from past trials (videos, sensor data, rewards), reasons about what could help, and then writes an update to the robot’s control algorithms. For example, an agent might start by coding a few hand-crafted rules (e.g. “if force spike, back off and try approach again”), and if that doesn’t succeed it might implement a behavior cloning step using the past successful demonstrations. If needed, the agent can even start an on-line reinforcement learning process: it will generate policy code, let the rollout trials run a while, then evaluate and adjust. Each code revision is tested immediately, so the agent is constantly experimenting. Because multiple agents and robots operate in parallel, they occasionally pick different ideas. This variation is healthy: one might focus on vision refinements while another explores dynamics adjustments. After each round, the Evolution module compares the branches. Suppose one agent’s tweak turns out to raise success from 60% to 80%. That improved code is automatically merged into the others’ workflows. At the same time, branches that consistently fail on hardware are pruned. In this way, the collective moves toward a working solution. ENPIRE even introduces a new benchmark for tracking this process. The team calls it AutoEnvBench – it plots success rate over actual wall-clock time for each agent’s progress on a task. In effect it visualizes the “idea tree” of the research: branches represent different hypotheses or training recipes, and whenever one spikes the success curve it is marked as a breakthrough (often shown as a green ring in the paper’s figures). This framework makes it possible to analyze how the agents solved the tasks. For example, the NVIDIA write-up notes that one of the most impactful single ideas came from the agents themselves: they introduced a behavior-cloning regularization during training, which immediately lifted the average success by about 10.8 percentage points. They also found that knowledge transfers across tasks: experience gained while inserting pins helped speed up learning to insert GPUs. Importantly, the coding agents operate under realistic constraints. Each has a limit on the number of tokens or computation it can use per hour, and they do not run at absurd speeds like our fastest supercomputers. Yet even with these limits, the experiments converged. In all the reported trials, the agents drove the robots to very high performance: with up to 8 attempts allowed per trial, they saw a “pass@8” success of roughly 99% on the test tasks. (Pass@8 means at least one of eight tries succeeded on a given subtask, simulating a promise that the system can retry intelligently based on past failures in that trial.) In plain terms, across scores of experimental runs the robot almost always managed at least one success by the eighth try. Hitting 99% on contact-rich, precision tasks – without any human corrections – is a striking achievement. Hard Tasks on the Bench: GPUs, Pins, and Zipties. What exactly were the agents teaching the robots to do? The demonstration covered several classic manipulation challenges that require dexterous, high-precision control – the sort of tasks humans find fiddly. One video shows the robots inserting a bulky graphics card into a motherboard slot. If you recall plugging in a PC GPU, you know how sensitive it is: the card’s pins must align precisely with the slot. The agents managed to teach each robot to pick up the card (even if it fell off), align it, and push it in fully. This was done without any prebuilt motion plan – purely with the agent-generated control code and vision feedback. Another task involved pin insertion: the robots had to sort or insert hundreds of tiny metal pins into a tray with 4mm holes. This is a standard dexterity benchmark in robotics labs. The success rate needs to be near-perfect because even a slight misalignment breaks the pin. Yet the agents converged on flawless policies here as well. The report mentions that on pin insertion, the agent policies reached near-perfect reliability faster than a comparable method where a human-tweaked algorithm was updated each morning. A third challenge was tie a zip-tie – or actually cut several zip ties – using an actual cutter tool (two hands, two cameras per station). The robots had to reach around and cinch a cable tie tight against a fixed post, then be able to cut it with a blade and clear it away. Here, the auto-evaluation pipeline (the “frozen reward”) was crucial: vision detected the zip-tie’s head and strap, and only if the strap was looped all the way through did the system register success. According to the reports, this task achieved about 99% success as well under the pass@8 metric. There was also Push-T, a standard benchmark where the robot pushes a T-shaped block into a target zone purely by nudges. (In one description, “Push-T” requires accurately sliding a T-shaped object to a precise goal using only pushing actions.) In simulation all the coding agents solved Push-T easily. However in the real world, the results buckled under unpredictable physics: two out of three agents who succeeded in sim could not make it work consistently on hardware. This highlighted the core challenge ENPIRE is built for: unexpected friction, bumps, and manufacturing variability. The system’s advantage is that it discovers these sim-to-real gaps by running on hardware and then corrects for them, rather than relying on a perfect simulator. In summary, the demonstration tasks were realistic and challenging: seating a GPU card, threading pins, managing zip ties, and the like. These are not trivial contactless pick-and-place jobs. Yet with the agentic loop, the fleet achieved exceptionally high success. As the press reports note, this suggests coding agents can essentially teach robots new complex skills by themselves ― something that used to require human hours of engineering each day. Scaling Up: The Physical “Scaling Law”. An intriguing outcome of ENPIRE is how performance scales with more robots. The team tested configurations of 1, 4, and 8 parallel robots (each with its own agent). Unsurprisingly, more robots meant faster learning – a kind of parallel speedup. In one result, using eight robots instead of one cut the time to learn a working Push-T policy from about five hours down to roughly two hours. For pin insertion, the time to reach near-perfect success also dropped by roughly half (from over 90 minutes to about 40) when scaling from one robot to eight. This is what the authors call a “physical scaling law”: more parallel hardware accelerates solution-finding significantly. However, as with all scaling laws, there was a cost. The team tracked new metrics – Mean Robot Utilization (MRU) and Mean Token Utilization (MTU) – to quantify efficiency. They found that adding robots does shorten convergence time, but each agent spends less of the wall-clock time actually driving a robot. In a bigger fleet, the models spend more time idle, waiting for each other’s code revisions or doing computations. As HotHardware explains, larger agent teams use more total compute without proportionally better robot usage. For example, eight agents consumed far more LLM tokens than one, because coordination overhead increased. The practical takeaway is a classic trade-off: expanding the fleet yields faster solutions, but the total resource bill – GPUs and API calls – climbs steeply. In scenarios where compute and tokens are cheap but robot-hours are scarce, ENPIRE’s scaling is a boon. In other settings, one would have to consider whether the extra hardware cost is justified for the speedup. In short, ENPIRE empirically shows that “speedup is real, but not free.” It quantifies a robotics-world equivalent of Amdahl’s law: doubling robots didn’t halve token cost, and beyond a point adding more arms had diminishing returns in MRU. Still, the results are impressive enough to merit the name of a “scaling law”: physical resources, like coding agents and real arms, can now be traded for faster research. Safety, Autonomy and “Frozen” Classifiers. Running robots fully autonomously does raise safety questions – literally, can the robot damage itself or the environment? ENPIRE addresses this with multiple safety layers. The Environment module includes explicit safety checks: for example, after each reset it verifies that the robot and objects are in expected poses before proceeding. During trials, the reward classifiers provide an indirect safety net: if an attempt goes wildly wrong (leaving things out of place), the trial is simply marked as failure, and the next reset is triggered. This keeps the workshop under control. Intriguingly, some of the agents even developed their own “safety harnesses.” In the pin insertion experiments, one coding agent discovered a contact-force-based control law. Essentially, it programmed the robot to monitor insertion force and avoid exceeding a threshold, which protected the delicate pins. As the CU-Berkeley co-authors note, this custom safety controller outperformed just fiddling with normal RL parameters. In practice, that meant the robot would automatically back off if it felt too much resistance, a self-imposed guardrail it invented on-the-fly. In effect, the system engendered its own low-level fail-safes without human design. Behind this is a general principle: the agents treat safe execution as just another part of the task. The reward model only gives credit when the outcome is valid, so heavy collisions only ever produce zeros. The codewriters, aiming to maximize reward, naturally converge on behaviors that avoid catastrophic faults. Combined with engineered constraints (speed limits, pause buttons, etc.), this has kept the robots reliably inside the lab bench boundaries. According to NVIDIA, the overnight runs caused no damage: one can go home at night knowing that the AI loop has its own protective “harness” built in. How It Came Together: From Idea to Open Source. ENPIRE didn’t emerge in a vacuum; it builds on recent ideas in autonomous agent research. For context, NVIDIA’s own 2023 “Eureka” system had already used a language model to write reward functions for robots in simulation. ENPIRE pushes all these concepts into the real world and broadens them: now the agent designs not only rewards but the experimental policy itself, executes it on hardware, and even codes the resets. The results were premiered at NVIDIA’s GTC conference. CEO Jensen Huang and others have been talking up “physical AI” – the vision that robots in factories and labs can learn continuously and autonomously. ENPIRE is the most concrete evidence yet of that vision. NVIDIA’s press materials make a big deal about making this available to the community: the team plans to open-source the entire framework so that “universities, startups, and hobbyists” can run their own self-improving robot labs. Imagine a college lab at home: eight off-the-shelf arms plugged into a powerful PC with ENPIRE running, quietly iterating skills while students sleep. Of course, there remain limitations and unanswered questions. The authors are forthright: each new task must still be engineered to be “self-resetting.” The lab can only automate what someone has set up. For GPU insertion, the robot still needed a handcrafted routine to unplug and re-grasp the card every trial. And unpredictable physics still bite: as noted, two of the agents that aced the Push-T in sim flunked it on the real table. This shows there is no magical cure for the real-world variations and noise. What ENPIRE buys us is speed: it lets agents discover and adapt to those variations far faster by doing real experiments day and night. But the underlying physics still govern success. Importantly, the extra power comes at a cost in complexity and resources. Running eight large LLMs, juggling logs, and coordinating via Git is a heavy engineering effort. Researchers with limited compute might find that a single agent (one robot) is more token-efficient, while those with plenty of GPUs and cloud access would favor the speedy fleet. The paper’s own “Limitations” section discusses filled robot time versus idle time and the fact that the system is currently tailor-made for tasks that can be reset. Future work will need to broaden the kinds of tasks and make setup easier. Why ENPIRE Matters. For the robotics community, ENPIRE is a watershed. It shows that physical AutoResearch – an AI system autonomously forming hypotheses and testing them on real hardware – is finally possible. Removing the human from the trial loop is a game-changer: rather than waiting for the morning shift, researchers can send robots home on Friday and trust them to debug themselves by Monday. This dramatically increases the pace of experimentation. In practice, ENPIRE can be thought of as taking advantage of the same brute-force that revolutionized the software world. Just as compilers and continuous integration let developers iterate code a thousand times overnight, ENPIRE lets us iterate on robot algorithms a thousand times on hardware. The work cites an oft-repeated maxim: in robot labs, the scarce resource is robot time, not computation. By buying more concurrent robot time, ENPIRE accelerates convergence. That said, it acknowledges that humans are still needed for ingenuity and the final touches – this system multiplies a team’s effort, it doesn’t replace all engineering. Looking forward, there are hints of even bigger ideas. The “AutoEnvBench” timeline suggests we might measure robot R&D progress more rigorously, as one would measure ML training curves. The idea of an “idea tree” also hybridizes code version control with research development history – the teams can literally see which agent’s change moved the needle. We may even see competitions of agent teams: which coding agent finds a solution fastest on new robotics challenges? (NVIDIA’s comparisons indicate Codex/GPT-5.5 often led Push-T, for instance, but leave open which agent is “best” overall.). On the safety front, ENPIRE highlights one more necessity: as robots become capable of unsupervised learning, verifying their behavior becomes crucial. The system’s vision-based reward classifiers and self-generated safety controllers are, in effect, the first guardrails. Industry reports already note that deploying AI-driven autonomous robots will make safety testing more complex and necessary. ENPIRE itself could be a platform to run such safety validations: since agents try everything repetitively, any dangerous corner case is likely to be discovered during overnight runs (as long as the safety monitor flags failures rather than letting damage happen). In conclusion, ENPIRE demonstrates an ambitious advance: turning entire robot labs into self-driven research machines run by AI coders. For the first time, and quite dramatically, NVIDIA’s team has shown that AI agents can “find terrible ideas” and debug them in closed loop on hardware. The robots learned GPU insertion and pin threading at a near-production level of reliability, all orchestrated by code generated on the fly. This doesn’t mean we’ve conquered robotics – far from it – but it does suggest that the era of overnight robot training is beginning. As Nvidia puts it, the goal is a day where even Jensen Huang would not notice the lab is still running on its own. For researchers and engineers, ENPIRE offers a glimpse of labs where new skills can be developed without nights of human babysitting. It opens the door to a future where physics experiments proceed at software-like speed, accelerating innovation in manipulation, assembly, and beyond.