Latest / Embodied AI 101 / EVA-Client: A Unified Framework for Real-Robot Policy Iteration

Transcript

EVA-Client: A Unified Framework for Real-Robot Policy Iteration · Embodied AI 101

• EVA-Client: A Unified Framework for Real-Robot Policy Iteration. Modern roboticists often have powerful policy-training tools – vision-language-action models, world models, and large-scale imitation/RL pipelines – yet struggle to deploy those policies cleanly on physical robots. The real-robot side of the loop (connecting a trained model to hardware, running trials, collecting new data, etc.) is still often handled by ad-hoc scripts or bespoke code. EVA-Client, an open-source project from the CoLA Lab group, fills this gap by providing one unified client that handles data collection, deployment, and evaluation for embodied policies on real robots. In short, EVA-Client “closes the loop” of embodied policy iteration: the policy iteration loop becomes a single software pipeline rather than a loose patchwork. The system manages teleoperation data capture, dataset preparation, model checkpoint deployment (with latency compensation and smoothing), real-time control, and logging – feeding each evaluation run’s data back into the next training cycle. In effect, EVA-Client sits between your learned policy (or human teleoperator) and the robot, mediating every real-time execution step in a consistent way. The Iterative Loop in Embodied AI. To understand EVA-Client, recall that an embodied policy (e.g. a robot manipulation model) is not “trained once and done” – it typically requires a cycle of collecting real-world data, training, deploying, and then making fresh evaluations or collecting more data. As the EVA-Client docs explain: “An embodied policy is produced by a closed loop, not a single training run: collect demonstration data on the real robot, train a policy from it, deploy the checkpoint back to the robot, then run on-robot evaluation; whatever the eval exposes feeds the next round of collection, and the model gets stronger loop by loop”. In practice, modern training toolkits (OpenPI, LeRobot, StarVLA, VLA Foundry, etc.) have solved much of the offline training and dataset tooling, but the real-robot execution side has remained fragmented. EVA-Client’s mission is to unify exactly that missing part: data collection, deployment, and evaluation on hardware, all in one framework. EVA-Client views the entire process as one continuous loop. Your robot provides fresh observations and, if in teleop mode, demonstration actions – EVA captures and packages these into a LeRobot-format dataset. The external training stack processes that data and produces a new policy checkpoint. EVA-Client then deploys that checkpoint back onto the robot (potentially substituting human tele-op with model inference) and runs it under controlled conditions. Crucially, every evaluation is also recorded: EVA-Client logs full trajectories (observations, predicted actions, applied commands, camera video, etc.) in a format ready for training. This way, each “test run” doesn’t end as an impressionistic note – it generates a reusable dataset for the next training iteration. In summary, EVA-Client automates the steps of (1) collect/tele-op, (2) show/publish data, (3) deploy policy and execute, and (4) evaluate & log — looping back to (1) again. Architecture and Design. At the heart of EVA-Client is a component-decoupled, modular architecture. The design separates what is controlling the robot (the policy or teleoperator) from how commands are sent and executed. In practice, EVA-Client has multiple pluggable layers:. Transport Layer: abstracts the communication channels. EVA supports transports like ROS1, ROS2, ZeroMQ, or even a “dataset replay” mode. For example, real robot streams via ROS or ZMQ can be captured and buffered into synchronous observation packets. A special transport simply reads pre-recorded episodes for offline testing. Robot Description: a declarative description of your specific hardware (actuator groups, sensor topics, joint limits, etc.). EVA-Client includes built-in support for robots like Dual-AgileX, ARX R5, Franka Emika arms, AgiBot, UR5e, R1 Lite, etc. Each robot class lists its joints, cameras, and how to read/write ROS/ZMQ messages. Adding a new robot is as simple as writing a short description class – the core EVA code then treats it uniformly. Policy Client (Signal Source): the external decision-making engine. This might be a trained VLA (Vision-Language-Action) model running in a separate server (for example, OpenPI, StarVLA, GR00T, DreamZero, etc.), or a human teleoperation source. EVA-Client sends synchronized observations (sensor readings, images, etc.) to the policy backend and receives “action chunks” in return. There are also mock backends: a policy that emits dummy actions and a backend that simply replays actions from recorded data. Inference Strategy: manages how incoming action chunks from the policy are scheduled and applied. EVA-Client supports multiple real-time execution strategies (see next section). This is also pluggable: you can switch between synchronous (one-shot) control, asynchronous prefetching, temporal ensembling, real-time chunking, etc., all through configuration. CLI/Web Interface: a persistent control console (served via web) that drives the session. The EVA-Client UI handles user commands (start/stop collection, step robot, change modes, etc.) and renders a 3D visualization and camera streams. Everything runs through this interface, which loads a Python config file specifying the choice of robot, transport, policy backend, and inference strategy. In practice, a single EVA-Client config file ties together these components. For example, a config might say “use the ROS2 transport to talk to an ARX robot, connect to an OpenPI policy server on port 5000, use the ‘act’ async strategy, and command joint targets.” Swapping the config to a different robot or strategy is all you need to do to change behavior – no code edits required. EVA-Client’s component grid ensures that adding a fresh robot description or an extra inference method “touches only its own layer”. This orthogonal design means any combination of supported robot backends, transports, or policies “runs out of the box,” making the whole system robot- and source-agnostic. Building Blocks and Supported Systems. EVA-Client comes bundled with many defaults for ease of use. For instance, its config system lists:. Transports: , , , or replay. Policy Backends: known model servers like (the CoLA Open-PI server), , , ; plus (dummy actions) and (offline actions). Inference Strategies: , , , , and (see below). Action Spaces: either (send joint angles) or (send end-effector pose). If EEF space is used, EVA-Client runs a continuous inverse kinematics solver (via PyRoki) to convert it to joint commands smoothly. By default EVA-Client includes robot configurations for manipulators like dual AgileX arms, Franka Panda, UR5e with gripper, the ARX R5 industrial arm, AgiBot G2, etc. New robots are supported by simply writing a class (perhaps by copying one of the examples and editing topic names); the framework’s abstraction means no core code changes are needed to integrate a new arm. Additionally, EVA-Client uses Python and a tool called (from Astral Labs) for its environment, and robot-specific drivers (e.g. Franka control) can be added via pip extras. Crucially, EVA-Client itself does not include low-level motor drivers; rather, it assumes each robot has its own running driver program publishing sensor topics and listening to commands. EVA-Client simply “bridges” the gap (over ROS, ZMQ, etc.) between policy outputs and the robot’s control interface. Interactive Collection, Deployment, and Evaluation. EVA-Client operates via a web-based console with three main workflow “tabs”: Collect, Debug/Deploy, and Eval/Result. This unified GUI is one of the system’s key conveniences — it ties together everything from teleop to scoring in one place. Collection: Here, a human operator can teleoperate the robot to gather demonstration data. EVA-Client supports a “slider-driven” manual mode to first verify basic control links (useful for safety checks). Then, in the Collect tab, the human drives the actual task while EVA-Client records each attempt as an “episode” in LeRobot dataset format. For example, in a “leader-follower” setup one arm is moved by the operator and the other mirrors it; EVA-Client logs synchronized (joint states, images, etc.) along with the teleop commands. Completed episodes are automatically run through QC algorithms: the system checks timestamps, frame integrity, data formats, etc., flagging any problem (e.g. dropped frames). The operator can then replay episodes open-loop in the console (without the robot or policy) to annotate each as pass/fail or add notes for any anomalies. All good episodes become structured training data: each step has state and action vectors and video frames on disk, ready for behavior-cloning or offline RL. Replay (Offline): Even without a robot, a user can load a recorded dataset into EVA-Client using the transport. This replays episodes into the 3D visualizer and QC tools for inspection. It’s a convenient way to relabel data or debug trajectories without live hardware. Debug/Deployment: In this mode, the trained policy (policy server) is connected and the robot actually executes actions. EVA-Client provides a range of control granularities. In sim-only mode, policy actions route only to the 3D scene (no real hardware) for a preliminary check. There is also a “step-by-chunk” mode: EVA-Client sends one action-chunk to the robot, pauses, then waits for user confirmation before continuing – this lets the operator verify each chunk’s effect (e.g. each waypoint) before proceeding. Finally, a full continuous execution mode smoothly steps at whatever rate the policy can produce. Throughout, the same console UI shows live camera streams, joint charts, and the 3D preview so the user can monitor the run. All action commands, raw predictions, and actual executed trajectories are logged for later analysis. Evaluation: EVA-Client treats evaluation as a first-class citizen. In the EVAL tab, one can load multiple model checkpoints (scored as “Model A, B, C…” etc) and run them under identical conditions on the robot. The system drives all models through a set of scripted test scenes or instruction-filled tasks, earning a score or success metric for each. EVA-Client logs three parallel action streams: (a) the raw model prediction chunks, (b) the smoothed control commands after the strategy, and (c) the robot’s executed joint commands. In the RESULT tab, a synchronized viewer plays back these parallel streams and camera video side-by-side. This lets the researcher drill into exactly where different models succeeded or failed. EVA-Client’s goal is to turn subjective, bench-top robot tests into “auditable, video-linked records” — making experiments reproducible by design. At the end of an evaluation run, all logged episodes join the dataset for future training, literally ready to feed the next iteration. In short, the EVA console brings together tele-op data collection, deployment debugging, and checkpoint evaluation in one integrated UI. Users never have to write new scripts to record one thing or the other; the same session and config handles it all. Switch from recording to testing simply by clicking a different tab or loading a different config. For people used to juggling ROS nodes and OpenCV windows, this unified interface is a major productivity boost. Real-Time Inference Strategies. A central challenge for real-robot deployment is dealing with latency and timing. Learned policies often produce actions in short “chunks” or segments, but inference takes time, and humans can’t quickly flip modes. EVA-Client provides multiple built-in strategies to handle this, essentially different scheduling/smoothing methods for the chunked output. These have been unified under one interface, letting a user pick or swap among them without rewriting code. The main strategies are:. Synchronous execution: The simplest mode. EVA sends a chunk of predicted commands (say, 0.5 seconds of motion) to the robot and waits until it completes before requesting the next chunk. This guarantees no command overlap, but produces a “pause-and-go” effect in the motion: the robot moves, then stops while the policy computes the next piece. It’s easy to implement and sometimes OK for slow teleop policies, but can feel jerky for faster controllers. Asynchronous (linear-overlap) execution: EVA will request the next action chunk before the first finishes, overlapping computation and execution. To handle overlap, the new chunk comes in trimmed by the known policy latency, and the two overlapping segments are linearly blended over their intersection. The result is continuous, smooth motion. In other words, EVA filters out the inference delay by partially overlapping commands. Temporal ensembling (ACT-style): EVA performs an exponential moving average over overlapping chunks. When a new chunk is predicted, the final command sent to the robot is a weighted average of the fresh chunk and the tail of the old one (with weights decayed over time). This smoothing by ensembling helps mask sudden changes between predictions. It’s similar to methods used in online continual control (like ACT training). Naive asynchronous: A simpler form of overlap: EVA immediately replaces the current buffer with the new chunk without any special smoothing (beyond the intrinsic physics). This tends to give quick response but can jump if predictions vary – it essentially “chunk-replaces” the motion. Real-Time Chunking (RTC): A hybrid approach that also uses overlap smoothing. With RTC, the policy server itself conditions on more context (essentially running a partial rollout), then EVA applies a linear blending to the result. It’s the only strategy that does smoothing both on the server side and client side. The goal is to achieve seamless continuous motion while giving the policy the maximum real-time context. By default, EVA-Client’s dispatcher can switch among these strategies on the fly, and the evaluation logs include all intermediate streams so you can debug which strategy was in use. In practice, the async methods with blending or ensembling yield fluid motion on hardware; synchronous mode is more of a baseline or debug mode. Figure illustrations in the docs show how a single planned trajectory can look in each mode. The key point is that EVA-Client supports them uniformly: you configure “which strategy” and EVA handles the scheduling accordingly. This way, researchers can experiment with these different real-time execution methods in one codebase and directly compare their impact on task success. Logging, Visualization, and Feedback. Beyond real-time control, EVA-Client emphasizes visibility and data for analysis. Every run (collection or evaluation) produces comprehensive logs: sensor readings, raw and filtered actions, joint states, and rendered video. EVA automatically timestamps everything and enforces a fixed-rate assumption by inserting synthetic timestamps when needed. These logs feed both the 3D visualizer and the episode dataset, ensuring nothing is lost. Crucially, EVA-Client’s evaluation mode provides a side-by-side viewer for inspecting multiple model checkpoints. After an evaluation sweep, you get synchronized playback of each model’s performance: video streams, a 3D scene, and even plotted joint trajectories, all aligned step-by-step. This “auditable evidence” lets you pinpoint exactly where model A did better than model B (for example, one spilled an object and the other didn’t) under identical conditions. In short, EVA-Client transforms messy hardware tests into structured experiments with clear metrics. Because every evaluation run is logged, it becomes additional training data out of the box. As the authors note, “every evaluation run doubles as a data collection, recording full rollouts in training-ready format alongside exhaustive logs. so each evaluation feeds the next round of training rather than ending as an unrecorded impression”. This seamless feedback loop is valuable for iterative improvement: if a model fails in a certain scenario, you already have that failure trace to inject into the training set and try again. Integration and Usage. Getting started with EVA-Client is straightforward for anyone familiar with ROS or Python. The whole system runs as a Python 3.10+ package. You install it (via the tool) and then run a single command:. Each config is just a small Python file that names the robot and selects the transport, policy backend, and strategy. Multiple example config templates are provided for supported robots. After launching, EVA prints a local URL (e.g. ) where the web console is accessible. Because it’s MIT-licensed and on GitHub, EVA-Client can be forked or adjusted as needed. The core team maintains ready-made configs for common tasks (the docs show one for an ARX arm, one for dual AgileX, etc.), making it easy to test on your own hardware. For new robots or sensors, a user only needs to write or adapt the small “robot description” and install any needed ROS drivers. In fact, the documentation explicitly notes: “EVA is only the connective layer. each robot talks over its own middleware. so adding a robot or strategy touches only the corresponding layer”. The project page even has a clickable element “Add your own” to guide writing a new robot spec. The UI console hides most of the complexity. In the Debug tab you can ramp up from simulation preview to full real-time as a safe progression. In the Collect tab you simply press a button to start/stop recording an episode (with QC checks happening automatically). In the Eval tab you load a list of checkpoint folders and it sweeps through them, keeping all trials aligned. Useful utility: during collection or evaluation, EVA-Client logs the raw images as H.264 videos (instead of per-frame dumps) – much easier for bandwidth – and syncs them with the data logs. Overall, EVA-Client tries to be flexible to common robotics setups. It works with ROS1 or ROS2 topics, or even bare-metal ZMQ for custom channels. It is agnostic to the type of policy server (it could even be a future RL server or a proprietary controller) as long as it outputs actions in time. It has built-in IK solvers so even if your policy only outputs an end-effector pose, EVA-Client can control a joint-arm smoothly. The system also includes development fixtures: a policy for dry runs, a “dataset” transport to test everything with a phantom replay, and extras like faster 3D visualization if desired. Impact and Future Directions. EVA-Client addresses a long-standing need in embodied AI. As the team notes, “training frameworks have converged. solve much of the training-side stack. The real-robot side should not still be a patchwork of project-specific scripts”. By open-sourcing this unified client, the CoLA Lab group is effectively providing the missing infrastructure for real-robot policy iteration. This matters because it lowers the barrier to real-hardware research. Instead of every lab writing its own data-logging sneakers, they can share EVA-Client config recipes. It also makes published results more reproducible: a paper could tag the EVA setup and others could replay the exact same evaluation scenario. We should note EVA-Client draws inspiration from prior works. LeRobot format (used here for episodes) and frameworks like OpenPI / StarVLA / GR00T have built trust in how to train policies; now EVA-Client completes the circle by handling real-time execution with those policies. In fact, EVA-Client was written by teams (from Beihang University and collaborators) who also developed LeRobot and OpenPI, so it slots neatly into that ecosystem. For example, the EVA-Client docs explicitly show integration with OpenPI, DreamZero, StarVLA, etc. This compatibility means if you trained a generalist VLA policy in one of those systems, you can use EVA-Client to plug-and-play it onto a real arm with minimal glue code. In terms of experiments, the EVA-Client report (and ongoing development) mention examples like collecting teleop driving policies or running pick-and-place trials on different robots. The focus is on the tooling rather than beating benchmarks. One demonstration noted in the docs is: “The paper uses the dataset to train a driving policy; no explicit training split or mixture ratios are described, but the data is prepared in a task-specific, training-ready format”. The fact that EVA-Client is already being used to bootstrap such tasks speaks to its utility. In practice, metrics would be task-specific (task success rates, speed, etc.), but EVA-Client’s innovation is that it exposes those metrics in a standardized way (e.g. side-by-side runs, scoring per checkpoint). Looking forward, EVA-Client is poised to become a common backend for real-robot evaluation. Projects that develop new VLAs or visuomotor policies can integrate EVA-Client to “spin up” hardware tests quickly. The plugin-like architecture means it could grow to support new domains (e.g. mobile robots, drones) or new sensing modalities with only local additions. Moreover, since it is open-source, other labs can contribute new inference strategies or quality-control rules. The developers even note the system is under continuous improvement (“ongoing development… with regular updates”). For the embodied-AI field, such a tool can accelerate progress. It aligns with the trend of “closing the reality gap” by making real-world experiments as routine as simulations. By automatically logging real-world runs as training data, EVA-Client also helps treat the physical environment as part of the learning loop. Ultimately, it helps ensure that advances in vision-language or world models actually translate to real robot capabilities, without being bottlenecked by fragile deployment code. In summary, EVA-Client provides “the missing real-robot infrastructure” for policy iteration. It unifies teleoperation capture, deployment, and evaluation in a modular architecture, offers powerful debug modes (from sim preview to full run), and automatically produces high-quality logged data for continuous learning. For practitioners in robotics and embodied AI, EVA-Client can save months of engineering effort and make experiments cleaner and repeatable. As such, it represents a valuable step toward standardized, end-to-end robotic learning pipelines. References: See the EVA-Client project page and documentation for in-depth details. The code is open on GitHub, and an arXiv manuscript (Yang et al. 2026) provides a full write-up of the system. Citing this infrastructure will be important as it gains adoption.