Latest / Embodied AI 101 / RoboMeter: Learning Dense Rewards from Successes and Failures

Transcript

RoboMeter: Learning Dense Rewards from Successes and Failures · Embodied AI 101

• Robometer: Learning Dense Rewards from Successes and Failures. Designing reward functions is a central challenge in robotics. In traditional reinforcement learning (RL), great effort goes into hand-crafting a reward that guides a policy to the goal. If we use imitation, we simply clone demonstrations, sidestepping reward design at the cost of generality. A more recent trend is to learn reward models from data – analogous to how large language models use human feedback – so that a single reward function can serve many tasks and robots. General-purpose reward modeling holds the promise of flexible, language-conditioned objectives: give the robot a video and a text instruction, and the reward model tells it how well the video fulfills the instruction at every moment. Most prior “foundational” approaches to learned rewards have, however, trained on only clean, expert demonstrations. These methods (e.g. progress predictors or vision-language critics) regress on the fraction of a task completed in an expert trajectory, producing a per-frame progress signal. That works well on perfect runs, but breaks down in the real world where we collect many failed or sub-optimal episodes. Those failures are ambiguous: you can’t easily label how “50% done” a crashed attempt is, nor assign dense feedback to them. As Anthony Liang and colleagues point out, this leaves huge amounts of robotic data unusable. Yet in practice, robots fail a lot during learning. Enter Robometer (Liang et al., RSS 2026) – a new framework for reward modeling that can make use of both the good and the bad trajectories. The key insight is deceptively simple: train the model not just to predict progress from expert demos, but also to compare pairs of trajectories and say which one is better. In RLHF style, relative preferences provide supervision where absolute labels are hard. The result is a dense, general-purpose reward model that is trained on over a million video trajectories (the new RBM-1M dataset), including many failures, and that yields much more robust performance. Liang et al. show that Robometer’s learned reward generalizes better – both quantitatively and in downstream robot learning – than previous methods. In this article we unpack Robometer’s approach and results. We’ll explain how it combines frame-level progress regression on expert trajectories with trajectory-level preference ranking across all data. We’ll look at the architecture (a video-language model with special tokens for progress and preference), the massive mixed-quality dataset, and the training recipe (including clever “video rewind” augmentations). Finally we survey the evaluations: Robometer achieves dramatically higher alignment with true progress on held-out data, and it unlocks gains in zero-shot success detection, imitation retrieval, and RL policy learning. Throughout, we compare to recent baselines like RoboReward, GVL, VLAC, RoboDopamine, and TOPReward, highlighting how Robometer improves upon them in practice. Beyond Progress: The Case for Preference Supervision. The usual way to train a vision-language reward model is to take expert demonstrations for a given task (a video of a robot completing “Put the red block on the blue plate”, for example), label the start as “0% done” and the end as “100% done,” and have the model predict a smoothly increasing score. This gives a dense, shaped reward signal for an agent: you know precisely how far along the task has been completed at any frame. This approach has worked surprisingly well when we only care about the expert runs. However, Liang et al. argue that this “progress-only” paradigm hits a wall when scaling to large, diverse data from the wild. In typical robotics datasets, expert data is scarce relative to all the trials you gather. Imagine a robot learning to stack blocks: you might collect thousands of attempts, but only a handful are perfect. The rest might put the wrong block, drop something, miss the target, or even do nothing. Labeling each frame of all those videos with a percentage complete is infeasible and ambiguous. What does “30% done” mean for a failure?. As a result, most approaches simply throw away the sub-optimal data, or at best use it extremely coarsely. Liang et al. note that existing reward models (like the earlier RoboReward or GVL methods) rely on such frame-wise progress labels and thus “scale poorly” to rich datasets where failed and suboptimal runs are the norm. The consequence is that the learned reward lacks robustness: it may do well on nice demos but fail to understand when things go wrong, or how badly. Robometer’s core idea is to treat trajectory-level comparisons as a second, complementary supervision. Instead of needing to assign an absolute score to every frame of a bad run, we can simply let the model see two runs of the same task (or one run and a slightly altered version) and teach it “trajectory A was better than trajectory B.” This is akin to how humans learn; if you watch two people attempt a task, you can usually tell which one did better without saying exactly how much they completed. By combining these two signals – frame-level progress on known-good runs and pairwise preference judgments on pairs of runs – Robometer can filter in massive amounts of otherwise unusable data. The authors describe this as a dual-objective reward modeling framework:. A frame-level progress loss that anchors the reward magnitude to expert data. Concretely, the model is trained to predict a progress percentage on each frame of an expert demonstration, via a categorical (C51-style) distribution. This sets an absolute scale: 0% at the start and 100% at task completion. A trajectory-comparison preference loss that imposes a global ordering on pairs of videos for the same task. Given two videos for “Task X,” the model learns to prefer the one where the robot did a better job. This can apply to one expert vs. one near-expert, or even to two suboptimal attempts. Crucially, this preference labeling is binary (which is better: video A or B?), so it can be obtained far more cheaply at scale. With these, Robometer can literally learn from failures. In the paper’s words, the preference loss “imposes global ordering constraints across trajectories of the same task, enabling effective learning from both real and augmented failed trajectories”. In practice, the authors form preference pairs in three ways (we’ll detail these below), each of which adds different insights. Some pairs contrast a perfect demo against a flawed one (strong supervision). Others pair two different tasks with mis-matched instructions (to ensure the model doesn’t reward a video for the wrong goal). And some artificially corrupt a video to simulate a failure (using “video rewind”) so the model sees an almost exact pair where one is purposefully worse. Importantly, by anchoring the reward scale on expert runs and only requiring relative judgments elsewhere, Robometer sidesteps the need to manually label progress on failed runs. This unlocks enormous flexibility: one can train on hundreds of thousands of trajectories of varying quality, gleaning signal from each. As Liang et al. note, this makes the rewards both well-calibrated (thanks to the anchored progress) and dense (thanks to all the preference data), enabling zero-shot policy learning. In short, the dual-objective – progress + preference – is the innovation that lets Robometer be “general purpose” and scalable. The next sections explain how to turn that idea into an actual model and dataset. RBM-1M: A Million Trajectories of Robot Behavior. A common theme in recent robotics is massive data. Robometer embodies this: the authors assembled RBM-1M, a collection of over one million trajectory videos spanning everything from simulators to real robots, across at least 21 different embodiments and environments. Importantly, this dataset deliberately mixes expert, noisy, human, simulated, and failed episodes. The goal is to give the reward model a truly broad playground so it can generalize widely. Some highlights of RBM-1M:. Diversity of robots and tasks. The dataset includes many kinds of manipulators (e.g. Franka Panda arms, wrist robots, hand models), mobile manipulators, and even human videos. Task categories range from placing and stacking to open-ended object interactions. By covering “an unprecedented diversity of embodiments and task categories,” Robometer is exposed to very different backgrounds, camera angles, object sets, and failure modes. Mixed quality. Instead of filtering only the clean demos, RBM-1M contains a large fraction of suboptimal and failed trials. For example, in stacking tasks you’ll find videos where the robot almost finishes but drops the block at the end, or where it grabs the wrong object, or simply fails to move. In peg-in-hole tasks there are binding failures, mis-alignments, and so on. This “richness” is exactly what preference training can exploit. Scale. By collecting over a million trajectories, Liang et al. ensure that the model sees enough examples of each situation to learn statistically meaningful signals. Even rare failure modes appear multiple times. Figure 1 in the paper (and similar t-SNE plots) illustrates how RBM-1M spans a broad range of scenarios. You can see clusters of “expert” vs “novel failure” in that embedding space. In one summary, Robometer’s creators highlight that RBM-1M has tens of thousands of trajectories from mixed-expertise datasets, showing just how comprehensive it is. This dataset is a significant resource in itself. The authors plan to release RBM-1M (they already share an out-of-distribution eval set, and promise the training data soon). In effect, they’ve built a foundation on which their model sits. But of course, a dataset alone does nothing without the right model to consume it – which brings us to the architecture of Robometer. The Robometer Model: A Video-Language Transformer with Special Tokens. At its heart, Robometer is a vision-language transformer – specifically, a fine-tuned Qwen3-VL-4B model (a 4-billion-parameter causal VLM). The model takes as input a text instruction plus either one or two video trajectories, and produces outputs that correspond to (a) the progress at each frame of the first video, (b) a binary preference result for which of two videos is better, and (c) a final success indicator. Let’s unpack how this is done. Backbone and Input. Robometer starts from Qwen3-VL-4B-Instruct, a pretrained video-language model that can ingest frames and text. The inputs are:. Language instruction. This is the natural-language task description (e.g. “Place the red block on the blue plate.”). It’s fed in as usual to the transformer’s text encoder portion. Video frames. Each trajectory is a sequence of images. In practice, Robometer subsamples each video down to 8 frames (so it sees a coarse version of the motion). Each frame is tokenized (often via a pre-trained image tokenizer) and fed into the model in sequence, interleaved after the text. For preference training, two videos are fed one after the other in the same prompt (so the model effectively “watches” both in series). Robometer uses causal attention, meaning that at any point it only sees past and current frames (not future frames in the same video). This is appropriate since we want to predict progress sequentially. The “causal masker” ensures that the outputs respect the video order. Special Tokens for Progress and Preference. Crucially, the model is augmented with learned tokens that signal where and what to output. There are two key types of extra tokens inserted into the input stream:. Progress tokens: After each frame of the first trajectory (the one we’re getting a reward for), Robometer inserts a special learned token (let’s call it T_progress). The hidden state corresponding to this token is trained to output the percentage of the task completed at that frame, on a discrete scale from 0 to 100%. In practice this is done as a classification into, say, 51 bins (like the C51 distributional RL trick). By placing one token per frame, the model learns a frame-level reward curve. Success token: At the very end of the first trajectory, an additional learned “success” token is appended. Its hidden state goes through a binary classifier that predicts whether the task has been achieved (a success signal). This allows the model to explicitly output 0/1 at the end, complementing the progress percentage. Preference token: If two trajectories are fed in, Robometer finally appends one more learned token after the second video. This token’s hidden state is trained to output which of the two videos better satisfies the instruction. Practically, this is done via a binary cross-entropy loss: the model’s output for “video 1 is better” vs “video 2 is better” is compared to the human-generated label on that pair. These tokens are clever because they let the same underlying transformer architecture produce multiple signals. Figure 2 in the paper (and on the project website) illustrates this: the video frames go through the VLM, but at each frame step the progress token can “read off” the hidden embedding and learn to predict how far along the task is. Meanwhile, the preference token at the very end attends over both trajectories (so it sees a global summary of each) and learns a comparative judgment [20†L96-L104]. Because the model has a large vision-language backbone, it can ground these tokens in both pixel and language semantics: e.g., the progress token at frame t might attend to the presence of the block, the gripper position, etc. And the preference token uses cross-attention across videos to say which scene looks closer to completion, given the text prompt. Losses and Training Heads. Summing up, Robometer has two main “heads” with loss functions:. A progress head (categorical C51 style) for the progress tokens, using cross-entropy. During training, the expert (fully successful) demonstration provides “ground truth” progress: at frame 0 progress=0%, at final frame progress=100%, and linear interpolation in between. Thus the model learns to produce an increasing curve. A preference head (binary classifier) for the preference token, using binary cross-entropy (BCE). Each training pair of trajectories comes with a label (1 = first video is better, 0 = second is better). Optionally, the success token can use another BCE for end success (though the paper emphasizes the progress/regression aspects more). In implementation, they set and when training, indicating both losses are active. The total loss is a weighted sum of these two (often just added together). One nice detail: by discretizing progress into bins, they avoid regression instability and allow the model to predict a full distribution over percent-complete, which empirically works better. The final reward signal (used for any downstream policy) can still be taken as the expectation of that distribution. In summary, Robometer modifies a standard VLM by planting learned tokens into the causal sequence, enabling it to simultaneously score individual frames and compare entire videos. This multi-output design is a departure from vanilla language models: it’s effectively multitask-learning embedded in the sequence. Training with Mixed Trajectories: Pairing Strategies. Having set up the architecture, the next question is how to generate training examples for the preference comparisons. Liang et al. use three main strategies to form pairs of videos (and instruct which one is better):. Progress-based comparisons (expert vs suboptimal). For a given task instruction, they take one expert demonstration (where the task is completed) and pair it with a suboptimal or failed trajectory of the same task. Obviously, the label is “expert is better.” This most directly teaches the model that doing the job fully is superior to doing it poorly. Since RBM-1M includes many trajectories labeled by success or known quality, these pairs are easy to form. Instruction negatives (different tasks). To ensure the model learns to align the correct video with the given instruction, they also pair videos from different tasks. For example, one video might show “stack block,” another shows “open drawer,” but the caption instructs “stack the block.” Clearly the correct match is the video that actually does stack. Training the preference head on these mismatched pairs teaches global semantics: the model must learn what each instruction means and recognize which video fulfills it. It says “A is about the right task, B is not; prefer A.”. Video rewind augmentations (simulated failures). This is a clever synthetic strategy borrowed from the earlier ReWiND work: take a good trajectory and inject a failure artificiality. For instance, reverse a segment in the middle, like having the block pile drop off two frames early. This creates a new video that looks almost identical to the original demo except for a glitch. Then the model is asked to prefer the uncorrupted original over this “rewound” one. This teaches the model that interruptions and backtracking are bad – it should learn that progress can decrease if something is undone, which it otherwise might never see. These pairing schemes cover a lot of ground. The first uses actual labels (human/expert vs non-expert). The second uses the fact that each video has an associated task caption. The third injects even more variety by data augmentation. Crucially, all of these labels are obtained without any extra annotation effort beyond what’s already in RBM-1M (trajectories, success labels, task text). In practice, any given training step on Robometer samples some batch of (video, instruction) pairs and forms a mix of tasks: some examples feed only one video through the model (for progress training on an expert demo), while others feed two videos (for preference training). The preference examples come with labels as above. The losses from progress and preference are both backpropagated into the shared transformer backbone, gradually shaping it. One nice case: Because masters of different expertise are paired, the system learns reward calibration. The progress loss on experts pins the scale (ensuring 0% start and 100% goal), so that when it later sees a pair like (expert vs failed) it doesn’t just trivially predict “expert is better” without any magnitude sense – it has a consistent scale of progress to compare. Liang et al. note that the progress head therefore “anchors the magnitude of predicted rewards”, while the preference loss “imposes global ordering constraints”. Together they yield a well-calibrated score. Finally, it’s worth mentioning that Robometer can be fine-tuned or adapted to subsets of tasks. They provide, for example, a version “Robometer-LIBERO” fine-tuned on specific categories. But the core model is multi-domain. Qualitative Behavior: Reward Curves and Failure Modes. What do Robometer’s rewards actually look like on real videos? The paper shows some illuminating examples. Imagine an expert demo alongside a near-miss failure, like stacking a block but dropping it at the end. A naive progress model would mostly be fooled, continuing to give high reward almost to the end because it only knows about perfectly success-aligned frames. In contrast, Robometer’s reward usage of preference training gives it a “sense” of failure. For example, Liang et al. illustrate a case where a robot is placing an object but then drops it in mid-task. The Robometer-predicted reward curve climbs as in the successful demo, then drops sharply at the point of failure, correctly indicating a regression in progress. The older model (without preference training) might have gone through the roof blindly. This shows Robometer actually learning that missteps rewind your progress and deserve punishment. Another scenario is quasi-success (almost there) versus perfect success. The model gives slightly lower final reward to the imperfect run (thanks to the preference signal), even if the forward motion looked similar. Or consider an irrelevant video paired with a task: Robometer learns to assign near-zero rewards to a video that doesn’t match the instruction at all, because during training it saw many “instruction negative” examples. In short, Robometer’s reward predictions are more nuanced on novel trajectories. It can detect subtle failures (wrong object, stalling, oscillation) as visible dips or plateaus in the curve. The project website shows animations where the score drops exactly when an object tumbles or a grasp fails – something learned from the rewind and failure examples. These qualitative gains translate into better ranking and discrimination. In one offline evaluation, the authors scored every video–instruction pair and checked if the highest reward occurred on the correct demo. Robometer produced very sharp “diagonal” confusion matrices – it almost always gave higher reward to the true match than to other combinations. In contrast, previous models exhibited blurrier matrices. In essence, Robometer learned a much clearer mapping: a video is worth high reward only if it actually aligns with the language goal and doesn’t contain obvious mistakes. Quantitative Results: We Beat the Baselines. Robometer’s improvements aren’t just qualitative – they shine in numbers too. The authors evaluated reward-model performance on several fronts:. Reward alignment (Value Order Correlation): This metric (akin to Pearson correlation) measures how well the model’s predicted reward ranks episodes in the same order as the ground-truth progress. Two benchmark datasets were used: RBM-EVAL-ID (in-distribution) and RBM-EVAL-OOD (out-of-distribution, with new robots and tasks). Across the board, Robometer vastly outperforms prior work. For RBM-EVAL-OOD, Robometer’s predicted rewards have r ≈ 0.95 correlation with true progress. By contrast, strong earlier models like RoboReward (trained on only ~100K demos) topped out around r ≈ 0.88, and simple baselines like GVL or VLAC were near 0.2. This shows Robometer is capturing task progress far more faithfully, even on unseen conditions. Trajectory ranking (Kendall’s τ): For each task, they also check if the model correctly sorts a set of trajectories by quality. Robometer achieved τ ≈ 0.66 on the OOD set, whereas RoboReward got ≈0.5 and other methods were even lower. In practical terms, if you ask Robometer to pick the best trajectory out of a mixed bag, it’s right more than twice as often as before. Failure detection: By thresholding the reward (not directly supervised for this, but emergent), Robometer can label an episode as failed if its final score is low. In a zero-shot test on new tasks, the model’s failure-vs-success classification F1 score was higher than any competitor. It spotted irreversible failures (like dropped objects) with particularly high recall – exactly as the preference training intended. RoboRewardBench (external test): On an independent benchmark of robot tasks, Robometer achieved a mean absolute error (MAE) of 0.72 in predicting progress, better than RoboReward’s 0.85 and competitive with even larger closed-source models. We can sum up: Robometer yields significantly better alignment with “ground truth” reward than any past generalist reward model. This is not just due to more parameters or data (though it uses both); it specifically comes from the pairwise training. The paper reports controlled ablations confirming this: even without any added failure data, simply adding preference supervision on the expert-only subset improved ranking (on LIBERO tasks) from τ=0.63 to 0.74. When reality’s failures are added, performance skyrocketed (τ=0.92 on that set). Downstream: Better RL, IL, and More. A reward model must prove useful for control. Liang et al. test Robometer rewards in a variety of robotics learning settings, showing large gains across the board. Online RL (Model-Free and Model-Based). In continuous simulation tasks (like those in the LIBERO suite), Robometer is plugged into a diffusion-based policy learner (DSRL). Without human intervention, it provides the reward signal that the agent uses to explore and optimize. Remarkably, this automates the training loop. The authors report that policies guided by Robometer rewards achieve 85% success in single-stage tasks (versus only 20% with a naive baseline), and similarly big improvements in multi-stage tasks. That’s over a 4× jump in performance. Moreover, a direct comparison to the best prior reward model shows a 2.5× win: Robometer outperforms the RoboReward-driven agent by a factor of 2.5 in success rate. These are huge margins: the lesson is that a scorched-earth policy of train-from-failure pays off. They also tried model-based RL. Incorporating Robometer into a Dreamer-like planner (DreamZero) improved the planner’s ability to rank trajectories and pick good actions. One example given is a block insertion task: using Robometer increased the number of successful insertion attempts compared to using a sparse or poorly shaped reward. (Numerical details are in the appendix of the paper.) Again, the intuition holds: a dense, informative reward accelerates any RL that uses it. Offline RL and Imitation. Since Robometer provides a learned reward that aligns with instructions, it can be used for offline RL from datasets that mix high- and low-quality data. Using Implicit Q-Learning (IQL) on buffers of expert+noisy trajectories, policies learned under Robometer’s reward achieved 2.4× higher normalized returns than those using a naïve or sparse reward. Notably, the gains were especially large at lower discount factors (γ), where intermediate reward guidance matters more – exactly where a dense Robometer signal helps the agent learn every little step. For imitation learning, Robometer can filter or rank demonstrations. The team demonstrated a retrieval scenario: given a library of subtrajectories, use Robometer to pick the ones that best fit the current task. Training an IL agent on the top-ranked segments alone led to much better behavior. Specifically, Robometer-based retrieval boosted downstream success rates by 4.5× over a baseline selection strategy. Almost as importantly, they found that both progress-based and preference-based retrieval objectives worked well, meaning you can use either signal or both to select good demos. Failure Detection and Other Utilities. Beyond learning policies, Robometer’s rewards serve as a tool for monitoring. Because the model explicitly signals regressions, it excels at zero-shot failure detection: simply threshold the reward curve to detect when a trial has gone off the rails. On held-out tasks, Robometer achieved the highest average F1 score of any method for identifying failures. It flagged not only catastrophic drops but also more subtle “non-terminal” failures (like doing the wrong motion). This could be very useful in practice for robotics debugging or safety. Putting It All Together: Comparisons and Conclusions. How far have we come? The results speak clearly. On the standard benchmarks, general-purpose reward models used to have, say, only ~0.2 correlation (VOC) with true progress, or moderate ranking (τ≈0.4). Robometer pushes those numbers into the 0.9 range, essentially saturating what a learned model can reasonably do without full on-policy RL. For context, the paper’s tables show:. RBM-EVAL-OOD VOC: Robometer ≈0.93–0.95 (Pearson r) versus RoboReward ≈0.88, TOPReward 0.78, RoboDopamine 0.74, and GVL/VLAC around 0.15–0.2. RBM-EVAL-OOD rank (Kendall τ): Robometer ≈0.66 versus RoboReward ≈0.5, TOPReward 0.22, RoboDopamine 0.39, etc. These gains are not minor tweaks; they represent a qualitatively better reward signal. In their words, “Robometer learns more generalizable reward functions than prior methods and improves robot learning performance across a diverse set of applications”. The improvement comes from two sources: data and training. Robometer trained on 1M trajectories -- an order of magnitude above earlier efforts -- and on that data it applied comparisons. Crucially, even if one had unlimited compute, you'd still need to decide how to supervise the model: Robometer’s method of dual supervision is the novel part. The ablations in the paper confirm this: they tried a “ReWiND” model (basically the same architecture without preference loss) and a pretrained Qwen3-VL with only progress; these lagged far behind the full Robometer model. In other words, the architecture is powerful, but without the preference loss it doesn’t achieve this high level. The authors are careful to point out some limitations. For example, because they subsample just 8 frames per video, very fast or subtle failures might go unseen. Also, the model only sees camera images; it has no direct notion of forces or contacts. So phenomena that require proprioception or fine timing may be missed. They suggest future work could involve denser video, multi-modal inputs (force sensors, depth, etc.), or question-answering style feedback to cover rare cases. Another interesting point: Robometer, like many VLM-based methods, relies on strong pretraining. The authors experimented with training a bigger “ReWind 34B” model from scratch and found it did worse. This suggests that the pretraining on internet-scale image/text corpora was still a big factor. (In other words, this is “fine-tuning a foundation model” more than training tabula rasa.). But with those caveats, the outlook is bright. Robometer represents a new paradigm: “reward functions” are no longer handcrafted or sparse or low-dimensional. They become learned, high-bandwidth functions that can incorporate language and vision. Liang et al. end by stating that with an approach like this, huge robotic datasets are no longer bottlenecked by ambiguous reward labels; preference comparisons unlock dense performance signals even from failed behaviors. That could be transformative for both RL and imitation learning: rather than needing human engineers to label or filter things, we have models that can do it at scale. In practical terms, Robometer and its RBM-1M dataset provide a ready-to-use core reward model for many tasks (and they’ve released the code, weights, and tools). A roboticist could take the 4B-parameter checkpoint, plug in their own videos and instructions via the provided inference server, and get immediate reward curves. They even supply a browser-based “RewardEval” demonstration. One can also fine-tune with LoRA on a new dataset to adapt to novel domains. Looking ahead, it’s easy to imagine extending this approach further. Could we add richer supervision, like asking “how many blocks are stacked?” or “is the object at location X?” as additional heads? Could we train temporally finer-grained models that don't subsample, using better video codecs or attention tricks? And of course, applying this at the real-robot level (continuous states, unpredictable lighting, etc) will test its limits. What Robometer clearly shows is that preference-based reward learning – a small idea in principle – scales up very powerfully when given vast data and a strong model. By formalizing a hybrid of “reflective feedback” and “expert grading,” it turns messy data into a consistent reward landscape. For many downstream tasks (RL, offline RL, imitation retrieval, failure detection), our performance improves by orders of magnitude. That is a strong case that this approach is on to something valuable. In summary, Robometer demonstrates that a reward model can be both dense and general-purpose by combining two training signals. It learns a frame-by-frame progress score like previous models, but it also learns to compare whole trajectories, learning from successes and failures alike. This lets it ingest large uncurated datasets and output reward functions that make sense zero-shot on new robots and tasks. The resulting rewards align very well with true task progress, enabling faster learning and more robust policies in practice. For researchers and roboticists, this means off-the-shelf reward models can now come equipped with much more ‘common sense’ about when things go wrong – a major step toward truly general embodied AI. References: Liang et al., Robometer: Scaling General-Purpose Robotic Reward Models via Trajectory Comparisons, RSS 2026 (and project site). They provide code and data at https://robometer.github.io.