Latest / Embodied AI 101 / ComBodied Agents: When the Human Becomes the World Model
Transcript
- ComBodied Agents: When the Human Becomes the World Model. ComBodied Agents: a New Paradigm of Human-Centric Agentic AI • Qianggang Ding, Xingyao Wang, Rui Feng, Zhibin Wang, Feixiang Wang, et al. • Affiliations include Université de Montréal, Mila, A*STAR IAIC, and 13 collaborating universities and institutes • arXiv preprint • 2026. The Missing State in Agentic AI. Suppose an older adult misses a medication dose. A digital agent can send another notification. A mobile manipulator can retrieve the pill bottle. Either system might complete its immediate task while completely misunderstanding the situation. Perhaps the person forgot. Perhaps they are confused. Perhaps the medication caused a side effect. Or perhaps they understood perfectly and chose not to take it. That ambiguity is the starting point for ComBodied Agents. The paper argues that most agent architectures are organized around the wrong state variable. Digital agents optimize transformations of software state: pages, files, tickets, repositories, messages, and database entries. Embodied agents optimize transformations of physical state: robot pose, object configuration, navigation progress, contact state, and task completion. Neither category necessarily treats the person’s evolving physiological, cognitive, emotional, behavioral, and social condition as the primary object of modeling and evaluation. The proposed alternative is a ComBodied Agent: an agent whose sensors, language models, software tools, wearables, robots, and human-service interfaces are all subordinated to one higher-level purpose—supporting a beneficial human trajectory while preserving the person’s agency. The name combines “companion” and “body,” although the proposed category is much broader than conversational companions or wearable assistants. This is an unusually fresh paper. It was submitted on August 11, 2026, and presents a 38-page research agenda rather than an experimentally validated system. That distinction matters throughout the discussion. What the Paper Actually Contributes. First, this is not a new vision-language-action model, a shared policy trained across browsers and robots, or an end-to-end system demonstrated in a household environment. There is no trained ComBodied model, robot evaluation, user study, benchmark leaderboard, or released dataset. The public Papers with Code entry currently lists the arXiv manuscript and PDF, while the paper proposes—but does not yet release—a future evaluation suite called CombodiedBench. Its contribution is instead architectural and conceptual. The authors provide a formal definition, a closed-loop reference architecture, a model of edge–cloud deployment, an evaluation framework, a taxonomy of applications and relationships, and a research agenda covering causal learning, privacy, consent, governance, and long-term human outcomes. That also qualifies the claim that ComBodied Agents “unify” digital and physical action spaces. The paper does not construct a common low-level action representation in the style of a generalist VLA policy. It unifies them at the decision and objective level. A browser operation, spoken explanation, robot motion, caregiver notification, and decision to remain silent are all candidate intervention channels whose value is judged by their effect on the person. The common currency is not an action token. It is the predicted human-state trajectory and the authority under which the intervention occurs. The formal definition has five jointly important properties. The human is the primary state being modeled; interaction is longitudinal rather than episodic; the system can intervene rather than merely answer; work is divided through co-agency rather than default human replacement; and assistance must preserve autonomy, dignity, control, relationships, and long-term capability. These properties define a center of gravity, not a clean product category. A chatbot is not ComBodied merely because it remembers a preference. A wearable is not ComBodied merely because it senses heart rate. Conversely, a system need not have a robot body. It could act through conversation, software, a phone, a wearable, a home robot, or an authorized human caregiver. The Action-Substrate Shift. The paper’s most useful conceptual device is the action substrate. This means the class of target states around which an agent organizes perception, planning, intervention, and evaluation. It is explicitly different from the agent’s immediate interface or actuator. A digital agent’s substrate is primarily digital state. An embodied agent’s substrate is primarily physical or simulated physical state. A ComBodied Agent’s substrate is the evolving human state and the person’s ability to exercise agency. These categories overlap in implementation: a health agent may use a browser and a robot, while an embodied assistant may maintain user preferences. The distinction is what ultimately determines success. For an ML researcher, the resulting problem resembles a partially observed, nonstationary, multi-objective control problem. The true human state is latent. Observations are sparse, noisy, multimodal, and sometimes self-reported. Actions have delayed and uncertain effects. The policy must trade off immediate benefit, safety, capability development, autonomy, and relationship quality while respecting hard constraints. But the human is not merely another environment. The person acts independently, can reject the intervention, revise their goals, correct the model, withdraw consent, or deliberately behave in ways the agent did not predict. The paper’s transition model therefore includes the user’s action and exogenous influences alongside the agent’s intervention. There is also a reflexive quality absent from many conventional world models. A prediction can change the person who hears it. A reminder can cause annoyance; a confident health forecast can cause anxiety; an empathic response can strengthen attachment to the system. The agent’s relationship with the user becomes part of the dynamics it is attempting to model. This reframing explains why conventional task success is insufficient. An AI system can produce a stronger document while leaving its author less able to reason about the subject. It can make the right decision while teaching the user to defer every future decision. It can keep a lonely person engaged while gradually displacing human relationships. The paper’s target is therefore immediate benefit plus what remains with the human after repeated assistance. Inside the Closed Loop. The proposed architecture has four functional blocks: human-state perception, longitudinal memory, a Personal World Model, and an intervention policy. Feedback from the person and environment then updates the loop. Consent, uncertainty, safety, reversibility, and user control constrain every block rather than appearing as a final output filter. From Raw Sensing to Governed Events. The perception layer may receive language, speech, video, physiological measurements, motion traces, communication metadata, environmental context, and institutional records. The paper’s important move is to reject the idea that these should simply be poured into a large context window. Instead, observations are converted into event-evidence records. Such a record identifies what was observed, when and how it was acquired, which interpretation it may support, its confidence and quality, contradictory evidence, sensitivity, consent basis, retention policy, and permissible use. The distinction between stages is fundamental:. observation → reconstructed event → inferred state → predicted trajectory → authorized intervention. An elevated heart-rate reading is an observation. Elevated heart rate followed by unusually slow recovery after exercise is an event. Fatigue is a possible inferred state. Continued poor recovery under further exertion is a predicted trajectory. Advising the person to stop is an intervention requiring separate authorization. This separation prevents two common category errors. Sensor confidence does not establish semantic certainty, and semantic certainty does not establish permission to act. Several weak sensors agreeing with each other may still be wrong, while one explicit correction from the user may outweigh them. More sensing can also increase privacy risk without improving the decision. Consider a hypothetical rehabilitation robot. A slower walking speed, unusual pauses, a statement such as “I’m fine,” and a recent medication change might jointly support several explanations: fatigue, pain, fear of falling, distraction, or a deliberate change in pace. A ComBodied system should preserve those alternatives rather than prematurely collapsing them into “low motivation” and escalating its coaching. Memory as Temporal Evidence, Not a Scrapbook. The memory layer is correspondingly richer than conversation recall. The paper separates episodic events, relatively stable person knowledge, state trajectories, goals and commitments, relationships, prior interventions and responses, and user-control records such as corrections or deletion rules. Intervention-response memory is particularly important. It connects an action to acceptance, rejection, benefit, burden, or harm. A reminder that worked last week should not become a permanent policy without considering context. Likewise, if a user repeatedly dismisses an intervention, the agent should not blindly increase its intensity in pursuit of adherence. Memory must remain inspectable and correctable. The user should be able to determine whether a stored claim came from something they explicitly said, an imported record, a sensor-derived event, or a model inference. Corrections should propagate into retrieval, prediction, and future intervention while retaining enough provenance to audit why the error occurred. The paper later adds a relational requirement: a single global personal memory is unsafe. Health disclosures, workplace context, family events, and intimate conversations may require separate visibility and access boundaries. The fact that information concerns the same person does not make it transferable across every role in that person’s life. The Personal World Model. The Personal World Model, or PWM, is the central predictive component. Given governed event history, current context, user goals, and a candidate scenario, it predicts distributions over future human states, observable events, and relevant outcomes. Candidate scenarios may vary the user’s decision, the agent’s intervention, its timing or intensity, and environmental or social conditions. For example, a PWM might compare remaining silent, asking why a medication was skipped, issuing a reminder, bringing the medication, or contacting an authorized caregiver. Its outputs might include likely adherence, distress, safety, autonomy, relationship effects, and future willingness to engage. The model is explicitly not just a user profile, memory retriever, next-message predictor, or plausible human simulator. Its functional contract is person-specific, context-sensitive, action-conditioned prediction. It must discriminate between alternative interventions and be validated against subsequent trajectories and intervention responses. Nor is it intended to be a complete human digital twin. The paper repeatedly advocates purpose-bounded models: a short-horizon fatigue model, medication-support model, learning-capability model, or social-dependence model rather than one supposedly complete simulation of a person. A domain-specific digital twin might supply physiological predictions, but it would not by itself determine memory boundaries, authority, consent, or the appropriate human–agent relationship. Implementation is deliberately open. A PWM could combine latent dynamics models, generative rollouts, mechanistic physiology, temporal knowledge graphs, causal models, foundation-model priors, and lightweight personalized adapters. The authors are explicit that PWM is an organizing abstraction, not a newly demonstrated estimator. Its validity would have to be established separately for each population, target, horizon, and authority level. The causal problem is severe. Observing that a person improved after a reminder does not show that the reminder caused the improvement. Motivation, hidden context, prior conversations, health changes, and selective engagement all confound the result. The paper correctly warns that writing a causal intervention symbol does not identify an individual counterfactual. Randomized, micro-randomized, or N-of-1 designs may help, but high-stakes agents cannot freely experiment on users simply to improve their models. Prediction Is Not Permission. The intervention policy is separate from the PWM. This may be the paper’s strongest architectural decision. The PWM can predict that an action would probably help. The policy must still decide whether that action is admissible. Its allowed set is constrained by consent, domain scope, uncertainty, safety, reversibility, role boundaries, and escalation requirements. It explicitly includes not intervening, asking for clarification, requesting confirmation, and referring to a qualified human. Within that admissible set, the authors propose preserving a vector of objectives rather than collapsing everything into one reward. Benefit, capability, autonomy, safety, and relationship quality can remain explicit trade-offs. The system identifies nondominated options, after which a user-approved rule is still required to choose among them. Safety and consent cannot simply be exchanged for a higher average utility estimate. This creates a useful safety-kernel interpretation: the predictive model proposes futures, but a smaller, more inspectable authority layer determines what the system may do. For robotic deployments, that authority layer should sit above—but not replace—conventional collision avoidance, force limits, emergency stops, verified motion constraints, and other physical safety mechanisms. What This Changes for Roboticists. In conventional embodied AI, a robot commonly estimates the scene, predicts environmental dynamics, and selects actions that achieve a physical goal. Human preferences or intent may condition the policy, but the benchmark typically ends when the object reaches the target pose or the navigation episode succeeds. A ComBodied architecture adds another hierarchy. The robot controller still needs geometric perception and reliable manipulation, but physical action becomes one instrument of a person-level policy. The higher layer asks whether manipulation should occur at all, what it communicates, how much autonomy it removes, and what happens to the person after repeated assistance. Take a robot supporting post-stroke rehabilitation. Fetching every requested object may maximize short-term task success while reducing opportunities to practice reaching and walking. Refusing assistance would be equally crude. The relevant action might be full execution, partial physical support, verbal coaching, a slower demonstration, a safety warning, or escalation to a therapist. The appropriate choice depends on fatigue, risk, rehabilitation goals, recent progress, user preference, and the long-term value of independent practice. This is adjacent to existing human-in-the-loop assistive AI. Bellos and colleagues, for example, developed an AR-based agent that guides people through physical procedures and evaluated task performance, error reduction, and learning outcomes. ComBodied Agents generalize that basic concern from bounded procedural episodes to long-term, multimodal, relationship-aware assistance. For HRI, the proposed shift is therefore less about a new robot foundation model than a new systems boundary. The robot’s state estimator is no longer sufficient; the system needs a governed belief about the person. Robot success rate is no longer sufficient; evaluation must include burden, independence, capability retention, dignity, refusal handling, and appropriate human escalation. Why the Personal Model Moves to the Edge. Because these agents retain intimate longitudinal evidence, the paper argues that authoritative personal intelligence should eventually be local-first. It outlines three deployment stages: cloud-centric agents that reconstruct the user from supplied context, hybrid systems where the edge mediates memory and authority, and edge-native systems where the authoritative memory, PWM, boundaries, and intervention policy reside on trusted user-side devices. Cloud models can still provide broad knowledge or expensive reasoning. The edge sends a purpose-limited representation, receives an abstracted result, and interprets that result locally before protected memory is changed or action is authorized. The important property is not that every matrix multiplication happens locally. It is that an external service cannot silently redefine the user’s goals, overwrite corrections, or bypass the intervention policy. The authors also avoid claiming that local computation automatically provides privacy. Device compromise, shared-device use, unsafe personalization, synchronization conflicts, model drift, and incorrect local memories remain serious risks. An edge-native medical agent also does not acquire clinical authority merely by keeping its inference on-device. For robotics platforms, this raises a practical architectural question: does the robot vendor own the human model, or does the user? A local personal model that can migrate between a phone, wearable, wheelchair, home robot, and vehicle would invert the usual arrangement in which every device reconstructs the person independently inside a vendor cloud. Evaluation Beyond Task Success. The paper organizes evaluation across four layers: perception and memory, PWM quality, intervention behavior, and human outcome and agency. Each must be tested at interaction, episode, and longitudinal horizons. A system might ground one event correctly yet accumulate a false personal narrative over months; choose one reasonable intervention yet develop a pattern of overreach; or improve immediate performance while producing skill erosion. Agency preservation is treated as multidimensional. The proposed metrics include meaningful user control, correction and contestability, informed decision-making, capability growth, dependence risk, reversibility, respect for boundaries and consent, and preservation of the user’s human relationships. These dimensions cannot always be averaged safely. High mean performance should not compensate for an irreversible unauthorized action, refusal failure, privacy violation, manipulation, or induced dependence. The paper calls these non-compensatory critical failures, which is closer to a safety-case mindset than a conventional scalar benchmark. CombodiedBench is proposed as a modular suite covering state perception, memory continuity, goal negotiation, intervention appropriateness, agency preservation, relationship boundaries, escalation, and delayed outcomes. Its basic evaluation unit would be a longitudinal scenario containing prior trajectory, current context, available evidence, permissible actions, authority level, and a delayed outcome window. At present, however, this is a benchmark specification, not a released benchmark. The suggested validation methods include field studies, expert audits, simulated users, randomized trials, and N-of-1 protocols. Existing work on just-in-time adaptive interventions provides a particularly relevant methodology: micro-randomized trials repeatedly randomize intervention opportunities to estimate whether, when, and under what conditions a form of support is effective. The challenge is that the most important outcomes are delayed and endogenous. If an agent becomes better at maintaining engagement, it may collect more data and appear increasingly personalized even while making the user more dependent. A benchmark must therefore measure whether the person functions well when assistance is reduced or removed, not simply whether interaction becomes smoother. The Intellectual Lineage. ComBodied Agents do not emerge from nowhere. One direct precursor is Bang Liu’s Human-Centered Agents: From Delegation to Human Growth. That framework models the person through condition, continuity, character, commitments, control, and competence; distinguishes proxy execution from capability-building assistance; and proposes evaluating what users can do after AI support is removed. Liu is also a corresponding author of the ComBodied paper. A second close neighbor is A Framework for Longitudinal Health AI Agents, published earlier in 2026. It organizes repeated health interaction around adaptation, coherence, continuity, and agency, showing that the move beyond isolated medical conversations was already becoming an explicit research program. ComBodied Agents broaden that logic beyond health into learning, emotional support, life management, eldercare, advocacy, and protective systems. The just-in-time adaptive intervention community has long studied changing internal state, wearable and mobile sensing, intervention timing, burden, and causal estimation. Assistive robotics and shared human–AI systems similarly investigate adjustable support, learning, and division of labor. Human digital twins contribute longitudinal predictive representations, while memory agents, personalized LLMs, digital phenotyping, companion systems, and generative human simulations each provide other pieces. The paper’s novelty is therefore primarily synthesis and reorganization. It places these fragmented traditions inside one agent loop and insists that prediction, authority, intervention, and agency-preserving evaluation be designed together. Whether “ComBodied” becomes a durable research category will depend less on the name than on whether this integrated contract produces systems and benchmarks that existing labels fail to capture. Where the Framework May Break. The first problem is scope. Because the authors define ComBodied Agents as a center of gravity rather than a strict category, many existing adaptive systems could be redescribed as partially ComBodied. That openness helps coalition-building but weakens falsifiability. The field will eventually need minimum operational criteria: perhaps persistent person-state estimation, intervention-response modeling, user-correctable memory, an explicit authority policy, and longitudinal agency evaluation. The second problem is the ontology of human state. Physiological quantities may be measurable, but concepts such as motivation, confusion, loneliness, trust, capability, and relationship quality are contested and context-dependent. A well-calibrated classifier can still encode the wrong construct. Provenance and uncertainty help, but they do not prevent an agent from turning a provisional interpretation into a durable identity claim. Third, personalized dynamics are exceptionally difficult to identify. Each user supplies little data; sensing is missing not at random; behavior changes across devices, relationships, and life stages; and the intervention itself changes the distribution. Population priors can dominate the individual, while aggressive adaptation can overfit temporary distress. The paper recognizes these problems, but a practical PWM will probably look less like one grand neural simulator and more like an ensemble of small, horizon-specific models with substantial abstention. Consent is another unresolved systems problem. Permission to sense is different from permission to infer, remember, share, recommend, or execute. Consent may also change with user capacity and context. A person could authorize family notification during a medical emergency but not routine disclosure; a worker could own a wearable while their employer controls the surrounding infrastructure. Multi-party events additionally contain information about people who never consented to the agent. The Pareto formulation preserves trade-offs but does not resolve them. Who decides between autonomy and safety when the user’s current decision conflicts with an earlier commitment? Which preference is authoritative when a caregiver, clinician, and user disagree? A “user-approved selection rule” is straightforward for low-risk scheduling and much harder under cognitive decline, coercion, addiction, or crisis. Reversibility is similarly more complicated than an undo button. A message can be deleted but not unread. A robot can return an object but not erase embarrassment caused by publicly intervening. An emotional companion can retract a statement but not necessarily undo attachment or social withdrawal. Intervention policies will need models of social and psychological irreversibility, not merely whether an external state can be restored. Finally, the business model may dominate the technical design. A system that knows when a person is lonely, anxious, exhausted, or cognitively overloaded has extraordinary persuasive power. If its actual objective is retention, advertising, sales, or data extraction, the same personalization machinery intended for support becomes an optimization engine for vulnerability. The authors explicitly reject engagement as a proxy for benefit, but operationalizing that principle may require product governance and regulation, not just better loss functions. How I Would Build a Minimum Viable ComBodied Robot. For researchers interested in turning this agenda into a testable system, the safest path is deliberately narrow. Specify the human-state contract. Choose one target, one time horizon, and one authority level. A rehabilitation system might model short-term fatigue and exercise capability, but not infer personality, family relationships, or general mental health. State what the system may sense, predict, recommend, and physically execute. Create a governed event schema before choosing the foundation model. Each event should distinguish observation from inference and carry timestamp, source, subject attribution, signal quality, uncertainty, sensitivity, consent basis, retention rule, and correction history. Raw audio or video should not become permanent memory merely because storage is inexpensive. Separate memories by function and relationship. Maintain event history, user-stated goals, inferred state, intervention-response history, and protected boundaries as different stores. Make corrections first-class transactions that propagate into retrieval and prediction. Relation-scoped access should be enforced technically rather than left to prompting. Start with a modest PWM. Combine a population prior with per-user calibration and explicit uncertainty. Predict only outcomes for which future observations can provide meaningful validation. Compare a small set of scenarios—no action, clarification, low-intensity support, high-intensity support, and escalation—rather than generating unconstrained narratives about the user’s future. Place an admissibility shield between prediction and action. The action set should always contain abstention, clarification, confirmation, and human referral. Physical execution should then pass through a separate robot safety stack. No world-model prediction should directly command an irreversible actuator solely because its expected reward is high. Validate in stages. Begin with retrospective replay and adversarial scenario testing, move to scripted and simulated interaction, then conduct controlled human studies measuring burden, understanding, refusal handling, and independent capability. Where appropriate, N-of-1 or micro-randomized designs can estimate intervention effects. Only then should the system enter longitudinal field deployment, with rollback, audit, and predefined stopping rules. This would not implement the paper’s entire vision. It would, however, make the central claims measurable: whether person-specific prediction adds value, whether the authority layer prevents overreach, and whether repeated robotic assistance leaves the user safer and more capable. The Verdict. ComBodied Agents is best read as a manifesto and systems blueprint, not evidence that a new agent architecture has been solved. Its most valuable contribution is the separation of what the model predicts from what the system is permitted to do, combined with a shift from task completion toward longitudinal human outcomes. For embodied AI, that shift is consequential. A robot should not be judged only by whether it moves the right object, reaches the right location, or completes the requested workflow. We should also ask what repeated interaction leaves behind in the person: greater understanding or less, stronger capability or dependence, meaningful control or quiet displacement. The paper’s central challenge is therefore a productive one. Once an agent can alter both the world and the human acting within it, safety is no longer just about preventing physical failure. It is about ensuring that the person remains able to understand, choose, refuse, correct, recover, and grow.