Latest / Embodied AI 101 / Learning the Rule, Not the Gait: Stick-Insect Behavior as a Portable Robot Objective
Transcript
- Learning the Rule, Not the Gait: Stick-Insect Behavior as a Portable Robot Objective. From Insect Behavior to Transferable Robot Locomotion: Inferring Embodied Locomotor Principles from Limited Data via Adversarial Inverse Reinforcement Learning • Yuchen Wang, Thirawat Chuthong, Mitsuhiro Hayashibe, Poramate Manoonpong, and Dai Owaki • Tohoku University and Vidyasirimedhi Institute of Science and Technology • Bioinspiration & Biomimetics • 2026. The real target is not an insect-shaped trajectory. A stick insect presents robotics with an uncomfortable comparison. It coordinates six articulated legs, negotiates changes in contact and terrain, and shifts among gait patterns without carrying anything remotely comparable to a modern robot’s compute stack. We can debate how much of that performance belongs to neural circuitry, local sensory loops, mechanics, or the statistics of the animal’s environment. But the engineering fact remains: remarkably adaptive behavior emerges from a controller that is small, distributed, and inseparable from its body. The paper at the center of this episode, published online on July 24, 2026, attacks that comparison from an unusually productive angle. Rather than asking how to copy a particular insect gait, Yuchen Wang and colleagues ask whether a robot can infer the latent objective behind observed insect walking. Their proposed framework uses adversarial inverse reinforcement learning, or AIRL, to learn both a continuous locomotion policy and a reward representation from a limited segment of stick-insect behavior. The authors report environmental adaptation beyond the flat-ground demonstration condition, transfer of the inferred reward across dynamic systems and robot morphologies, faster learning than manually shaped reinforcement learning, and a preliminary deployment on a physical bio-inspired robot. That distinction between copying a gait and learning its objective is the conceptual heart of the work. A policy is an embodiment-specific solution. It maps the observations available to one body into commands for that body’s actuators. Change the link lengths, masses, motor dynamics, contact geometry, or action interface, and the mapping may cease to make sense. A reward, by contrast, can potentially express something more stable: maintain useful support, coordinate leg transitions, advance the body, avoid dynamically poor contact arrangements, and respond smoothly when the environment changes. If AIRL really recovers that kind of objective, then the insect trajectory is no longer merely a motion clip. It becomes a compressed specification of a locomotor strategy. From gait interpretation to continuous robot control. This paper is best understood as the next step in a sequence of work by Wang and colleagues. Their 2025 paper, Encoding Flexible Gait Strategies in Stick Insects through Data-Driven Inverse Reinforcement Learning, used maximum-entropy IRL to study gait selection. Walking velocity, direction, acceleration, and antennal motion were included in the state representation, while gait patterns formed the action space. The inferred rewards and policies were used to analyze canonical and noncanonical gait transitions, including the apparent role of sensory information in probabilistic gait modulation. That earlier work was primarily an analysis of behavioral choice: given observable locomotor and sensory state, which gait does the insect select? An associated 2025 conference contribution similarly represented gait patterns as discrete actions and examined tactile-responsive flexibility. The new paper moves down the control hierarchy. It is no longer just asking why an insect selects tripod-like versus tetrapod-like coordination. It is learning a continuous locomotion controller and attempting to carry the inferred objective into new simulated bodies and, preliminarily, a physical robot. The move from discrete gait classification to continuous control is important because the convenient boundaries between named gaits largely disappear at this level. Leg phases drift, contact events are asynchronous, and useful coordination often lies between textbook categories. This also changes what “biological imitation” means. A controller need not replay a canonical tripod gait to be biologically grounded. It could instead reproduce the insect’s distribution of contact relations, its willingness to use noncanonical patterns, or the way coordination reorganizes when sensory and mechanical conditions change. In that interpretation, the target is not a periodic template but a family of viable coordination strategies. Turning a biological recording into an inverse-control problem. At a high level, the framework has four stages. First, measured insect walking is transformed into demonstrations suitable for a sequential decision-making problem. Second, AIRL alternates between learning a policy that produces locomotion and learning a discriminator that distinguishes the policy’s behavior from the biological demonstrations. Third, the structured discriminator is decomposed so that part of it can be interpreted as a recovered reward rather than merely a classifier score. Finally, that reward network is carried into changed environments or different robot models, where a new policy can be trained under the new dynamics. The first stage is easy to summarize and hard to do well. Biological measurements and robot control variables rarely have a natural one-to-one correspondence. An insect’s joint trajectories cannot simply be issued to a robot with different kinematics. Raw motor commands are even less portable. A useful representation must preserve behaviorally meaningful structure while discarding details tied to the source animal. Possible shared quantities include normalized limb phase, contact state, foot motion relative to the body, body velocity, duty factor, local support relationships, and transitions between stance and swing. Those examples are not claims about the paper’s exact feature vector. They illustrate the central design constraint: whatever the reward observes must have consistent semantics across the insect demonstration, the source simulation, and the target robot. This representation question becomes especially important when evaluating the morphology-transfer claim. If a reward consumes the absolute angle of “front-left joint two,” then moving it to a body with a different joint layout is mostly a software exercise, not meaningful transfer. If it consumes normalized contact and body-motion descriptors, then it can potentially express a body-independent preference while allowing each morphology to discover its own realization. The source policy and the reward therefore play very different roles. The policy answers, “What should this particular body do now?” The reward answers, “How desirable is the resulting coordination?” That separation is what allows the paper to frame insect behavior as a source of transferable locomotor principles. Why AIRL rather than behavior cloning?. The simplest learning-from-demonstration baseline would be behavior cloning. Given an observed state, predict the demonstrated action. This can work extremely well when demonstrations densely cover deployment conditions and the learner shares the demonstrator’s observation and action spaces. Neither condition is naturally satisfied here. The demonstrations are limited, the test environments vary, and the target robot may have different dynamics or morphology. Once a cloned policy visits a state absent from the biological data, its prediction errors can change the next state, producing further errors and eventually pushing the system outside the training distribution. The morphology problem is more fundamental: an action appropriate for the insect may not even exist in the target robot’s action space. Generative adversarial imitation learning, or GAIL, addresses some of behavior cloning’s distribution-shift problem by letting the learner interact with its environment. A discriminator distinguishes expert transitions from learner transitions, while the policy tries to generate behavior that the discriminator regards as expert-like. In effect, the learner attempts to match the expert’s occupancy distribution rather than independently predicting every recorded action. But standard adversarial imitation primarily gives us a policy and an imitation signal tied to the training domain. AIRL modifies the discriminator so that it can recover a reward intended to survive changes in dynamics. The method was introduced specifically to distinguish an underlying objective from reward-shaping effects that happen to make learning convenient in one environment. Here, “adversarial” does not mean adversarial perturbations or malicious attacks. It refers to a two-player training process. The discriminator learns to tell biological demonstrations from policy-generated behavior. The policy receives a signal derived from that discriminator and tries to produce trajectories that erase the distinction. The critical detail is that AIRL does not treat the discriminator as an unconstrained black box. Its score is structured as a candidate reward plus a difference of learned state potentials. In plain language, one component is supposed to capture what states or actions are intrinsically preferred, while the potential component absorbs incentives that merely help navigate the source dynamics. Consider a robot receiving a large intermediate bonus whenever it moves closer to a target. That distance-reduction term may accelerate learning, but it is not necessarily the true task. If the dynamics or obstacles change, blindly transferring the shaped signal can prescribe the wrong behavior. AIRL tries to recover the more stable objective beneath such source-specific guidance. This is why AIRL is a natural match for the paper’s ambition. Direct imitation asks a new body to reproduce an old body’s answer. AIRL attempts to transfer the question. What the experimental claims actually establish. The paper’s reported evidence can be read as a progression through increasingly difficult forms of generalization. Learning from a short flat-ground demonstration. The first result is that a relatively short segment of flat-terrain insect walking is sufficient to initiate the learning process. This matters because high-quality biological motion data can be expensive to acquire, synchronize, clean, and map into control-relevant variables. A method that required recordings for every terrain, speed, and disturbance would provide little advantage over building a large gait library. There is, however, an important sample-efficiency distinction. Limited demonstration data does not necessarily mean limited environment interaction. AIRL can reuse the same expert transitions while producing many simulation rollouts from the current policy. The biological-data requirement may be small even if the overall reinforcement-learning budget remains substantial. The demonstrations also provide more than target trajectories. Through the discriminator, every learner transition receives information about whether it belongs near the demonstrated behavior distribution. That can be substantially denser than a conventional hand-designed objective composed of forward speed, uprightness, energy, slip, and foot-clearance terms. Generalizing beyond the demonstrated environment. The authors report that a policy learned from flat-ground data develops adaptive and flexible coordination under changed environmental conditions. This is stronger than replaying the training sequence, but it should be interpreted separately from morphology transfer. Environmental generalization can arise from several sources. The policy may use feedback rather than open-loop phase playback. The simulation may expose it to a range of states while AIRL is training. The biological objective may permit multiple coordination patterns rather than enforcing a single reference trajectory. And the mechanics of the body may convert local reactions into useful global reorganization. That last point is particularly relevant to insect-inspired control. Earlier stick-insect robotics work combined biological foot trajectories with decentralized central-pattern-generator networks and foot-contact feedback. Flexible interlimb coordination emerged through interactions among independent leg controllers, body mechanics, and the environment, rather than from an explicit global gait schedule. The AIRL paper appears to be exploiting a complementary idea: instead of prescribing the coordination architecture or its parameters, infer an objective under which useful coordination can emerge. Generalization then comes not from memorizing every condition, but from allowing the policy and embodiment to find another high-reward solution. Transferring the reward across dynamics and morphology. This is the strongest claim in the paper. The authors report transferring the learned reward network to different dynamic systems and robot models with different morphologies, then using it to guide target-policy learning. The wording matters. This is not necessarily zero-shot transfer of the original controller. The reward transfers; the policy is learned or adapted for the new body. That is exactly the distinction IRL is supposed to enable. A long-legged robot and a short-legged robot should not produce identical joint trajectories. They may not even use the same contact timing. But if both are optimizing a shared notion of stable, adaptive progression, each can find a morphology-appropriate gait. Morphology transfer is also a more convincing test of reward quality than running the source policy under modest parameter randomization. If a reward only encodes quirks of the source body, target optimization should fail, converge slowly, or produce pathological motion. Successful target learning suggests that the representation captures at least some structure that survives the change of embodiment. It does not prove that the network has recovered the insect nervous system’s actual objective. IRL cannot provide that kind of biological identification from behavior alone. But it does suggest that the inferred objective is operationally useful: it compresses something about the demonstration that remains actionable after the original kinematics are removed. Beating manually shaped reinforcement learning. The paper compares its approach with methods relying solely on hand-designed reward shaping and reports faster convergence together with more biologically consistent gait coordination. This result addresses two familiar weaknesses of locomotion RL. First, a manual reward is usually an uneasy bundle of partially conflicting terms: velocity tracking, orientation, torque cost, smoothness, foot clearance, slip penalties, contact-force penalties, and survival. The coefficients define the behavior as much as the algorithm does. Small changes can produce shuffling, hopping, excessively conservative stance, or high-frequency exploitation. Second, task performance alone says little about motion character. Many dynamically valid ways exist to move a multilegged body forward. A speed reward may produce a gait that is effective yet unlike the biological strategy of interest. Demonstrations narrow that solution set without requiring the designer to articulate every coordination preference. For biological consistency, visual resemblance is the weakest possible metric. The more informative quantities are interleg phase relations, distributions of stance duration, contact-sequence diversity, support transitions, and how those statistics change with speed or perturbation. The value of the result ultimately depends on whether such coordination-level measures remain consistent on held-out conditions, rather than only during the demonstration window. Crossing into a physical robot. Finally, the authors report a preliminary deployment on a physical bio-inspired robot. That is deliberately modest language, and it should be read that way. It demonstrates that the learned controller is not restricted to an idealized simulator, but it is not yet evidence for production-level sim-to-real robustness. For this class of controller, the reality gap is not just uncertain mass and friction. Contact timing changes with actuator latency, compliance, backlash, state-estimation delay, and foot-ground geometry. Because coordination is coupled through those contacts, a small local discrepancy can reorganize the whole gait. Even a preliminary deployment is therefore useful. It tests whether the learned policy has enough feedback structure and margin to remain meaningful when contact events stop occurring exactly where the simulator predicted them. What is an “embodied locomotor principle”?. The phrase risks sounding more interpretable than a neural reward network actually is. A learned scalar function does not automatically become a concise biological law. The most useful interpretation is functional. An embodied locomotor principle is a preference that cannot be reduced to a desired trajectory independently of the body. It assigns value to the relationship among action, mechanical state, sensory feedback, and environmental consequence. For example, “maintain tripod gait” is a symbolic prescription. “Prefer contact transitions that preserve support while permitting progression” is closer to an embodied principle. The latter can produce tripod-like coordination at one speed, tetrapod-like coordination at another, and irregular intermediate patterns during a disturbance. Similarly, an explicit phase offset between two legs is controller-specific. A reward that values compatible stance-swing relationships may allow the phase difference to emerge from body mechanics and local feedback. That makes the resulting coordination adaptable rather than merely variable. This framing also prevents a common category error. AIRL is producing a normative model: a compact account of what objective could make the observed behavior sensible. It is not directly identifying synapses, central pattern generator connectivity, or the biological reward signals used by the insect. Behavioral equivalence does not imply mechanistic equivalence. That limitation is familiar in computational ethology. Earlier animal-behavior IRL work showed that reward-based models can identify strategies from behavioral time series, but the inferred reward remains an explanatory model under a chosen state and action representation. Change the representation and the apparent strategy may change with it. How this differs from nearby locomotion approaches. The closest family of alternatives uses explicit bio-inspired control structure. In the stick-insect-like controller reported by Larsen and colleagues, real insect foot trajectories trained radial-basis-function premotor networks for individual legs. Decentralized pattern generators, contact feedback, and continuous phase modulation then produced self-organized interlimb coordination in simulation. The resulting controller generated both tripod and non-tripod patterns without requiring a fixed global gait schedule. That approach has strong inductive bias. We know where rhythm generation lives, how foot contact modulates it, and how biological trajectories enter the controller. Its disadvantage is that the architecture already contains a hypothesis about the solution. The AIRL approach relocates the inductive bias. It assumes that the demonstration can be explained by reward-directed behavior and that the structured adversarial learner can separate a portable objective from source-specific shaping. The learned policy is less mechanistically constrained, but the recovered reward offers a potential route across embodiments. Another neighboring family uses adversarial motion priors. These methods train a discriminator on reference motion and turn its output into a style reward. They have produced natural, agile locomotion and multiple switchable motion styles, including physical quadruped demonstrations. Animal-data adversarial branches have also been combined with teacher-student locomotion pipelines to make quadruped behavior more natural while retaining robustness on challenging terrain. The key distinction is objective recovery. Motion-prior methods primarily ask whether generated behavior resembles a reference distribution. AIRL’s structured discriminator is intended to expose a reward component that can be removed from the source policy and used to train another one. In this paper, that distinction is not philosophical decoration; it is what supports the morphology-transfer experiment. What is genuinely new here. None of the individual ingredients is unprecedented. Inverse reinforcement learning from animal behavior is established. AIRL was designed for transferable reward recovery. Adversarial imitation is widely used for natural locomotion. Bio-inspired hexapod control and sim-to-real reinforcement learning are mature research areas. The contribution is the way these pieces are assembled around an unusually demanding transfer chain:. limited biological behavior → continuous simulated control → environmental adaptation → reward transfer across bodies → physical deployment. Most animal-imitation systems stop after reproducing motion in one embodiment. Most locomotion-reward-learning systems start from human or algorithmic experts already expressed in the robot’s own state-action space. Most bio-inspired controllers transfer an explicitly designed coordination rule rather than an inferred latent objective. Here, the biological data is treated as evidence about what locomotion should optimize. If the reported morphology transfer is robust, that is a meaningful advance: the result would suggest that small behavioral datasets can supervise objectives more portable than the motions from which they were learned. The important caveats. The first caveat is reward identifiability. Many reward functions can explain the same demonstrations. Potential-based shaping is only one source of ambiguity; limited state coverage and correlated features create many more. AIRL reduces a particular transfer problem through its discriminator structure, but it does not make inverse reinforcement learning uniquely solvable. Later analyses of AIRL have also argued that policy optimization quality, reward decomposition, and training variance can materially affect practical reward transfer. Some work recommends separating source-domain reward recovery from target-domain policy optimization, potentially using different RL algorithms for the two stages. The second caveat is representation dependence. A reward transfers only to the extent that its inputs retain the same meaning across bodies. Normalization does not automatically create semantic correspondence. A foot-contact pattern may mean something different when the target body has changed mass distribution, compliance, workspace, or number of effective limbs. The third is that expert insect behavior need not be optimal for the robot. Biological movement reflects evolutionary history, metabolic constraints, safety margins, sensory limitations, and multifunctional requirements that may not apply to an engineered platform. IRL will faithfully preserve undesirable compromises if the state representation and demonstration data make them look intentional. The fourth is demonstration coverage. A short flat-ground sequence cannot reveal every preference relevant to slopes, obstacles, damage, or low-friction contacts. Generalization outside the demonstration distribution relies on the learned reward extrapolating sensibly. Neural reward models can become confident for the wrong reasons in precisely those regions where no expert data exists. The fifth is adversarial training stability. With limited demonstrations, the discriminator can memorize the expert samples or exploit superficial differences between biological and simulated trajectories. If that happens, the policy may optimize artifacts of preprocessing rather than locomotor structure. Held-out demonstrations, discriminator regularization, temporal augmentation, and careful source-target normalization are therefore essential. Finally, a preliminary real-robot result should motivate, not conclude, the sim-to-real story. The next tests should include repeated trials, parameter and surface variation, external pushes, sensor noise, actuator degradation, and transfers to bodies whose morphology differs enough to rule out trivial reuse. How a robotics group could adapt the idea. For researchers interested in building on this work, the most important decision is not which AIRL implementation to clone. It is the choice of interface between biological behavior and robot optimization. A practical pipeline would look like this:. Define a shared behavioral space. Prefer body-frame velocity, normalized limb phase, contact transitions, local foot motion, support relations, and other quantities that can be computed consistently across embodiments. Keep morphology-specific joint variables inside the policy, not necessarily inside the transferable reward. Separate task validity from biological preference. Falls, self-collisions, actuator limits, and hardware safety may be better handled as constraints or termination conditions. The learned reward can then focus on coordination instead of being forced to rediscover basic feasibility from a tiny dataset. Train and audit the source model. Policy performance is not enough. Probe the reward on counterfactual trajectories: correct speed with poor coordination, stable standing with no progression, reversed contact sequences, artificial phase locking, and high-frequency motion. A portable reward should rank these cases sensibly. Freeze the intended reward component before transfer. Do not transfer the raw discriminator without understanding which terms contain source-policy probabilities or shaping information. AIRL’s point is to isolate the reward-like component. Retrain a target-specific policy. Let each morphology discover its own action-level solution. Evaluate from scratch as well as from transferred initialization so that reward transfer is not confused with policy or representation reuse. Test increasingly causal changes. Begin with mass and actuator variations, then alter link proportions, joint layout, compliance, or available contacts. The more the target body departs from the source while preserving performance, the stronger the evidence that the reward captures a genuine cross-embodiment regularity. For evaluation, success rate and forward velocity are insufficient. Measure coordination distributions, transition diversity, recovery after contact errors, energy and peak-load consequences, and how quickly a target morphology learns compared with manual rewards, motion discriminators, and task-only RL. A particularly revealing ablation would compare three objects transferred from the source domain: the policy, an unstructured imitation discriminator, and the AIRL reward. That would directly separate controller reuse, motion-style matching, and objective transfer. The broader lesson. The most interesting claim in this paper is not that robots should walk like stick insects. It is that animal behavior may be useful as objective data. A biological trajectory is only one realization of a strategy under one body and one environment. Replaying it preserves the realization. Inverse reinforcement learning tries to preserve the preference that selected it. That is a much more ambitious form of biomimetics. Traditional bio-inspired robotics often copies morphology, neural circuits, reflex rules, or observed kinematics. Wang and colleagues instead treat behavior as evidence from which a controller can infer what matters. The robot is then free to express that preference through its own mechanics. The work remains subject to the standard difficulties of IRL: ambiguity, representation dependence, adversarial instability, and uncertain extrapolation. Its real-robot evidence is still preliminary. Yet the direction is compelling. If the reward—not the joint trajectory, oscillator network, or gait label—is the transferable unit, then a few seconds of animal behavior could become a reusable specification for many robotic bodies. The insect would not be providing the robot with a gait to copy. It would be providing a clue about how locomotion should be organized.