Latest / Embodied AI 101 / Taming the Flying Tip: Constrained MPC for Dynamic Deformable Linear Objects

Transcript

Taming the Flying Tip: Constrained MPC for Dynamic Deformable Linear Objects · Embodied AI 101

• Taming the Flying Tip: Constrained MPC for Dynamic Deformable Linear Objects. Constrained Close Loop Model Predictive Control Architecture for Deformable Linear Objects for Dynamic Motions • Marc Kilian Klankers and Jochen J. Steil • Institute for Robotics and Process Control, Technische Universität Braunschweig • Robotics (MDPI) • 2026. Move a rigid tool quickly and, within the limits of the robot’s bandwidth, the tool usually goes where its kinematics say it should. Move the base of a cable, hose, rope, or flexible rod quickly, and the interesting part begins only after the robot has moved. Momentum travels along the object. Bending energy accumulates and releases. Oscillatory modes interact. The free endpoint may still be accelerating while the gripper is already reversing direction. That makes high-speed manipulation of a deformable linear object, or DLO, a particularly unforgiving control problem. The actuator is located at one boundary, the quantity we often care about is at the other boundary, and most of the object’s dynamical state lies somewhere in between. In control terms, it is a distributed, underactuated, non-collocated system whose relevant state is difficult to measure. The work by Marc Kilian Klankers and Jochen J. Steil addresses this problem as a complete feedback architecture rather than as an isolated trajectory-generation trick. The same authors presented the architecture’s core at ICRA 2026 under the title Closed-Loop Trajectory Optimization of Deformable Linear Objects for Dynamic Motions. That abstract identifies four central ingredients: a floating-base kinematic-chain representation of the DLO, a data-driven approximation of its dynamics, an optimal control problem solved online using linear model predictive control and differential dynamic programming, and validation in both simulation and hardware on dynamic two-dimensional endpoint tracking. The central idea is straightforward to state but difficult to execute: give the deformable object an explicit, control-oriented state; learn how that state evolves when the robot moves; estimate it fast enough to close the loop; and repeatedly optimize the robot’s motion subject to operational constraints. Why Dynamic DLO Control Is a Different Problem. A large fraction of robotic DLO manipulation is quasi-static. The robot moves slowly, waits for the cable or rod to settle, observes the resulting shape, and then chooses another action. Under that assumption, inertia and transient vibration can be neglected or treated as disturbances. Planning becomes a sequence of approximately equilibrated shapes. That formulation is appropriate for tasks such as routing a wire through clips, shaping a cable on a table, or positioning a flexible object around obstacles. For example, existing cable-shape planners divide large deformations into intermediate profiles and optimize dual-arm motions that progressively reduce shape error. Other methods derive planar shape and endpoint forces from elastic energy and Lagrangian mechanics. Both are useful, but they operate in a regime where the DLO has time to relax toward equilibrium. The dynamic regime removes that convenience. A fast robot motion deliberately excites the object. Success may depend on exploiting, rather than suppressing, elastic energy. The robot must time its acceleration and reversal so that a wave arrives at the free endpoint at the correct instant. A small error in stiffness, damping, latency, or initial shape can produce a large endpoint error several hundred milliseconds later. Three difficulties then become tightly coupled. First, there is the representation problem. A DLO is physically continuous, but an online controller needs a finite-dimensional state. Raw images are too indirect. Dense finite-element states can be too large. A task-specific latent vector may be compact but gives the optimizer little geometric meaning. Second, there is the observability problem. Position alone is not sufficient in a dynamic task. Two identical-looking shapes can evolve differently if their internal velocities differ. Yet directly measuring the velocity and deformation state of every point along a real cable is generally impractical. Third, there is the optimization problem. Even with a usable state and model, the controller must predict far enough ahead to understand oscillatory motion, solve quickly enough to react, and keep the commanded robot trajectory within position, velocity, acceleration, and workspace limits. The paper’s contribution is valuable because it treats these as one architectural problem. The spatial model is chosen to support state estimation. The estimated state is chosen to support dynamics learning. The learned dynamics are chosen to support differentiation and online optimal control. And the controller is formulated so constraints can be represented directly rather than approximated through carefully tuned penalties. Turning a Continuum into a Control State. The authors represent the DLO as a floating-base kinematic chain. Conceptually, the object’s centerline is divided into ordered segments connected by virtual joints. The floating base describes the pose of the driven boundary—typically the part attached to the robot—while the relative joint variables describe how the rest of the object bends away from that boundary. This is not a claim that a cable is literally a rigid serial manipulator. It is a coordinate system. The chain supplies a finite set of geometrically meaningful variables from which the position of every represented point, including the free endpoint, can be reconstructed by forward kinematics. That choice has several advantages. Most importantly, it preserves spatial ordering. Segment three is always between segments two and four. Information does not have to be rediscovered from an unordered point cloud at every control step. Adjacency, length, tangent direction, and endpoint location are built into the representation. It also separates the motion imposed by the robot from the object’s internal deformation. The floating base follows the manipulated boundary, while the virtual joints encode the lagging, bending response of the DLO. This decomposition is particularly natural for endpoint control because the optimizer can reason about the causal path from robot motion, through internal deformation, to free-tip motion. The representation is also compatible with constraints. Limits on robot motion remain limits on base variables or commands. Excessive virtual-joint deflection can be used as a proxy for extreme curvature. Geometric quantities such as endpoint position, segment clearance, or workspace occupancy can be computed directly from the chain. Finally, the state remains interpretable. If the learned model fails, a researcher can inspect which virtual joints diverged, whether an estimated velocity became noisy, or whether the model predicted the wrong phase of an oscillation. That is much harder with an unconstrained visual latent state. The price is discretization. Too few segments cannot express the relevant bending modes. Too many enlarge the estimator, the dynamics model, and the online optimization problem. The right resolution is therefore not simply the finest one the perception system can observe. It is the coarsest one that still captures the modes affecting the controlled output. That is an important control-oriented distinction. If the task is to place the free endpoint, the model does not need to reproduce every small wrinkle along the object. It needs to preserve the deformation modes that materially change endpoint prediction over the MPC horizon. Other representations make different trade-offs. DLOFTBs, for example, fits B-splines for fast spatial tracking, providing a smooth and compact description of the observed centerline. A spline is excellent for perception and shape comparison. A virtual chain is often more convenient for recursive kinematics, explicit generalized coordinates, and dynamics optimization. The two need not be competitors: a spline tracker could provide spatial measurements that are subsequently projected into the chain state used by the controller. The deeper lesson is that “generic” does not mean one physical model that works unchanged for every cable. It means a reusable coordinate template. Objects with different lengths, stiffnesses, and damping can share the same representational form even though their learned transition models differ. Learning the Dynamics Without Abandoning Structure. Once the DLO has a state, the next question is how that state changes under robot motion. A full analytical model is possible in principle. One could use a dynamic Cosserat rod, discrete elastic rods, finite elements, a mass–spring system, or a multibody chain with elastic joints. But accurate analytical modeling requires material parameters, boundary conditions, damping laws, and contact properties that are difficult to identify. High-fidelity integration can also be expensive inside a controller that must perform many model rollouts per update. A purely black-box alternative would learn a mapping directly from recent images and robot commands to future endpoint positions. That might work inside a narrow dataset, but it discards most of the useful structure. It also makes state constraints, physical interpretation, and transfer between perception setups much harder. Klankers and Steil occupy the middle ground. The geometry remains explicit, while the difficult transition behavior is approximated from data. The ICRA description calls this a data-driven approximation of the DLO’s hybrid dynamics. Although the short abstract does not fully define “hybrid,” the important architectural fact is clear: the model acts on an explicit chain state rather than on pixels, and it is suitable for both local linearization and nonlinear trajectory optimization. The authors’ preceding ICRA 2025 paper, Modeling of Deformable Linear Objects Under Incomplete State Information, focused specifically on the modeling challenge that precedes this controller. The progression is significant: first establish a control-oriented dynamics model when the full physical state cannot be directly observed, then embed that model in a receding-horizon controller. For this kind of model, one-step accuracy is not enough. A predictor can have a low average next-state error and still be useless for MPC if small errors accumulate with the wrong phase. Flexible systems are especially sensitive to this. Slightly underestimating the period of an oscillation may look harmless for one step, yet place the predicted free endpoint on the opposite side of its trajectory after a longer rollout. Training and validation should therefore emphasize multi-step behavior, phase fidelity, damping, and response to reversals. The model must also cover the state–action region that the controller will visit. If training consists only of gentle random motions, MPC may exploit unsupported high-acceleration regions because the learned model predicts unrealistically favorable behavior there. Differentiability matters as well. Differential dynamic programming requires derivatives of the transition model, while linear MPC needs local state and input Jacobians. A differentiable learned model makes both available through automatic differentiation, avoiding separate finite-difference identification at every control cycle. There are higher-fidelity alternatives. DEFORM, for example, combines differentiable discrete-elastic-rod physics with learning to obtain accurate, real-time DLO prediction over long horizons. Such a model could potentially replace the learned dynamics block in the Klankers–Steil architecture. The trade-off would be greater physical structure and possibly better extrapolation in exchange for more calibration and computational complexity. The important contribution here is therefore not allegiance to a particular neural architecture. It is the interface: explicit state in, explicit state out, robot motion as the input, and gradients available for optimization. State Estimation Is the Critical Middle Layer. A dynamics model is useful only if the controller knows where in that model’s state space the physical object currently is. For a floating-base chain, estimation means recovering the chain configuration from spatial observations and supplying the dynamic quantities—most importantly velocities—needed for prediction. The base pose may be available from robot kinematics, but the internal chain state must be inferred from sensing. The earlier incomplete-state paper makes clear that the authors regard this as a fundamental part of the modeling problem, not as a trivial preprocessing step. This layer has to reconcile several incompatible requirements. A camera or motion-capture system provides noisy, delayed samples. Numerical differentiation amplifies that noise. Aggressive filtering reduces noise but adds phase delay, exactly what a high-speed flexible-object controller cannot tolerate. Meanwhile, the optimizer needs a temporally coherent state that lies on the chain’s kinematic manifold. The chain representation helps because state estimation can be framed as structured geometric fitting rather than unrestricted reconstruction. Observed points or curves can be projected onto a fixed-length ordered chain. Once the configuration is expressed in generalized coordinates, a model-based observer or filtered history can estimate the corresponding velocities. But representation does not eliminate ambiguity. A planar centerline does not reveal torsion. Occlusion can hide internal segments. Sparse endpoint observations may not distinguish between internal modes that produce similar instantaneous tip positions. In practice, the estimator and controller must therefore be co-designed around what the sensors can reliably observe. This is one reason the work’s two-dimensional scope is sensible. Planar dynamics remove torsional ambiguity and simplify the mapping between an observed centerline and virtual-joint coordinates. It is not the final industrial problem, but it is an appropriate setting in which to establish whether learned closed-loop optimal control can tame the basic flexible dynamics. For practitioners, estimator latency should be treated as part of the plant. A model trained on synchronized, low-latency data can fail when deployed behind a slower vision pipeline. Useful extensions would include explicit observation-delay compensation, uncertainty propagation, or a moving-horizon estimator sharing the same dynamics model as the controller. From Prediction to Constrained Receding-Horizon Control. With an estimated state and a differentiable model, the endpoint-tracking problem becomes a finite-horizon optimal control problem. At each update, the controller predicts how candidate robot motions will propagate through the DLO and move the free endpoint. The objective penalizes deviation from the desired endpoint trajectory over the horizon. It can also regularize control effort, rapid changes in command, internal deformation, or departures from a preferred robot configuration. The optimizer returns a sequence of commands, but only the first part is executed before sensing and optimization are repeated. That last step is what turns trajectory optimization into feedback control. A model error that would ruin an open-loop whipping motion becomes a deviation that can be observed and corrected at the next update—provided the system still has enough controllability and remaining horizon to recover. The paper investigates two solution strategies: linear MPC and differential dynamic programming, or DDP. Linear MPC. Linear MPC locally approximates the learned nonlinear transition model around the current or nominal trajectory. Once the dynamics and output map have been linearized and the objective made quadratic, the online problem becomes a quadratic program. This brings three practical benefits. First, quadratic programs can be solved quickly and predictably using mature real-time solvers. Second, warm-starting from the previous control solution is straightforward. Third, linear state and input constraints can be imposed directly. That is crucial when aggressive DLO motion is generated by an expensive manipulator whose velocity, acceleration, workspace, and safety limits must not be violated. The local approximation is also less restrictive than it may initially sound. MPC repeatedly re-linearizes around the latest state. The controller does not need a single global linear model of the DLO. It needs a locally adequate approximation over the distance traveled before the next feedback update. The weakness appears when the prediction horizon crosses strongly nonlinear regions. A planned command may reverse the object’s curvature, excite a different mode, or move sufficiently far that the local Jacobian ceases to describe the dynamics. Shortening the horizon improves local validity but removes the foresight needed for oscillatory motion. Dynamic DLO control therefore exposes the classic MPC tension between model fidelity, horizon length, and solve time in a particularly sharp form. Differential Dynamic Programming. DDP retains the nonlinear model during its forward rollout. It then performs a backward pass that constructs locally quadratic approximations of the cost-to-go and updates the control sequence. For smooth systems with accurate derivatives, it can capture nonlinear behavior more faithfully than a single linearized MPC problem. That makes DDP a useful counterpart to linear MPC. If DDP produces substantially better predictions or tracking, the local linear approximation is probably the bottleneck. If both perform similarly, the simpler constrained QP may be the better engineering choice. DDP’s difficulty is constraint handling. Bounds can be introduced through penalties, projections, box-constrained variants, or more elaborate constrained formulations, but the resulting solver is generally less straightforward than conventional constrained linear MPC. This helps explain why the paper’s architecture is interesting beyond a mere controller comparison: it separates the nonlinear modeling problem from the choice of optimizer, allowing different solvers to share the same state and learned dynamics. The two approaches also expose different failure modes. Linear MPC may fail because its approximation is too local. DDP may fail because its nonlinear optimization converges too slowly or enters a poor local minimum. A robust implementation should monitor predicted improvement, constraint margins, and model residuals rather than assuming every optimization result is safe to execute. What the Experiments Establish. The authors validate the approach in simulation and on hardware, controlling the free endpoint of a DLO to follow dynamic planar references. The official conference description specifically reports dynamic two-dimensional endpoint tracking and compares online optimal-control implementations based on linear MPC and DDP. Hardware validation matters unusually strongly here. Flexible-object experiments contain modeling errors that are easy to miss in simulation: nonlinear damping, compliance in the attachment, imperfect homogeneity, small out-of-plane motions, sensing delay, robot servo dynamics, and changes in the object caused by repeated bending. Closing the loop on a real free endpoint is therefore qualitatively stronger evidence than demonstrating that a simulated optimizer can generate a plausible motion. The result to focus on is the integration. The representation, estimator, learned transition model, optimizer, and robot controller all have to operate on compatible timescales. A good model evaluated offline is not enough. A fast optimizer with a delayed state is not enough. Accurate perception with an unsuitable state representation is not enough. The hardware demonstration shows that the complete chain can be made operational. At the same time, endpoint tracking is narrower than full-shape control. The internal DLO configuration is principally valuable because it predicts the controlled endpoint. The experiments do not, by themselves, establish arbitrary three-dimensional shape regulation, contact-rich routing, knot manipulation, or collision-aware operation in clutter. That distinction is important. The work advances dynamic feedback control without pretending to solve the entire DLO-manipulation problem. How This Differs from Closely Related Directions. The DLO literature is moving along several complementary axes, and this architecture occupies a distinctive point among them. One direction emphasizes high-fidelity differentiable simulation. DEFORM learns around a differentiable discrete-elastic-rod model, prioritizing accurate long-horizon prediction while retaining physical structure. That is attractive when transfer across trajectories and physically meaningful parameter adaptation are central. Klankers and Steil instead prioritize a compact control state and an end-to-end real-time architecture. A second direction emphasizes visual shape tracking and quasi-static manipulation. B-spline trackers and learned three-dimensional DLO models provide smooth geometric estimates that generalize across appearance or support bimanual shaping. These methods solve crucial perception and model-learning problems, but they are generally not centered on repeatedly solving a constrained dynamic endpoint-control problem at high speed. A third direction learns the manipulation policy itself. DexDLO learns goal-conditioned policies for dynamic dexterous manipulation, while DynDLO uses reinforcement learning in MuJoCo with B-spline-parameterized robot trajectories and reports sim-to-real transfer across representative dynamic tasks. These approaches can absorb complex behavior without requiring an explicit online optimizer. Their cost is that constraints, extrapolation, and failure interpretation are usually less transparent than in MPC. The Klankers–Steil architecture makes a different bet: learn the part that is hardest to model, but retain explicit online decision-making. The learned component answers, “What will the DLO do?” The optimizer separately answers, “Given the current target and constraints, what should the robot do next?”. That separation is valuable when targets change online, when actuator limits vary, or when new safety constraints must be added without retraining a policy. It also supports a more traditional verification workflow: validate state estimation, then model rollouts, then optimization, and finally closed-loop behavior. The price is online computation and dependence on model quality. A policy amortizes optimization into training and may run in milliseconds. MPC pays the optimization cost during every control cycle. But MPC can adapt immediately to a new reference or constraint set, whereas a policy may require retraining or careful conditioning. This is not a question of model-based control versus learning. It is a question of where learning belongs in the stack. Limitations and Open Questions. The clearest scope boundary is dimensionality. Dynamic planar endpoint tracking removes the hardest ambiguities in three-dimensional DLO state estimation. A spatial extension would have to represent bending in multiple directions, torsion, out-of-plane self-occlusion, and potentially axial deformation. It would also need sensing capable of resolving those quantities at control rate. A second limitation is task definition. Controlling one endpoint does not guarantee a desirable full-body shape. Two DLO configurations may place the free tip at the same position while wrapping around an obstacle differently or storing very different amounts of elastic energy. Contact-aware tasks will require costs and constraints that reason about the entire chain, not just its terminal output. Third, learned dynamics are generally tied to an operating distribution. Changes in material, length, grasp point, temperature, aging, or attached payload can shift the dynamics. The chain representation is reusable, but that does not automatically make the transition model object-independent. Parameter-conditioned models, online adaptation, ensembles, or residual learning around an analytical rod model would be natural extensions. Fourth, constraints are only as meaningful as the model through which they are evaluated. A predicted curvature bound does not protect the physical object if state estimation is biased or the learned model underestimates bending. Safety-critical deployment would benefit from uncertainty-aware tightening, robust MPC, independent robot-level safety filters, and conservative fallback behavior when model residuals grow. Fifth, local optimization remains local. Flexible systems have many ways to exchange kinetic and elastic energy, and the finite-horizon objective may contain several qualitatively different solutions. Warm-starting helps computationally but can also keep the controller trapped in the same family of motions. Finally, the field needs standardized dynamic benchmarks. Endpoint RMSE alone does not capture peak error, phase lag, recovery from disturbances, constraint violations, sensitivity to initial shape, or generalization across DLOs. A useful benchmark would vary object properties, reference frequency, sensing delay, and initial deformation while reporting both task performance and computational deadline misses. These are not reasons to discount the work. They identify the research program that becomes possible once closed-loop dynamic control has been demonstrated. How to Adapt the Architecture in Your Lab. For researchers interested in reproducing or extending this approach, the safest route is to preserve its modularity. Define the controlled output before choosing the model. Decide whether the task requires free-tip position, endpoint pose, several keypoints, or full-shape tracking. A state representation that is sufficient for endpoint control may be inadequate for obstacle avoidance or insertion. Use the lowest chain resolution that captures the relevant modes. Increase the number of virtual joints until endpoint rollout accuracy stops improving meaningfully over the intended MPC horizon. Do not optimize resolution solely for visual reconstruction quality. Characterize the sensing pipeline as part of the dynamics. Measure capture delay, processing delay, timestamp error, and noise under actual motion. Validate velocity estimates on aggressive trajectories, not only on slow calibration motions. Collect persistently exciting but safe training data. Include changes of direction, multiple amplitudes and frequencies, and states near the intended operating limits. If the eventual controller will exploit resonance, the dataset must contain enough dynamic excitation to identify it. Evaluate models through closed-loop-relevant rollouts. Report multi-step endpoint error, phase error, damping behavior, and Jacobian quality. A model with slightly worse one-step loss may be much better for MPC if its long-horizon gradients are more faithful. Introduce constraints and speed gradually. Begin with conservative robot limits and slow references. Increase aggressiveness only after checking deadline compliance, estimator residuals, prediction error, and constraint margins. Maintain an independent emergency stop and robot-level safety controller. Several substitutions are possible without changing the overall architecture. A B-spline perception front end could replace marker-based spatial reconstruction. DEFORM or another differentiable rod model could replace the learned transition function. An ensemble could provide uncertainty estimates for tube MPC. A nonlinear constrained solver could replace both linear MPC and conventional DDP if sufficient compute is available. For adaptation across objects, I would avoid immediately training one enormous universal model. A more practical first step would combine a shared geometric representation with a small object-specific dynamics module. That module could be identified through a brief excitation routine and updated online as model residuals accumulate. For contact-rich tasks, the chain should be augmented with signed-distance, contact-mode, or force features. But contact introduces discontinuities that challenge both local linearization and smooth DDP. A hierarchical design may be preferable: a discrete planner selects contact modes or motion primitives, and continuous MPC handles dynamic tracking within each mode. The Broader Lesson for Embodied AI. This paper is a useful example of structured learning in robotics. It neither attempts to derive every aspect of flexible-object physics from first principles nor asks a neural policy to absorb the entire control problem. Instead, it assigns each component a specific job. Geometry supplies a meaningful state. Data supplies difficult dynamics. State estimation closes the reality gap. Optimization handles changing objectives and constraints. Feedback corrects what the model gets wrong. That decomposition is particularly compelling for embodied systems whose dynamics are too complex for exact analytical modeling but whose safety and task structure are too important to hide inside an end-to-end policy. The most important conceptual shift is from dynamic DLO manipulation as a repeatable open-loop trick to dynamic DLO manipulation as a feedback-control problem. Once the free endpoint can be estimated, predicted, and controlled online, the robot no longer needs every throw, flick, or whip to unfold exactly as rehearsed. The current result is still deliberately bounded: a finite chain, learned dynamics, planar motion, and endpoint objectives. But it establishes the right architectural foundation. The route to faster and more capable DLO manipulation is not simply a larger simulator or a larger policy. It is a representation, estimator, model, and constrained controller designed to work together—and fast enough that the flexible object cannot outrun the feedback loop.