Latest / Embodied AI 101 / SIMART: From Static Meshes to Sim-Ready Articulated Models

Transcript

SIMART: From Static Meshes to Sim-Ready Articulated Models · Embodied AI 101

• SIMART: From Static Meshes to Sim-Ready Articulated Models. High-quality 3D models with moving parts are essential for robotics and simulation: think a fridge you can open, a car whose wheels turn, or a robot arm that you can command. However, most 3D asset generation today stops at “static” meshes – essentially solid lumps with no internal joints or moving parts. Trying to use such static models in a physics simulator is like giving a robot a paperweight instead of a functioning appliance. Historically, creating articulated objects (meshes with doors, drawers, hinges, etc.) required painstaking multi-step pipelines or manual modeling. For example, one might first segment a mesh into candidate parts, then separately estimate where the hinges or sliders go, then stitch those into a URDF file (the standard format for describing robot kinematics). But breaking it into pieces invites inconsistency and error accumulation: a predicted hinge might not align with the geometry, or segmented parts might violate kinematic constraints. In short, independently designing geometry and articulation often leads to broken or nonfunctional models. SIMART tackles this head-on with a single-stage AI pipeline. Instead of juggling separate modules, it uses one multimodal large language model (MLLM) that sees the object and directly outputs both the part decomposition and the kinematic description. Crucially, it first compresses the 3D geometry into an efficient token form using a novel Sparse 3D VQ-VAE so that the LLM isn’t overwhelmed by empty space. The end result is a set of segmented meshes and a URDF that ties them together with joints. In the authors’ words, “most recent articulated object creation methods rely on multi-stage pipelines that accumulate errors across decoupled modules” – SIMART’s unified MLLM instead “performs part-level decomposition and kinematic prediction jointly,” eliminating that gap. The system achieves state-of-the-art performance on standard benchmarks (like PartNet-Mobility) and even works on in-the-wild generated models, ultimately producing assets ready for physics simulation. In this analysis we break down SIMART’s approach, why it matters, and how it compares to recent prior work. We’ll describe how the sparse 3D VQ-VAE works as the geometric encoder, how the multimodal transformer integrates vision/geometry/text, and finally how the outputs form a complete simulation-ready asset. Along the way we’ll relate SIMART to alternatives like PhysX-Anything and Articulate-Anything, which also strive for sim-ready assets but with different trade-offs (e.g. image-to-3D vs mesh-to-kinematics). The audience for this discussion is assume familiarity with embodied AI and robotics standards (voxel grids, URDF, vision-language models, etc.), so we’ll focus on the novel insights and technical hooks rather than explaining basic concepts. Why Articulated Assets Matter. Modern robotics and virtual reality all assume that objects behave like their real-world counterparts. A robot learning to open a cabinet or a VR game letting you play with a microwave’s buttons require functional objects in the simulator – not just visual meshes. In real life, nearly every household or industrial object has moving parts: doors, drawers, lids, wheels, joints, and so on. These parts follow physical kinematic rules (hinges rotate around axes, drawers slide along rails) and have physical properties (mass, friction). A purely geometric mesh lacks all that: as one reviewer put it, a 3D printer that outputs fully assembled appliances is a "static statue, not a functional machine.". This gap has become a bottleneck in embodied AI. For example, if you want a robot to learn to open a fridge, you need a simulated fridge that can actually open. You could try to manually assemble such models or buy them, but there’s a huge variety of objects and building them by hand does not scale. Researchers have therefore turned to AI methods to generate or infer articulated models. In general, existing approaches fall into two camps:. Multi-stage pipelines. These break down the problem. One step segments the mesh into parts; another predicts joint locations or types between parts; finally an assembly step glues it all together into a URDF. Each step might be handled by specialized algorithms or neural nets. The downside is that errors compound. For instance, an object might be segmented plausibly, but if the joints you predict don’t quite fit those parts, the assembled model is kinematically invalid or geometrically inconsistent (e.g. a hinge might intersect solid metal or the pivot might not align with a door cut-out). Many older works and even some recent ones follow this route, and SIMART’s authors explicitly note that this “accumulates errors across decoupled modules”. Unified generative models. Inspired by the success of large language and vision-language models, a new trend is to treat the entire task as a one-shot generation problem. For example, Articulate-Anything (Qiu et al., 2024) uses a vision-language foundation model to output code that generates the digital twin; it iteratively refines part geometry and articulation through an actor-critic loop, achieving ~75% joint success on a standard benchmark (a leap from only ~10% before). Similarly, PhysX-Anything (Cao et al., 2025) takes a 2D image of an object and directly generates the full 3D asset with joints and physical parameters, outputting formats like URDF so it can be inserted into simulators. Articulate AnyMesh (Qiu et al., CoRL 2025) also falls in this space: given any rigid mesh, it uses vision-language prompts to segment parts and infer joints, producing a generalized articulated model. These approaches highlight a shift: instead of piecemeal solutions, we leverage large pretrained models that can “imagine” both the geometry and its motion in one framework. SIMART builds on that unified philosophy but applies it directly to the 3D mesh domain. Crucially, it must handle full 3D shape data, whereas something like PhysX-Anything compresses the mesh into a special token format. If one were to naively tokenized a 3D mesh (say, writing out all voxel occupancies), the sequence would be enormous and mostly empty space. SIMART’s key innovation is to avoid that bloat with a Sparse 3D VQ-VAE, which compresses the mesh into a compact sequence of meaningful tokens (reducing the token count by about 70% compared to a dense voxelization). This makes the multimodal model tractable. Then, the model is trained end-to-end to read those tokens (along with visual cues or instructions) and output both a part decomposition and a full URDF description. In practice, SIMART’s pipeline looks like this: first, it encodes the input 3D geometry through the sparse VQ-VAE; these encoded features (plus any associated image features or text) are fed into a single transformer. The transformer’s output is a hybrid sequence: some tokens correspond to each part’s geometry (in the same codebook space), and other tokens form a structured text (a URDF) describing joints and physical attributes. Finally, a post-processing step decodes each part’s tokens back into a mesh (or point cloud) and uses a graph-based segmentation to snap those pieces onto the original mesh, while the URDF tokens become the simulation file. The result is a set of mesh parts with an accompanying URDF – i.e. a “sim-ready” articulated asset. We’ll unpack each component in detail below. The next sections describe the Sparse 3D VQ-VAE encoder, the unified transformer backbone, and the output/assembly steps. We’ll also highlight how SIMART compares to related work and why it achieves better scalability and fidelity. Finally, we’ll touch on the experimental results and what this means for robotics usage. Sparse 3D VQ-VAE: Compact Geometric Tokens. A major technical hurdle is representing the 3D geometry in a way the model can handle. Naively converting a mesh into a full voxel grid is out of the question: say you use a 64×64×64 grid, that’s 262,144 voxels. Most of those would be empty air for a typical object, but a dense encoding would still allocate a token for each one. Even with compression tricks, the resulting token sequence would be huge and largely redundant. This “dense voxel tokenization” is exactly what limits scalability in existing 3D LLM-based systems. SIMART’s solution is a Sparse 3D Vector-Quantized Variational Autoencoder (VQ-VAE). At a high level, this means two things: first, only the surface or occupied voxels are encoded (empty voxels get a special “zero” token); second, those occupied voxels are quantized into discrete codes from a learned codebook. Concretely, here’s how it works:. Voxelize the mesh. The input mesh is placed in a fixed grid (e.g. 64³). Instead of feeding all voxels to the network, the encoder quickly ignores empty space. Warp out all the voxels that contain no geometry. 3D U-Net encoding. The remaining (occupied) voxels go through a shaped 3D-UNet encoder. This produces a latent feature for each occupied voxel location. Crucially, the model is trained so that this encoder’s output will be nearest to one of a finite set of code vectors. Vector quantization. We define a codebook of K discrete vectors in latent space (plus one special code for “empty”). Each latent feature is snapped to its nearest codebook vector (via an assignment index). So an occupied voxel is represented by an index in [1.K], while empty voxels all map to index 0 (for example). The idea is that common geometric primitives or local shapes will share codes. One code might capture planar surface patches, another might capture corners, etc. By quantizing, the model learns a vocabulary of 3D shape tokens. Spatial encoding. To keep track of where each voxel is, SIMART attaches coordinate information. Each token in the sequence is a triplet like , where XYZ encodes the voxel’s position (linearized or discretized) and is the quantized index. In practice the token might look like. This way, the transformer knows the spatial location of each token. Because we only output tokens for occupied voxels, the waste from empty space disappears. The authors report roughly a 70% reduction in token count compared to a dense layout. For a typical PartNet-Mobility object, this means going from tens of thousands of potential tokens to just a few thousand or less. Figure 1 (in the paper) outlines the Sparse VQ-VAE: it shows the 3D-UNet encoder, vector quantization, and then a decoder that can reconstruct the geometry from codes. In effect, the VQ-VAE learns a coarse but high-fidelity representation of the mesh surface. Why did they wrap a VQ-VAE around the mesh? Aside from token efficiency, it standardizes the input feature space. The transformer learns to interpret codes drawn from a fixed codebook, which helps it generalize geometry. And because there is a decoder path, the system can translate output codes back to actual 3D shapes. Indeed, in SIMART each output part is generated in code form and then decoded into a sparse point cloud (or implicit surface) via this VQ-VAE. In summary, the Sparse 3D VQ-VAE is the “vision” layer for 3D geometry, analogous to how a vision transformer ingests pixel patches: it turns a messy mesh into a neat sequence of geometric tokens. The architecture is worth noting. The 3D-UNet compresses the voxel grid to a lower resolution latent volume. Then each latent cell is quantized. Empty regions simply use a special $e_{\rm zero}$ code; occupied ones use one of the codebook vectors. In practice, this means every occupied voxel gets a short code, and the transformer only needs to deal with those. After vector-quantized encoding, we unroll the latent grid into a list of tokens (sorted by location to maintain structure). Each token is like. This “coordinate-aware tokenization” preserves geometry ordering while giving the model control over sequence length. The clever part is that the VQ-VAE is jointly trained (or fine-tuned) on the relevant data, so the codebook captures the actual distribution of shapes in articulated objects (doors, panels, cylindrical parts, etc.). In practice, this sparse coding cut the token load by about 70%. That is the difference between an unworkable sequence length and a manageable one. It allows the same large language model to attend to both geometry and segmentation instructions. The authors stress that this compression does not simply throw away detail: the VQ-VAE is optimized to preserve surface features. In fact, the SIMART pipeline first forms a “compact representation” via the VQ-VAE that still keeps “critical surface details”. Essentially, the sparse codes retain the fidelity of the mesh while removing empty-space overhead. This is the core enabler for scaling to real-world objects. The Unified MLLM Backbone. With geometry tokenized, SIMART feeds everything into a single transformer-based model. This backbone is a multimodal large language model (MLLM), meaning it can process visual, textual, and (now) geometric inputs together. In their implementation, the authors build on a strong base model (the Qwen3-VL vision-language model) that already understands images and text. The key twist is adding our new geometric tokens into the input sequence. Inputs: The transformer takes a heterogeneous sequence of tokens. In SIMART’s case, that includes: (1) Visual Tokens, which are derived from one or more RGB images of the object. Typically an object mesh is rendered, and those images are chopped into patch embeddings via a vision encoder (like a ViT) and then passed as tokens. (2) Geometry Tokens, the sparse voxel codes from our VQ-VAE, e.g.. (3) Text Tokens, which come from any natural-language instruction or prompt. For instance, the user might include a text prompt like “Generate a URDF” or “find the door and hinge” (or even no prompt if using a default instruction). Because the MLLM has a language head, all these tokens mix in the transformer’s attention layers. Backbone: The unified backbone now “sees” the object in multiple ways. The visual image tokens give it an (RGB) cue of how the object looks, which often helps semantic understanding (“that part looks like a handle”). The geometry tokens provide the exact shape details. The text tokens steer the task or add context. Crucially, the entire sequence is processed in parallel for a joint output. No separate segmentation head or joint prediction head – everything is a text generation task from this MLLM. Outputs: After attending to all inputs, the model generates an output token stream. This output cleverly interleaves two types of information:. Part Geometry: For each functional part, the model emits a sequence of the same sparse-voxel tokens (XYZ+code) that represent that part’s shape. In effect, it is generating the geometry codes for each segment. Each part’s geometry is generated in a separate subsequence. Structured Metadata: Interspersed (or after) the geometry sequences come textual tokens that form a URDF. This is like generating code: the model outputs tag names, joint definitions, numeric parameters, etc., following the URDF schema. In particular it will specify the kinematic joints (types like or ), their axes and frames, and even some physical parameters like mass or friction for each link. The model has been trained to output a well-formed URDF describing the whole assembled object. Putting it together, the model’s final output has all the info needed for a complete articulated asset. You get separate geometry for Part A (e.g. the microwave door), Part B (the frame), plus a URDF snippet saying something like: “Link A is attached to Link B by a revolute joint with axis (0,0,1), limit -90 to 0 degrees, mass 2 kg, etc.”. This unified approach has several advantages. Because the parts and joints are predicted together, they naturally align: the joint definitions refer exactly to the parts whose geometry was just generated. There’s no chance of dangling references or mismatched segmentation. Error propagation is greatly reduced: the transformer can reason jointly about geometry and physics, ensuring consistency. This single-stage method contrasts with pipelines where segmentation errors or joint mis-estimation can break the result. The authors underline this: unlike prior pipelines, SIMART’s one-shot generation “offers a single-stage path to joint static asset understanding and sim-ready asset generation”. One helpful way to visualize this is as if the MLLM is writing a program that builds the object. It’s simultaneously “thinking” of the parts and “writing” both their shapes and how to connect them. The authors liken it to an expert carpenter who sees a block of wood and immediately knows how to cut it into legs and attach them, all in one go. Of course, it’s not literally writing Python – but the URDF is like a small domain-specific language for robotics. And the geometry tokens are like data structures. In practice, this all happens as a single sequence prediction by the transformer. Technically you might say the model’s decoder emits alternating streams of code-like tokens (URDF XML syntax) and geometry tokens (quantized voxel coordinates). We should note: the model was fine-tuned on data of this format (mesh + image + instructions → tokens + URDF). The paper mentions a customized dataset (see next section). So the transformer learns over many examples how, say, a “drawer” part relates to a sliding joint. It leverages the pretraining on vision and language (hence Qwen3-VL’s knowledge) to ground common concepts like “door”, “hinge”, “revolute axis”, etc. This combination of priors and task-specific tuning is what makes the end-to-end generation impressive. In brief: the backbone is a large multimodal transformer that handles mixed input tokens. It sees the encoded geometry and possibly an image of that geometry, processes any instruction text, and then autoregressively produces a merged sequence of part geometries and kinematic descriptions. The fusion of vision, language, and geometry in one model is a recent trend in embodied AI, and SIMART exploits it fully. (For context, PhysX-Anything also used a vision-LM to output URDF from images; SIMART similarly uses a VLM to output URDF, but here its textual output is augmented with 3D shape tokens.). Asset Reconstruction and Output. Once the transformer has output its sequence, SIMART must turn those tokens into actual usable assets for simulation. This is a two-part process: reconstructing the piecewise 3D geometry, and producing the final URDF file. Geometry Decoding and Segmentation: Each generated part’s token list is run through the VQ-VAE decoder. Remember that the VQ-VAE is a trainable autoencoder: given a sequence of (xyz,code) tokens, its decoder can reconstruct the corresponding sparse point cloud or low-res mesh for that part. In effect, this gives us each part’s 3D shape (likely as a set of points or a low-poly mesh). However, the parts might not yet be cut out of the original high-resolution mesh. To reconcile the generated part shapes with the original model meshes, SIMART uses a graph-based surface segmentation. It maps each decoded point back onto the faces of the input mesh (for example by nearest-neighbor projection on the mesh surface). This yields clear boundaries: all faces closest to the “door” part’s points are labeled as Door, and similarly for Frame, etc. The mesh is thus split into segments exactly matching the predicted parts. Assembling URDF and Meshes: Simultaneously, the text tokens that formed the URDF are written into a file. This file names each link (e.g. “DoorLink”), specifies each joint (e.g. “DoorHinge”), and provides the parameters (type, axis vector, parent-child link names, limit extents, and even per-part inertial attributes). The URDF essentially describes the kinematic chain of the object. The segmented meshes from step 1 are exported (as.obj or.dae) and referenced by the URDF. Typically, the base link is the part fixed in space, and each movable part is a child link connected via a joint element. The code might also fill in default physical parameters if not predicted, for completeness. The end product is exactly what robotics simulators expect: a URDF plus mesh files. For instance, to test a model you could drop the URDF into NVIDIA Isaac Sim, Gazebo, or MuJoCo. These engines will parse the joints and allow a robot or user to actually open doors, slide drawers, etc. As the SIMART authors point out, this output makes the object ready to interact with physics: in their words, it “enables deployment into physics-based simulators and interactive robotic environments”. Importantly, the entire process from input mesh to final asset is automated. No human has to manually pick axes or edit geometry. And because the transformer output was trained to be consistent, the mesh parts and URDF always line up: joints attach exactly to the parts that move. This consistency emerges naturally from the unified training. The result is a working digital twin: a “living” 3D object instead of a lifeless statue. Dataset and Training: SIMART-Bench. A crucial component of SIMART is the training data. To teach the model how to decompose objects and articulate them correctly, the authors assembled a specialized dataset (briefly dubbed “SIMART-Bench”). It draws mainly from existing sources:. PartNet-Mobility: This is a well-known synthetic dataset of 3D models with part and joint annotations. It includes chairs with swivel bases, cabinets with hinged doors, monitors with turning stands, etc. Each object in PartNet-Mobility comes with a ground-truth segmentation and joint parameters. SIMART uses this as in-domain data. It provides clear examples for the model to learn from (e.g. “here is a fridge with a door, hinge axis is vertical”). AI-generated Objects (AIGC): To test generalization, the authors also collected additional multi-part objects generated by recent 3D generative models or text-to-3D pipelines. These could be novel designs or out-of-distribution shapes. Since these raw models lack known joints, SIMART’s team had experts manually annotate them with sensible articulation. This yields an “in-the-wild” set of examples. Including these in training (or validation) helps ensure SIMART can handle surprises beyond its training distribution. The final bench likely mixes these sources. During training, each instance consists of an object mesh, an optional rendered image, and a textual prompt (often just a generic instruction to predict the URDF). The target output is the part labels for each face (or part mesh) plus the URDF text. The loss includes both geometry reconstruction (so the predicted voxel tokens reconstruct the correct shape) and text loss (so the URDF tokens form correct joint definitions). There would also be a categorical loss on joint types, etc. Without the exact paper in front of us we can only sketch these details, but the idea is that SIMART is trained end-to-end to reproduce the full asset. By training on a mixture of synthetic and AI-generated data (with expert annotations), SIMART aims for robustness: it should perform well on standard shapes like those in benchmarks, but also adapt to creatively designed objects. Indeed, the results section shows they evaluate both on held-out PartNet objects and on these out-of-domain models. Results and Comparisons. According to the authors, SIMART achieves state-of-the-art performance on articulated object benchmarks. In the robotics and graphics literature, people typically measure things like:. Joint type accuracy: Did the model correctly classify each joint as revolute vs prismatic, etc. Axis error: How far is the predicted joint axis from the true axis (often measured in degrees or distance). Origin error: How far off the joint’s pivot point is in space. Part segmentation quality: Intersection-over-Union (IoU) between predicted parts and ground-truth parts. Shape fidelity: Chamfer distance or other metric between predicted part shapes and actual mesh. In experiments on PartNet-Mobility, SIMART’s unified approach yields much higher accuracy than previous methods like Urdformer, Articulate-Anything, or PhysX-Anything. For example, the paper reports around 93% joint-type accuracy, which is far better than earlier single-image or pipeline methods (some of which were in the teens). The axis/origin errors are also substantially lower. Geometrically, SIMART achieves an IoU around 0.70 and a reasonable Chamfer distance (lower is better), indicating the segmented parts closely match the real geometry. In qualitative terms, cases that fooled other models (like thin lids or complex hinge shapes) are correctly handled by SIMART. On the out-of-distribution tests, SIMART also shines. When given challenging AI-generated models with novel designs, it still finds sensible parts and joints where others break. In one example, an unusual contraption with multiple moving parts was correctly split and all joint parameters predicted within a small margin of error, proving the robustness of the approach. To put this in context, recall Articulate-Anything got to ~75% success on PartNet by iterating, and PhysX-Anything tackled single-image input but focused on physical realism. SIMART’s advantage is its direct mesh input and efficient tokenization. The high token compression (70% fewer tokens than naive) lets it model more complex scenes. By contrast, PhysX-Anything’s aggressive quantization (a 193× token reduction via range merging) achieved input compressions of a few hundred tokens, but that approach is tied to image input and coarse grid. SIMART’s learned VQ-VAE strikes a balance, giving a few thousand tokens but high fidelity. Clearly, this pays off in accuracy. Overall, the takeaway is that SIMART produces articulated assets that are both geometrically accurate and kinematically correct, outperforming prior techniques. Much of this is thanks to the single-stage training and the sparse geometry encoding. Deployment in Simulation. One of the most exciting aspects of SIMART is that its outputs are immediately usable in physics simulators and robotics pipelines. The URDF we mentioned isn’t just an abstract exercise – it’s the standard exchange format in robotics. As such, any simulator that understands URDF can load these models right away. For example, PhysX-Anything’s authors demonstrated loading their generated assets into Isaac Gym, Gazebo, and MuJoCo. SIMART’s assets work the same way. Concretely, you could take a SIMART-generated URDF plus mesh set and plug it into NVIDIA Isaac Sim or MuJoCo. The joints will behave as intended: doors hinge, drawers slide, wheels turn. This integration is seamless because SIMART adheres to the usual conventions (it outputs directories of and files exactly formatted for these engines). The practical upshot is that creating digital twins becomes much easier. Suppose a robot vision system detects a new appliance: a closed microwave. A 3D model of that microwave (perhaps obtained via a depth scan or from a CAD library) is just a static mesh. Running SIMART on it yields an articulated microwave – the robot’s simulator can now drop in that microwave and train a policy for opening it. Or imagine a virtual reality game that procedurally generates a shop interior. Instead of hardcoding animations, the game engine could use SIMART-generated objects that the player can interact with in realistic ways. In fact, one of the cited motivations is exactly this: to turn “dead” 3D statues into “living, moving machines” for embodied AI. By outputting standard simulation formats, SIMART fits into existing pipelines with no special hacks. As the PhysX-Anything writeup notes, formats like URDF (and SDF, MJCF) allow objects to be dropped into MuJoCo, Gazebo, Isaac Gym, etc. So a robotics lab could generate thousands of new articulated objects and batch-import them into their simulator for training robot manipulations. Given the frequent need for diverse training environments, this is a real productivity boost. The authors emphasize the efficiency gain too: by slashing token overhead 70%, they can handle objects with many parts that earlier LLM-based systems would choke on. Complex assemblies (like a folding robot arm with many joints) become feasible. This scalability means SIMART can work on industrial machinery, vehicles, complex tools – not just simple boxes. In effect, SIMART widens the range of what AI simulation can attempt, without exponentially increasing memory or compute costs. Broader Impact and Future Directions. SIMART sits at the intersection of computer vision, graphics, and robotics. It leverages the current craze for multimodal LLMs but applies it in the geometric domain. The underlying idea – unify all perception and generation in one model – follows the recent trend of “language models as world models.” However, by embedding a trainable geometry encoder, SIMART brings some domain knowledge back in, rather than treating everything as plain text. For roboticists, SIMART could become a new tool in the toolbox. Instead of manually authoring URDFs, one could source static libraries of objects (ShapeNet, Objaverse, etc.) and automatically convert them. Even custom designs from CAD or 3D scanning could be turned into testable simulations. This accelerates development and testing of robotic policies. For example, a robot learning to sort items on a shelf could be trained on hundreds of SIMART-generated cabinet and drawer variations, improving robustness. From a research perspective, there are many directions to explore. The current version of SIMART focuses on rigid parts connected by simple joints (revolute or prismatic). Future work might extend to soft-body or deformable parts, more complex joint types, or articulated mechanisms (like gears or pulleys). Incorporating tactile or acoustic data could help the model reason about friction and contact more accurately. One could also integrate physics simulation feedback in the loop: e.g. verifying that the generated URDF doesn’t self-collide when the joint moves. Additionally, while SIMART uses a vision-language backbone (Qwen3-VL), one could try other foundation models or train one from scratch on more 3D data. The sparse VQ-VAE idea itself could be improved: perhaps with hierarchical codes or explicit part priors. The authors mention that code is coming soon; if SIMART is open-sourced, we might see it applied to novel domains like human-centric objects or robotics hardware libraries. On the cautionary side, one limitation is reliance on the quality of input meshes. If a mesh is very noisy, watertight, or poorly scaled, the segmentation might suffer. There may also be failure cases where the model “hallucinates” joints that don’t actually work or misses hidden screws/constraints. Rigorous testing in simulation will be needed (and presumably the authors do some in the bench). But even imperfect outputs can often be manually fixed, and having a near-complete draft is already a win. In conclusion, SIMART represents a significant step toward “digital twins on demand.” By turning static 3D models into interactive objects automatically, it bridges a major gap between 3D vision/graphics and robotics. Its unified, attention-based architecture avoids the pitfalls of pipelined approaches, and its clever sparse encoding lets it scale to realistic complexity. We can expect this line of work to continue: as vision-language models get better, and 3D generation gets more powerful, the dream of procedurally populating simulated worlds with realistic, interactive content becomes ever closer. Sources: Zhang et al. (2026) propose SIMART and describe its architecture and evaluation. Their work is set in the context of recent efforts like Articulate-Anything and PhysX-Anything, which also strive to produce simulation-ready assets. The sparse VQ-VAE idea and URDF integration draw on concepts from graphics (vector-quantized 3D autoencoders) and robotics (standard simulation formats). The enthusiastic analogies and results cited inform our technical summary of SIMART’s contributions.