Latest / Embodied AI 101 / TriSplat: Feed-Forward 3D Reconstruction with Triangulated Meshes

Transcript

TriSplat: Feed-Forward 3D Reconstruction with Triangulated Meshes · Embodied AI 101

• TriSplat: Feed-Forward 3D Reconstruction with Triangulated Meshes. Reconstructing a 3D scene from a handful of images is a classic problem in vision. In robotics and simulation, the goal is not just impressive renderings but a usable mesh – one that you can load into a simulator or physics engine and immediately start interacting with. Traditional pipelines (think multi-view stereo plus Poisson surface reconstruction, or optimizing a NeRF) can yield high-fidelity models, but they are often slow and brittle. In recent years, a new paradigm has emerged: feed-forward neural models that map images directly to 3D geometry without per-scene optimization. Many of these models represent scenes as collections of Gaussian “splats” or other volumetric primitives, using differentiable rendering to train. Unfortunately, Gaussian primitives only give implicit geometry – you still need TSDF fusion, marching cubes, or Poisson reconstruction to extract an actual mesh. That defeats the low-latency, plug-and-play promise of feed-forward methods. TriSplat, a 2026 method by Wang et al. (Zhejiang, ETH, Microsoft, Monash), breaks this cycle by working with triangles natively. Instead of drifting between implicit densities or fuzzy Gaussians, TriSplat’s network directly predicts a set of oriented triangle primitives that represent the scene surface. In one forward pass on a few images (poses unknown!), TriSplat outputs a triangle mesh that you can immediately drop into Unity, PyBullet, Isaac Sim, or any renderer. There’s no need for post hoc TSDF fusion or Poisson reconstruction – the triangles are already surface elements. This “simulation-ready” output is a boon for robotics: you get explicit collision geometry and textures without extra processing. In brief, TriSplat reads multiple uncalibrated images and jointly estimates the camera parameters, a dense local 3D point map for each view, and the parameters of a triangle at every pixel. Each predicted triangle is anchored in space via a normal direction derived from the point map, refined by an image-conditioned normal network. During training, TriSplat uses a clever curriculum – starting with broad, semi-transparent triangles and gradually sharpening them – so that the triangles gradually “snap” into a crisp mesh. The authors evaluate on indoor-scene datasets (RealEstate10K, DL3DV) and show their meshes are more faithful to the true geometry than Gaussian-based baselines, while rendering novel views almost as well. Crucially, inference is very fast: TriSplat produces a full mesh in about 1.3 seconds, whereas Gaussian-splat pipelines typically need tens or hundreds of seconds to convert to mesh. This opens the door to real-time or batch offline scanning of environments for simulation. Below we unpack TriSplat’s approach in detail, contrast it with existing methods, and discuss why predicting triangles directly is such a compelling idea for robotics and graphics. The Challenge of Feed-Forward 3D Scene Reconstruction. Modern 3D reconstruction has two primary regimes. One is optimization-based: given a set of images and their known or estimated poses, one runs bundle adjustment, dense stereo, or neural rendering optimization (NeRF). These can yield very high-quality models (often meshes after post-processing), but each new scene requires minutes or hours of computation. This is fine for offline content creation, but it’s unsuitable for applications like robot vision or interactive scanning. The other regime is feed-forward (or generalizable) reconstruction. A single neural network is trained on many scenes so that, at test time, it can take pictures of a new scene and (in one forward pass) output a 3D model. This approach is much faster at inference and can work even when camera poses or intrinsics are unknown. Recent work in this space has tried various representations: implicit fields (like a plug-and-play NeRF model), point clouds, voxels, or collections of primitives. Notably, several methods have converged on splatting representations: they predict a set of small primitives (often Gaussians or oriented ellipsoids) that are rendered and composited together. Methods like Splat-SAP or PreF3R show that you can predict Gaussians and then render novel views with quality approaching NeRFs, but their output is still a volume-like representation. The problem is that simulation and physics need meshes, not a cloud of Gaussians. Converting Gaussians to a closed mesh is expensive. One typically does something like converting to a TSDF (volumetric grid) and then running Marching Cubes or Poisson reconstruction. These post-processing steps take time and often introduce artifacts (holes, soft edges, smoothing, etc). In the TriSplat paper, the authors emphasize this point: Gaussian splats “expose surfaces only indirectly,” and you still need “expensive post-hoc steps” to get a mesh. Their insight is to cut out the middleman: why not predict triangles directly? If the network’s rendering primitives are already triangles, then the forward pass itself produces an explicit surface. No external reconstruction needed; the network’s output is instantly a mesh you can export. This idea of using triangles is reminiscent of older graphics pipelines and recent “neural triangle” works. In fact, a physics-based renderer inherently works with triangles (GPU rasterization, etc.), suggesting a natural fit. We see motivational echoes in Held et al.’s “Triangle Splatting” (3DV 2026): that work shows triangle primitives can be used in differentiable rendering to great effect, bridging the gap between neural rendering and classical pipelines. TriSplat takes the triangle-native philosophy into the feed-forward reconstruction domain. TriSplat’s Architecture and Pipeline. At a high level, TriSplat’s network takes (N) images (even if the camera poses are unknown) and outputs:. A dense 3D point map for each view (essentially a depth estimate for every pixel). Triangle attributes for each pixel (which determine how a small triangle at that pixel should be rendered). The camera poses (and optionally intrinsics) for each input view. From these outputs we immediately have a rendered mesh: each pixel’s triangle is placed in 3D, oriented by its normal, and has color/opacity. A single forward pass yields a fully triangulated scene. The core components are:. Backbone and Transformer Fusion. TriSplat uses a DINOv2-pretrained vision backbone (ResNet-like) to extract image features, with added 2D positional encodings. Unlike many methods that assume fixed camera pose, TriSplat must handle unposed images. The network takes as input multiple images, but it does not know their relative position apriori. To fuse the information, TriSplat uses a transformer-style decoder. The design alternates between self-attention over each view and cross-attention between views, so that features can align pixels across images. The transformer outputs shared feature tokens, from which three “heads” emerge: one head predicts the 3D point map, one head predicts the camera parameters, and one head predicts triangle parameters. Point Head – Local 3D Geometry. The point head produces, for each pixel in each view, a three-dimensional point (\mathbf{P}(u,v)) in that camera’s local coordinate frame. More precisely, for pixel ((u,v)) it predicts ((\Delta u,\Delta v, z')), where (z = \exp(z')) is depth and ((u,v)) off-center coordinates may allow sub-pixel shifts. In effect, each pixel learns where in 3D its “triangle center” should be. These dense point maps from each view implicitly reconstruct the scene geometry, and they also anchor the triangles in space. Camera Head – Relative Poses and Intrinsics. A parallel head regresses the camera extrinsics (and optional intrinsics) from the same fused feature tokens. Because no poses are given at inference, the network must figure them out. Training on synthetic or real scenes with known poses helps it learn to predict these. In practice TriSplat demonstrates it can generalize to environments like ScanNet with zero-shot (using a model trained on RealEstate10K plus DL3DV). The output is a set of relative poses that reproject the multi-view geometry coherently. Triangle Head – Surface and Appearance Attributes. The final head yields the parameters of the triangle at each pixel. What exactly are these “triangle attributes”? In principle they include things like the triangle’s opacity, its color or texture, and perhaps a blur or blend radius. The idea is that each pixel’s triangle is initially semi-transparent and softly tapered, and the network predicts how opaque (and how big) it should be. Geometry Anchoring via Normals. A crucial step is deciding how each triangle is oriented in 3D. Rather than learning an arbitrary orientation, TriSplat anchors each triangle’s normal to the scene geometry. It first computes a raw normal from the point map: for each pixel, it looks at neighboring point predictions (via finite differences) to form a little local plane. This gives an initial “geometry normal.” Then, an image-conditioned U-Net (trained as a “normal head”) refines these normals: it takes the raw normals and also the RGB appearance and outputs final, smoothed normals. Training uses a monocular-normal bootstrap: early in training, the network leans on a pretrained monocular normal estimator (from Omnidata) to provide supervision, and gradually shifts to its own predictions. This curriculum stabilizes learning. Once we have a final normal ( \mathbf{n}(u,v) ) for each pixel, that normal defines the orientation of the triangle at (\mathbf{P}(u,v)). In effect, each triangle lives in the tangent plane of the local surface. Rendering and Losses. With oriented triangles at each pixel, TriSplat can render views differentiably. The pipeline uses a custom triangle rasterizer (inspired by works like YoNoSplat and Triangle Splatting) to composite all triangles with alpha-blending. It outputs a rendered RGB image, a depth map, and a normal map for each view. These are compared to ground truth (or nearest-neighbor warps of ground truth, in unposed settings). Photometric and geometric losses train the network end to end: it tries to match the input images and enforce depth/normal consistency. Because everything is differentiable, the triangles’ positions and opacities adjust to explain the images. Progressive Sharpening. Training a mesh directly can be tricky if you force hard edges too soon. TriSplat eases into it with a progressive sharpening schedule on each triangle’s opacity and blur. Initially, triangles have large, blurred footprints and low opacity (so they overlap softly), making it easy to explain the scene with many soft elements. Over training, a schedule slowly increases a “temperature” parameter that sharpens the triangle edges and boosts opacity. By the end, each triangle is sharp and opaque, giving crisp surfaces. This annealing helps the network converge stably: early on it can rely on fuzzy splats to capture coarse shape, then refine to sharp mesh. Mesh Export. After a forward pass, we have a dense cloud of oriented triangles (one per pixel). To extract a standard 3D mesh, TriSplat essentially “bakes” these out. It removes any triangles that remained very low-opacity (they contributed nothing), then corrects face winding (ensuring consistency of normals), and merges nearby vertices (since adjacent triangles share vertices). The result is a triangular mesh with vertex colors. The authors denote this the “direct mesh” and also a cleaned “_post” mesh with small components removed. That mesh can immediately be saved as PLY/OBJ/PLY. The full pipeline is shown in the TriSplat teaser: from sparsely sampled unposed images, the network outputs depth and normals per pixel, plus a collection of tiny oriented triangles that exactly trace out the scene geometry. Because these triangles were the rendering primitive all along, the final mesh is identical to what the network implicitly knew about the surface. No extra reconstruction step is needed; the feed-forward promise stays intact. Triangles vs. Gaussians: Why Mesh-Native Representations Matter. Much of the recent literature on feed-forward reconstruction has revolved around Gaussian Splatting or related primitives. In that world, a network might predict thousands of little 3D Gaussians scattered in space, each with a covariance and color. Rendering them (by projecting a 3D Gaussian to an elliptical silhouette on the image and blending) produces high-quality novel views in real time. Examples include methods like YoNoSplat and other “3D Gaussian Splatting” approaches. These systems can be fast and yield great images, but end users (e.g., a robot or simulator) still need a mesh. Extracting a mesh from Gaussians is an extra step: either integrate the Gaussian field into a volumetric TSDF and run Marching Cubes, or treat each Gaussian as a pseudo-point cloud and do Poisson surface reconstruction. Both are computationally heavy and often do not capture fine details well; they also smooth the result. In short, Gaussian-based reconstructions “expose surfaces only indirectly” and require expensive post-hoc meshing. That breaks the feed-forward promise that the network alone should produce the final output. TriSplat sidesteps this by embracing triangles as first-class citizens. What benefits does this bring?. Direct Mesh Output. Because the output primitives are triangles, TriSplat can export a mesh with no intermediate fusion. Each predicted triangle becomes one face (or part of a face) in the mesh. The authors note that the simulator-ready output can be “directly ingested by physics engines, collision detectors, and standard rendering pipelines” with no conversion. In practice this means you get a standard mesh whose vertices and faces you can feed into Unity, Blender, Nvidia Isaac Sim, PyBullet, etc. The TriSplat code base even demonstrates exactly this: the refined mesh can be opened by any simulator. By contrast, using a Gaussian-based method would have required an extra processing step that could take tens of seconds or more, and might lose surface accuracy. Physical Consistency and Collision. In robotics, having an explicit collision mesh is important. A collection of fuzzy Gaussians doesn’t lend itself naturally to strict intersections. A triangle mesh, on the other hand, can be used directly for collision detection and physical simulation. In the TriSplat write-up, they highlight an interactive demo where multiple agents drive around on the predicted mesh surface. This demonstrates that TriSplat’s outputs truly behave like physical platforms or walls, not just an image impression. Sharp Geometry and Detail. By anchoring triangle normals to the predicted geometry, TriSplat enforces that adjacent triangles align into a smooth surface. The progressive sharpening schedule then lets the model carve out crisp edges. Empirically, the authors show that TriSplat’s reconstructions are more geometry-faithful than Gaussian baselines. Gaussian splatting tends to blur corners and small features, whereas TriSplat’s explicit mesh can preserve facets and corners directly. This is particularly important for robotic tasks that rely on accurate geometry (e.g. grasping a corner or sliding along a wall). Computational Efficiency. It may seem like triangles – each rendered individually – could be expensive. Surprisingly, TriSplat is extremely fast at inference. On RealEstate10K scenes it can generate the mesh in about 1.3 seconds (on modern hardware), including rasterizing all triangles into RGB, depth, and normal images and converting to a mesh. By comparison, the same scene reconstructed with a Gaussian-splat pipeline (followed by TSDF meshing) could take tens or hundreds of seconds. This speedup is due in part to TriSplat’s feed-forward design (no optimization loops) and in part to the lean representation (no costly volumetric computations or Poisson solves). Triangles turn out to be very GPU-friendly: they leverage the same rasterization tricks that real-time graphics hardware has optimized for decades. In fact, other work (Held et al.) has shown triangle-based rendering can achieve thousands of frames per second once trained. Compatibility with Graphics Hardware. Triangles are the native primitive of GPUs. TriSplat’s triangles could in principle be rendered by standard graphics APIs with no extra overhead. This means the pipeline easily interfaces with existing graphics stacks. For example, the exported mesh can be rendered with OpenGL or Unreal Engine without custom shaders. This is harder with arbitrary Gaussians. Of course, TriSplat also brings its own engineering complexity (more on that below). But the key is: by training the network to produce triangles in the first place, TriSplat bakes the mesh into its very representation. In a sense, it unifies neural reconstruction with classical mesh representations, achieving the best of both worlds. Training TriSplat: Anchoring Normals and Progressive Focusing. Predicting triangles requires care in training. If we just naively had a neural network spit out a bunch of triangles and hoped for the best, the gradients might be messy and surfaces could flicker. TriSplat addresses this with two important techniques: normal anchoring and progressive sharpening. Geometry–Anchored Normals. A triangle in space is defined by three points (vertices). TriSplat chooses to represent a triangle at each pixel by its center point and a normal direction (plus some side length or blur). How to get that normal? Rather than treating the normal as a free output dimension (which could drift arbitrarily during training), TriSplat computes normals from the predicted geometry itself. This is called normal anchoring. Concretely, once the network predicts the local 3D point map (\mathbf{P}(u,v)), one can compute a quick estimate of the normal at (u,v) by looking at neighboring points. For example, using finite differences: [ \mathbf{n}(u,v) \approx \bigl(\mathbf{P}(u+1,v)-\mathbf{P}(u,v)\bigr) \times \bigl(\mathbf{P}(u,v+1)-\mathbf{P}(u,v)\bigr). ] This gives a raw normal vector (before normalization). Doing this densely yields a “geometry normal map” over each view. In practice this raw normal is noisy and might have “hard-edged” artifacts where geometry is uneven. TriSplat refines this with a lightweight normal refinement network. This is a small U-Net (all convolutional) that takes as input the raw geometry normals, a smoothed (pooling-averaged) version of those normals, the rendered RGB appearance, and the visibility mask (indicating which points are valid). Its output is a smoother normal map. Early in training, TriSplat even uses a pretrained monocular normal estimator (OmniData’s DPT-normal model) to help the normals start in a reasonable configuration. They call this a “mono-normal bootstrap”: at the beginning, the loss encourages the network normals to match those from the pretrained model, and then gradually Anneal to let the network predict its own normals. By anchoring the triangles to these geometry-aligned normals, TriSplat ensures the triangles conform to the surface shape and avoids arbitrary tilting. The authors note that turning the triangle orientation into an explicit function of the geometry (instead of an unconstrained latent) yields much more stable training and higher fidelity. In practice, each triangle sits flush with the local surface it represents, like a tiny planar patch tessellating the scene. Progressive Opacity and Blur Scheduling. Even with anchored normals, asking the network to learn crisp mesh geometry from day one is hard. You want it to gradually “sharpen” into a mesh. TriSplat does this with a temperature schedule on the triangles’ opacity and edge blur. In essence, the network learns parameters (like an “opacity temperature” per triangle) that start low (opaque and blurry) and end high (sharp edges). During early iterations, triangles overlap widely, making training forgiving: errors average out and large parts of geometry can be explained by big soft splats. Over the course of training (they use about 16K steps), they steadily increase the temperature so triangles become smaller and more opaque. This forces the solution to refine to a fine-grained mesh. In the final stages, each triangle is nearly fully opaque and has crisp boundaries. This schedule is implemented via a smooth ramp from temperature 1.0 to 5.0 (for instance) as training progresses. Ablations in the paper show that fixing the temperature (either too low or too high) hurts results. A low temperature (always fuzzy triangles) yields coarse surfaces, and a fixed high temperature (always sharp) makes optimization unstable. The annealed schedule strikes a balance, yielding the best geometry and rendering metrics. The net effect is that the network can use big soft strokes initially, then taper to fine brushwork—just like an artist refining a sketch. Differentiable Triangle Rendering. How does the network see that its triangles match the images? TriSplat uses a custom differentiable triangle renderer. Essentially, it takes all predicted triangles from all views, projects them into each camera, and alpha-blends them (with z-sorting or a sort of per-tile painter’s algorithm) to form an RGB image. It also produces a synthetic depth and normal map in this way. All these outputs are compared to the actual input images (or known ground-truth) using losses on color, depth, and normals. Because this renderer is differentiable, errors propagate back to update the triangle parameters and the network weights. This is similar in spirit to Gaussian splatting renderers or mesh renderers like Soft Rasterizer, but specialized for small triangles. Interestingly, each triangle’s contribution to the image is computed by what they call a “tile-based” approach, ensuring real-time performance. The bottom line is that TriSplat end-to-end learns from image comparisons just as a NeRF or Gaussian splatting network would, but its primitives are triangles instead of volumetric elements. Summary of the Output. After a forward pass, TriSplat produces per-pixel triangles in space. To form the final mesh, the authors do a quick cleanup: they discard any triangle that ended up mostly transparent (opacity below a threshold). They fix the winding order of each triangle’s vertices (ensuring consistency of normal direction). Then they merge vertices that are very close (within 1e-5 in hashed quantized space) to eliminate duplicates. The result is a watertight—or nearly watertight—triangle mesh with vertex colors. They call this the “direct mesh.” A small post-processing step (keeping only the largest connected components, etc.) yields the final “_post” mesh, which is what one would typically use in simulation. Because the triangles were rendered during training, this mesh lines up exactly with the learned radiance. In fact, TriSplat provides not only visualizations of their meshes, but even a live interactive demo: you can “drive multiple agents” on the reconstructed surface, observing through a first-person camera. The submitted code and model also include example files that you can open directly. In this way, TriSplat outputs are ready to go into any standard 3D workflow. Advantages Over Existing Methods. TriSplat’s mesh-native approach carries several practical advantages, as well as some inherent trade-offs. From the paper and experiments, the key benefits are:. No Expensive Mesh Extraction: Unlike Gaussian or implicit methods, TriSplat produces a mesh outright. The paper emphasizes that Gaussian splatting pipelines “require post-hoc mesh extraction,” whereas TriSplat “directly exports simulation-ready mesh scenes from a single forward pass”. In practice, they show TriSplat runs in ~1.3 seconds per scene while Gaussian baselines take tens to hundreds of seconds to give a usable mesh. This speedup alone makes TriSplat appealing for any application that values quick turnaround. Physics Engine Compatibility: A recurring motivation is making the output easy to feed into downstream systems. The authors point out that since TriSplat’s primitives “are themselves surface triangles,” the mesh can be loaded into physics engines or collision checkers as is. The code’s documentation confirms this: the final can be imported into Blender, Unity, NVIDIA Isaac Sim, or even PyBullet, and immediately used for simulation. This plug-and-play integration is a huge benefit for embodied AI or robotics researchers who want to simulate around real scenes. Geometry Fidelity: In quantitative evaluations on RealEstate10K and DL3DV, TriSplat’s reconstructions were measurably more accurate in geometry than Gaussian-based feed-forward baselines. For example, metrics like Chamfer distance and normal consistency favored TriSplat, especially as more views were provided. Intuitively, because TriSplat’s triangles are constrained to fit each local surface (via the anchored normals), they can capture edges and flat patches cleanly. The Gaussian approaches tended to blur details or weld surfaces together softly. Importantly, TriSplat maintains competitive novel-view synthesis quality (PSNR/LPIPS) even though its goal is primarily geometry. This means it didn’t sacrifice too much visual fidelity to get a mesh. Generalization: The network was trained on indoor datasets (RealEstate10K and DL3DV) but tested on a completely different one (ScanNet) without fine-tuning. The zero-shot generalization was quite strong: TriSplat produced reasonable meshes on ScanNet scenes. This suggests the triangle-based representation didn’t overfit to the training domain. It even means a robot could use a pre-trained TriSplat model to scan a new building and get an immediate mesh. Speed and Scalability: We mentioned the raw speed improvement, but it’s worth highlighting that TriSplat’s performance scales well as you add more input views. The authors measured reconstructions with different numbers of input images. TriSplat’s quality improved consistently and robustly with more views, while its inference time scaled linearly (still very fast). Gaussian baselines also improved with more views but never surpassed TriSplat’s geometry metrics. These advantages make TriSplat particularly suitable for tasks like: real-time environment mapping, virtual reality world capture, simulating robot interaction with recently observed scenes, or any scenario where quick mesh output from photos is needed. On the other hand, there are trade-offs and limitations to be aware of (even if not all are explicitly discussed in the paper):. Training Complexity: TriSplat’s sophisticated pipeline (transformer fusion, normal U-Net, schedules) means it’s not a trivial “one-line” model to train. The code includes multi-GPU training scripts, downloading specific pretrained weights, etc. Achieving the full performance requires careful implementation. Fixed Triangle Density: The number of triangles is tied to image resolution. In practice, TriSplat produces one triangle per pixel (before pruning). To cover very large or detailed scenes, you need high-res views. There might be inefficiencies compared to adaptive methods that concentrate geometry where needed. Dynamic Scenes: TriSplat is demonstrated on static scenes. Handling dynamic objects (moving around) would require a more complex scheme (maybe temporal extension, which is left for future work). Texture vs Geometry: While TriSplat outputs vertex-colored meshes, it doesn’t produce high-frequency textures beyond the vertex colors. If one wanted to relight or apply fancy rendering, the mesh would need additional texturing cleanup. Occlusions and Visibility: As with any multi-view method, TriSplat can only reconstruct what is seen. Hidden or thin structures might not be captured unless images come from enough angles. Despite these caveats, TriSplat’s approach is elegant and effective for the regime it targets: moderately complex indoor scenes viewed from a handful of angles. It essentially moves much of the complexity of scene representation into the network itself, yielding nice geometric output. Relation to Other Approaches. It’s helpful to contrast TriSplat with a few categories of related work:. Gaussian Splatting Networks (e.g. YoNoSplat, PreF3R, Splat-SAP, etc.): These are the most direct predecessors. They also predict primitives (Gaussians) and camera poses in a feed-forward way. For instance, Ye et al.’s YoNoSplat (ICLR 2026) similarly reconstructs Gaussians from images and handles unposed input. The big difference is that YoNoSplat’s output is still a volumetric model; one must ray-trace the Gaussians to render, and convert to mesh via TSDF. TriSplat replaces those Gaussians with triangles, eliminating the volumetric representation. When trained on the same data, TriSplat’s output is a consistent mesh whereas YoNoSplat’s output would require further processing. According to the authors, TriSplat achieves more faithful geometry than these Gaussian baselines, likely because of the normal anchoring and sharpening. Neural Implicit Methods (NeRFs): Classic NeRF is per-scene optimization, not directly comparable. However, there is work on generalizable NeRF (which uses learned priors to speed up new scenes) like pixelNeRF, but those still optimize density per scene. TriSplat is fully feed-forward and does not produce a volumetric density. In terms of output, a NeRF (or Plenoxels) typically also needs Marching Cubes to get a mesh. So TriSplat’s key novelty here is staying mesh-native, unlike anything in the NeRF camp. Mesh Prediction Networks: There is a long history of networks that predict explicit meshes (e.g. AtlasNet, Pixel2Mesh, etc) but those are usually object-level, category-specific, and start from a mesh template. They also typically assume known poses. TriSplat is markably different: it’s scene-level, pose-free, and does not start from a template mesh at all. Triangle-based Neural Rendering: As mentioned earlier, Held et al. (Triangle Splatting, 3DV 2026) also uses triangles, but as a differentiable renderer-only method (for novel view synthesis, not feed-forward reconstruction). They emphasize that triangles can achieve high FPS and fidelity superior to Gaussians in rendering tasks. TriSplat borrows the same spirit for representing geometry, but it’s trained via images to reconstruct 3D, not to optimize for a given scene. Interestingly, both lines of work converge on the idea that triangles are an efficient primitive for neural graphics. FTSplat (March 2026): An independent concurrent work (Jinlin et al.) also proposed a “Feed-forward Triangle Splatting Network.” FTSplat, unlike TriSplat, assumes calibrated cameras (known poses) and focuses on producing continuous triangle surfaces from multiple views. They introduce a pixel-aligned triangle generation and supervise with known point clouds. TriSplat’s pose-free design distinguishes it: TriSplat learns the poses internally. FTSplat emphasizes its own speed and mesh compatibility, which are similar goals. It would be interesting future work to see how the methods compare directly; for now we simply note that the idea of triangle-based feed-forward reconstruction is gaining traction across labs. Performance and Results. On standard indoor scene benchmarks, TriSplat shows strong performance. Qualitatively, their reconstructions look quite clean: walls are planar, corners are sharp, and furniture has coherent surfaces. Quantitatively, the authors report metrics like Chamfer distance and normal consistency. Key observations include:. Geometry Accuracy: TriSplat achieves lower Chamfer error (compared to ground truth mesh) than its Gaussian-splatting baselines across view counts. Even with only two views, TriSplat outperforms; with more views (4 or 6), the gap widens. In fact, TriSplat “obtains best surface geometry metrics” at all tested view counts. This suggests its explicit normal anchoring and training regime really pay off in terms of surface alignment. Visual Quality: Measured by metrics like PSNR or LPIPS on novel-view synthesis, TriSplat remains competitive. It may not always top the Gaussian methods in pure pixel error (since the Gaussians are optimized for rendering), but it is very close. The difference is small enough that for applications focused on geometry, the sacrifice is worth it. Runtime: The head-to-head comparison is compelling. Gaussian-splatting pipelines generally need to refine a 3D field or, if fed directly from a network, still require heavy mesh extraction. TriSplat’s feed-forward runs in ~1.3 s on a high-end GPU. By contrast, typical Gaussian approaches (if they produce a mesh) can take tens of seconds to extract and optimize even on the same hardware. The paper includes a plot showing that TriSplat’s mesh export time is negligible compared to alternatives. Generalization: Perhaps most impressively, a single TriSplat model trained on RealEstate10K (a largely synthetic-looking indoor dataset) generalizes to real ScanNet scenes. The output mesh quality on ScanNet is quite reasonable given no fine-tuning. This zero-shot ability confirms that the approach is not overfitting to one data distribution. The combination of high geometry fidelity, simulation compatibility, and speed clearly distinguishes TriSplat from prior feed-forward schemes. It essentially delivers the best-case scenario for scene reconstruction: accurate mesh ready for immediate use. Potential Applications. The designers of TriSplat explicitly target applications in robotics and embodied AI, and the method has several attributes that make it attractive in those fields:. Simulation and Digital Twins: A robot often needs a model of its environment for tasks like planning, exploration, and control. With TriSplat, one could quickly scan a few photos and get a high-quality mesh in seconds. This mesh can then be imported into simulators (Isaac, Gazebo, Unity) to simulate the robot’s movements or interactions before trying them in the real world. In a sense, TriSplat automates the creation of a “digital twin” of a scene from camera data. Reinforcement Learning in Simulation: Rapidly generating new simulated environments is valuable for RL, especially in vision-based tasks. TriSplat could allow researchers to scan real indoor spaces and instantly produce simulated variants (with textures, geometry, collisions) to train agents. Since the output is a static triangle mesh, one could randomize textures or lighting on top of it, then run RL in those virtual worlds. Augmented Reality (AR) and VR: For AR, you often need a mesh of the real world to do occlusion and interaction. TriSplat’s run-time speed hints at possible on-device or edge scenarios: a tablet or AR headset could capture a room and within a second or two have a coarse mesh to use as the basis for placing virtual objects. (Though, in practice, TriSplat currently needs a GPU, but hardware is always improving.). Indoor Mapping for Analytics: Beyond robotics, any field that uses indoor mapping (architecture, interior design, real estate) could use TriSplat for quick 3D scans. It could complement LIDAR scans or deliver a purely vision-based scan. The fact that TriSplat is trained on datasets like RealEstate10K suggests it’s already geared for houses and apartments, making it relevant to real-estate 3D modeling. Real-time Visual Navigation: While not directly outputting a topological map, TriSplat could serve as a forward model: imagine a mobile robot that takes occasional snapshots as it moves. TriSplat could incrementally fuse these into a cumulative mesh of the environment, helping with navigation or obstacle avoidance. Because it’s feed-forward, running TriSplat repeatedly on adjacent frames might even be feasible for near-real-time updates (especially if the model is further optimized). Robotics Benchmarks: In research, TriSplat itself could become a baseline for scene reconstruction. Robotics simulators could incorporate TriSplat as a data generator. Also, since the output is differentiable to the input images, it could be used in hybrid training loops (for example, adversarial or self-supervised setups where an agent learns to adjust the scene). The paper’s interactive demo (driving agents on the mesh) showcases exactly this pipeline: capture images, reconstruct, simulate. The ease of integration—no special format conversion—is a practical win. Conclusion. TriSplat introduces a clever twist to the growing field of feed-forward 3D reconstruction: use triangles instead of Gaussians or voxels. This seemingly simple idea has big payoffs. By designing the network to predict geometry in the form of oriented triangles, the model natively outputs a mesh that is automatically ready for downstream use. There’s no need for an external mesh-fusion step that could bottleneck or corrupt the results. The technical integration of feature transformers, geometry-based normals, and curriculum learning culminates in a system that is both fast and accurate. On challenging datasets, TriSplat outperforms previous feed-forward methods in geometric accuracy, while producing competitive renderings. And the result is a tangible mesh file, suitable for physics simulation and rendering. For any researcher or developer needing quick 3D models from images – especially if those models need to be “useful” to a simulator – TriSplat offers an appealing new tool. The authors have released code and pretrained models, making it accessible to try out on custom scenes. As feed-forward methods mature, having mesh-in-hand instantly is a significant step. TriSplat’s triangle-based philosophy bridges the gap between neural implicit reconstructions and the tried-and-true mesh world, enabling a new class of vision-to-simulation applications. Sources: TriSplat’s own project page and arXiv details describe the method and results. The authors emphasize that Gaussian-based reconstructions require costly post-processing while TriSplat’s direct triangular output “can be directly ingested by physics engines”. They show it runs an order of magnitude faster than Gaussian pipelines and produces more faithful geometric detail. TriSplat’s code even notes that the final mesh can be loaded into Unity, PyBullet, NVIDIA Isaac Sim, etc. – a clear sign of its simulation-ready nature. Overall, TriSplat represents a compelling advance for real-time, robot-ready 3D scene capture.