2D Triangle Splatting for Direct Differentiable Mesh Training

TL;DR

The paper introduces 2D Triangle Splatting (2DTS), a novel differentiable rendering method replacing Gaussian primitives with triangle primitives to produce mesh-like reconstructions from multi-view images.

Contribution

It presents a new triangle-based representation that maintains differentiability, produces opaque mesh faces without post-processing, and bridges differentiable rendering with traditional mesh-based methods.

Findings

Achieves competitive visual quality with Gaussian-based methods.

Produces fully opaque triangle meshes at the end of training.

Provides a direct link to mesh-based rendering applications.

Abstract

Differentiable rendering with 3D Gaussian primitives has emerged as a powerful method for reconstructing high-fidelity 3D scenes from multi-view images. While it offers improvements over NeRF-based methods, this representation still encounters challenges with rendering speed and advanced rendering effects, such as relighting and shadow rendering, compared to mesh-based models. In this paper, we propose 2D Triangle Splatting (2DTS), a novel method that replaces 3D Gaussian primitives with 2D triangle primitives. This representation naturally forms a discrete mesh-like structure while retaining the benefits of continuous volumetric modeling. Through the incorporation and controlled annealing of a compactness parameter, our method maintains differentiability during training while producing triangle meshes with fully opaque faces at the end of optimization without the need for additional post-processing. Experimental results demonstrate that our triangle-based representation achieves competitive visual quality with Gaussian-based methods while providing a more direct bridge to mesh-based representations. Our method bridges the gap between differentiable rendering and traditional mesh-based rendering, offering a promising solution for applications requiring renderable mesh-like reconstructions. Please visit our project page at https://gaoderender.github.io/triangle-splatting.