Gaussian Mesh Renderer for Lightweight Differentiable Rendering

TL;DR

The paper introduces Gaussian Mesh Renderer (GMR), a lightweight differentiable renderer that combines Gaussian and mesh representations to improve gradient smoothness and optimization efficiency in 3D surface rendering.

Contribution

It presents a novel GMR method that integrates Gaussian primitives with mesh models, enabling faster, smoother, and more memory-efficient differentiable rendering.

Findings

Achieves smoother gradients compared to traditional mesh renderers.

Enables effective optimization with smaller batch sizes.

Provides a lightweight and efficient rendering pipeline.

Abstract

3D Gaussian Splatting (3DGS) has enabled high-fidelity virtualization with fast rendering and optimization for novel view synthesis. On the other hand, triangle mesh models still remain a popular choice for surface reconstruction but suffer from slow or heavy optimization in traditional mesh-based differentiable renderers. To address this problem, we propose a new lightweight differentiable mesh renderer leveraging the efficient rasterization process of 3DGS, named Gaussian Mesh Renderer (GMR), which tightly integrates the Gaussian and mesh representations. Each Gaussian primitive is analytically derived from the corresponding mesh triangle, preserving structural fidelity and enabling the gradient flow. Compared to the traditional mesh renderers, our method achieves smoother gradients, which especially contributes to better optimization using smaller batch sizes with limited memory. Our implementation is available in the public GitHub repository at https://github.com/huntorochi/Gaussian-Mesh-Renderer.