3D Gaussian Splatting with Normal Information for Mesh Extraction and Improved Rendering

TL;DR

This paper introduces a regularization technique using normal information derived from a signed distance function to enhance 3D Gaussian splatting, leading to improved rendering quality and more accurate mesh extraction from complex scenes.

Contribution

It proposes a novel normal supervision regularization method that improves both rendering quality and mesh reconstruction in differentiable 3D Gaussian splatting.

Findings

Higher photorealism scores on benchmark datasets

Improved mesh accuracy and quality

Effective in complex scene reconstruction

Abstract

Differentiable 3D Gaussian splatting has emerged as an efficient and flexible rendering technique for representing complex scenes from a collection of 2D views and enabling high-quality real-time novel-view synthesis. However, its reliance on photometric losses can lead to imprecisely reconstructed geometry and extracted meshes, especially in regions with high curvature or fine detail. We propose a novel regularization method using the gradients of a signed distance function estimated from the Gaussians, to improve the quality of rendering while also extracting a surface mesh. The regularizing normal supervision facilitates better rendering and mesh reconstruction, which is crucial for downstream applications in video generation, animation, AR-VR and gaming. We demonstrate the effectiveness of our approach on datasets such as Mip-NeRF360, Tanks and Temples, and Deep-Blending. Our method scores higher on photorealism metrics compared to other mesh extracting rendering methods without compromising mesh quality.