Does 3D Gaussian Splatting Need Accurate Volumetric Rendering?

TL;DR

This paper analyzes whether replacing 3D Gaussian Splatting's approximations with accurate volumetric rendering improves 3D scene reconstruction, finding that current approximations are sufficient for high-quality results due to optimization efficiency.

Contribution

The paper provides an in-depth analysis of 3D Gaussian Splatting's approximations and demonstrates that its current approach outperforms more accurate volumetric rendering methods.

Findings

More accurate volumetric rendering benefits low primitive counts.

3DGS outperforms volumetric rendering despite approximations.

Efficient optimization enables high-quality 3D reconstructions.

Abstract

Since its introduction, 3D Gaussian Splatting (3DGS) has become an important reference method for learning 3D representations of a captured scene, allowing real-time novel-view synthesis with high visual quality and fast training times. Neural Radiance Fields (NeRFs), which preceded 3DGS, are based on a principled ray-marching approach for volumetric rendering. In contrast, while sharing a similar image formation model with NeRF, 3DGS uses a hybrid rendering solution that builds on the strengths of volume rendering and primitive rasterization. A crucial benefit of 3DGS is its performance, achieved through a set of approximations, in many cases with respect to volumetric rendering theory. A naturally arising question is whether replacing these approximations with more principled volumetric rendering solutions can improve the quality of 3DGS. In this paper, we present an in-depth analysis of the various approximations and assumptions used by the original 3DGS solution. We demonstrate that, while more accurate volumetric rendering can help for low numbers of primitives, the power of efficient optimization and the large number of Gaussians allows 3DGS to outperform volumetric rendering despite its approximations.