The Role and Relationship of Initialization and Densification in 3D Gaussian Splatting

TL;DR

This paper investigates how initialization and densification influence 3D Gaussian Splatting, revealing current densification methods' limitations in leveraging dense initializations for scene reconstruction.

Contribution

It introduces a benchmark to systematically evaluate various initialization and densification combinations in 3D Gaussian Splatting.

Findings

Current densification methods do not fully utilize dense initializations.

Dense initializations often do not significantly outperform sparse SfM initializations.

The benchmark will be publicly available for further research.

Abstract

3D Gaussian Splatting (3DGS) has become the method of choice for photo-realistic 3D reconstruction of scenes, due to being able to efficiently and accurately recover the scene appearance and geometry from images. 3DGS represents the scene through a set of 3D Gaussians, parameterized by their position, spatial extent, and view-dependent color. Starting from an initial point cloud, 3DGS refines the Gaussians' parameters as to reconstruct a set of training images as accurately as possible. Typically, a sparse Structure-from-Motion point cloud is used as initialization. In order to obtain dense Gaussian clouds, 3DGS methods thus rely on a densification stage. In this paper, we systematically study the relation between densification and initialization. Proposing a new benchmark, we study combinations of different types of initializations (dense laser scans, dense (multi-view) stereo point clouds, dense monocular depth estimates, sparse SfM point clouds) and different densification schemes. We show that current densification approaches are not able to take full advantage of dense initialization as they are often unable to (significantly) improve over sparse SfM-based initialization. We will make our benchmark publicly available.