ResGS: Residual Densification of 3D Gaussian for Efficient Detail Recovery

TL;DR

ResGS introduces a residual densification technique for 3D Gaussian Splatting that adaptively enhances detail recovery and geometry coverage, leading to state-of-the-art view synthesis quality.

Contribution

The paper proposes a novel residual split densification operation and a progressive pipeline to improve 3D-GS detail capture and geometry completeness.

Findings

Achieves state-of-the-art rendering quality.

Improves detail recovery in 3D-GS.

Enhances various 3D-GS variants with residual split.

Abstract

Recently, 3D Gaussian Splatting (3D-GS) has prevailed in novel view synthesis, achieving high fidelity and efficiency. However, it often struggles to capture rich details and complete geometry. Our analysis reveals that the 3D-GS densification operation lacks adaptiveness and faces a dilemma between geometry coverage and detail recovery. To address this, we introduce a novel densification operation, residual split, which adds a downscaled Gaussian as a residual. Our approach is capable of adaptively retrieving details and complementing missing geometry. To further support this method, we propose a pipeline named ResGS. Specifically, we integrate a Gaussian image pyramid for progressive supervision and implement a selection scheme that prioritizes the densification of coarse Gaussians over time. Extensive experiments demonstrate that our method achieves SOTA rendering quality. Consistent performance improvements can be achieved by applying our residual split on various 3D-GS variants, underscoring its versatility and potential for broader application in 3D-GS-based applications.