CoR-GS: Sparse-View 3D Gaussian Splatting via Co-Regularization

TL;DR

This paper introduces CoR-GS, a co-regularization method for sparse-view 3D Gaussian Splatting that suppresses inaccuracies by analyzing disagreements between dual radiance fields, leading to improved scene reconstruction and rendering quality.

Contribution

It proposes a novel co-regularization framework that identifies and suppresses inaccuracies in sparse-view 3D Gaussian Splatting using disagreement metrics between two radiance fields.

Findings

Achieves state-of-the-art results on multiple datasets.

Effectively regularizes scene geometry with sparse views.

Improves reconstruction accuracy and rendering quality.

Abstract

3D Gaussian Splatting (3DGS) creates a radiance field consisting of 3D Gaussians to represent a scene. With sparse training views, 3DGS easily suffers from overfitting, negatively impacting rendering. This paper introduces a new co-regularization perspective for improving sparse-view 3DGS. When training two 3D Gaussian radiance fields, we observe that the two radiance fields exhibit point disagreement and rendering disagreement that can unsupervisedly predict reconstruction quality, stemming from the randomness of densification implementation. We further quantify the two disagreements and demonstrate the negative correlation between them and accurate reconstruction, which allows us to identify inaccurate reconstruction without accessing ground-truth information. Based on the study, we propose CoR-GS, which identifies and suppresses inaccurate reconstruction based on the two disagreements: (1) Co-pruning considers Gaussians that exhibit high point disagreement in inaccurate positions and prunes them. (2) Pseudo-view co-regularization considers pixels that exhibit high rendering disagreement are inaccurate and suppress the disagreement. Results on LLFF, Mip-NeRF360, DTU, and Blender demonstrate that CoR-GS effectively regularizes the scene geometry, reconstructs the compact representations, and achieves state-of-the-art novel view synthesis quality under sparse training views.