LoopSparseGS: Loop Based Sparse-View Friendly Gaussian Splatting

TL;DR

LoopSparseGS introduces a loop-based framework that enhances sparse-view 3D Gaussian splatting for photorealistic novel view synthesis by iteratively densifying points and leveraging depth regularization.

Contribution

It proposes a novel loop-based approach with progressive initialization, depth regularization, and sparse-friendly sampling to improve sparse-view 3D Gaussian splatting performance.

Findings

Outperforms state-of-the-art methods on four datasets

Effective in indoor, outdoor, and object-level scenes

Handles various image resolutions successfully

Abstract

Despite the photorealistic novel view synthesis (NVS) performance achieved by the original 3D Gaussian splatting (3DGS), its rendering quality significantly degrades with sparse input views. This performance drop is mainly caused by the limited number of initial points generated from the sparse input, insufficient supervision during the training process, and inadequate regularization of the oversized Gaussian ellipsoids. To handle these issues, we propose the LoopSparseGS, a loop-based 3DGS framework for the sparse novel view synthesis task. In specific, we propose a loop-based Progressive Gaussian Initialization (PGI) strategy that could iteratively densify the initialized point cloud using the rendered pseudo images during the training process. Then, the sparse and reliable depth from the Structure from Motion, and the window-based dense monocular depth are leveraged to provide precise geometric supervision via the proposed Depth-alignment Regularization (DAR). Additionally, we introduce a novel Sparse-friendly Sampling (SFS) strategy to handle oversized Gaussian ellipsoids leading to large pixel errors. Comprehensive experiments on four datasets demonstrate that LoopSparseGS outperforms existing state-of-the-art methods for sparse-input novel view synthesis, across indoor, outdoor, and object-level scenes with various image resolutions.