MVPGS: Excavating Multi-view Priors for Gaussian Splatting from Sparse Input Views

TL;DR

MVPGS is a novel few-shot neural view synthesis method that combines multi-view priors with Gaussian Splatting, leveraging multi-view stereo and geometric constraints to achieve real-time, high-quality rendering from sparse inputs.

Contribution

It introduces a multi-view prior excavation framework for Gaussian Splatting, incorporating geometric initialization, appearance constraints, and view-consistent geometry to improve few-shot novel view synthesis.

Findings

Achieves state-of-the-art results in few-shot NVS.

Enables real-time rendering with high quality.

Effectively mitigates overfitting with geometric and appearance constraints.

Abstract

Recently, the Neural Radiance Field (NeRF) advancement has facilitated few-shot Novel View Synthesis (NVS), which is a significant challenge in 3D vision applications. Despite numerous attempts to reduce the dense input requirement in NeRF, it still suffers from time-consumed training and rendering processes. More recently, 3D Gaussian Splatting (3DGS) achieves real-time high-quality rendering with an explicit point-based representation. However, similar to NeRF, it tends to overfit the train views for lack of constraints. In this paper, we propose \textbf{MVPGS}, a few-shot NVS method that excavates the multi-view priors based on 3D Gaussian Splatting. We leverage the recent learning-based Multi-view Stereo (MVS) to enhance the quality of geometric initialization for 3DGS. To mitigate overfitting, we propose a forward-warping method for additional appearance constraints conforming to scenes based on the computed geometry. Furthermore, we introduce a view-consistent geometry constraint for Gaussian parameters to facilitate proper optimization convergence and utilize a monocular depth regularization as compensation. Experiments show that the proposed method achieves state-of-the-art performance with real-time rendering speed. Project page: https://zezeaaa.github.io/projects/MVPGS/