Binocular-Guided 3D Gaussian Splatting with View Consistency for Sparse View Synthesis

TL;DR

This paper introduces a novel binocular-guided 3D Gaussian Splatting method for sparse view synthesis that leverages stereo view consistency and Gaussian opacity constraints, eliminating the need for external neural priors and achieving superior results.

Contribution

The method utilizes inherent stereo self-supervision and Gaussian opacity constraints to improve 3D Gaussian inference without external priors, outperforming existing approaches.

Findings

Significantly outperforms state-of-the-art methods on LLFF, DTU, and Blender datasets.

Effectively leverages stereo view consistency for self-supervision.

Improves robustness and efficiency in 3D Gaussian inference from sparse views.

Abstract

Novel view synthesis from sparse inputs is a vital yet challenging task in 3D computer vision. Previous methods explore 3D Gaussian Splatting with neural priors (e.g. depth priors) as an additional supervision, demonstrating promising quality and efficiency compared to the NeRF based methods. However, the neural priors from 2D pretrained models are often noisy and blurry, which struggle to precisely guide the learning of radiance fields. In this paper, We propose a novel method for synthesizing novel views from sparse views with Gaussian Splatting that does not require external prior as supervision. Our key idea lies in exploring the self-supervisions inherent in the binocular stereo consistency between each pair of binocular images constructed with disparity-guided image warping. To this end, we additionally introduce a Gaussian opacity constraint which regularizes the Gaussian locations and avoids Gaussian redundancy for improving the robustness and efficiency of inferring 3D Gaussians from sparse views. Extensive experiments on the LLFF, DTU, and Blender datasets demonstrate that our method significantly outperforms the state-of-the-art methods.