SmileSplat: Generalizable Gaussian Splats for Unconstrained Sparse Images

TL;DR

SmileSplat introduces a novel generalizable Gaussian Splatting method for reconstructing radiance fields from unconstrained sparse multi-view images, enabling high-quality novel view synthesis without requiring ground-truth camera parameters.

Contribution

The paper proposes SmileSplat, a new Gaussian Splatting approach that predicts pixel-aligned surfels with enhanced multi-view consistency and optimizes camera parameters for improved 3D scene reconstruction.

Findings

Achieves state-of-the-art results in novel view rendering.

Demonstrates high-quality depth map prediction.

Effective on diverse public datasets.

Abstract

Sparse Multi-view Images can be Learned to predict explicit radiance fields via Generalizable Gaussian Splatting approaches, which can achieve wider application prospects in real-life when ground-truth camera parameters are not required as inputs. In this paper, a novel generalizable Gaussian Splatting method, SmileSplat, is proposed to reconstruct pixel-aligned Gaussian surfels for diverse scenarios only requiring unconstrained sparse multi-view images. First, Gaussian surfels are predicted based on the multi-head Gaussian regression decoder, which can are represented with less degree-of-freedom but have better multi-view consistency. Furthermore, the normal vectors of Gaussian surfel are enhanced based on high-quality of normal priors. Second, the Gaussians and camera parameters (both extrinsic and intrinsic) are optimized to obtain high-quality Gaussian radiance fields for novel view synthesis tasks based on the proposed Bundle-Adjusting Gaussian Splatting module. Extensive experiments on novel view rendering and depth map prediction tasks are conducted on public datasets, demonstrating that the proposed method achieves state-of-the-art performance in various 3D vision tasks. More information can be found on our project page (https://yanyan-li.github.io/project/gs/smilesplat)