RegGS: Unposed Sparse Views Gaussian Splatting with 3DGS Registration

TL;DR

RegGS introduces a registration-based framework for reconstructing 3D scenes from unposed sparse images by aligning local Gaussians with a novel optimal transport-based method, improving pose estimation and scene quality.

Contribution

It presents a novel registration framework using optimal transport for aligning Gaussian representations in 3DGS, enabling effective reconstruction from unposed sparse views.

Findings

Accurately estimates camera poses in sparse view scenarios.

Achieves high-quality novel-view synthesis.

Demonstrates superior registration fidelity on benchmark datasets.

Abstract

3D Gaussian Splatting (3DGS) has demonstrated its potential in reconstructing scenes from unposed images. However, optimization-based 3DGS methods struggle with sparse views due to limited prior knowledge. Meanwhile, feed-forward Gaussian approaches are constrained by input formats, making it challenging to incorporate more input views. To address these challenges, we propose RegGS, a 3D Gaussian registration-based framework for reconstructing unposed sparse views. RegGS aligns local 3D Gaussians generated by a feed-forward network into a globally consistent 3D Gaussian representation. Technically, we implement an entropy-regularized Sinkhorn algorithm to efficiently solve the optimal transport Mixture 2-Wasserstein $(\text{MW}_2)$ distance, which serves as an alignment metric for Gaussian mixture models (GMMs) in $\mathrm{Sim}(3)$ space. Furthermore, we design a joint 3DGS registration module that integrates the $\text{MW}_2$ distance, photometric consistency, and depth geometry. This enables a coarse-to-fine registration process while accurately estimating camera poses and aligning the scene. Experiments on the RE10K and ACID datasets demonstrate that RegGS effectively registers local Gaussians with high fidelity, achieving precise pose estimation and high-quality novel-view synthesis. Project page: https://3dagentworld.github.io/reggs/.