No Pose at All: Self-Supervised Pose-Free 3D Gaussian Splatting from Sparse Views

TL;DR

SPFSplat is a novel framework that enables 3D Gaussian splatting and view synthesis from sparse multi-view images without requiring camera pose supervision, achieving state-of-the-art results.

Contribution

It introduces a pose-free training method for 3D Gaussian splatting that jointly predicts primitives and camera poses in a single feed-forward pass.

Findings

Achieves state-of-the-art novel view synthesis without pose supervision.

Outperforms recent pose estimation methods trained with geometry priors.

Effective even with limited image overlap and significant viewpoint changes.

Abstract

We introduce SPFSplat, an efficient framework for 3D Gaussian splatting from sparse multi-view images, requiring no ground-truth poses during training or inference. It employs a shared feature extraction backbone, enabling simultaneous prediction of 3D Gaussian primitives and camera poses in a canonical space from unposed inputs within a single feed-forward step. Alongside the rendering loss based on estimated novel-view poses, a reprojection loss is integrated to enforce the learning of pixel-aligned Gaussian primitives for enhanced geometric constraints. This pose-free training paradigm and efficient one-step feed-forward design make SPFSplat well-suited for practical applications. Remarkably, despite the absence of pose supervision, SPFSplat achieves state-of-the-art performance in novel view synthesis even under significant viewpoint changes and limited image overlap. It also surpasses recent methods trained with geometry priors in relative pose estimation. Code and trained models are available on our project page: https://ranrhuang.github.io/spfsplat/.