Sparfels: Fast Reconstruction from Sparse Unposed Imagery

TL;DR

Sparfels introduces a fast, efficient method for reconstructing 3D shapes from sparse, unposed images using a novel splatting approach and a 3D foundation model, achieving state-of-the-art results in under 3 minutes.

Contribution

The paper presents a novel, simple pipeline leveraging a 3D foundation model and a new splatted color variance formulation for rapid sparse view shape reconstruction.

Findings

Achieves reconstruction in under 3 minutes on a consumer GPU.

Outperforms existing methods in sparse uncalibrated settings.

Demonstrates state-of-the-art results on multi-view datasets.

Abstract

We present a method for Sparse view reconstruction with surface element splatting that runs within 3 minutes on a consumer grade GPU. While few methods address sparse radiance field learning from noisy or unposed sparse cameras, shape recovery remains relatively underexplored in this setting. Several radiance and shape learning test-time optimization methods address the sparse posed setting by learning data priors or using combinations of external monocular geometry priors. Differently, we propose an efficient and simple pipeline harnessing a single recent 3D foundation model. We leverage its various task heads, notably point maps and camera initializations to instantiate a bundle adjusting 2D Gaussian Splatting (2DGS) model, and image correspondences to guide camera optimization midst 2DGS training. Key to our contribution is a novel formulation of splatted color variance along rays, which can be computed efficiently. Reducing this moment in training leads to more accurate shape reconstructions. We demonstrate state-of-the-art performances in the sparse uncalibrated setting in reconstruction and novel view benchmarks based on established multi-view datasets.