SparsePose: Sparse-View Camera Pose Regression and Refinement

TL;DR

SparsePose is a novel method that accurately estimates camera poses from fewer than 10 wide-baseline images, enabling high-quality 3D reconstruction with minimal views.

Contribution

It introduces a learning-based approach for initial pose regression and iterative refinement tailored for sparse-view scenarios, outperforming existing methods.

Findings

Outperforms conventional and learning-based baselines in pose accuracy

Enables high-fidelity 3D reconstruction with only 5-9 images

Demonstrates effectiveness on large-scale object dataset Co3D

Abstract

Camera pose estimation is a key step in standard 3D reconstruction pipelines that operate on a dense set of images of a single object or scene. However, methods for pose estimation often fail when only a few images are available because they rely on the ability to robustly identify and match visual features between image pairs. While these methods can work robustly with dense camera views, capturing a large set of images can be time-consuming or impractical. We propose SparsePose for recovering accurate camera poses given a sparse set of wide-baseline images (fewer than 10). The method learns to regress initial camera poses and then iteratively refine them after training on a large-scale dataset of objects (Co3D: Common Objects in 3D). SparsePose significantly outperforms conventional and learning-based baselines in recovering accurate camera rotations and translations. We also demonstrate our pipeline for high-fidelity 3D reconstruction using only 5-9 images of an object.