FLARE: Feed-forward Geometry, Appearance and Camera Estimation from Uncalibrated Sparse Views

TL;DR

FLARE is a fast, feed-forward model that accurately estimates camera poses and 3D geometry from a small number of uncalibrated images, enabling practical applications in real-world scenarios.

Contribution

It introduces a cascaded learning framework that jointly estimates camera poses and 3D structure from sparse views, achieving state-of-the-art results efficiently.

Findings

State-of-the-art accuracy in pose estimation and 3D reconstruction

Inference time less than 0.5 seconds

Effective with as few as 2-8 input images

Abstract

We present FLARE, a feed-forward model designed to infer high-quality camera poses and 3D geometry from uncalibrated sparse-view images (i.e., as few as 2-8 inputs), which is a challenging yet practical setting in real-world applications. Our solution features a cascaded learning paradigm with camera pose serving as the critical bridge, recognizing its essential role in mapping 3D structures onto 2D image planes. Concretely, FLARE starts with camera pose estimation, whose results condition the subsequent learning of geometric structure and appearance, optimized through the objectives of geometry reconstruction and novel-view synthesis. Utilizing large-scale public datasets for training, our method delivers state-of-the-art performance in the tasks of pose estimation, geometry reconstruction, and novel view synthesis, while maintaining the inference efficiency (i.e., less than 0.5 seconds). The project page and code can be found at: https://zhanghe3z.github.io/FLARE/