SPARF: Neural Radiance Fields from Sparse and Noisy Poses

TL;DR

SPARF enables high-quality novel view synthesis from only a few wide-baseline images with noisy camera poses by jointly refining scene and camera parameters using multi-view geometry constraints.

Contribution

The paper introduces SPARF, a method that jointly optimizes neural radiance fields and camera poses from sparse, noisy inputs, advancing the sparse-view synthesis capability.

Findings

Sets new state-of-the-art in sparse-view novel view synthesis

Effective refinement of camera poses from minimal and noisy data

Achieves photorealistic rendering with as few as 3 input images

Abstract

Neural Radiance Field (NeRF) has recently emerged as a powerful representation to synthesize photorealistic novel views. While showing impressive performance, it relies on the availability of dense input views with highly accurate camera poses, thus limiting its application in real-world scenarios. In this work, we introduce Sparse Pose Adjusting Radiance Field (SPARF), to address the challenge of novel-view synthesis given only few wide-baseline input images (as low as 3) with noisy camera poses. Our approach exploits multi-view geometry constraints in order to jointly learn the NeRF and refine the camera poses. By relying on pixel matches extracted between the input views, our multi-view correspondence objective enforces the optimized scene and camera poses to converge to a global and geometrically accurate solution. Our depth consistency loss further encourages the reconstructed scene to be consistent from any viewpoint. Our approach sets a new state of the art in the sparse-view regime on multiple challenging datasets.