ZeroRF: Fast Sparse View 360{\deg} Reconstruction with Zero Pretraining

TL;DR

ZeroRF introduces a fast, pretraining-free method for 360-degree scene reconstruction from sparse views by integrating a deep image prior into a neural radiance field, outperforming existing approaches in quality and speed.

Contribution

ZeroRF is the first to combine a deep image prior with a factorized NeRF for efficient, pretraining-free sparse view 360{ extdegree} reconstruction.

Findings

ZeroRF achieves state-of-the-art reconstruction quality.

ZeroRF significantly reduces computational time.

ZeroRF demonstrates versatility across diverse datasets.

Abstract

We present ZeroRF, a novel per-scene optimization method addressing the challenge of sparse view 360{\deg} reconstruction in neural field representations. Current breakthroughs like Neural Radiance Fields (NeRF) have demonstrated high-fidelity image synthesis but struggle with sparse input views. Existing methods, such as Generalizable NeRFs and per-scene optimization approaches, face limitations in data dependency, computational cost, and generalization across diverse scenarios. To overcome these challenges, we propose ZeroRF, whose key idea is to integrate a tailored Deep Image Prior into a factorized NeRF representation. Unlike traditional methods, ZeroRF parametrizes feature grids with a neural network generator, enabling efficient sparse view 360{\deg} reconstruction without any pretraining or additional regularization. Extensive experiments showcase ZeroRF's versatility and superiority in terms of both quality and speed, achieving state-of-the-art results on benchmark datasets. ZeroRF's significance extends to applications in 3D content generation and editing. Project page: https://sarahweiii.github.io/zerorf/