ActiveNeRF: Learning where to See with Uncertainty Estimation

TL;DR

ActiveNeRF introduces an active learning framework for NeRF that uses uncertainty estimation to select the most informative samples, significantly improving 3D scene reconstruction with limited data.

Contribution

This paper presents a novel ActiveNeRF framework that integrates uncertainty estimation and active learning to enhance NeRF performance under limited training data.

Findings

Improves NeRF quality with fewer training images.

Uncertainty-based sample selection enhances scene reconstruction.

Validated on both synthetic and real scenes.

Abstract

Recently, Neural Radiance Fields (NeRF) has shown promising performances on reconstructing 3D scenes and synthesizing novel views from a sparse set of 2D images. Albeit effective, the performance of NeRF is highly influenced by the quality of training samples. With limited posed images from the scene, NeRF fails to generalize well to novel views and may collapse to trivial solutions in unobserved regions. This makes NeRF impractical under resource-constrained scenarios. In this paper, we present a novel learning framework, ActiveNeRF, aiming to model a 3D scene with a constrained input budget. Specifically, we first incorporate uncertainty estimation into a NeRF model, which ensures robustness under few observations and provides an interpretation of how NeRF understands the scene. On this basis, we propose to supplement the existing training set with newly captured samples based on an active learning scheme. By evaluating the reduction of uncertainty given new inputs, we select the samples that bring the most information gain. In this way, the quality of novel view synthesis can be improved with minimal additional resources. Extensive experiments validate the performance of our model on both realistic and synthetic scenes, especially with scarcer training data. Code will be released at \url{https://github.com/LeapLabTHU/ActiveNeRF}.