SO-NeRF: Active View Planning for NeRF using Surrogate Objectives

TL;DR

This paper introduces SOAR, a set of interpretable surrogate objectives for active view planning in NeRF, enabling fast and effective view selection to improve reconstruction quality without extensive prior visits.

Contribution

It proposes SOAR, a novel framework with learned surrogate scores for efficient view planning in NeRF, generalizable across different NeRF models.

Findings

SOARNet achieves ~80x speed-up over baselines.

SOAR outperforms baselines in reconstruction quality.

It generalizes across neural-implicit and explicit approaches.

Abstract

Despite the great success of Neural Radiance Fields (NeRF), its data-gathering process remains vague with only a general rule of thumb of sampling as densely as possible. The lack of understanding of what actually constitutes good views for NeRF makes it difficult to actively plan a sequence of views that yield the maximal reconstruction quality. We propose Surrogate Objectives for Active Radiance Fields (SOAR), which is a set of interpretable functions that evaluates the goodness of views using geometric and photometric visual cues - surface coverage, geometric complexity, textural complexity, and ray diversity. Moreover, by learning to infer the SOAR scores from a deep network, SOARNet, we are able to effectively select views in mere seconds instead of hours, without the need for prior visits to all the candidate views or training any radiance field during such planning. Our experiments show SOARNet outperforms the baselines with $\sim$80x speed-up while achieving better or comparable reconstruction qualities. We finally show that SOAR is model-agnostic, thus it generalizes across fully neural-implicit to fully explicit approaches.