Pre-NeRF 360: Enriching Unbounded Appearances for Neural Radiance Fields

TL;DR

Pre-NeRF 360 introduces a novel framework that significantly enhances neural radiance fields for open scenes, enabling better handling of multiple inputs, pose ambiguity, and large-scale environments, with improved results over previous methods.

Contribution

The paper presents Pre-NeRF 360, a new framework that addresses key limitations of NeRF in unbounded scenes, including multi-input handling and pose ambiguity, and introduces an updated dataset for evaluation.

Findings

Superior rendering quality in open scenes compared to prior NeRF models.

Effective handling of multiple video inputs and ambiguous poses.

Enhanced dataset (N5k360) for large-scale scene reconstruction.

Abstract

Neural radiance fields (NeRF) appeared recently as a powerful tool to generate realistic views of objects and confined areas. Still, they face serious challenges with open scenes, where the camera has unrestricted movement and content can appear at any distance. In such scenarios, current NeRF-inspired models frequently yield hazy or pixelated outputs, suffer slow training times, and might display irregularities, because of the challenging task of reconstructing an extensive scene from a limited number of images. We propose a new framework to boost the performance of NeRF-based architectures yielding significantly superior outcomes compared to the prior work. Our solution overcomes several obstacles that plagued earlier versions of NeRF, including handling multiple video inputs, selecting keyframes, and extracting poses from real-world frames that are ambiguous and symmetrical. Furthermore, we applied our framework, dubbed as "Pre-NeRF 360", to enable the use of the Nutrition5k dataset in NeRF and introduce an updated version of this dataset, known as the N5k360 dataset.