NeRF On-the-go: Exploiting Uncertainty for Distractor-free NeRFs in the Wild

TL;DR

NeRF On-the-go introduces an uncertainty-aware method that effectively removes distractors and enhances view synthesis in complex, real-world scenes from casual image sequences, with faster convergence and improved quality.

Contribution

The paper presents a novel NeRF approach that leverages uncertainty to handle distractors and occlusions in dynamic environments, improving robustness and efficiency.

Findings

Significant improvement over state-of-the-art NeRF methods.

Effective elimination of distractors in complex scenes.

Faster convergence speed in training.

Abstract

Neural Radiance Fields (NeRFs) have shown remarkable success in synthesizing photorealistic views from multi-view images of static scenes, but face challenges in dynamic, real-world environments with distractors like moving objects, shadows, and lighting changes. Existing methods manage controlled environments and low occlusion ratios but fall short in render quality, especially under high occlusion scenarios. In this paper, we introduce NeRF On-the-go, a simple yet effective approach that enables the robust synthesis of novel views in complex, in-the-wild scenes from only casually captured image sequences. Delving into uncertainty, our method not only efficiently eliminates distractors, even when they are predominant in captures, but also achieves a notably faster convergence speed. Through comprehensive experiments on various scenes, our method demonstrates a significant improvement over state-of-the-art techniques. This advancement opens new avenues for NeRF in diverse and dynamic real-world applications.