NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections

TL;DR

This paper introduces NeRF-W, an extension of Neural Radiance Fields, capable of synthesizing realistic novel views from unstructured, uncontrolled internet photo collections by addressing real-world variability.

Contribution

NeRF-W extends NeRF to handle uncontrolled image collections by modeling transient occlusions and variable illumination, enabling photorealistic reconstructions from internet photos.

Findings

NeRF-W produces more photorealistic and consistent novel views than previous methods.

The system effectively handles real-world phenomena like lighting changes and occlusions.

Results demonstrate significant improvements on landmark photo collections.

Abstract

We present a learning-based method for synthesizing novel views of complex scenes using only unstructured collections of in-the-wild photographs. We build on Neural Radiance Fields (NeRF), which uses the weights of a multilayer perceptron to model the density and color of a scene as a function of 3D coordinates. While NeRF works well on images of static subjects captured under controlled settings, it is incapable of modeling many ubiquitous, real-world phenomena in uncontrolled images, such as variable illumination or transient occluders. We introduce a series of extensions to NeRF to address these issues, thereby enabling accurate reconstructions from unstructured image collections taken from the internet. We apply our system, dubbed NeRF-W, to internet photo collections of famous landmarks, and demonstrate temporally consistent novel view renderings that are significantly closer to photorealism than the prior state of the art.