wildNeRF: Complete view synthesis of in-the-wild dynamic scenes captured using sparse monocular data

TL;DR

wildNeRF introduces a self-supervised neural radiance model capable of synthesizing novel views of complex, dynamic scenes from sparse monocular data, achieving state-of-the-art results efficiently.

Contribution

It presents a novel, end-to-end trainable neural radiance model that differentiates static and dynamic pixels for high-quality view synthesis from limited data.

Findings

Achieves state-of-the-art performance on NVIDIA Dynamic Scenes Dataset.

Efficient training: static scenes in seconds, dynamic scenes in minutes.

Performs well on real-world datasets like Cholec80 and SurgicalActions160.

Abstract

We present a novel neural radiance model that is trainable in a self-supervised manner for novel-view synthesis of dynamic unstructured scenes. Our end-to-end trainable algorithm learns highly complex, real-world static scenes within seconds and dynamic scenes with both rigid and non-rigid motion within minutes. By differentiating between static and motion-centric pixels, we create high-quality representations from a sparse set of images. We perform extensive qualitative and quantitative evaluation on existing benchmarks and set the state-of-the-art on performance measures on the challenging NVIDIA Dynamic Scenes Dataset. Additionally, we evaluate our model performance on challenging real-world datasets such as Cholec80 and SurgicalActions160.