RustNeRF: Robust Neural Radiance Field with Low-Quality Images

TL;DR

RustNeRF introduces a robust neural radiance field model that effectively handles real-world low-quality images by incorporating degradation modeling and multi-view guidance, enhancing 3D scene reconstruction and view synthesis.

Contribution

The paper proposes RustNeRF, a novel approach that improves NeRF robustness to real-world image degradation through a 3D-aware preprocessing network and implicit multi-view guidance.

Findings

RustNeRF outperforms existing methods on degraded image datasets.

The approach effectively models real-world image degradation.

Experimental results show improved 3D reconstruction quality.

Abstract

Recent work on Neural Radiance Fields (NeRF) exploits multi-view 3D consistency, achieving impressive results in 3D scene modeling and high-fidelity novel-view synthesis. However, there are limitations. First, existing methods assume enough high-quality images are available for training the NeRF model, ignoring real-world image degradation. Second, previous methods struggle with ambiguity in the training set due to unmodeled inconsistencies among different views. In this work, we present RustNeRF for real-world high-quality NeRF. To improve NeRF's robustness under real-world inputs, we train a 3D-aware preprocessing network that incorporates real-world degradation modeling. We propose a novel implicit multi-view guidance to address information loss during image degradation and restoration. Extensive experiments demonstrate RustNeRF's advantages over existing approaches under real-world degradation. The code will be released.