Towards Degradation-Robust Reconstruction in Generalizable NeRF

TL;DR

This paper introduces a large-scale dataset and a simple module to improve the robustness of generalizable NeRF models against various image degradations, enhancing 3D reconstruction quality in real-world conditions.

Contribution

It provides the Objaverse Blur Dataset with diverse blur degradations and proposes a model-agnostic module to enhance GNeRF robustness to such degradations.

Findings

The dataset enables training and evaluation of degradation-robust GNeRF models.

The proposed module improves reconstruction quality across different degradation levels.

Enhanced models outperform baseline methods in both quantitative metrics and visual quality.

Abstract

Generalizable Neural Radiance Field (GNeRF) across scenes has been proven to be an effective way to avoid per-scene optimization by representing a scene with deep image features of source images. However, despite its potential for real-world applications, there has been limited research on the robustness of GNeRFs to different types of degradation present in the source images. The lack of such research is primarily attributed to the absence of a large-scale dataset fit for training a degradation-robust generalizable NeRF model. To address this gap and facilitate investigations into the degradation robustness of 3D reconstruction tasks, we construct the Objaverse Blur Dataset, comprising 50,000 images from over 1000 settings featuring multiple levels of blur degradation. In addition, we design a simple and model-agnostic module for enhancing the degradation robustness of GNeRFs. Specifically, by extracting 3D-aware features through a lightweight depth estimator and denoiser, the proposed module shows improvement on different popular methods in GNeRFs in terms of both quantitative and visual quality over varying degradation types and levels. Our dataset and code will be made publicly available.