Bayesian Monocular Depth Refinement via Neural Radiance Fields

TL;DR

This paper introduces MDENeRF, a Bayesian framework that refines monocular depth maps by integrating neural radiance fields with uncertainty modeling, enhancing geometric detail for better scene understanding.

Contribution

The paper presents a novel iterative method combining monocular depth estimates with NeRF-derived information and uncertainty to improve depth map accuracy.

Findings

Significant improvement in depth detail and accuracy on SUN RGB-D dataset.

Effective integration of NeRF uncertainty for iterative depth refinement.

Enhanced global structure preservation in depth maps.

Abstract

Monocular depth estimation has applications in many fields, such as autonomous navigation and extended reality, making it an essential computer vision task. However, current methods often produce smooth depth maps that lack the fine geometric detail needed for accurate scene understanding. We propose MDENeRF, an iterative framework that refines monocular depth estimates using depth information from Neural Radiance Fields (NeRFs). MDENeRF consists of three components: (1) an initial monocular estimate for global structure, (2) a NeRF trained on perturbed viewpoints, with per-pixel uncertainty, and (3) Bayesian fusion of the noisy monocular and NeRF depths. We derive NeRF uncertainty from the volume rendering process to iteratively inject high-frequency fine details. Meanwhile, our monocular prior maintains global structure. We demonstrate improvements on key metrics and experiments using indoor scenes from the SUN RGB-D dataset.