RGB-D-Fusion: Image Conditioned Depth Diffusion of Humanoid Subjects

TL;DR

RGB-D-Fusion introduces a multi-stage diffusion-based approach to generate high-resolution depth maps from low-resolution RGB images of humanoid subjects, enhancing depth estimation with novel augmentation techniques.

Contribution

It proposes a two-stage diffusion model for depth super-resolution conditioned on RGB images, incorporating a new depth noise augmentation for robustness.

Findings

Achieves high-quality depth maps from low-res RGB inputs.

Demonstrates improved robustness with depth noise augmentation.

Outperforms existing depth super-resolution methods.

Abstract

We present RGB-D-Fusion, a multi-modal conditional denoising diffusion probabilistic model to generate high resolution depth maps from low-resolution monocular RGB images of humanoid subjects. RGB-D-Fusion first generates a low-resolution depth map using an image conditioned denoising diffusion probabilistic model and then upsamples the depth map using a second denoising diffusion probabilistic model conditioned on a low-resolution RGB-D image. We further introduce a novel augmentation technique, depth noise augmentation, to increase the robustness of our super-resolution model.