Self-Supervised Deep Depth Denoising

TL;DR

This paper introduces a self-supervised deep autoencoder that effectively denoises depth maps by leveraging multi-view data, differentiable rendering, and geometric priors, without requiring clean ground truth data during training.

Contribution

It proposes a novel self-supervised deep autoencoder for depth denoising that uses multi-view consistency and differentiable rendering, overcoming the need for clean ground truth data.

Findings

Effective noise suppression in depth maps

Improved 3D reconstruction quality

Operates without clean ground truth during training

Abstract

Depth perception is considered an invaluable source of information for various vision tasks. However, depth maps acquired using consumer-level sensors still suffer from non-negligible noise. This fact has recently motivated researchers to exploit traditional filters, as well as the deep learning paradigm, in order to suppress the aforementioned non-uniform noise, while preserving geometric details. Despite the effort, deep depth denoising is still an open challenge mainly due to the lack of clean data that could be used as ground truth. In this paper, we propose a fully convolutional deep autoencoder that learns to denoise depth maps, surpassing the lack of ground truth data. Specifically, the proposed autoencoder exploits multiple views of the same scene from different points of view in order to learn to suppress noise in a self-supervised end-to-end manner using depth and color information during training, yet only depth during inference. To enforce selfsupervision, we leverage a differentiable rendering technique to exploit photometric supervision, which is further regularized using geometric and surface priors. As the proposed approach relies on raw data acquisition, a large RGB-D corpus is collected using Intel RealSense sensors. Complementary to a quantitative evaluation, we demonstrate the effectiveness of the proposed self-supervised denoising approach on established 3D reconstruction applications. Code is avalable at https://github.com/VCL3D/DeepDepthDenoising