Multi-resolution Monocular Depth Map Fusion by Self-supervised Gradient-based Composition

TL;DR

This paper introduces a self-supervised, real-time depth map fusion method that combines multi-resolution monocular depth estimations using gradient-based composition, significantly enhancing detail and noise immunity.

Contribution

It proposes a novel self-supervised fusion framework leveraging guided filtering and gradient domain integration, achieving superior detail preservation and noise robustness in depth estimation.

Findings

Outperforms state-of-the-art fusion methods in noise immunity

Runs 80 times faster than existing methods

Achieves state-of-the-art detail enhancement in depth maps

Abstract

Monocular depth estimation is a challenging problem on which deep neural networks have demonstrated great potential. However, depth maps predicted by existing deep models usually lack fine-grained details due to the convolution operations and the down-samplings in networks. We find that increasing input resolution is helpful to preserve more local details while the estimation at low resolution is more accurate globally. Therefore, we propose a novel depth map fusion module to combine the advantages of estimations with multi-resolution inputs. Instead of merging the low- and high-resolution estimations equally, we adopt the core idea of Poisson fusion, trying to implant the gradient domain of high-resolution depth into the low-resolution depth. While classic Poisson fusion requires a fusion mask as supervision, we propose a self-supervised framework based on guided image filtering. We demonstrate that this gradient-based composition performs much better at noisy immunity, compared with the state-of-the-art depth map fusion method. Our lightweight depth fusion is one-shot and runs in real-time, making our method 80X faster than a state-of-the-art depth fusion method. Quantitative evaluations demonstrate that the proposed method can be integrated into many fully convolutional monocular depth estimation backbones with a significant performance boost, leading to state-of-the-art results of detail enhancement on depth maps.