Dynamic Spatial Propagation Network for Depth Completion

TL;DR

The paper introduces DySPN, an efficient, attention-based dynamic spatial propagation network for depth completion that improves accuracy and reduces over-smoothing by estimating adaptive affinity matrices and employing diffusion suppression.

Contribution

It proposes a novel non-linear propagation model with dynamic affinity estimation and diffusion suppression, enhancing depth completion performance and efficiency over existing fixed-affinity SPNs.

Findings

Outperforms state-of-the-art methods on KITTI Depth Completion dataset.

Achieves state-of-the-art results on NYU Depth v2 dataset.

Requires fewer iterations while maintaining high accuracy.

Abstract

Image-guided depth completion aims to generate dense depth maps with sparse depth measurements and corresponding RGB images. Currently, spatial propagation networks (SPNs) are the most popular affinity-based methods in depth completion, but they still suffer from the representation limitation of the fixed affinity and the over smoothing during iterations. Our solution is to estimate independent affinity matrices in each SPN iteration, but it is over-parameterized and heavy calculation. This paper introduces an efficient model that learns the affinity among neighboring pixels with an attention-based, dynamic approach. Specifically, the Dynamic Spatial Propagation Network (DySPN) we proposed makes use of a non-linear propagation model (NLPM). It decouples the neighborhood into parts regarding to different distances and recursively generates independent attention maps to refine these parts into adaptive affinity matrices. Furthermore, we adopt a diffusion suppression (DS) operation so that the model converges at an early stage to prevent over-smoothing of dense depth. Finally, in order to decrease the computational cost required, we also introduce three variations that reduce the amount of neighbors and attentions needed while still retaining similar accuracy. In practice, our method requires less iteration to match the performance of other SPNs and yields better results overall. DySPN outperforms other state-of-the-art (SoTA) methods on KITTI Depth Completion (DC) evaluation by the time of submission and is able to yield SoTA performance in NYU Depth v2 dataset as well.