Revisiting Deformable Convolution for Depth Completion

TL;DR

This paper introduces a novel depth completion method using deformable convolution as a single-pass refinement, achieving state-of-the-art results efficiently by addressing limitations of iterative propagation methods.

Contribution

It proposes a deformable convolution-based architecture for depth completion that outperforms existing methods in accuracy and speed, with systematic analysis of deformable convolution strategies.

Findings

Deformable convolution improves depth map refinement in a single pass.

Applying deformable convolution on denser estimated depth maps yields better results.

Achieves state-of-the-art accuracy and inference speed on KITTI dataset.

Abstract

Depth completion, which aims to generate high-quality dense depth maps from sparse depth maps, has attracted increasing attention in recent years. Previous work usually employs RGB images as guidance, and introduces iterative spatial propagation to refine estimated coarse depth maps. However, most of the propagation refinement methods require several iterations and suffer from a fixed receptive field, which may contain irrelevant and useless information with very sparse input. In this paper, we address these two challenges simultaneously by revisiting the idea of deformable convolution. We propose an effective architecture that leverages deformable kernel convolution as a single-pass refinement module, and empirically demonstrate its superiority. To better understand the function of deformable convolution and exploit it for depth completion, we further systematically investigate a variety of representative strategies. Our study reveals that, different from prior work, deformable convolution needs to be applied on an estimated depth map with a relatively high density for better performance. We evaluate our model on the large-scale KITTI dataset and achieve state-of-the-art level performance in both accuracy and inference speed. Our code is available at https://github.com/AlexSunNik/ReDC.