LRRU: Long-short Range Recurrent Updating Networks for Depth Completion

TL;DR

The paper introduces LRRU, a lightweight recurrent network that efficiently refines initial depth maps from sparse data using content-adaptive kernels, achieving state-of-the-art results with less computation.

Contribution

The novel LRRU framework performs iterative depth refinement with dynamic kernels, reducing complexity while maintaining high accuracy in depth completion tasks.

Findings

Achieves state-of-the-art performance on NYUv2 and KITTI datasets.

Outperforms existing methods with fewer parameters and faster inference.

Effectively captures long-to-short range dependencies in depth maps.

Abstract

Existing deep learning-based depth completion methods generally employ massive stacked layers to predict the dense depth map from sparse input data. Although such approaches greatly advance this task, their accompanied huge computational complexity hinders their practical applications. To accomplish depth completion more efficiently, we propose a novel lightweight deep network framework, the Long-short Range Recurrent Updating (LRRU) network. Without learning complex feature representations, LRRU first roughly fills the sparse input to obtain an initial dense depth map, and then iteratively updates it through learned spatially-variant kernels. Our iterative update process is content-adaptive and highly flexible, where the kernel weights are learned by jointly considering the guidance RGB images and the depth map to be updated, and large-to-small kernel scopes are dynamically adjusted to capture long-to-short range dependencies. Our initial depth map has coarse but complete scene depth information, which helps relieve the burden of directly regressing the dense depth from sparse ones, while our proposed method can effectively refine it to an accurate depth map with less learnable parameters and inference time. Experimental results demonstrate that our proposed LRRU variants achieve state-of-the-art performance across different parameter regimes. In particular, the LRRU-Base model outperforms competing approaches on the NYUv2 dataset, and ranks 1st on the KITTI depth completion benchmark at the time of submission. Project page: https://npucvr.github.io/LRRU/.