Gated Cross-Attention Network for Depth Completion

TL;DR

This paper introduces a Gated Cross-Attention Network that effectively fuses color and depth features for fast, accurate depth completion, outperforming existing methods on benchmark datasets.

Contribution

The paper proposes a novel Gated Cross-Attention Network with a gating mechanism and Transformer-based global feature fusion, achieving state-of-the-art depth completion without extra post-processing.

Findings

Achieves Pareto-optimal speed and accuracy trade-offs.

Ranks first on KITTI depth completion benchmark.

Performs well on indoor and outdoor datasets.

Abstract

Depth completion is a popular research direction in the field of depth estimation. The fusion of color and depth features is the current critical challenge in this task, mainly due to the asymmetry between the rich scene details in color images and the sparse pixels in depth maps. To tackle this issue, we design an efficient Gated Cross-Attention Network that propagates confidence via a gating mechanism, simultaneously extracting and refining key information in both color and depth branches to achieve local spatial feature fusion. Additionally, we employ an attention network based on the Transformer in low-dimensional space to effectively fuse global features and increase the network's receptive field. With a simple yet efficient gating mechanism, our proposed method achieves fast and accurate depth completion without the need for additional branches or post-processing steps. At the same time, we use the Ray Tune mechanism with the AsyncHyperBandScheduler scheduler and the HyperOptSearch algorithm to automatically search for the optimal number of module iterations, which also allows us to achieve performance comparable to state-of-the-art methods. We conduct experiments on both indoor and outdoor scene datasets. Our fast network achieves Pareto-optimal solutions in terms of time and accuracy, and at the time of submission, our accurate network ranks first among all published papers on the KITTI official website in terms of accuracy.