AGG-Net: Attention Guided Gated-convolutional Network for Depth Image Completion

TL;DR

AGG-Net is a novel deep learning model that enhances depth image completion by effectively fusing RGB and depth data using attention-guided gated convolutions, outperforming existing methods on multiple benchmarks.

Contribution

The paper introduces AGG-Net, a new architecture with attention-guided gated convolution modules for improved depth image completion from RGB-D data.

Findings

Outperforms state-of-the-art methods on NYU-Depth V2, DIML, and SUN RGB-D benchmarks.

Effectively reduces negative impacts of invalid depth data during reconstruction.

Utilizes a UNet-like architecture with novel attention-guided modules.

Abstract

Recently, stereo vision based on lightweight RGBD cameras has been widely used in various fields. However, limited by the imaging principles, the commonly used RGB-D cameras based on TOF, structured light, or binocular vision acquire some invalid data inevitably, such as weak reflection, boundary shadows, and artifacts, which may bring adverse impacts to the follow-up work. In this paper, we propose a new model for depth image completion based on the Attention Guided Gated-convolutional Network (AGG-Net), through which more accurate and reliable depth images can be obtained from the raw depth maps and the corresponding RGB images. Our model employs a UNet-like architecture which consists of two parallel branches of depth and color features. In the encoding stage, an Attention Guided Gated-Convolution (AG-GConv) module is proposed to realize the fusion of depth and color features at different scales, which can effectively reduce the negative impacts of invalid depth data on the reconstruction. In the decoding stage, an Attention Guided Skip Connection (AG-SC) module is presented to avoid introducing too many depth-irrelevant features to the reconstruction. The experimental results demonstrate that our method outperforms the state-of-the-art methods on the popular benchmarks NYU-Depth V2, DIML, and SUN RGB-D.