TDCNet: Transparent Objects Depth Completion with CNN-Transformer Dual-Branch Parallel Network

TL;DR

TDCNet is a dual-branch CNN-Transformer network designed to improve depth completion for transparent objects, effectively utilizing both partial depth data and RGB-D images to achieve state-of-the-art results.

Contribution

The paper introduces a novel dual-branch CNN-Transformer architecture that better leverages original depth maps for transparent object depth completion.

Findings

Achieves state-of-the-art performance on multiple datasets.

Effectively combines partial depth and RGB-D information.

Demonstrates superior accuracy over existing methods.

Abstract

The sensing and manipulation of transparent objects present a critical challenge in industrial and laboratory robotics. Conventional sensors face challenges in obtaining the full depth of transparent objects due to the refraction and reflection of light on their surfaces and their lack of visible texture. Previous research has attempted to obtain complete depth maps of transparent objects from RGB and damaged depth maps (collected by depth sensor) using deep learning models. However, existing methods fail to fully utilize the original depth map, resulting in limited accuracy for deep completion. To solve this problem, we propose TDCNet, a novel dual-branch CNN-Transformer parallel network for transparent object depth completion. The proposed framework consists of two different branches: one extracts features from partial depth maps, while the other processes RGB-D images. Experimental results demonstrate that our model achieves state-of-the-art performance across multiple public datasets. Our code and the pre-trained model are publicly available at https://github.com/XianghuiFan/TDCNet.