HDCNet: A Hybrid Depth Completion Network for Grasping Transparent and Reflective Objects

TL;DR

HDCNet is a novel hybrid neural network that combines Transformer, CNN, and Mamba architectures to improve depth completion and robotic grasping of transparent and reflective objects, outperforming existing methods.

Contribution

The paper introduces HDCNet, a hybrid depth completion network that effectively integrates multiple architectures for enhanced accuracy and robustness in challenging perception tasks.

Findings

HDCNet achieves state-of-the-art depth completion performance.

The network significantly improves grasp success rates on transparent and reflective objects.

Robotic experiments show up to 60% increase in grasp success.

Abstract

Depth perception of transparent and reflective objects has long been a critical challenge in robotic manipulation.Conventional depth sensors often fail to provide reliable measurements on such surfaces, limiting the performance of robots in perception and grasping tasks. To address this issue, we propose a novel depth completion network,HDCNet,which integrates the complementary strengths of Transformer,CNN and Mamba architectures.Specifically,the encoder is designed as a dual-branch Transformer-CNN framework to extract modality-specific features. At the shallow layers of the encoder, we introduce a lightweight multimodal fusion module to effectively integrate low-level features. At the network bottleneck,a Transformer-Mamba hybrid fusion module is developed to achieve deep integration of high-level semantic and global contextual information, significantly enhancing depth completion accuracy and robustness. Extensive evaluations on multiple public datasets demonstrate that HDCNet achieves state-of-the-art(SOTA) performance in depth completion tasks.Furthermore,robotic grasping experiments show that HDCNet substantially improves grasp success rates for transparent and reflective objects,achieving up to a 60% increase.