Rethinking Transparent Object Grasping: Depth Completion with Monocular Depth Estimation and Instance Mask

TL;DR

This paper introduces ReMake, a depth completion framework that uses instance masks and monocular depth estimation to improve the accuracy and generalization of transparent object grasping in robotic systems.

Contribution

The proposed ReMake framework explicitly distinguishes transparent regions to enhance depth completion and generalization, outperforming existing methods in real-world scenarios.

Findings

Outperforms existing approaches on benchmark datasets

Achieves higher accuracy in real-world transparent object grasping

Demonstrates improved generalization to complex lighting conditions

Abstract

Due to the optical properties, transparent objects often lead depth cameras to generate incomplete or invalid depth data, which in turn reduces the accuracy and reliability of robotic grasping. Existing approaches typically input the RGB-D image directly into the network to output the complete depth, expecting the model to implicitly infer the reliability of depth values. However, while effective in training datasets, such methods often fail to generalize to real-world scenarios, where complex light interactions lead to highly variable distributions of valid and invalid depth data. To address this, we propose ReMake, a novel depth completion framework guided by an instance mask and monocular depth estimation. By explicitly distinguishing transparent regions from non-transparent ones, the mask enables the model to concentrate on learning accurate depth estimation in these areas from RGB-D input during training. This targeted supervision reduces reliance on implicit reasoning and improves generalization to real-world scenarios. Additionally, monocular depth estimation provides depth context between the transparent object and its surroundings, enhancing depth prediction accuracy. Extensive experiments show that our method outperforms existing approaches on both benchmark datasets and real-world scenarios, demonstrating superior accuracy and generalization capability. Code and videos are available at https://chengyaofeng.github.io/ReMake.github.io/.