Transparent Object Depth Completion

TL;DR

This paper introduces an end-to-end neural network for completing depth maps of transparent objects, combining single-view and multi-view estimations with confidence-based refinement to improve accuracy and robustness in complex scenes.

Contribution

The novel depth completion network effectively fuses single-view and multi-view depth estimates with confidence-based refinement for transparent objects.

Findings

Achieves superior accuracy on ClearPose and TransCG datasets.

Outperforms state-of-the-art methods in occluded scenarios.

Demonstrates robustness in complex environments.

Abstract

The perception of transparent objects for grasp and manipulation remains a major challenge, because existing robotic grasp methods which heavily rely on depth maps are not suitable for transparent objects due to their unique visual properties. These properties lead to gaps and inaccuracies in the depth maps of the transparent objects captured by depth sensors. To address this issue, we propose an end-to-end network for transparent object depth completion that combines the strengths of single-view RGB-D based depth completion and multi-view depth estimation. Moreover, we introduce a depth refinement module based on confidence estimation to fuse predicted depth maps from single-view and multi-view modules, which further refines the restored depth map. The extensive experiments on the ClearPose and TransCG datasets demonstrate that our method achieves superior accuracy and robustness in complex scenarios with significant occlusion compared to the state-of-the-art methods.