ContactDexNet: Multi-fingered Robotic Hand Grasping in Cluttered Environments through Hand-object Contact Semantic Mapping

TL;DR

This paper introduces ContactDexNet, a novel approach for multi-fingered robotic hand grasping in cluttered environments using contact semantic mapping, a specialized variational autoencoder, and an improved grasp evaluation model, achieving higher success rates.

Contribution

The paper presents a new contact semantic map generation method, a contact semantic conditional variational autoencoder, and an enhanced grasp evaluation model, advancing grasping in cluttered scenes.

Findings

Achieved 81.0% grasp success rate in real-world single-object scenarios.

Attained 75.3% grasp success rate in cluttered environments.

Outperformed state-of-the-art methods by at least 4.65%.

Abstract

The deep learning models has significantly advanced dexterous manipulation techniques for multi-fingered hand grasping. However, the contact information-guided grasping in cluttered environments remains largely underexplored. To address this gap, we have developed a method for generating multi-fingered hand grasp samples in cluttered settings through contact semantic map. We introduce a contact semantic conditional variational autoencoder network (CoSe-CVAE) for creating comprehensive contact semantic map from object point cloud. We utilize grasp detection method to estimate hand grasp poses from the contact semantic map. Finally, an unified grasp evaluation model PointNetGPD++ is designed to assess grasp quality and collision probability, substantially improving the reliability of identifying optimal grasps in cluttered scenarios. Our grasp generation method has demonstrated remarkable success, outperforming state-of-the-art methods by at least 4.65% with 81.0% average grasping success rate in real-world single-object environment and 75.3% grasping success rate in cluttered scenes. We also proposed the multi-modal multi-fingered grasping dataset generation method. Our multi-fingered hand grasping dataset outperforms previous datasets in scene diversity, modality diversity. The dataset, code and supplementary materials can be found at https://sites.google.com/view/contact-dexnet.