DexH2R: A Benchmark for Dynamic Dexterous Grasping in Human-to-Robot Handover

TL;DR

This paper introduces DexH2R, a real-world dataset for dynamic human-to-robot handovers using a dexterous robotic hand, along with a new grasping method and thorough evaluation to advance the field.

Contribution

It provides a high-quality, real-world dataset for dynamic handovers, and proposes DynamicGrasp, an effective approach evaluated against state-of-the-art methods.

Findings

DexH2R captures diverse objects and motions with detailed annotations.

DynamicGrasp outperforms existing approaches in human-like dexterous grasping.

Comprehensive evaluation offers insights into effective strategies for human-robot handover.

Abstract

Handover between a human and a dexterous robotic hand is a fundamental yet challenging task in human-robot collaboration. It requires handling dynamic environments and a wide variety of objects and demands robust and adaptive grasping strategies. However, progress in developing effective dynamic dexterous grasping methods is limited by the absence of high-quality, real-world human-to-robot handover datasets. Existing datasets primarily focus on grasping static objects or rely on synthesized handover motions, which differ significantly from real-world robot motion patterns, creating a substantial gap in applicability. In this paper, we introduce DexH2R, a comprehensive real-world dataset for human-to-robot handovers, built on a dexterous robotic hand. Our dataset captures a diverse range of interactive objects, dynamic motion patterns, rich visual sensor data, and detailed annotations. Additionally, to ensure natural and human-like dexterous motions, we utilize teleoperation for data collection, enabling the robot's movements to align with human behaviors and habits, which is a crucial characteristic for intelligent humanoid robots. Furthermore, we propose an effective solution, DynamicGrasp, for human-to-robot handover and evaluate various state-of-the-art approaches, including auto-regressive models and diffusion policy methods, providing a thorough comparison and analysis. We believe our benchmark will drive advancements in human-to-robot handover research by offering a high-quality dataset, effective solutions, and comprehensive evaluation metrics.