HRDexDB: A Large-Scale Dataset of Dexterous Human and Robotic Hand Grasps

TL;DR

HRDexDB is a comprehensive, multi-modal dataset of human and robotic hand grasps across diverse objects, enabling advanced research in dexterous manipulation and policy learning.

Contribution

It introduces a large-scale, multi-modal dataset with high-fidelity grasping sequences, including tactile, visual, and kinematic data for both humans and robots.

Findings

Provides 1.4K grasping trials with success and failure data.

Includes synchronized multi-modal data: vision, tactile, and motion.

Enables cross-domain dexterous manipulation research.

Abstract

We present HRDexDB, a large-scale, multi-modal dataset of high-fidelity dexterous grasping sequences featuring both human and diverse robotic hands. Unlike existing datasets, HRDexDB provides a comprehensive collection of grasping trajectories across human hands and multiple robot hand embodiments, spanning 100 diverse objects. Leveraging state-of-the-art vision methods and a new dedicated multi-camera system, our HRDexDB offers high-precision spatiotemporal 3D ground-truth motion for both the agent and the manipulated object. To facilitate the study of physical interaction, HRDexDB includes high-resolution tactile signals, synchronized multi-view video, and egocentric video streams. The dataset comprises 1.4K grasping trials, encompassing both successes and failures, each enriched with visual, kinematic, and tactile modalities. By providing closely aligned captures of human dexterity and robotic execution on the same target objects under comparable grasping motions, HRDexDB serves as a foundational benchmark for multi-modal policy learning and cross-domain dexterous manipulation.