Generate, Transfer, Adapt: Learning Functional Dexterous Grasping from a Single Human Demonstration

TL;DR

This paper introduces CorDex, a framework that learns dexterous robotic grasps from a single human demonstration by generating synthetic data and integrating visual and geometric reasoning, enabling generalization to new objects.

Contribution

CorDex is the first method to learn functional grasps from a single demonstration using synthetic data generation and multimodal prediction, addressing data scarcity and reasoning integration.

Findings

Outperforms state-of-the-art baselines in grasping accuracy.

Generalizes well to unseen objects across categories.

Efficiently predicts grasps using local-global fusion and importance sampling.

Abstract

Functional grasping with dexterous robotic hands is a key capability for enabling tool use and complex manipulation, yet progress has been constrained by two persistent bottlenecks: the scarcity of large-scale datasets and the absence of integrated semantic and geometric reasoning in learned models. In this work, we present CorDex, a framework that robustly learns dexterous functional grasps of novel objects from synthetic data generated from just a single human demonstration. At the core of our approach is a correspondence-based data engine that generates diverse, high-quality training data in simulation. Based on the human demonstration, our data engine generates diverse object instances of the same category, transfers the expert grasp to the generated objects through correspondence estimation, and adapts the grasp through optimization. Building on the generated data, we introduce a multimodal prediction network that integrates visual and geometric information. By devising a local-global fusion module and an importance-aware sampling mechanism, we enable robust and computationally efficient prediction of functional dexterous grasps. Through extensive experiments across various object categories, we demonstrate that CorDex generalizes well to unseen object instances and significantly outperforms state-of-the-art baselines.