FunGrasp: Functional Grasping for Diverse Dexterous Hands

TL;DR

FunGrasp is a system that enables diverse dexterous robotic hands to perform functional grasping of unseen objects from a single RGBD image, using human-to-robot grasp transfer and reinforcement learning for control.

Contribution

It introduces a novel approach combining human grasp estimation, grasp retargeting, and reinforcement learning for functional dexterous grasping across various robot hands.

Findings

Enables functional grasping of unseen objects from a single RGBD image.

Successfully transfers human grasp poses to different robotic hands.

Demonstrates robust sim-to-real transfer with multiple techniques.

Abstract

Functional grasping is essential for humans to perform specific tasks, such as grasping scissors by the finger holes to cut materials or by the blade to safely hand them over. Enabling dexterous robot hands with functional grasping capabilities is crucial for their deployment to accomplish diverse real-world tasks. Recent research in dexterous grasping, however, often focuses on power grasps while overlooking task- and object-specific functional grasping poses. In this paper, we introduce FunGrasp, a system that enables functional dexterous grasping across various robot hands and performs one-shot transfer to unseen objects. Given a single RGBD image of functional human grasping, our system estimates the hand pose and transfers it to different robotic hands via a human-to-robot (H2R) grasp retargeting module. Guided by the retargeted grasping poses, a policy is trained through reinforcement learning in simulation for dynamic grasping control. To achieve robust sim-to-real transfer, we employ several techniques including privileged learning, system identification, domain randomization, and gravity compensation. In our experiments, we demonstrate that our system enables diverse functional grasping of unseen objects using single RGBD images, and can be successfully deployed across various dexterous robot hands. The significance of the components is validated through comprehensive ablation studies. Project page: https://hly-123.github.io/FunGrasp/ .