Cross-Embodiment Dexterous Grasping with Reinforcement Learning

TL;DR

This paper introduces a reinforcement learning approach for cross-embodiment dexterous grasping, enabling a universal policy that generalizes across different robotic hands using eigengrasps and simplified proprioception.

Contribution

It proposes a universal action space based on eigengrasps and a unified observation space, allowing a single policy to control diverse dexterous hands with high success and zero-shot generalization.

Findings

Achieved 80% grasp success rate across four robot embodiments.

Zero-shot generalization to two unseen robot hands.

Significant improvement in finetuning efficiency.

Abstract

Dexterous hands exhibit significant potential for complex real-world grasping tasks. While recent studies have primarily focused on learning policies for specific robotic hands, the development of a universal policy that controls diverse dexterous hands remains largely unexplored. In this work, we study the learning of cross-embodiment dexterous grasping policies using reinforcement learning (RL). Inspired by the capability of human hands to control various dexterous hands through teleoperation, we propose a universal action space based on the human hand's eigengrasps. The policy outputs eigengrasp actions that are then converted into specific joint actions for each robot hand through a retargeting mapping. We simplify the robot hand's proprioception to include only the positions of fingertips and the palm, offering a unified observation space across different robot hands. Our approach demonstrates an 80% success rate in grasping objects from the YCB dataset across four distinct embodiments using a single vision-based policy. Additionally, our policy exhibits zero-shot generalization to two previously unseen embodiments and significant improvement in efficient finetuning. For further details and videos, visit our project page https://sites.google.com/view/crossdex.