PCHands: PCA-based Hand Pose Synergy Representation on Manipulators with N-DoF

TL;DR

PCHands introduces a unified PCA-based hand pose representation that captures manipulation synergies across diverse robotic hands, improving learning efficiency and robustness in RL tasks and real-world experiments.

Contribution

It proposes a novel unified hand pose representation that generalizes across manipulators with different structures and degrees of freedom.

Findings

Outperforms joint space learning in RL efficiency

Robust in RL from demonstration across different manipulators

Effective in real-world experiments with various robotic hands

Abstract

We consider the problem of learning a common representation for dexterous manipulation across manipulators of different morphologies. To this end, we propose PCHands, a novel approach for extracting hand postural synergies from a large set of manipulators. We define a simplified and unified description format based on anchor positions for manipulators ranging from 2-finger grippers to 5-finger anthropomorphic hands. This enables learning a variable-length latent representation of the manipulator configuration and the alignment of the end-effector frame of all manipulators. We show that it is possible to extract principal components from this latent representation that is universal across manipulators of different structures and degrees of freedom. To evaluate PCHands, we use this compact representation to encode observation and action spaces of control policies for dexterous manipulation tasks learned with RL. In terms of learning efficiency and consistency, the proposed representation outperforms a baseline that learns the same tasks in joint space. We additionally show that PCHands performs robustly in RL from demonstration, when demonstrations are provided from a different manipulator. We further support our results with real-world experiments that involve a 2-finger gripper and a 4-finger anthropomorphic hand. Code and additional material are available at https://hsp-iit.github.io/PCHands/.