Dexterous Teleoperation of 20-DoF ByteDexter Hand via Human Motion Retargeting

TL;DR

This paper presents a novel teleoperation system that enables high-fidelity, real-time control of a 20-DoF robotic hand through human motion retargeting, facilitating dexterous manipulation and data collection.

Contribution

It introduces a biomimetic 20-DoF robotic hand with an optimization-based retargeting system for seamless human motion transfer in teleoperation.

Findings

Successful in-hand manipulation and organization tasks

High-quality demonstration data generated in real-time

Intuitive interface enabling complex dexterous control

Abstract

Replicating human--level dexterity remains a fundamental robotics challenge, requiring integrated solutions from mechatronic design to the control of high degree--of--freedom (DoF) robotic hands. While imitation learning shows promise in transferring human dexterity to robots, the efficacy of trained policies relies on the quality of human demonstration data. We bridge this gap with a hand--arm teleoperation system featuring: (1) a 20--DoF linkage--driven anthropomorphic robotic hand for biomimetic dexterity, and (2) an optimization--based motion retargeting for real--time, high--fidelity reproduction of intricate human hand motions and seamless hand--arm coordination. We validate the system via extensive empirical evaluations, including dexterous in-hand manipulation tasks and a long--horizon task requiring the organization of a cluttered makeup table randomly populated with nine objects. Experimental results demonstrate its intuitive teleoperation interface with real--time control and the ability to generate high--quality demonstration data. Please refer to the accompanying video for further details.