ExoStart: Efficient learning for dexterous manipulation with sensorized exoskeleton demonstrations

TL;DR

ExoStart introduces a scalable framework that uses sensorized exoskeleton demonstrations and simulation-based filtering to efficiently learn dexterous robotic hand manipulation skills with zero-shot transfer to real robots.

Contribution

The paper presents a novel learning pipeline combining human demonstrations via a wearable exoskeleton, a dynamics filter, and auto-curriculum reinforcement learning for dexterous robotic hand control.

Findings

Achieves over 50% success rate on complex manipulation tasks

Demonstrates zero-shot transfer of policies to real robotic hands

Enables learning of diverse dexterous manipulation skills

Abstract

Recent advancements in teleoperation systems have enabled high-quality data collection for robotic manipulators, showing impressive results in learning manipulation at scale. This progress suggests that extending these capabilities to robotic hands could unlock an even broader range of manipulation skills, especially if we could achieve the same level of dexterity that human hands exhibit. However, teleoperating robotic hands is far from a solved problem, as it presents a significant challenge due to the high degrees of freedom of robotic hands and the complex dynamics occurring during contact-rich settings. In this work, we present ExoStart, a general and scalable learning framework that leverages human dexterity to improve robotic hand control. In particular, we obtain high-quality data by collecting direct demonstrations without a robot in the loop using a sensorized low-cost wearable exoskeleton, capturing the rich behaviors that humans can demonstrate with their own hands. We also propose a simulation-based dynamics filter that generates dynamically feasible trajectories from the collected demonstrations and use the generated trajectories to bootstrap an auto-curriculum reinforcement learning method that relies only on simple sparse rewards. The ExoStart pipeline is generalizable and yields robust policies that transfer zero-shot to the real robot. Our results demonstrate that ExoStart can generate dexterous real-world hand skills, achieving a success rate above 50% on a wide range of complex tasks such as opening an AirPods case or inserting and turning a key in a lock. More details and videos can be found in https://sites.google.com/view/exostart.