Tilde: Teleoperation for Dexterous In-Hand Manipulation Learning with a DeltaHand

TL;DR

This paper introduces Tilde, a system combining teleoperation and imitation learning with diffusion policies to enable dexterous in-hand manipulation on a low-cost robotic hand, achieving high success rates in complex tasks.

Contribution

The paper presents a novel system integrating teleoperation, a soft robotic hand, and diffusion-based imitation learning for dexterous manipulation.

Findings

Achieved an average 90% success rate across seven manipulation tasks.

Demonstrated effective autonomous deployment of learned policies.

Enabled precise data collection via a twin teleoperation interface.

Abstract

Dexterous robotic manipulation remains a challenging domain due to its strict demands for precision and robustness on both hardware and software. While dexterous robotic hands have demonstrated remarkable capabilities in complex tasks, efficiently learning adaptive control policies for hands still presents a significant hurdle given the high dimensionalities of hands and tasks. To bridge this gap, we propose Tilde, an imitation learning-based in-hand manipulation system on a dexterous DeltaHand. It leverages 1) a low-cost, configurable, simple-to-control, soft dexterous robotic hand, DeltaHand, 2) a user-friendly, precise, real-time teleoperation interface, TeleHand, and 3) an efficient and generalizable imitation learning approach with diffusion policies. Our proposed TeleHand has a kinematic twin design to the DeltaHand that enables precise one-to-one joint control of the DeltaHand during teleoperation. This facilitates efficient high-quality data collection of human demonstrations in the real world. To evaluate the effectiveness of our system, we demonstrate the fully autonomous closed-loop deployment of diffusion policies learned from demonstrations across seven dexterous manipulation tasks with an average 90% success rate.