A Wearable Robotic Hand for Hand-over-Hand Imitation Learning

TL;DR

This paper introduces HIRO Hand, a wearable robotic hand that captures expert hand movements and tactile feedback for improved imitation learning in dexterous manipulation tasks.

Contribution

It presents a novel wearable robotic hand that directly captures tactile and motion data, overcoming limitations of data gloves for imitation learning.

Findings

Successfully achieved grasping and in-hand manipulation.

Enabled more accurate imitation through tactile feedback.

Integrated data collection with dexterous operation capabilities.

Abstract

Dexterous manipulation through imitation learning has gained significant attention in robotics research. The collection of high-quality expert data holds paramount importance when using imitation learning. The existing approaches for acquiring expert data commonly involve utilizing a data glove to capture hand motion information. However, this method suffers from limitations as the collected information cannot be directly mapped to the robotic hand due to discrepancies in their degrees of freedom or structures. Furthermore,it fails to accurately capture force feedback information between the hand and objects during the demonstration process. To overcome these challenges, this paper presents a novel solution in the form of a wearable dexterous hand, namely Hand-over-hand Imitation learning wearable RObotic Hand (HIRO Hand),which integrates expert data collection and enables the implementation of dexterous operations. This HIRO Hand empowers the operator to utilize their own tactile feedback to determine appropriate force, position, and actions, resulting in more accurate imitation of the expert's actions. We develop both non-learning and visual behavior cloning based controllers allowing HIRO Hand successfully achieves grasping and in-hand manipulation ability.