EyeSight Hand: Design of a Fully-Actuated Dexterous Robot Hand with Integrated Vision-Based Tactile Sensors and Compliant Actuation

TL;DR

The EyeSight Hand is a fully-actuated humanoid robot hand with integrated vision-based tactile sensors and a novel actuation scheme, demonstrating improved dexterous manipulation through tactile feedback in complex tasks.

Contribution

We designed a new humanoid hand with integrated tactile sensors and quasi-direct drive actuation, enabling advanced manipulation and data collection for learning tasks.

Findings

Tactile sensing significantly improves task success rates.

The hand achieves human-like strength and speed.

Vision dropout enhances imitation learning performance.

Abstract

In this work, we introduce the EyeSight Hand, a novel 7 degrees of freedom (DoF) humanoid hand featuring integrated vision-based tactile sensors tailored for enhanced whole-hand manipulation. Additionally, we introduce an actuation scheme centered around quasi-direct drive actuation to achieve human-like strength and speed while ensuring robustness for large-scale data collection. We evaluate the EyeSight Hand on three challenging tasks: bottle opening, plasticine cutting, and plate pick and place, which require a blend of complex manipulation, tool use, and precise force application. Imitation learning models trained on these tasks, with a novel vision dropout strategy, showcase the benefits of tactile feedback in enhancing task success rates. Our results reveal that the integration of tactile sensing dramatically improves task performance, underscoring the critical role of tactile information in dexterous manipulation.