A Humanoid Visual-Tactile-Action Dataset for Contact-Rich Manipulation

TL;DR

This paper introduces a comprehensive humanoid visual-tactile-action dataset for contact-rich manipulation of deformable objects, addressing the lack of diverse pressure condition data in robot learning.

Contribution

It provides a new multi-modal dataset collected via teleoperation, enabling research on manipulation of soft objects with diverse tactile signals.

Findings

Dataset captures multi-modal interactions under varying pressures

Facilitates development of models leveraging tactile complexity

Supports research on deformable object manipulation

Abstract

Contact-rich manipulation has become increasingly important in robot learning. However, previous studies on robot learning datasets have focused on rigid objects and underrepresented the diversity of pressure conditions for real-world manipulation. To address this gap, we present a humanoid visual-tactile-action dataset designed for manipulating deformable soft objects. The dataset was collected via teleoperation using a humanoid robot equipped with dexterous hands, capturing multi-modal interactions under varying pressure conditions. This work also motivates future research on models with advanced optimization strategies capable of effectively leveraging the complexity and diversity of tactile signals.