REASSEMBLE: A Multimodal Dataset for Contact-rich Robotic Assembly and Disassembly

TL;DR

REASSEMBLE is a comprehensive multimodal dataset designed to facilitate research in contact-rich robotic assembly and disassembly tasks, capturing complex physical interactions with diverse sensor data.

Contribution

The paper introduces REASSEMBLE, a new large-scale dataset specifically for contact-rich manipulation tasks, including multiple sensor modalities and a variety of assembly actions.

Findings

Contains 4,551 demonstrations with diverse successful and unsuccessful attempts.

Includes multi-modal data such as event cameras, force-torque sensors, microphones, and RGB videos.

Supports research in contact-rich manipulation, task segmentation, and learning from complex physical interactions.

Abstract

Robotic manipulation remains a core challenge in robotics, particularly for contact-rich tasks such as industrial assembly and disassembly. Existing datasets have significantly advanced learning in manipulation but are primarily focused on simpler tasks like object rearrangement, falling short of capturing the complexity and physical dynamics involved in assembly and disassembly. To bridge this gap, we present REASSEMBLE (Robotic assEmbly disASSEMBLy datasEt), a new dataset designed specifically for contact-rich manipulation tasks. Built around the NIST Assembly Task Board 1 benchmark, REASSEMBLE includes four actions (pick, insert, remove, and place) involving 17 objects. The dataset contains 4,551 demonstrations, of which 4,035 were successful, spanning a total of 781 minutes. Our dataset features multi-modal sensor data, including event cameras, force-torque sensors, microphones, and multi-view RGB cameras. This diverse dataset supports research in areas such as learning contact-rich manipulation, task condition identification, action segmentation, and task inversion learning. The REASSEMBLE will be a valuable resource for advancing robotic manipulation in complex, real-world scenarios. The dataset is publicly available on our project website: https://tuwien-asl.github.io/REASSEMBLE_page/.