RoboCOIN: An Open-Sourced Bimanual Robotic Data Collection for Integrated Manipulation

TL;DR

RoboCOIN is a comprehensive, open-source dataset of over 180,000 bimanual manipulation demonstrations across 15 robotic platforms, designed to advance research in multi-embodiment robotic manipulation.

Contribution

It introduces RoboCOIN, a large-scale, diverse bimanual manipulation dataset with hierarchical annotations and an efficient processing pipeline, filling a critical gap in available robotic datasets.

Findings

Enhances manipulation model performance across various robots

Provides detailed hierarchical annotations for tasks and actions

Demonstrates the dataset's utility through extensive experiments

Abstract

Despite the critical role of bimanual manipulation in endowing robots with human-like dexterity, large-scale and diverse datasets remain scarce due to the significant hardware heterogeneity across bimanual robotic platforms. To bridge this gap, we introduce RoboCOIN, a large-scale multi-embodiment bimanual manipulation dataset comprising over 180,000 demonstrations collected from 15 distinct robotic platforms. Spanning 16 diverse environments-including residential, commercial, and industrial settings-the dataset features 421 bimanual tasks systematically categorized by 39 bimanual collaboration actions and 432 objects. A key innovation of our work is the hierarchical capability pyramid, which provides granular annotations ranging from trajectory-level concepts to segment-level subtasks and frame-level kinematics. Furthermore, we present CoRobot, an efficient data processing pipeline powered by the Robot Trajectory Markup Language (RTML), designed to facilitate quality assessment, automated annotation, and unified multi-embodiment and data management. Extensive experiments demonstrate the effectiveness of RoboCOIN in enhancing the performance of various bimanual manipulation models across a wide spectrum of robotic embodiments. The entire dataset and codebase are fully open-sourced, providing a valuable resource for advancing research in bimanual and multi-embodiment manipulation.