DexHub and DART: Towards Internet Scale Robot Data Collection

TL;DR

This paper introduces DART, a cloud-based teleoperation platform utilizing AR for scalable robot data collection, and DexHub, a cloud database for sharing this data, to advance generalist robotic systems.

Contribution

The paper presents DART, a novel AR-enabled teleoperation system for scalable robot data collection, and DexHub, a cloud platform for sharing and accessing this data.

Findings

DART enables higher data throughput and reduces physical fatigue.

Policies trained on DART data transfer successfully to real robots.

DART-collected data is robust to visual disturbances.

Abstract

The quest to build a generalist robotic system is impeded by the scarcity of diverse and high-quality data. While real-world data collection effort exist, requirements for robot hardware, physical environment setups, and frequent resets significantly impede the scalability needed for modern learning frameworks. We introduce DART, a teleoperation platform designed for crowdsourcing that reimagines robotic data collection by leveraging cloud-based simulation and augmented reality (AR) to address many limitations of prior data collection efforts. Our user studies highlight that DART enables higher data collection throughput and lower physical fatigue compared to real-world teleoperation. We also demonstrate that policies trained using DART-collected datasets successfully transfer to reality and are robust to unseen visual disturbances. All data collected through DART is automatically stored in our cloud-hosted database, DexHub, which will be made publicly available upon curation, paving the path for DexHub to become an ever-growing data hub for robot learning. Videos are available at: https://dexhub.ai/project