AR2-D2:Training a Robot Without a Robot

TL;DR

AR2-D2 is an iOS app that enables users to record demonstrations of object manipulation without specialized training or real robots, facilitating diverse data collection for robot learning.

Contribution

We introduce AR2-D2, a novel system that collects manipulation demonstrations via videos without requiring real robots or specialized training, expanding data collection capabilities.

Findings

Training with AR data is as effective as real robot demonstrations.

Users find AR2-D2 intuitive and require no training.

The system enables manipulation of diverse objects.

Abstract

Diligently gathered human demonstrations serve as the unsung heroes empowering the progression of robot learning. Today, demonstrations are collected by training people to use specialized controllers, which (tele-)operate robots to manipulate a small number of objects. By contrast, we introduce AR2-D2: a system for collecting demonstrations which (1) does not require people with specialized training, (2) does not require any real robots during data collection, and therefore, (3) enables manipulation of diverse objects with a real robot. AR2-D2 is a framework in the form of an iOS app that people can use to record a video of themselves manipulating any object while simultaneously capturing essential data modalities for training a real robot. We show that data collected via our system enables the training of behavior cloning agents in manipulating real objects. Our experiments further show that training with our AR data is as effective as training with real-world robot demonstrations. Moreover, our user study indicates that users find AR2-D2 intuitive to use and require no training in contrast to four other frequently employed methods for collecting robot demonstrations.