Augmented Reality Demonstrations for Scalable Robot Imitation Learning

TL;DR

This paper introduces an AR-assisted framework that enables non-expert users to collect diverse robot demonstrations using devices like HoloLens 2, enhancing the scalability of imitation learning for manipulation tasks.

Contribution

It presents a novel AR-based system for demonstration collection, reducing the need for expert operators and facilitating scalable robot imitation learning.

Findings

Robots successfully performed reach, push, and pick-and-place tasks using demonstrations collected via AR.

The AR framework enables non-experts to generate effective demonstrations for robot learning.

Demonstrations collected through AR led to successful task execution on real robots.

Abstract

Robot Imitation Learning (IL) is a widely used method for training robots to perform manipulation tasks that involve mimicking human demonstrations to acquire skills. However, its practicality has been limited due to its requirement that users be trained in operating real robot arms to provide demonstrations. This paper presents an innovative solution: an Augmented Reality (AR)-assisted framework for demonstration collection, empowering non-roboticist users to produce demonstrations for robot IL using devices like the HoloLens 2. Our framework facilitates scalable and diverse demonstration collection for real-world tasks. We validate our approach with experiments on three classical robotics tasks: reach, push, and pick-and-place. The real robot performs each task successfully while replaying demonstrations collected via AR.