AVID: Learning Multi-Stage Tasks via Pixel-Level Translation of Human Videos

TL;DR

This paper introduces AVID, a framework that enables robots to learn multi-stage tasks from human videos by automatically translating them into robot videos, simplifying task specification and reducing human effort in reinforcement learning.

Contribution

AVID presents an automated method using pixel-level image translation to convert human demonstration videos into robot-compatible videos for autonomous multi-stage task learning.

Findings

Successfully learned complex tasks like operating a coffee machine

Minimal human input needed: 20 minutes for demonstrations

Total training time around 180 minutes of robot interaction

Abstract

Robotic reinforcement learning (RL) holds the promise of enabling robots to learn complex behaviors through experience. However, realizing this promise for long-horizon tasks in the real world requires mechanisms to reduce human burden in terms of defining the task and scaffolding the learning process. In this paper, we study how these challenges can be alleviated with an automated robotic learning framework, in which multi-stage tasks are defined simply by providing videos of a human demonstrator and then learned autonomously by the robot from raw image observations. A central challenge in imitating human videos is the difference in appearance between the human and robot, which typically requires manual correspondence. We instead take an automated approach and perform pixel-level image translation via CycleGAN to convert the human demonstration into a video of a robot, which can then be used to construct a reward function for a model-based RL algorithm. The robot then learns the task one stage at a time, automatically learning how to reset each stage to retry it multiple times without human-provided resets. This makes the learning process largely automatic, from intuitive task specification via a video to automated training with minimal human intervention. We demonstrate that our approach is capable of learning complex tasks, such as operating a coffee machine, directly from raw image observations, requiring only 20 minutes to provide human demonstrations and about 180 minutes of robot interaction.