BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning

TL;DR

This paper presents BC-Z, a vision-based robotic imitation learning system capable of generalizing to new tasks by leveraging large-scale data and diverse task conditioning, achieving notable success without direct demonstrations.

Contribution

Introduces BC-Z, a scalable imitation learning framework that generalizes to novel tasks using diverse data and task conditioning, without requiring demonstrations for new tasks.

Findings

System learned from over 100 tasks

Achieved 44% success on 24 unseen tasks

Effective use of natural language and video embeddings

Abstract

In this paper, we study the problem of enabling a vision-based robotic manipulation system to generalize to novel tasks, a long-standing challenge in robot learning. We approach the challenge from an imitation learning perspective, aiming to study how scaling and broadening the data collected can facilitate such generalization. To that end, we develop an interactive and flexible imitation learning system that can learn from both demonstrations and interventions and can be conditioned on different forms of information that convey the task, including pre-trained embeddings of natural language or videos of humans performing the task. When scaling data collection on a real robot to more than 100 distinct tasks, we find that this system can perform 24 unseen manipulation tasks with an average success rate of 44%, without any robot demonstrations for those tasks.