Zero-Shot Visual Imitation

TL;DR

This paper introduces a zero-shot imitation learning method where an agent learns to imitate tasks from visual demonstrations without expert action data, using exploration and goal-conditioned policies, applicable to robotics and navigation.

Contribution

The authors propose a novel framework for zero-shot visual imitation that relies on exploration and goal communication, eliminating the need for expert action supervision during training and inference.

Findings

Effective in complex rope manipulation tasks with a Baxter robot.

Successful navigation in unseen office environments with TurtleBot.

Improved policy performance through enhanced exploration mechanisms.

Abstract

The current dominant paradigm for imitation learning relies on strong supervision of expert actions to learn both 'what' and 'how' to imitate. We pursue an alternative paradigm wherein an agent first explores the world without any expert supervision and then distills its experience into a goal-conditioned skill policy with a novel forward consistency loss. In our framework, the role of the expert is only to communicate the goals (i.e., what to imitate) during inference. The learned policy is then employed to mimic the expert (i.e., how to imitate) after seeing just a sequence of images demonstrating the desired task. Our method is 'zero-shot' in the sense that the agent never has access to expert actions during training or for the task demonstration at inference. We evaluate our zero-shot imitator in two real-world settings: complex rope manipulation with a Baxter robot and navigation in previously unseen office environments with a TurtleBot. Through further experiments in VizDoom simulation, we provide evidence that better mechanisms for exploration lead to learning a more capable policy which in turn improves end task performance. Videos, models, and more details are available at https://pathak22.github.io/zeroshot-imitation/