Zero-Shot Transfer in Imitation Learning

TL;DR

This paper introduces a zero-shot transfer algorithm for imitation learning that leverages disentangled representations and a single Q-function, enabling domain transfer without retraining, which is crucial for real-world robotic applications.

Contribution

The paper proposes a novel imitation learning method combining AnnealedVAE for disentangled states and a single Q-function for transfer, avoiding adversarial training.

Findings

Effective transfer across three diverse environments

Avoids retraining in new domains

Utilizes disentangled representations for robust transfer

Abstract

We present an algorithm that learns to imitate expert behavior and can transfer to previously unseen domains without retraining. Such an algorithm is extremely relevant in real-world applications such as robotic learning because 1) reward functions are difficult to design, 2) learned policies from one domain are difficult to deploy in another domain and 3) learning directly in the real world is either expensive or unfeasible due to security concerns. To overcome these constraints, we combine recent advances in Deep RL by using an AnnealedVAE to learn a disentangled state representation and imitate an expert by learning a single Q-function which avoids adversarial training. We demonstrate the effectiveness of our method in 3 environments ranging in difficulty and the type of transfer knowledge required.