ADAIL: Adaptive Adversarial Imitation Learning

TL;DR

ADAIL introduces an adaptive adversarial imitation learning algorithm that enables policies to transfer across environments with different dynamics by using a dynamics embedding and domain-adversarial training, demonstrated on simulated control tasks.

Contribution

The paper proposes a novel method for adaptive imitation learning that generalizes policies across varying dynamics using adversarial training and dynamics embeddings.

Findings

Outperforms recent baselines in simulated control tasks with varying dynamics

Learns dynamics-invariant policies effective across multiple environments

Demonstrates robustness in transferring learned policies to new dynamics

Abstract

We present the ADaptive Adversarial Imitation Learning (ADAIL) algorithm for learning adaptive policies that can be transferred between environments of varying dynamics, by imitating a small number of demonstrations collected from a single source domain. This is an important problem in robotic learning because in real world scenarios 1) reward functions are hard to obtain, 2) learned policies from one domain are difficult to deploy in another due to varying source to target domain statistics, 3) collecting expert demonstrations in multiple environments where the dynamics are known and controlled is often infeasible. We address these constraints by building upon recent advances in adversarial imitation learning; we condition our policy on a learned dynamics embedding and we employ a domain-adversarial loss to learn a dynamics-invariant discriminator. The effectiveness of our method is demonstrated on simulated control tasks with varying environment dynamics and the learned adaptive agent outperforms several recent baselines.