Diverse Imitation Learning via Self-Organizing Generative Models

TL;DR

This paper introduces a novel encoder-free generative model for imitation learning that effectively captures diverse behaviors and improves robustness, outperforming existing methods in multiple experiments.

Contribution

It proposes an encoder-free generative approach combined with GAIL to better imitate multiple expert behaviors and reduce compounding errors.

Findings

Significantly outperforms state-of-the-art methods

Effectively distinguishes and imitates different behavior modes

Improves robustness against unseen states

Abstract

Imitation learning is the task of replicating expert policy from demonstrations, without access to a reward function. This task becomes particularly challenging when the expert exhibits a mixture of behaviors. Prior work has introduced latent variables to model variations of the expert policy. However, our experiments show that the existing works do not exhibit appropriate imitation of individual modes. To tackle this problem, we adopt an encoder-free generative model for behavior cloning (BC) to accurately distinguish and imitate different modes. Then, we integrate it with GAIL to make the learning robust towards compounding errors at unseen states. We show that our method significantly outperforms the state of the art across multiple experiments.