Sample-efficient Adversarial Imitation Learning

TL;DR

This paper introduces a self-supervised representation-based adversarial imitation learning method that significantly improves sample efficiency and performance in non-image control tasks, requiring fewer expert demonstrations.

Contribution

It proposes a novel self-supervised learning approach with a new corruption method for robust state-action representations, enhancing imitation learning with fewer samples.

Findings

39% relative improvement over existing methods on MuJoCo

Effective with only 100 expert state-action pairs

Robust representations improve imitation performance

Abstract

Imitation learning, in which learning is performed by demonstration, has been studied and advanced for sequential decision-making tasks in which a reward function is not predefined. However, imitation learning methods still require numerous expert demonstration samples to successfully imitate an expert's behavior. To improve sample efficiency, we utilize self-supervised representation learning, which can generate vast training signals from the given data. In this study, we propose a self-supervised representation-based adversarial imitation learning method to learn state and action representations that are robust to diverse distortions and temporally predictive, on non-image control tasks. In particular, in comparison with existing self-supervised learning methods for tabular data, we propose a different corruption method for state and action representations that is robust to diverse distortions. We theoretically and empirically observe that making an informative feature manifold with less sample complexity significantly improves the performance of imitation learning. The proposed method shows a 39% relative improvement over existing adversarial imitation learning methods on MuJoCo in a setting limited to 100 expert state-action pairs. Moreover, we conduct comprehensive ablations and additional experiments using demonstrations with varying optimality to provide insights into a range of factors.