Hindsight Generative Adversarial Imitation Learning

TL;DR

This paper introduces HGAIL, a novel imitation learning algorithm that eliminates the need for expert demonstrations by combining hindsight with GAIL, enabling effective policy training in data-scarce scenarios.

Contribution

The paper proposes HGAIL, a new imitation learning method that removes the requirement for expert demonstrations by integrating hindsight into the GAIL framework.

Findings

HGAIL achieves comparable performance to traditional imitation learning methods.

The method inherently incorporates curriculum learning, aiding policy development.

Experiments validate HGAIL's effectiveness in demonstration-free scenarios.

Abstract

Compared to reinforcement learning, imitation learning (IL) is a powerful paradigm for training agents to learn control policies efficiently from expert demonstrations. However, in most cases, obtaining demonstration data is costly and laborious, which poses a significant challenge in some scenarios. A promising alternative is to train agent learning skills via imitation learning without expert demonstrations, which, to some extent, would extremely expand imitation learning areas. To achieve such expectation, in this paper, we propose Hindsight Generative Adversarial Imitation Learning (HGAIL) algorithm, with the aim of achieving imitation learning satisfying no need of demonstrations. Combining hindsight idea with the generative adversarial imitation learning (GAIL) framework, we realize implementing imitation learning successfully in cases of expert demonstration data are not available. Experiments show that the proposed method can train policies showing comparable performance to current imitation learning methods. Further more, HGAIL essentially endows curriculum learning mechanism which is critical for learning policies.