Goal-based Self-Adaptive Generative Adversarial Imitation Learning (Goal-SAGAIL) for Multi-goal Robotic Manipulation Tasks

TL;DR

This paper introduces Goal-SAGAIL, a novel goal-conditioned imitation learning framework that improves multi-goal robotic manipulation learning efficiency, especially with limited or suboptimal demonstrations, outperforming existing methods in complex tasks.

Contribution

The paper proposes Goal-SAGAIL, integrating self-adaptive learning with goal-conditioned GAIL to enhance imitation learning in multi-goal robot manipulation tasks.

Findings

Significantly improved learning efficiency in complex manipulation tasks.

Effective with limited and suboptimal demonstration data.

Validated on both simulation and real-world scenarios.

Abstract

Reinforcement learning for multi-goal robot manipulation tasks poses significant challenges due to the diversity and complexity of the goal space. Techniques such as Hindsight Experience Replay (HER) have been introduced to improve learning efficiency for such tasks. More recently, researchers have combined HER with advanced imitation learning methods such as Generative Adversarial Imitation Learning (GAIL) to integrate demonstration data and accelerate training speed. However, demonstration data often fails to provide enough coverage for the goal space, especially when acquired from human teleoperation. This biases the learning-from-demonstration process toward mastering easier sub-tasks instead of tackling the more challenging ones. In this work, we present Goal-based Self-Adaptive Generative Adversarial Imitation Learning (Goal-SAGAIL), a novel framework specifically designed for multi-goal robot manipulation tasks. By integrating self-adaptive learning principles with goal-conditioned GAIL, our approach enhances imitation learning efficiency, even when limited, suboptimal demonstrations are available. Experimental results validate that our method significantly improves learning efficiency across various multi-goal manipulation scenarios -- including complex in-hand manipulation tasks -- using suboptimal demonstrations provided by both simulation and human experts.