Online Adaptation for Enhancing Imitation Learning Policies

TL;DR

This paper introduces an online adaptation method that combines pre-trained policies with expert experience to improve imitation learning, especially in complex or poorly represented tasks, leading to better performance and robustness.

Contribution

The paper presents a novel online adaptation technique that enhances imitation learning policies by integrating expert experience, enabling recovery from failures and improving overall performance.

Findings

Adapted agents outperform pure imitation learning agents.

Adapted agents can succeed even when the base policy fails catastrophically.

The method improves robustness and generalization of imitation learning.

Abstract

Imitation learning enables autonomous agents to learn from human examples, without the need for a reward signal. Still, if the provided dataset does not encapsulate the task correctly, or when the task is too complex to be modeled, such agents fail to reproduce the expert policy. We propose to recover from these failures through online adaptation. Our approach combines the action proposal coming from a pre-trained policy with relevant experience recorded by an expert. The combination results in an adapted action that closely follows the expert. Our experiments show that an adapted agent performs better than its pure imitation learning counterpart. Notably, adapted agents can achieve reasonable performance even when the base, non-adapted policy catastrophically fails.