Explorative Imitation Learning: A Path Signature Approach for Continuous Environments

TL;DR

This paper introduces CILO, a novel imitation learning method that enhances exploration and encodes constraints via path signatures, reducing reliance on expert data and improving performance across multiple environments.

Contribution

CILO combines exploration and path signature encoding to improve imitation learning efficiency and effectiveness with fewer expert trajectories.

Findings

CILO outperforms baseline and leading methods in five environments.

CILO surpasses expert performance in two environments.

CILO requires fewer training iterations and expert trajectories.

Abstract

Some imitation learning methods combine behavioural cloning with self-supervision to infer actions from state pairs. However, most rely on a large number of expert trajectories to increase generalisation and human intervention to capture key aspects of the problem, such as domain constraints. In this paper, we propose Continuous Imitation Learning from Observation (CILO), a new method augmenting imitation learning with two important features: (i) exploration, allowing for more diverse state transitions, requiring less expert trajectories and resulting in fewer training iterations; and (ii) path signatures, allowing for automatic encoding of constraints, through the creation of non-parametric representations of agents and expert trajectories. We compared CILO with a baseline and two leading imitation learning methods in five environments. It had the best overall performance of all methods in all environments, outperforming the expert in two of them.