Compressed imitation learning

TL;DR

This paper introduces a novel imitation learning approach that leverages policy simplicity as a prior, inspired by compressed sensing, to achieve sample-efficient learning across linear, nonlinear, and partial observation scenarios.

Contribution

It extends the concept of compressed sensing to imitation learning by incorporating policy simplicity as a prior, improving sample efficiency over traditional methods.

Findings

Significantly higher scores than behavior cloning with limited demonstrations

Effective in linear, nonlinear, and partial observation scenarios

Demonstrates the feasibility of using policy simplicity as a prior

Abstract

In analogy to compressed sensing, which allows sample-efficient signal reconstruction given prior knowledge of its sparsity in frequency domain, we propose to utilize policy simplicity (Occam's Razor) as a prior to enable sample-efficient imitation learning. We first demonstrated the feasibility of this scheme on linear case where state-value function can be sampled directly. We also extended the scheme to scenarios where only actions are visible and scenarios where the policy is obtained from nonlinear network. The method is benchmarked against behavior cloning and results in significantly higher scores with limited expert demonstrations.