Stochastic Action Prediction for Imitation Learning

TL;DR

This paper investigates the inherent randomness in expert demonstrations for imitation learning and shows that modeling this stochasticity improves task success rates.

Contribution

It introduces methods to model stochasticity in demonstration data using autoregressive, GAN, and variational approaches, demonstrating their effectiveness.

Findings

Modeling stochasticity improves imitation learning success rates.

Autoregressive, GAN, and variational methods effectively capture data variability.

Accounting for stochasticity leads to substantial performance gains.

Abstract

Imitation learning is a data-driven approach to acquiring skills that relies on expert demonstrations to learn a policy that maps observations to actions. When performing demonstrations, experts are not always consistent and might accomplish the same task in slightly different ways. In this paper, we demonstrate inherent stochasticity in demonstrations collected for tasks including line following with a remote-controlled car and manipulation tasks including reaching, pushing, and picking and placing an object. We model stochasticity in the data distribution using autoregressive action generation, generative adversarial nets, and variational prediction and compare the performance of these approaches. We find that accounting for stochasticity in the expert data leads to substantial improvement in the success rate of task completion.