Towards Improving Learning from Demonstration Algorithms via MCMC Methods

TL;DR

This paper proposes enhancing learning from demonstration algorithms by using implicit energy-based policy models, which outperform traditional neural network models in complex robot policy learning scenarios, especially with discontinuous and multimodal functions.

Contribution

It introduces the use of implicit energy-based models for learning from demonstration, showing improved performance over explicit neural network models in complex tasks.

Findings

Implicit models outperform explicit neural networks in complex scenarios.

Energy-based policies better handle discontinuous and multimodal functions.

Results demonstrate improved learning efficiency and accuracy.

Abstract

Behavioral cloning, or more broadly, learning from demonstrations (LfD) is a priomising direction for robot policy learning in complex scenarios. Albeit being straightforward to implement and data-efficient, behavioral cloning has its own drawbacks, limiting its efficacy in real robot setups. In this work, we take one step towards improving learning from demonstration algorithms by leveraging implicit energy-based policy models. Results suggest that in selected complex robot policy learning scenarios, treating supervised policy learning with an implicit model generally performs better, on average, than commonly used neural network-based explicit models, especially in the cases of approximating potentially discontinuous and multimodal functions.