Continuous Inverse Optimal Control with Locally Optimal Examples

TL;DR

This paper presents a scalable probabilistic inverse optimal control method that learns reward functions from locally optimal demonstrations in large, continuous domains, relaxing the need for global optimality assumptions.

Contribution

It introduces a local approximation approach for inverse optimal control that handles large, continuous spaces and non-globally optimal demonstrations.

Findings

Scales well with high-dimensional tasks

Learns from locally optimal, not necessarily globally optimal, demonstrations

Effective in large, continuous domains

Abstract

Inverse optimal control, also known as inverse reinforcement learning, is the problem of recovering an unknown reward function in a Markov decision process from expert demonstrations of the optimal policy. We introduce a probabilistic inverse optimal control algorithm that scales gracefully with task dimensionality, and is suitable for large, continuous domains where even computing a full policy is impractical. By using a local approximation of the reward function, our method can also drop the assumption that the demonstrations are globally optimal, requiring only local optimality. This allows it to learn from examples that are unsuitable for prior methods.