Cost Function Estimation Using Inverse Reinforcement Learning with Minimal Observations

TL;DR

This paper introduces an efficient inverse reinforcement learning algorithm that infers cost functions from minimal observations in continuous spaces, using optimal control-generated trajectories and feature tuning for faster, more accurate learning.

Contribution

It proposes a novel iterative method that individually tunes observation effectiveness and avoids large sample sets, improving speed and accuracy over existing algorithms.

Findings

Faster learning with fewer observations.

More accurate cost function inference.

Effective in multiple simulated environments.

Abstract

We present an iterative inverse reinforcement learning algorithm to infer optimal cost functions in continuous spaces. Based on a popular maximum entropy criteria, our approach iteratively finds a weight improvement step and proposes a method to find an appropriate step size that ensures learned cost function features remain similar to the demonstrated trajectory features. In contrast to similar approaches, our algorithm can individually tune the effectiveness of each observation for the partition function and does not need a large sample set, enabling faster learning. We generate sample trajectories by solving an optimal control problem instead of random sampling, leading to more informative trajectories. The performance of our method is compared to two state of the art algorithms to demonstrate its benefits in several simulated environments.