Risk-Aware Active Inverse Reinforcement Learning

TL;DR

This paper introduces a risk-aware active inverse reinforcement learning algorithm that strategically queries human input in high-risk areas, improving learning efficiency and safety in robotic tasks.

Contribution

It presents a novel active IRL method that minimizes policy performance risk by focusing queries on high-error regions, with a performance-based stopping criterion.

Findings

Outperforms standard active IRL in gridworld, driving, and table setting tasks.

Provides a safety-aware stopping criterion for demonstrations.

Demonstrates improved generalization and risk management.

Abstract

Active learning from demonstration allows a robot to query a human for specific types of input to achieve efficient learning. Existing work has explored a variety of active query strategies; however, to our knowledge, none of these strategies directly minimize the performance risk of the policy the robot is learning. Utilizing recent advances in performance bounds for inverse reinforcement learning, we propose a risk-aware active inverse reinforcement learning algorithm that focuses active queries on areas of the state space with the potential for large generalization error. We show that risk-aware active learning outperforms standard active IRL approaches on gridworld, simulated driving, and table setting tasks, while also providing a performance-based stopping criterion that allows a robot to know when it has received enough demonstrations to safely perform a task.