Supervised Reward Inference

TL;DR

This paper introduces a supervised learning framework for reward inference from diverse human behaviors, including suboptimal actions, demonstrating asymptotic Bayes-optimality and effectiveness in robotic tasks.

Contribution

It proposes a unified supervised learning approach for reward inference that handles arbitrary behaviors and proves its asymptotic optimality under mild conditions.

Findings

Efficient reward inference from suboptimal demonstrations

Method achieves asymptotic Bayes-optimality

Effective in simulated robotic manipulation tasks

Abstract

Existing approaches to reward inference from behavior typically assume that humans provide demonstrations according to specific models of behavior. However, humans often indicate their goals through a wide range of behaviors, from actions that are suboptimal due to poor planning or execution to behaviors which are intended to communicate goals rather than achieve them. We propose that supervised learning offers a unified framework to infer reward functions from any class of behavior, and show that such an approach is asymptotically Bayes-optimal under mild assumptions. Experiments on simulated robotic manipulation tasks show that our method can efficiently infer rewards from a wide variety of arbitrarily suboptimal demonstrations.