Learning Performance Graphs from Demonstrations via Task-Based Evaluations

TL;DR

This paper introduces an algorithm to automatically learn performance graphs from demonstrations in robot learning, enabling more accurate evaluation of behaviors without manual specification, demonstrated through a highway driving domain.

Contribution

The paper presents a novel algorithm to learn performance graphs directly from demonstrations, reducing manual effort and improving evaluation accuracy in learning from demonstration frameworks.

Findings

Learned performance graphs match user-specified priorities in simulations.

Reward functions from learned graphs produce similar policies to manual graphs.

User study confirms accurate inference of behavior priorities.

Abstract

In the learning from demonstration (LfD) paradigm, understanding and evaluating the demonstrated behaviors plays a critical role in extracting control policies for robots. Without this knowledge, a robot may infer incorrect reward functions that lead to undesirable or unsafe control policies. Recent work has proposed an LfD framework where a user provides a set of formal task specifications to guide LfD, to address the challenge of reward shaping. However, in this framework, specifications are manually ordered in a performance graph (a partial order that specifies relative importance between the specifications). The main contribution of this paper is an algorithm to learn the performance graph directly from the user-provided demonstrations, and show that the reward functions generated using the learned performance graph generate similar policies to those from manually specified performance graphs. We perform a user study that shows that priorities specified by users on behaviors in a simulated highway driving domain match the automatically inferred performance graph. This establishes that we can accurately evaluate user demonstrations with respect to task specifications without expert criteria.