Interactive Robot Training for Non-Markov Tasks

TL;DR

This paper introduces a Bayesian interactive training framework for robots to learn non-Markov tasks from demonstrations and teacher feedback, improving learning efficiency and accuracy in real-world settings.

Contribution

It presents a novel active learning approach that combines demonstrations and teacher assessments to better infer task specifications for non-Markov tasks.

Findings

Active learning identifies task specifications with high accuracy.

The approach outperforms demonstration-only learning in similarity to intended tasks.

Successful real-world application in a dinner table setting.

Abstract

Defining sound and complete specifications for robots using formal languages is challenging, while learning formal specifications directly from demonstrations can lead to over-constrained task policies. In this paper, we propose a Bayesian interactive robot training framework that allows the robot to learn from both demonstrations provided by a teacher, and that teacher's assessments of the robot's task executions. We also present an active learning approach -- inspired by uncertainty sampling -- to identify the task execution with the most uncertain degree of acceptability. Through a simulated experiment, we demonstrate that our active learning approach identifies a teacher's intended task specification with an equivalent or greater similarity when compared to an approach that learns purely from demonstrations. Finally, we demonstrate the efficacy of our approach in a real-world setting through a user-study based on teaching a robot to set a dinner table.