Training People to Reward Robots

TL;DR

This paper explores how machine teaching can guide novice teachers to improve their demonstration quality in robot learning, significantly enhancing robot performance on both trained and unseen skills.

Contribution

It introduces a method using machine teaching to train novices in demonstration tasks, leading to substantial improvements in robot learning performance.

Findings

89% improvement on trained skills

70% improvement on unseen skills

MT-guidance enhances human teaching effectiveness

Abstract

Learning from demonstration (LfD) is a technique that allows expert teachers to teach task-oriented skills to robotic systems. However, the most effective way of guiding novice teachers to approach expert-level demonstrations quantitatively for specific teaching tasks remains an open question. To this end, this paper investigates the use of machine teaching (MT) to guide novice teachers to improve their teaching skills based on reinforcement learning from demonstration (RLfD). The paper reports an experiment in which novices receive MT-derived guidance to train their ability to teach a given motor skill with only 8 demonstrations and generalise this to previously unseen ones. Results indicate that the MT-guidance not only enhances robot learning performance by 89% on the training skill but also causes a 70% improvement in robot learning performance on skills not seen by subjects during training. These findings highlight the effectiveness of MT-guidance in upskilling human teaching behaviours, ultimately improving demonstration quality in RLfD.