Class Teaching for Inverse Reinforcement Learners

TL;DR

This paper introduces a novel machine teaching algorithm for multiple inverse reinforcement learners, addressing the challenge of teaching heterogeneous groups with a single demonstration and analyzing conditions for its success.

Contribution

It formally defines the problem of teaching multiple inverse reinforcement learners, identifies conditions for single demonstration teaching, and proposes an effective algorithm for heterogeneous groups.

Findings

Single demonstration teaching may not be feasible with highly heterogeneous agents.

The proposed algorithm effectively ensures all agents learn a compatible task.

Heterogeneity impacts the success of a unified teaching demonstration.

Abstract

In this paper we propose the first machine teaching algorithm for multiple inverse reinforcement learners. Specifically, our contributions are: (i) we formally introduce the problem of teaching a sequential task to a heterogeneous group of learners; (ii) we identify conditions under which it is possible to conduct such teaching using the same demonstration for all learners; and (iii) we propose and evaluate a simple algorithm that computes a demonstration(s) ensuring that all agents in a heterogeneous class learn a task description that is compatible with the target task. Our analysis shows that, contrary to other teaching problems, teaching a heterogeneous class with a single demonstration may not be possible as the differences between agents increase. We also showcase the advantages of our proposed machine teaching approach against several possible alternatives.