Bayesian multitask inverse reinforcement learning

TL;DR

This paper extends inverse reinforcement learning to multiple tasks and demonstrations, using structured priors to model task relatedness and policy optimality, enabling efficient learning and differentiation of goals among multiple experts.

Contribution

It formalizes multi-task inverse reinforcement learning as statistical preference elicitation with novel priors, including a natural prior on policy optimality, improving learning from multiple experts.

Findings

Efficiently learns from multiple experts

Differentiates between different task goals

Introduces a prior on policy optimality

Abstract

We generalise the problem of inverse reinforcement learning to multiple tasks, from multiple demonstrations. Each one may represent one expert trying to solve a different task, or as different experts trying to solve the same task. Our main contribution is to formalise the problem as statistical preference elicitation, via a number of structured priors, whose form captures our biases about the relatedness of different tasks or expert policies. In doing so, we introduce a prior on policy optimality, which is more natural to specify. We show that our framework allows us not only to learn to efficiently from multiple experts but to also effectively differentiate between the goals of each. Possible applications include analysing the intrinsic motivations of subjects in behavioural experiments and learning from multiple teachers.