Pragmatically Learning from Pedagogical Demonstrations in Multi-Goal Environments

TL;DR

This paper introduces a Bayesian model for goal inference that enhances learning efficiency in multi-goal environments by incorporating pedagogical and pragmatic mechanisms, especially effective with limited demonstrations.

Contribution

It presents a novel Bayesian Goal Inference model that integrates pedagogy and pragmatism, improving multi-goal learning from demonstrations with fewer examples.

Findings

Faster learning with BGI-agents compared to standard methods.

Reduced goal ambiguity in few demonstrations regimes.

Enhanced goal inference accuracy through pedagogical and pragmatic strategies.

Abstract

Learning from demonstration methods usually leverage close to optimal demonstrations to accelerate training. By contrast, when demonstrating a task, human teachers deviate from optimal demonstrations and pedagogically modify their behavior by giving demonstrations that best disambiguate the goal they want to demonstrate. Analogously, human learners excel at pragmatically inferring the intent of the teacher, facilitating communication between the two agents. These mechanisms are critical in the few demonstrations regime, where inferring the goal is more difficult. In this paper, we implement pedagogy and pragmatism mechanisms by leveraging a Bayesian model of Goal Inference from demonstrations (BGI). We highlight the benefits of this model in multi-goal teacher-learner setups with two artificial agents that learn with goal-conditioned Reinforcement Learning. We show that combining BGI-agents (a pedagogical teacher and a pragmatic learner) results in faster learning and reduced goal ambiguity over standard learning from demonstrations, especially in the few demonstrations regime. We provide the code for our experiments (https://github.com/Caselles/NeurIPS22-demonstrations-pedagogy-pragmatism), as well as an illustrative video explaining our approach (https://youtu.be/V4n16IjkNyw).