Overcoming Referential Ambiguity in Language-Guided Goal-Conditioned Reinforcement Learning

TL;DR

This paper explores how cognitive science concepts like pedagogy and pragmatism can reduce referential ambiguity in language-guided reinforcement learning, improving training efficiency in robotic tasks.

Contribution

It introduces a novel approach applying pedagogy and pragmatism to resolve referential ambiguity in language-based RL tasks, enhancing sample efficiency.

Findings

Pedagogy and pragmatism improve learning efficiency.

The approach reduces misunderstandings in language instructions.

Sample efficiency increases in simulated robotic tasks.

Abstract

Teaching an agent to perform new tasks using natural language can easily be hindered by ambiguities in interpretation. When a teacher provides an instruction to a learner about an object by referring to its features, the learner can misunderstand the teacher's intentions, for instance if the instruction ambiguously refer to features of the object, a phenomenon called referential ambiguity. We study how two concepts derived from cognitive sciences can help resolve those referential ambiguities: pedagogy (selecting the right instructions) and pragmatism (learning the preferences of the other agents using inductive reasoning). We apply those ideas to a teacher/learner setup with two artificial agents on a simulated robotic task (block-stacking). We show that these concepts improve sample efficiency for training the learner.