Language-Conditioned Reinforcement Learning to Solve Misunderstandings with Action Corrections

TL;DR

This paper introduces a reinforcement learning framework for robots to incrementally resolve misunderstandings in language instructions through action corrections, emphasizing the importance of the ongoing process of establishing understanding.

Contribution

It formalizes and experimentally validates incremental action-repair in language-conditioned reinforcement learning, and provides benchmark environments for evaluating such methods.

Findings

RL agents can learn to understand incremental language corrections

Benchmark environments facilitate evaluation of action correction methods

Demonstrates the importance of incremental understanding in robotic instruction-following

Abstract

Human-to-human conversation is not just talking and listening. It is an incremental process where participants continually establish a common understanding to rule out misunderstandings. Current language understanding methods for intelligent robots do not consider this. There exist numerous approaches considering non-understandings, but they ignore the incremental process of resolving misunderstandings. In this article, we present a first formalization and experimental validation of incremental action-repair for robotic instruction-following based on reinforcement learning. To evaluate our approach, we propose a collection of benchmark environments for action correction in language-conditioned reinforcement learning, utilizing a synthetic instructor to generate language goals and their corresponding corrections. We show that a reinforcement learning agent can successfully learn to understand incremental corrections of misunderstood instructions.