Language-Conditioned Goal Generation: a New Approach to Language Grounding for RL

TL;DR

This paper introduces a novel approach where language conditions goal generators in reinforcement learning, enabling agents to produce diverse, language-agnostic goals and decouple sensorimotor learning from language understanding.

Contribution

It proposes using language to condition goal generators, allowing for diverse goal generation independent of language, and demonstrates benefits over traditional instruction-following methods.

Findings

Decouples sensorimotor learning from language acquisition.

Enables agents to generate diverse goals based on instructions.

Improves flexibility in language-conditioned reinforcement learning.

Abstract

In the real world, linguistic agents are also embodied agents: they perceive and act in the physical world. The notion of Language Grounding questions the interactions between language and embodiment: how do learning agents connect or ground linguistic representations to the physical world ? This question has recently been approached by the Reinforcement Learning community under the framework of instruction-following agents. In these agents, behavioral policies or reward functions are conditioned on the embedding of an instruction expressed in natural language. This paper proposes another approach: using language to condition goal generators. Given any goal-conditioned policy, one could train a language-conditioned goal generator to generate language-agnostic goals for the agent. This method allows to decouple sensorimotor learning from language acquisition and enable agents to demonstrate a diversity of behaviors for any given instruction. We propose a particular instantiation of this approach and demonstrate its benefits.