From Language to Goals: Inverse Reinforcement Learning for Vision-Based Instruction Following

TL;DR

This paper introduces language-conditioned reward learning (LC-RL), enabling machines to understand natural language commands as reward functions, which improves transferability to new tasks and environments in vision-based instruction following.

Contribution

The paper proposes LC-RL, a novel method that grounds language commands as reward functions using deep neural networks, enhancing transferability over language-conditioned policies.

Findings

LC-RL learns transferable reward functions for new tasks.

Language-conditioned rewards outperform policies in transfer scenarios.

Model demonstrates effectiveness in high-dimensional visual environments.

Abstract

Reinforcement learning is a promising framework for solving control problems, but its use in practical situations is hampered by the fact that reward functions are often difficult to engineer. Specifying goals and tasks for autonomous machines, such as robots, is a significant challenge: conventionally, reward functions and goal states have been used to communicate objectives. But people can communicate objectives to each other simply by describing or demonstrating them. How can we build learning algorithms that will allow us to tell machines what we want them to do? In this work, we investigate the problem of grounding language commands as reward functions using inverse reinforcement learning, and argue that language-conditioned rewards are more transferable than language-conditioned policies to new environments. We propose language-conditioned reward learning (LC-RL), which grounds language commands as a reward function represented by a deep neural network. We demonstrate that our model learns rewards that transfer to novel tasks and environments on realistic, high-dimensional visual environments with natural language commands, whereas directly learning a language-conditioned policy leads to poor performance.