Deep Reinforcement Learning with a Natural Language Action Space

TL;DR

This paper presents a deep reinforcement learning architecture that effectively handles natural language state and action spaces in text-based games, demonstrating superior performance and meaningful language understanding.

Contribution

Introduces the DRRN architecture that models natural language actions and states separately, improving learning and generalization in text-based reinforcement learning tasks.

Findings

DRRN outperforms other deep Q-learning models in text games

Model captures semantic meaning beyond string memorization

Effective handling of paraphrased action descriptions

Abstract

This paper introduces a novel architecture for reinforcement learning with deep neural networks designed to handle state and action spaces characterized by natural language, as found in text-based games. Termed a deep reinforcement relevance network (DRRN), the architecture represents action and state spaces with separate embedding vectors, which are combined with an interaction function to approximate the Q-function in reinforcement learning. We evaluate the DRRN on two popular text games, showing superior performance over other deep Q-learning architectures. Experiments with paraphrased action descriptions show that the model is extracting meaning rather than simply memorizing strings of text.