Perceiving the World: Question-guided Reinforcement Learning for Text-based Games

TL;DR

This paper introduces a question-guided reinforcement learning approach with world-perceiving modules and a two-phase training framework to enhance performance and sample efficiency in text-based game agents.

Contribution

It presents a novel method combining question-guided modules and a decoupled training framework to address challenges in sample efficiency and large action spaces in text-based games.

Findings

Significantly improves game-playing performance.

Enhances sample efficiency over baseline methods.

Robust against compound errors and limited pre-training data.

Abstract

Text-based games provide an interactive way to study natural language processing. While deep reinforcement learning has shown effectiveness in developing the game playing agent, the low sample efficiency and the large action space remain to be the two major challenges that hinder the DRL from being applied in the real world. In this paper, we address the challenges by introducing world-perceiving modules, which automatically decompose tasks and prune actions by answering questions about the environment. We then propose a two-phase training framework to decouple language learning from reinforcement learning, which further improves the sample efficiency. The experimental results show that the proposed method significantly improves the performance and sample efficiency. Besides, it shows robustness against compound error and limited pre-training data.