Algorithmic Improvements for Deep Reinforcement Learning applied to Interactive Fiction

TL;DR

This paper introduces algorithmic enhancements for deep reinforcement learning agents to better handle the unique challenges of text-based interactive fiction games, focusing on feedback-driven learning and structural game features.

Contribution

It proposes a contextualisation mechanism and action selection methods tailored for text-based games, improving agent performance in complex, partially observable environments.

Findings

Techniques improve baseline agent performance

Methods handle large state and action spaces

Approach effective in diverse text-based games

Abstract

Text-based games are a natural challenge domain for deep reinforcement learning algorithms. Their state and action spaces are combinatorially large, their reward function is sparse, and they are partially observable: the agent is informed of the consequences of its actions through textual feedback. In this paper we emphasize this latter point and consider the design of a deep reinforcement learning agent that can play from feedback alone. Our design recognizes and takes advantage of the structural characteristics of text-based games. We first propose a contextualisation mechanism, based on accumulated reward, which simplifies the learning problem and mitigates partial observability. We then study different methods that rely on the notion that most actions are ineffectual in any given situation, following Zahavy et al.'s idea of an admissible action. We evaluate these techniques in a series of text-based games of increasing difficulty based on the TextWorld framework, as well as the iconic game Zork. Empirically, we find that these techniques improve the performance of a baseline deep reinforcement learning agent applied to text-based games.