The Text-Based Adventure AI Competition

TL;DR

This paper presents a competition framework for AI agents to play text-based adventure games, serving as a new benchmark that emphasizes natural language understanding and generation challenges.

Contribution

It introduces a novel competition platform for text-based game AI, including open source tools and an evaluation methodology across multiple years and games.

Findings

Improved evaluation methods for text-based game agents

Open source competition framework available for research

Competitive agents demonstrate varying levels of success across games

Abstract

In 2016, 2017, and 2018 at the IEEE Conference on Computational Intelligence in Games, the authors of this paper ran a competition for agents that can play classic text-based adventure games. This competition fills a gap in existing game AI competitions that have typically focussed on traditional card/board games or modern video games with graphical interfaces. By providing a platform for evaluating agents in text-based adventures, the competition provides a novel benchmark for game AI with unique challenges for natural language understanding and generation. This paper summarises the three competitions ran in 2016, 2017, and 2018 (including details of open source implementations of both the competition framework and our competitors) and presents the results of an improved evaluation of these competitors across 20 games.