Towards a Deep Reinforcement Learning Approach for Tower Line Wars

TL;DR

This paper introduces a new game environment based on Tower Line Wars for Deep Reinforcement Learning research, demonstrating that a simplified Deep Q-Network can effectively learn and outperform standard methods in this setting.

Contribution

The paper presents a novel game environment for Deep Reinforcement Learning research and a simplified Deep Q-Network architecture that improves learning performance.

Findings

The environment effectively fosters Deep Reinforcement Learning research.

The proposed architecture scores 33% better than standard Deep Q-learning.

The environment balances complexity between simple and complex existing platforms.

Abstract

There have been numerous breakthroughs with reinforcement learning in the recent years, perhaps most notably on Deep Reinforcement Learning successfully playing and winning relatively advanced computer games. There is undoubtedly an anticipation that Deep Reinforcement Learning will play a major role when the first AI masters the complicated game plays needed to beat a professional Real-Time Strategy game player. For this to be possible, there needs to be a game environment that targets and fosters AI research, and specifically Deep Reinforcement Learning. Some game environments already exist, however, these are either overly simplistic such as Atari 2600 or complex such as Starcraft II from Blizzard Entertainment. We propose a game environment in between Atari 2600 and Starcraft II, particularly targeting Deep Reinforcement Learning algorithm research. The environment is a variant of Tower Line Wars from Warcraft III, Blizzard Entertainment. Further, as a proof of concept that the environment can harbor Deep Reinforcement algorithms, we propose and apply a Deep Q-Reinforcement architecture. The architecture simplifies the state space so that it is applicable to Q-learning, and in turn improves performance compared to current state-of-the-art methods. Our experiments show that the proposed architecture can learn to play the environment well, and score 33% better than standard Deep Q-learning which in turn proves the usefulness of the game environment.