Reinforcement Learning for High-Level Strategic Control in Tower Defense Games

TL;DR

This paper introduces an automated gameplay testing method combining reinforcement learning with scripted AI to improve robustness and performance in tower defense games, exemplified by Plants vs. Zombies.

Contribution

It presents a novel hybrid approach that integrates reinforcement learning with scripted AI for game testing and validation, enhancing agent performance and robustness.

Findings

Reinforcement learning combined with scripted AI outperforms heuristic AI.

Achieved a success rate of 57.12% versus 47.95%.

Demonstrated challenges in training a general agent for puzzle-like games.

Abstract

In strategy games, one of the most important aspects of game design is maintaining a sense of challenge for players. Many mobile titles feature quick gameplay loops that allow players to progress steadily, requiring an abundance of levels and puzzles to prevent them from reaching the end too quickly. As with any content creation, testing and validation are essential to ensure engaging gameplay mechanics, enjoyable game assets, and playable levels. In this paper, we propose an automated approach that can be leveraged for gameplay testing and validation that combines traditional scripted methods with reinforcement learning, reaping the benefits of both approaches while adapting to new situations similarly to how a human player would. We test our solution on a popular tower defense game, Plants vs. Zombies. The results show that combining a learned approach, such as reinforcement learning, with a scripted AI produces a higher-performing and more robust agent than using only heuristic AI, achieving a 57.12% success rate compared to 47.95% in a set of 40 levels. Moreover, the results demonstrate the difficulty of training a general agent for this type of puzzle-like game.