Split Moves for Monte-Carlo Tree Search

TL;DR

This paper explores how dividing complex moves into simpler sub-moves, called split moves, affects the performance of Monte-Carlo Tree Search agents across various board games, proposing a generalized MCTS framework for split moves.

Contribution

It introduces a generalized MCTS algorithm capable of handling arbitrarily split moves and evaluates the impact of split strategies on agent efficiency and strength.

Findings

Split moves can significantly improve agent performance.

The impact varies depending on move granularity and game type.

Split strategies enhance both single- and multi-action game agents.

Abstract

In many games, moves consist of several decisions made by the player. These decisions can be viewed as separate moves, which is already a common practice in multi-action games for efficiency reasons. Such division of a player move into a sequence of simpler / lower level moves is called \emph{splitting}. So far, split moves have been applied only in forementioned straightforward cases, and furthermore, there was almost no study revealing its impact on agents' playing strength. Taking the knowledge-free perspective, we aim to answer how to effectively use split moves within Monte-Carlo Tree Search (MCTS) and what is the practical impact of split design on agents' strength. This paper proposes a generalization of MCTS that works with arbitrarily split moves. We design several variations of the algorithm and try to measure the impact of split moves separately on efficiency, quality of MCTS, simulations, and action-based heuristics. The tests are carried out on a set of board games and performed using the Regular Boardgames General Game Playing formalism, where split strategies of different granularity can be automatically derived based on an abstract description of the game. The results give an overview of the behavior of agents using split design in different ways. We conclude that split design can be greatly beneficial for single- as well as multi-action games.