Monte Carlo Tree Search: A Review of Recent Modifications and Applications

TL;DR

This paper reviews recent modifications and applications of Monte Carlo Tree Search, highlighting domain-specific adaptations and hybrid approaches that enhance its effectiveness in complex games and practical decision-making problems.

Contribution

It provides a comprehensive overview of recent advances in MCTS, focusing on problem-dependent modifications and hybrid methods since 2012.

Findings

MCTS remains state-of-the-art for combinatorial games.

Domain-specific modifications improve efficiency in complex scenarios.

Hybrid approaches extend MCTS applicability to practical domains.

Abstract

Monte Carlo Tree Search (MCTS) is a powerful approach to designing game-playing bots or solving sequential decision problems. The method relies on intelligent tree search that balances exploration and exploitation. MCTS performs random sampling in the form of simulations and stores statistics of actions to make more educated choices in each subsequent iteration. The method has become a state-of-the-art technique for combinatorial games, however, in more complex games (e.g. those with high branching factor or real-time ones), as well as in various practical domains (e.g. transportation, scheduling or security) an efficient MCTS application often requires its problem-dependent modification or integration with other techniques. Such domain-specific modifications and hybrid approaches are the main focus of this survey. The last major MCTS survey has been published in 2012. Contributions that appeared since its release are of particular interest for this review.