An Extensible and Modular Design and Implementation of Monte Carlo Tree Search for the JVM

TL;DR

This paper presents mctreesearch4j, a flexible, modular, and extensible Java library for Monte Carlo Tree Search that adapts to various Markov Decision Processes and game domains, emphasizing design principles and customization.

Contribution

The paper introduces a novel JVM-based MCTS library with a modular, object-oriented design that facilitates customization and adaptation to diverse decision process domains.

Findings

The library is easy to adapt for different heuristics and MDPs.

It offers reasonable performance and accuracy on standard MDPs.

The design clarifies different types of MCTS algorithms.

Abstract

Flexible implementations of Monte Carlo Tree Search (MCTS), combined with domain specific knowledge and hybridization with other search algorithms, can be powerful for finding the solutions to problems in complex planning. We introduce mctreesearch4j, an MCTS implementation written as a standard JVM library following key design principles of object oriented programming. We define key class abstractions allowing the MCTS library to flexibly adapt to any well defined Markov Decision Process or turn-based adversarial game. Furthermore, our library is designed to be modular and extensible, utilizing class inheritance and generic typing to standardize custom algorithm definitions. We demonstrate that the design of the MCTS implementation provides ease of adaptation for unique heuristics and customization across varying Markov Decision Process (MDP) domains. In addition, the implementation is reasonably performant and accurate for standard MDP's. In addition, via the implementation of mctreesearch4j, the nuances of different types of MCTS algorithms are discussed.