Generalized Proof-Number Monte-Carlo Tree Search

TL;DR

This paper introduces a generalized Proof-Number Monte-Carlo Tree Search that improves search efficiency and applicability to multi-player games by tracking proof numbers per player, biasing selection strategies, and integrating score bounds, resulting in significant performance gains.

Contribution

It proposes three key modifications to combine proof-number search with MCTS, including multi-player proof tracking, new biasing methods, and integration with Score Bounded MCTS, enhancing performance and generality.

Findings

Achieved up to 80% performance improvement on 8 of 11 board games.

Simplified implementation by tracking proof numbers per player.

Enabled use of score bounds for more effective proof and pruning.

Abstract

This paper presents Generalized Proof-Number Monte-Carlo Tree Search: a generalization of recently proposed combinations of Proof-Number Search (PNS) with Monte-Carlo Tree Search (MCTS), which use (dis)proof numbers to bias UCB1-based Selection strategies towards parts of the search that are expected to be easily (dis)proven. We propose three core modifications of prior combinations of PNS with MCTS. First, we track proof numbers per player. This reduces code complexity in the sense that we no longer need disproof numbers, and generalizes the technique to be applicable to games with more than two players. Second, we propose and extensively evaluate different methods of using proof numbers to bias the selection strategy, achieving strong performance with strategies that are simpler to implement and compute. Third, we merge our technique with Score Bounded MCTS, enabling the algorithm to prove and leverage upper and lower bounds on scores - as opposed to only proving wins or not-wins. Experiments demonstrate substantial performance increases, reaching the range of 80% for 8 out of the 11 tested board games.