Batch Monte Carlo Tree Search

TL;DR

This paper introduces batched Monte Carlo Tree Search algorithms that leverage GPU efficiency for neural network inferences, combining search trees and transposition tables, and evaluates several heuristics in the game of Go.

Contribution

It proposes a novel MCTS algorithm using both search trees and transposition tables with batch inference, and analyzes multiple heuristics to enhance search performance.

Findings

Batched MCTS significantly speeds up inference on GPUs.

Combining search trees with transposition tables improves search efficiency.

Heuristics like the $PU$, Virtual Mean, Last Iteration, and Second Move enhance Go gameplay.

Abstract

Making inferences with a deep neural network on a batch of states is much faster with a GPU than making inferences on one state after another. We build on this property to propose Monte Carlo Tree Search algorithms using batched inferences. Instead of using either a search tree or a transposition table we propose to use both in the same algorithm. The transposition table contains the results of the inferences while the search tree contains the statistics of Monte Carlo Tree Search. We also propose to analyze multiple heuristics that improve the search: the $\mu$ FPU, the Virtual Mean, the Last Iteration and the Second Move heuristics. They are evaluated for the game of Go using a MobileNet neural network.