Dual Monte Carlo Tree Search

TL;DR

This paper introduces Dual MCTS, a neural Monte Carlo Tree Search algorithm that reduces computational requirements and converges faster than AlphaZero by using two search trees and a novel update technique.

Contribution

The paper proposes Dual MCTS, a new neural MCTS algorithm that improves efficiency and performance over AlphaZero through innovative tree update methods.

Findings

Dual MCTS outperforms AlphaZero in various games.

The new update technique reduces the number of tree updates.

Dual MCTS converges faster with less computational power.

Abstract

AlphaZero, using a combination of Deep Neural Networks and Monte Carlo Tree Search (MCTS), has successfully trained reinforcement learning agents in a tabula-rasa way. The neural MCTS algorithm has been successful in finding near-optimal strategies for games through self-play. However, the AlphaZero algorithm has a significant drawback; it takes a long time to converge and requires high computational power due to complex neural networks for solving games like Chess, Go, Shogi, etc. Owing to this, it is very difficult to pursue neural MCTS research without cutting-edge hardware, which is a roadblock for many aspiring neural MCTS researchers. In this paper, we propose a new neural MCTS algorithm, called Dual MCTS, which helps overcome these drawbacks. Dual MCTS uses two different search trees, a single deep neural network, and a new update technique for the search trees using a combination of the PUCB, a sliding-window, and the epsilon-greedy algorithm. This technique is applicable to any MCTS based algorithm to reduce the number of updates to the tree. We show that Dual MCTS performs better than one of the most widely used neural MCTS algorithms, AlphaZero, for various symmetric and asymmetric games.