Gaussian Process Aggregation for Root-Parallel Monte Carlo Tree Search with Continuous Actions

TL;DR

This paper introduces a Gaussian Process-based method for aggregating statistics in root-parallel Monte Carlo Tree Search with continuous actions, improving performance across multiple domains.

Contribution

The authors propose a novel Gaussian Process Regression approach to estimate values for untried actions, enhancing aggregation in parallel MCTS with continuous action spaces.

Findings

Outperforms existing aggregation strategies in 6 domains

Requires only a modest increase in inference time

Effective in environments with continuous action spaces

Abstract

Monte Carlo Tree Search is a cornerstone algorithm for online planning, and its root-parallel variant is widely used when wall clock time is limited but best performance is desired. In environments with continuous action spaces, how to best aggregate statistics from different threads is an important yet underexplored question. In this work, we introduce a method that uses Gaussian Process Regression to obtain value estimates for promising actions that were not trialed in the environment. We perform a systematic evaluation across 6 different domains, demonstrating that our approach outperforms existing aggregation strategies while requiring a modest increase in inference time.