Time-critical and confidence-based abstraction dropping methods

TL;DR

This paper introduces two novel abstraction dropping schemes for Monte Carlo Tree Search that improve performance and safety, especially in time-critical and fixed-iteration scenarios.

Contribution

The paper proposes OGA-IAAD and OGA-CAD, two new abstraction dropping methods that outperform previous approaches while ensuring no performance degradation.

Findings

OGA-IAAD enhances time-critical MCTS performance.

OGA-CAD improves MCTS efficiency with fixed iterations.

Both methods outperform Xu's abstraction dropping approach.

Abstract

One paradigm of Monte Carlo Tree Search (MCTS) improvements is to build and use state and/or action abstractions during the tree search. Non-exact abstractions, however, introduce an approximation error making convergence to the optimal action in the abstract space impossible. Hence, as proposed as a component of Elastic Monte Carlo Tree Search by Xu et al., abstraction algorithms should eventually drop the abstraction. In this paper, we propose two novel abstraction dropping schemes, namely OGA-IAAD and OGA-CAD which can yield clear performance improvements whilst being safe in the sense that the dropping never causes any notable performance degradations contrary to Xu's dropping method. OGA-IAAD is designed for time critical settings while OGA-CAD is designed to improve the MCTS performance with the same number of iterations.