Bilevel MCTS for Amortized O(1) Node Selection in Classical Planning

TL;DR

This paper introduces a bilevel MCTS approach with Tree Collapsing that achieves amortized O(1) node selection time in classical planning, significantly improving efficiency over traditional methods.

Contribution

It presents a novel bilevel modification to MCTS and a Tree Collapsing technique to reduce node selection complexity in classical planning.

Findings

Achieves amortized O(1) node selection runtime.

Reduces action selection steps with Tree Collapsing.

Improves planning efficiency in large-depth problems.

Abstract

We study an efficient implementation of Multi-Armed Bandit (MAB)-based Monte-Carlo Tree Search (MCTS) for classical planning. One weakness of MCTS is that it spends a significant time deciding which node to expand next. While selecting a node from an OPEN list with $N$ nodes has $O(1)$ runtime complexity with traditional array-based priority-queues for dense integer keys, the tree-based OPEN list used by MCTS requires $O(\log N)$, which roughly corresponds to the search depth $d$. In classical planning, $d$ is arbitrarily large (e.g., $2^k-1$ in $k$-disk Tower-of-Hanoi) and the runtime for node selection is significant, unlike in game tree search, where the cost is negligible compared to the node evaluation (rollouts) because $d$ is inherently limited by the game (e.g., $d\leq 361$ in Go). To improve this bottleneck, we propose a bilevel modification to MCTS that runs a best-first search from each selected leaf node with an expansion budget proportional to $d$, which achieves amortized $O(1)$ runtime for node selection, equivalent to the traditional queue-based OPEN list. In addition, we introduce Tree Collapsing, an enhancement that reduces action selection steps and further improves the performance.