Diverse, Top-k, and Top-Quality Planning Over Simulators

TL;DR

This paper introduces a Monte Carlo Tree Search-based method for generating diverse, top-k, and high-quality solution sets in sequential decision problems, especially where classical symbolic planners cannot be used.

Contribution

It presents a novel MCTS-based approach for diverse and top-quality planning over black-box simulators, with a procedure for extracting solution sets and measuring their quality.

Findings

Successfully applied to path planning with hidden information

Generated diverse, high-quality plan sets in non-symbolic domains

Enhanced MCTS to improve plan diversity

Abstract

Diverse, top-k, and top-quality planning are concerned with the generation of sets of solutions to sequential decision problems. Previously this area has been the domain of classical planners that require a symbolic model of the problem instance. This paper proposes a novel alternative approach that uses Monte Carlo Tree Search (MCTS), enabling application to problems for which only a black-box simulation model is available. We present a procedure for extracting bounded sets of plans from pre-generated search trees in best-first order, and a metric for evaluating the relative quality of paths through a search tree. We demonstrate this approach on a path-planning problem with hidden information, and suggest adaptations to the MCTS algorithm to increase the diversity of generated plans. Our results show that our method can generate diverse and high-quality plan sets in domains where classical planners are not applicable.