Solving Stochastic Orienteering Problems with Chance Constraints Using Monte Carlo Tree Search

TL;DR

This paper introduces an online Monte Carlo Tree Search algorithm tailored for stochastic orienteering problems with chance constraints, effectively balancing solution quality and computational efficiency through probabilistic pruning.

Contribution

The paper presents a novel MCTS algorithm that estimates violation probabilities and prunes trajectories, improving solution quality for stochastic problems with chance constraints.

Findings

Algorithm produces high-quality solutions quickly.

Competitive with optimal solutions but less computationally intensive.

Effective in managing probabilistic constraints in stochastic planning.

Abstract

We present a new Monte Carlo Tree Search (MCTS) algorithm to solve the stochastic orienteering problem with chance constraints, i.e., a version of the problem where travel costs are random, and one is assigned a bound on the tolerable probability of exceeding the budget. The algorithm we present is online and anytime, i.e., it alternates planning and execution, and the quality of the solution it produces increases as the allowed computational time increases. Differently from most former MCTS algorithms, for each action available in a state the algorithm maintains estimates of both its value and the probability that its execution will eventually result in a violation of the chance constraint. Then, at action selection time, our proposed solution prunes away trajectories that are estimated to violate the failure probability. Extensive simulation results show that this approach can quickly produce high-quality solutions and is competitive with the optimal but time-consuming solution.