Exploring search space trees using an adapted version of Monte Carlo tree search for combinatorial optimization problems

TL;DR

This paper introduces an adapted Monte Carlo tree search algorithm with enhancements tailored for combinatorial optimization, demonstrating superior or competitive results on scheduling and knapsack problems.

Contribution

It proposes a novel Monte Carlo tree search-based heuristic with problem-specific enhancements for combinatorial optimization, achieving state-of-the-art or near-optimal solutions efficiently.

Findings

Surpasses state-of-the-art in quay crane scheduling

Finds new best solutions for benchmark instances

Produces near-optimal solutions quickly for knapsack problem

Abstract

In this article we propose a heuristic algorithm to explore search space trees associated with instances of combinatorial optimization problems. The algorithm is based on Monte Carlo tree search, a popular algorithm in game playing that is used to explore game trees and represents the state-of-the-art algorithm for a number of games. Several enhancements to Monte Carlo tree search are proposed that make the algorithm more suitable in a combinatorial optimization context. These enhancements exploit the combinatorial structure of the problem and aim to efficiently explore the search space tree by pruning subtrees, using a heuristic simulation policy, reducing the domains of variables by eliminating dominated value assignments and using a beam width. The algorithm was implemented with its components specifically tailored to two combinatorial optimization problems: the quay crane scheduling problem with non-crossing constraints and the 0-1 knapsack problem. For the first problem our algorithm surpasses the state-of-the-art results and several new best solutions are found for a benchmark set of instances. For the second problem our algorithm typically produces near-optimal solutions that are slightly worse than the state-of-the-art results, but it needs only a small fraction of the time to do so. These results indicate that the algorithm is competitive with the state-of-the-art for two entirely different combinatorial optimization problems.