Investigating the Monte-Carlo Tree Search Approach for the Job Shop Scheduling Problem

TL;DR

This paper explores the application of Monte Carlo Tree Search (MCTS) to large-scale Job Shop Scheduling Problems, demonstrating its effectiveness over traditional methods in complex manufacturing scenarios.

Contribution

It introduces MCTS formulations for JSSP, proposes a new synthetic benchmark, and shows MCTS's superior performance on large, complex instances compared to constraint programming.

Findings

MCTS outperforms constraint programming on large JSSP instances.

The new benchmark captures real manufacturing complexity.

MCTS effectively handles recirculation in scheduling.

Abstract

The Job Shop Scheduling Problem (JSSP) is a well-known optimization problem in manufacturing, where the goal is to determine the optimal sequence of jobs across different machines to minimize a given objective. In this work, we focus on minimising the weighted sum of job completion times. We explore the potential of Monte Carlo Tree Search (MCTS), a heuristic-based reinforcement learning technique, to solve large-scale JSSPs, especially those with recirculation. We propose several Markov Decision Process (MDP) formulations to model the JSSP for the MCTS algorithm. In addition, we introduce a new synthetic benchmark derived from real manufacturing data, which captures the complexity of large, non-rectangular instances often encountered in practice. Our experimental results show that MCTS effectively produces good-quality solutions for large-scale JSSP instances, outperforming our constraint programming approach.