Offline reinforcement learning for job-shop scheduling problems

TL;DR

This paper presents a novel offline reinforcement learning approach tailored for complex combinatorial optimization problems like job-shop scheduling, effectively handling heterogeneous graph states and variable actions to outperform existing methods.

Contribution

Introduces an offline RL method that encodes actions as edge attributes and balances reward optimization with imitation, addressing limitations of prior RL and behavioral cloning approaches.

Findings

Achieves superior performance on job-shop scheduling benchmarks.

Effectively handles complex constraints and variable actions.

Outperforms state-of-the-art techniques in experiments.

Abstract

Recent advances in deep learning have shown significant potential for solving combinatorial optimization problems in real-time. Unlike traditional methods, deep learning can generate high-quality solutions efficiently, which is crucial for applications like routing and scheduling. However, existing approaches like deep reinforcement learning (RL) and behavioral cloning have notable limitations, with deep RL suffering from slow learning and behavioral cloning relying solely on expert actions, which can lead to generalization issues and neglect of the optimization objective. This paper introduces a novel offline RL method designed for combinatorial optimization problems with complex constraints, where the state is represented as a heterogeneous graph and the action space is variable. Our approach encodes actions in edge attributes and balances expected rewards with the imitation of expert solutions. We demonstrate the effectiveness of this method on job-shop scheduling and flexible job-shop scheduling benchmarks, achieving superior performance compared to state-of-the-art techniques.