Reinforcement Learning as an Improvement Heuristic for Real-World Production Scheduling

TL;DR

This paper presents a novel reinforcement learning approach using Transformer-based encoding to improve real-world multiobjective production scheduling solutions, outperforming traditional heuristics.

Contribution

It introduces a new RL-based improvement heuristic with Transformer encoding for complex scheduling problems, validated on real industry data.

Findings

Our method outperforms existing heuristics on real data.

Transformer encoding effectively captures job relationships.

The approach achieves significant solution quality improvements.

Abstract

The integration of Reinforcement Learning (RL) with heuristic methods is an emerging trend for solving optimization problems, which leverages RL's ability to learn from the data generated during the search process. One promising approach is to train an RL agent as an improvement heuristic, starting with a suboptimal solution that is iteratively improved by applying small changes. We apply this approach to a real-world multiobjective production scheduling problem. Our approach utilizes a network architecture that includes Transformer encoding to learn the relationships between jobs. Afterwards, a probability matrix is generated from which pairs of jobs are sampled and then swapped to improve the solution. We benchmarked our approach against other heuristics using real data from our industry partner, demonstrating its superior performance.