A Production Scheduling Framework for Reinforcement Learning Under Real-World Constraints

TL;DR

This paper introduces JobShopLab, a modular, open-source framework that extends classical job shop scheduling to include real-world constraints, enabling effective training and evaluation of reinforcement learning agents in complex production environments.

Contribution

The paper presents a comprehensive, customizable framework that incorporates real-world constraints into RL-based production scheduling, facilitating standardized testing and comparison of scheduling strategies.

Findings

Framework effectively models real-world production constraints.

Supports multi-objective optimization in scheduling.

Enables standardized evaluation of RL agents.

Abstract

The classical Job Shop Scheduling Problem (JSSP) focuses on optimizing makespan under deterministic constraints. Real-world production environments introduce additional complexities that cause traditional scheduling approaches to be less effective. Reinforcement learning (RL) holds potential in addressing these challenges, as it allows agents to learn adaptive scheduling strategies. However, there is a lack of a comprehensive, general-purpose frameworks for effectively training and evaluating RL agents under real-world constraints. To address this gap, we propose a modular framework that extends classical JSSP formulations by incorporating key real-world constraints inherent to the shopfloor, including transport logistics, buffer management, machine breakdowns, setup times, and stochastic processing conditions, while also supporting multi-objective optimization. The framework is a customizable solution that offers flexibility in defining problem instances and configuring simulation parameters, enabling adaptation to diverse production scenarios. A standardized interface ensures compatibility with various RL approaches, providing a robust environment for training RL agents and facilitating the standardized comparison of different scheduling methods under dynamic and uncertain conditions. We release JobShopLab as an open-source tool for both research and industrial applications, accessible at: https://github.com/proto-lab-ro/jobshoplab