Deep reinforcement learning for machine scheduling: Methodology, the state-of-the-art, and future directions

TL;DR

This paper reviews deep reinforcement learning methods for machine scheduling, comparing their methodologies and highlighting their advantages, limitations, and future research directions in optimizing complex manufacturing processes.

Contribution

It provides a comprehensive categorization and comparison of DRL-based scheduling approaches, emphasizing their performance and identifying key challenges for future work.

Findings

DRL methods outperform traditional solvers and heuristics in speed and near-optimal solutions

DRL approaches are effective for static and dynamic scheduling in various environments

Limitations include handling complex constraints and ensuring scalability and robustness

Abstract

Machine scheduling aims to optimize job assignments to machines while adhering to manufacturing rules and job specifications. This optimization leads to reduced operational costs, improved customer demand fulfillment, and enhanced production efficiency. However, machine scheduling remains a challenging combinatorial problem due to its NP-hard nature. Deep Reinforcement Learning (DRL), a key component of artificial general intelligence, has shown promise in various domains like gaming and robotics. Researchers have explored applying DRL to machine scheduling problems since 1995. This paper offers a comprehensive review and comparison of DRL-based approaches, highlighting their methodology, applications, advantages, and limitations. It categorizes these approaches based on computational components: conventional neural networks, encoder-decoder architectures, graph neural networks, and metaheuristic algorithms. Our review concludes that DRL-based methods outperform exact solvers, heuristics, and tabular reinforcement learning algorithms in terms of computation speed and generating near-global optimal solutions. These DRL-based approaches have been successfully applied to static and dynamic scheduling across diverse machine environments and job characteristics. However, DRL-based schedulers face limitations in handling complex operational constraints, configurable multi-objective optimization, generalization, scalability, interpretability, and robustness. Addressing these challenges will be a crucial focus for future research in this field. This paper serves as a valuable resource for researchers to assess the current state of DRL-based machine scheduling and identify research gaps. It also aids experts and practitioners in selecting the appropriate DRL approach for production scheduling.