Bilevel Learning Model Towards Industrial Scheduling

TL;DR

This paper introduces a novel bilevel deep reinforcement learning scheduler, BDS, for large-scale industrial job scheduling, significantly improving efficiency and solution quality over traditional heuristics and deep networks.

Contribution

The paper develops a bilevel deep reinforcement learning approach combining DDQN and GPN, with theoretical convergence guarantees, for large-scale industrial scheduling.

Findings

BDS outperforms heuristics and deep networks in industrial scenarios.

BDS reduces makespan by over 22% to 28% on large datasets.

Computational time is less than 200 seconds for large-scale problems.

Abstract

Automatic industrial scheduling, aiming at optimizing the sequence of jobs over limited resources, is widely needed in manufacturing industries. However, existing scheduling systems heavily rely on heuristic algorithms, which either generate ineffective solutions or compute inefficiently when job scale increases. Thus, it is of great importance to develop new large-scale algorithms that are not only efficient and effective, but also capable of satisfying complex constraints in practice. In this paper, we propose a Bilevel Deep reinforcement learning Scheduler, \textit{BDS}, in which the higher level is responsible for exploring an initial global sequence, whereas the lower level is aiming at exploitation for partial sequence refinements, and the two levels are connected by a sliding-window sampling mechanism. In the implementation, a Double Deep Q Network (DDQN) is used in the upper level and Graph Pointer Network (GPN) lies within the lower level. After the theoretical guarantee for the convergence of BDS, we evaluate it in an industrial automatic warehouse scenario, with job number up to $5000$ in each production line. It is shown that our proposed BDS significantly outperforms two most used heuristics, three strong deep networks, and another bilevel baseline approach. In particular, compared with the most used greedy-based heuristic algorithm in real world which takes nearly an hour, our BDS can decrease the makespan by 27.5\%, 28.6\% and 22.1\% for 3 largest datasets respectively, with computational time less than 200 seconds.