Vehicle management in a modular production context using Deep Q-Learning

TL;DR

This paper explores using Deep Q-Learning agents for vehicle scheduling in modular production, demonstrating comparable performance to heuristics and increased robustness to noise through simulation-based experiments.

Contribution

It introduces a DRL approach for job-shop scheduling in modular production, comparing it with traditional heuristics and analyzing its robustness and performance.

Findings

DRL agents perform comparably to heuristics.

DRL agents are more robust to noise.

Deep Q-Learning is viable for vehicle scheduling.

Abstract

We investigate the feasibility of deploying Deep-Q based deep reinforcement learning agents to job-shop scheduling problems in the context of modular production facilities, using discrete event simulations for the environment. These environments are comprised of a source and sink for the parts to be processed, as well as (several) workstations. The agents are trained to schedule automated guided vehicles to transport the parts back and forth between those stations in an optimal fashion. Starting from a very simplistic setup, we increase the complexity of the environment and compare the agents' performances with well established heuristic approaches, such as first-in-first-out based agents, cost tables and a nearest-neighbor approach. We furthermore seek particular configurations of the environments in which the heuristic approaches struggle, to investigate to what degree the Deep-Q agents are affected by these challenges. We find that Deep-Q based agents show comparable performance as the heuristic baselines. Furthermore, our findings suggest that the DRL agents exhibit an increased robustness to noise, as compared to the conventional approaches. Overall, we find that DRL agents constitute a valuable approach for this type of scheduling problems.