Exploring Multi-Agent Reinforcement Learning for Unrelated Parallel Machine Scheduling

TL;DR

This paper explores the application of Multi-Agent Reinforcement Learning to the complex Unrelated Parallel Machine Scheduling Problem, demonstrating the effectiveness of MARL approaches and highlighting challenges in cooperative learning.

Contribution

It introduces a MARL framework for UPMS with setup times, compares single- and multi-agent algorithms, and provides empirical insights into their performance and scalability.

Findings

MARL approaches outperform single-agent algorithms in scalable scenarios

Maskable PPO enhances single-agent scheduling performance

Multi-Agent PPO shows potential but faces cooperative learning challenges

Abstract

Scheduling problems pose significant challenges in resource, industry, and operational management. This paper addresses the Unrelated Parallel Machine Scheduling Problem (UPMS) with setup times and resources using a Multi-Agent Reinforcement Learning (MARL) approach. The study introduces the Reinforcement Learning environment and conducts empirical analyses, comparing MARL with Single-Agent algorithms. The experiments employ various deep neural network policies for single- and Multi-Agent approaches. Results demonstrate the efficacy of the Maskable extension of the Proximal Policy Optimization (PPO) algorithm in Single-Agent scenarios and the Multi-Agent PPO algorithm in Multi-Agent setups. While Single-Agent algorithms perform adequately in reduced scenarios, Multi-Agent approaches reveal challenges in cooperative learning but a scalable capacity. This research contributes insights into applying MARL techniques to scheduling optimization, emphasizing the need for algorithmic sophistication balanced with scalability for intelligent scheduling solutions.