Policy-Based Deep Reinforcement Learning Hyperheuristics for Job-Shop Scheduling Problems

TL;DR

This paper introduces a novel policy-based deep reinforcement learning hyper-heuristic framework for Job Shop Scheduling, incorporating mechanisms for feasible action filtering and heuristic switching regulation, leading to improved scheduling performance.

Contribution

It presents a new hyper-heuristic approach with action prefiltering and commitment strategies, advancing the application of deep RL in JSSP.

Findings

Outperforms traditional heuristics and metaheuristics

Effective in reducing makespan on benchmark problems

Demonstrates advantages of stochastic over deterministic action selection

Abstract

This paper proposes a policy-based deep reinforcement learning hyper-heuristic framework for solving the Job Shop Scheduling Problem. The hyper-heuristic agent learns to switch scheduling rules based on the system state dynamically. We extend the hyper-heuristic framework with two key mechanisms. First, action prefiltering restricts decision-making to feasible low-level actions, enabling low-level heuristics to be evaluated independently of environmental constraints and providing an unbiased assessment. Second, a commitment mechanism regulates the frequency of heuristic switching. We investigate the impact of different commitment strategies, from step-wise switching to full-episode commitment, on both training behavior and makespan. Additionally, we compare two action selection strategies at the policy level: deterministic greedy selection and stochastic sampling. Computational experiments on standard JSSP benchmarks demonstrate that the proposed approach outperforms traditional heuristics, metaheuristics, and recent neural network-based scheduling methods