Scheduling That Speaks: An Interpretable Programmatic Reinforcement Learning Framework

TL;DR

ProRL is an interpretable reinforcement learning framework that learns human-readable scheduling programs, combining domain-specific language, local search, and Bayesian optimization to outperform existing heuristics and DRL methods.

Contribution

It introduces a novel programmatic RL approach with a domain-specific language and optimization techniques for interpretable and efficient scheduling policies.

Findings

ProRL achieves strong performance on benchmark scheduling problems.

ProRL performs well with limited training data (only 100 episodes).

ProRL outperforms existing heuristics and DRL baselines.

Abstract

Deep reinforcement learning (DRL) has recently emerged as a promising approach to solve combinatorial optimization problems such as job shop scheduling. However, the policies learned by DRL are typically represented by deep neural networks (DNNs), whose opaque neural architectures and non-interpretable policy decisions can lead to critical trust and usability concerns for human decision makers. In addition, the computational requirements of DNNs can further hinder practical deployment in resource constrained environments. In this work, we propose ProRL, a novel interpretable programmatic reinforcement learning framework that achieves high-performance scheduling with human-readable and editable programmatic policies (i.e., programs). We first introduce a domain-specific language for scheduling (DSL-S) to represent scheduling strategies as structured programs. ProRL then explores the program space defined by DSL-S using local search to identify incomplete programs, which are subsequently completed by learning their parameters via Bayesian optimization. ProRL learns which scheduling heuristic rules to select, and hence, it naturally incorporates existing heuristics already used in industrial scenarios. Experiments on widely used benchmark instances demonstrate the strong performance of ProRL against existing heuristics and DRL baselines. Furthermore, ProRL performs well under strongly constrained computational resources, such as training with only 100 episodes. Our code is available at https://github.com/HcPlu/ProRL.