When does learning pay off? A study on DRL-based dynamic algorithm configuration for carbon-aware scheduling

TL;DR

This paper investigates whether deep reinforcement learning can develop generalizable policies for carbon-aware scheduling, demonstrating its effectiveness across diverse instances and problem complexities, justifying the training costs.

Contribution

It introduces a DRL-based dynamic algorithm configuration framework trained on simple instances and shows its strong transferability to complex, unseen problems in carbon-aware scheduling.

Findings

DRL policies outperform static tuning on complex instances.

Policies trained on simple instances generalize well to unseen problems.

DRL provides robust control policies effective across different problem scales.

Abstract

Deep reinforcement learning (DRL) has recently emerged as a promising tool for Dynamic Algorithm Configuration (DAC), enabling evolutionary algorithms to adapt their parameters online rather than relying on static tuned configurations. While DRL can learn effective control policies, training is computationally expensive. This cost may be justified if learned policies generalize, allowing the training effort to transfer across instance types and problem scales. Yet, for real-world optimization problems, it remains unclear whether this promise holds in practice and under which conditions the investment in learning pays off. In this work, we investigate this question in the context of the carbon-aware permutation flow-shop scheduling problem. We develop a DRL-based DAC framework and train it exclusively on small, simple instances. We then deploy the learned policy on both similar and more complex unseen instances and compare its performance against a static tuned baseline, which provides a fair point of comparison. Our findings show that the proposed method provides a strong dynamic algorithm control policy that can be effectively transferred to different unseen problem instances. Notably, on simple and cheap to compute instances, similar to those observed during training and tuning, DRL performs comparably with the statically tuned baseline. However, as instance characteristics diverge and computational complexities increase, the DRL-learned policy continuously outperforms static tuning. These results confirm that DRL can acquire robust and generalizable control policies which are effective beyond the training instance distributions. This ability to generalize across instance types makes the initial computational investment worthwhile, particularly in settings where static tuning struggles to adapt to changing problem scenarios.