Smaller Batches, Bigger Gains? Investigating the Impact of Batch Sizes on Reinforcement Learning Based Real-World Production Scheduling

TL;DR

This paper examines how different batch sizes affect reinforcement learning performance in real-world production scheduling, identifying optimal boundaries and proposing curriculum strategies to improve training with small batches.

Contribution

It introduces a systematic analysis of batch size effects in RL for scheduling and proposes two curriculum learning strategies to enhance training with small batches.

Findings

Identified boundaries for effective batch sizes balancing sample complexity and flexibility.

Demonstrated the effectiveness of curriculum strategies for small batch training.

Provided insights for practitioners to select appropriate batch sizes in industrial scheduling.

Abstract

Production scheduling is an essential task in manufacturing, with Reinforcement Learning (RL) emerging as a key solution. In a previous work, RL was utilized to solve an extended permutation flow shop scheduling problem (PFSSP) for a real-world production line with two stages, linked by a central buffer. The RL agent was trained to sequence equallysized product batches to minimize setup efforts and idle times. However, the substantial impact caused by varying the size of these product batches has not yet been explored. In this follow-up study, we investigate the effects of varying batch sizes, exploring both the quality of solutions and the training dynamics of the RL agent. The results demonstrate that it is possible to methodically identify reasonable boundaries for the batch size. These boundaries are determined on one side by the increasing sample complexity associated with smaller batch sizes, and on the other side by the decreasing flexibility of the agent when dealing with larger batch sizes. This provides the practitioner the ability to make an informed decision regarding the selection of an appropriate batch size. Moreover, we introduce and investigate two new curriculum learning strategies to enable the training with small batch sizes. The findings of this work offer the potential for application in several industrial use cases with comparable scheduling problems.