Reinforcement Learning Provides a Flexible Approach for Realistic Supply Chain Safety Stock Optimisation

TL;DR

This paper explores the use of Reinforcement Learning to optimize safety stock and order quantities in supply chains, offering a more realistic and flexible approach compared to traditional methods, though it requires more computational time.

Contribution

It demonstrates that RL can optimize both safety stock levels and order quantities simultaneously, unlike classical models that only optimize safety stock, thus enabling more complex supply chain modeling.

Findings

RL can optimize safety stock and order quantities together

RL models accommodate more realistic supply chain features

RL requires longer computation times

Abstract

Although safety stock optimisation has been studied for more than 60 years, most companies still use simplistic means to calculate necessary safety stock levels, partly due to the mismatch between existing analytical methods' emphases on deriving provably optimal solutions and companies' preferences to sacrifice optimal results in favour of more realistic problem settings. A newly emerging method from the field of Artificial Intelligence (AI), namely Reinforcement Learning (RL), offers promise in finding optimal solutions while accommodating more realistic problem features. Unlike analytical-based models, RL treats the problem as a black-box simulation environment mitigating against the problem of oversimplifying reality. As such, assumptions on stock keeping policy can be relaxed and a higher number of problem variables can be accommodated. While RL has been popular in other domains, its applications in safety stock optimisation remain scarce. In this paper, we investigate three RL methods, namely, Q-Learning, Temporal Difference Advantage Actor-Critic and Multi-agent Temporal Difference Advantage Actor-Critic for optimising safety stock in a linear chain of independent agents. We find that RL can simultaneously optimise both safety stock level and order quantity parameters of an inventory policy, unlike classical safety stock optimisation models where only safety stock level is optimised while order quantity is predetermined based on simple rules. This allows RL to model more complex supply chain procurement behaviour. However, RL takes longer time to arrive at solutions, necessitating future research on identifying and improving trade-offs between the use of AI and mathematical models are needed.