Behaviorally Grounded Model-Based and Model Free Cost Reduction in a Simulated Multi-Echelon Supply Chain

TL;DR

This paper compares behaviorally grounded model-based and model-free reinforcement learning methods to reduce the bullwhip effect in multi-echelon supply chains, highlighting their complementary strengths and insights.

Contribution

It introduces a dual deep Q-network reinforcement learning approach and compares it with traditional behavioral models for supply chain bullwhip mitigation.

Findings

Model-free RL effectively reduces bullwhip in complex supply chains.

Model-based approaches provide valuable behavioral insights.

Combining both approaches offers a comprehensive strategy.

Abstract

Amplification and phase shift in ordering signals, commonly referred to as bullwhip, are responsible for both excessive strain on real world inventory management systems, stock outs, and unnecessary capital reservation though safety stock building. Bullwhip is a classic, yet persisting, problem with reverberating consequences in inventory management. Research on bullwhip has consistently emphasized behavioral influences for this phenomenon and leveraged behavioral ordering models to suggest interventions. However more recent model-free approaches have also seen success. In this work, the author develops algorithmic approaches towards mitigating bullwhip using both behaviorally grounded model-based approaches alongside a model-free dual deep Q-network reinforcement learning approach. In addition to exploring the utility of this specific model-free architecture to multi-echelon supply chains with imperfect information sharing and information delays, the author directly compares the performance of these model-based and model-free approaches. In doing so, this work highlights both the insights gained from exploring model-based approaches in the context of prior behavioral operations management literature and emphasizes the complementary nature of model-based and model-free approaches in approaching behaviorally grounded supply chain management problems.