Reinforcement Learning for Multi-Objective Multi-Echelon Supply Chain Optimisation

TL;DR

This paper introduces a multi-objective reinforcement learning approach for complex supply chain optimization, effectively balancing economic, environmental, and social goals with superior performance over existing methods.

Contribution

It develops a novel multi-objective RL model for multi-echelon supply chains, outperforming traditional single-objective RL and evolutionary algorithms in complex scenarios.

Findings

Up to 75% higher hypervolume than MOEA-based approach

Solutions are approximately eleven times denser, indicating better robustness

Achieves stable production and inventory levels while minimizing demand loss

Abstract

This study develops a generalised multi-objective, multi-echelon supply chain optimisation model with non-stationary markets based on a Markov decision process, incorporating economic, environmental, and social considerations. The model is evaluated using a multi-objective reinforcement learning (RL) method, benchmarked against an originally single-objective RL algorithm modified with weighted sum using predefined weights, and a multi-objective evolutionary algorithm (MOEA)-based approach. We conduct experiments on varying network complexities, mimicking typical real-world challenges using a customisable simulator. The model determines production and delivery quantities across supply chain routes to achieve near-optimal trade-offs between competing objectives, approximating Pareto front sets. The results demonstrate that the primary approach provides the most balanced trade-off between optimality, diversity, and density, further enhanced with a shared experience buffer that allows knowledge transfer among policies. In complex settings, it achieves up to 75\% higher hypervolume than the MOEA-based method and generates solutions that are approximately eleven times denser, signifying better robustness, than those produced by the modified single-objective RL method. Moreover, it ensures stable production and inventory levels while minimising demand loss.