ORPR: An OR-Guided Pretrain-then-Reinforce Learning Model for Inventory Management

TL;DR

This paper introduces a novel OR-Guided Pretrain-then-Reinforce framework that combines simulation-augmented OR models with deep learning and reinforcement learning to improve inventory management, achieving significant real-world operational gains.

Contribution

It presents a new hybrid framework integrating OR and AI through pretraining and reinforcement learning, emphasizing structured guidance and expert-in-the-loop adaptation for supply chain optimization.

Findings

Achieved a 5.27-day reduction in inventory turnover time.

Increased in-stock rates by 2.29%.

Reduced holding costs by 29.95%.

Abstract

As the pursuit of synergy between Artificial Intelligence (AI) and Operations Research (OR) gains momentum in handling complex inventory systems, a critical challenge persists: how to effectively reconcile AI's adaptive perception with OR's structural rigor. To bridge this gap, we propose a novel OR-Guided "Pretrain-then-Reinforce" framework. To provide structured guidance, we propose a simulation-augmented OR model that generates high-quality reference decisions, implicitly capturing complex business constraints and managerial preferences. Leveraging these OR-derived decisions as foundational training labels, we design a domain-informed deep learning foundation model to establish foundational decision-making capabilities, followed by a reinforcement learning (RL) fine-tuning stage. Uniquely, we position RL as a deep alignment mechanism that enables the AI agent to internalize the optimality principles of OR, while simultaneously leveraging exploration for general policy refinement and allowing expert guidance for scenario-specific adaptation (e.g., promotional events). Validated through extensive numerical experiments and a field deployment at JD.com augmented by a Difference-in-Differences (DiD) analysis, our model significantly outperforms incumbent industrial practices, delivering real-world gains of a 5.27-day reduction in turnover and a 2.29% increase in in-stock rates, alongside a 29.95% decrease in holding costs. Contrary to the prevailing trend of brute-force model scaling, our study demonstrates that a lightweight, domain-informed model can deliver state-of-the-art performance and robust transferability when guided by structured OR logic. This approach offers a scalable and cost-effective paradigm for intelligent supply chain management, highlighting the value of deeply aligning AI with OR.