Inventory Management with Partially Observed Nonstationary Demand

TL;DR

This paper develops a model for inventory management under non-stationary, Markov-modulated demand with unobserved demand states, introducing active learning and deriving explicit filtering equations for optimal control.

Contribution

It introduces a novel approach combining active learning with Markov-modulated demand, deriving explicit filtering equations and characterizing optimal policies in a partially observed setting.

Findings

Explicit filtering equations for demand state inference

Optimal inventory policies derived for the model

Computational procedure for value function and policy calculation

Abstract

We consider a continuous-time model for inventory management with Markov modulated non-stationary demands. We introduce active learning by assuming that the state of the world is unobserved and must be inferred by the manager. We also assume that demands are observed only when they are completely met. We first derive the explicit filtering equations and pass to an equivalent fully observed impulse control problem in terms of the sufficient statistics, the a posteriori probability process and the current inventory level. We then solve this equivalent formulation and directly characterize an optimal inventory policy. We also describe a computational procedure to calculate the value function and the optimal policy and present two numerical illustrations.