Optimality Conditions for Inventory Control

TL;DR

This paper presents general optimality conditions for Markov Decision Processes with applications to inventory control, ensuring the validity of optimality equations and convergence of algorithms under mild conditions.

Contribution

It introduces broad optimality conditions for inventory control problems, applicable to both discounted and average-cost criteria without restrictive demand distribution assumptions.

Findings

Optimality equations and inequalities are validated.

Convergence of value iteration algorithms is established.

Applicable to partially observable inventory states.

Abstract

This tutorial describes recently developed general optimality conditions for Markov Decision Processes that have significant applications to inventory control. In particular, these conditions imply the validity of optimality equations and inequalities. They also imply the convergence of value iteration algorithms. For total discounted-cost problems only two mild conditions on the continuity of transition probabilities and lower semi-continuity of one-step costs are needed. For average-cost problems, a single additional assumption on the finiteness of relative values is required. The general results are applied to periodic-review inventory control problems with discounted and average-cost criteria without any assumptions on demand distributions. The case of partially observable states is also discussed.