Stochastic Depletion Problems: Effective Myopic Policies for a class of Dynamic Optimization Problems

TL;DR

This paper introduces Stochastic Depletion Problems, a class of dynamic stochastic optimization problems, and shows that simple myopic policies can achieve near-optimal performance under certain conditions, simplifying complex decision-making.

Contribution

The paper identifies two properties ensuring myopic policies are within 50% of optimal and verifies these for several problem families, providing practical near-optimal solutions.

Findings

Myopic policies are within 50% of optimal for certain stochastic depletion problems.

Two properties guarantee near-optimality of simple heuristics.

Efficient near-optimal policies are identified for multiple problem classes.

Abstract

This paper presents a general class of dynamic stochastic optimization problems we refer to as Stochastic Depletion Problems. A number of challenging dynamic optimization problems of practical interest are stochastic depletion problems. Optimal solutions for such problems are difficult to obtain, both from a pragmatic computational perspective as also from a theoretical perspective. As such, simple heuristics are highly desirable. We isolate two simple properties that, if satisfied by a problem within this class, guarantee that a myopic policy incurs a performance loss of at most 50 % relative to the optimal adaptive control policy for that problem. We are able to verify that these two properties are satisfied for several interesting families of stochastic depletion problems and as a consequence identify efficient near-optimal control policies for a number of interesting dynamic stochastic optimization problems.