Gradient-Bounded Dynamic Programming with Submodular and Concave Extensible Value Functions

TL;DR

This paper introduces a novel algorithm for high-dimensional dynamic programming problems with submodular and concave extensible value functions, providing bounds and demonstrating effectiveness in delivery slot pricing.

Contribution

The paper presents a new algorithm that computes bounds for complex dynamic programs with specific value function properties, ensuring finite termination.

Findings

Algorithm computes deterministic upper bounds and stochastic lower bounds.

Proven finite termination of the proposed algorithm.

Effective application demonstrated in high-dimensional delivery pricing example.

Abstract

We consider dynamic programming problems with finite, discrete-time horizons and prohibitively high-dimensional, discrete state-spaces for direct computation of the value function from the Bellman equation. For the case that the value function of the dynamic program is concave extensible and submodular in its state-space, we present a new algorithm that computes deterministic upper and stochastic lower bounds of the value function similar to dual dynamic programming. We then show that the proposed algorithm terminates after a finite number of iterations. Finally, we demonstrate the efficacy of our approach on a high-dimensional numerical example from delivery slot pricing in attended home delivery.