On the Out-of-Sample Performance of Stochastic Dynamic Programming and Model Predictive Control

TL;DR

This paper compares stochastic dynamic programming and model predictive control, revealing conditions where MPC outperforms SDP, especially with skewed distributions, and provides insights into their relative performance based on problem structure.

Contribution

The paper offers a theoretical analysis linking SDP and MPC through distributional ambiguity, explaining when MPC can outperform SDP in multistage stochastic optimization.

Findings

MPC can be seen as solving a distributionally ambiguous problem related to SDP.

MPC performs better when the underlying distribution is skewed or tail-heavy.

Performance guarantees depend on the convexity or concavity of the problem.

Abstract

Sample average approximation--based stochastic dynamic programming (SDP) and model predictive control (MPC) are two different methods for approaching multistage stochastic optimization. In this paper we investigate the conditions under which SDP may be outperformed by MPC. We show that, depending on the presence of concavity or convexity, MPC can be interpreted as solving a mean-constrained distributionally ambiguous version of the problem that is solved by SDP. This furnishes performance guarantees when the true mean is known and provides intuition for why MPC performs better in some applications and worse in others. We then study a multistage stochastic optimization problem that is representative of the type for which MPC may be the better choice. We find that this can indeed be the case when the probability distribution of the underlying random variable is skewed or has enough weight in the right-hand tail.