Sample Complexity of Data-driven Multistage Stochastic Programming under Markovian Uncertainty

TL;DR

This paper introduces the Markov Recombining Scenario Tree (MRST) method, a data-driven approach that reduces sample complexity from exponential to polynomial in multistage stochastic programming with Markovian uncertainty.

Contribution

The paper proposes the MRST method, which constructs an approximate problem using only two trajectories, overcoming the curse of dimensionality in multistage stochastic optimization.

Findings

MRST achieves polynomial sample complexity in the time horizon T.

Numerical experiments show MRST outperforms traditional SAA.

MRST effectively addresses the curse of dimensionality.

Abstract

This work is motivated by the challenges of applying the sample average approximation (SAA) method to multistage stochastic programming with an unknown continuous-state Markov process. While SAA is widely used in static and two-stage stochastic optimization, it becomes computationally intractable in general multistage settings as the time horizon $T$ increases. Indeed, the number of samples required to obtain a reasonably accurate solution grows exponentially$\text{ -- }$a phenomenon known as the curse of dimensionality with respect to the time horizon. To overcome this limitation, we propose a novel data-driven approach, the Markov Recombining Scenario Tree (MRST) method, which constructs an approximate problem using only two independent trajectories of historical data. Our analysis demonstrates that the MRST method achieves polynomial sample complexity in $T$, providing a more efficient alternative to SAA. Numerical experiments on the Linear Quadratic Gaussian problem show that MRST outperforms SAA, addressing the curse of dimensionality.