Value Function Gradient Learning for Large-Scale Multistage Stochastic Programming Problems

TL;DR

This paper introduces VFGL, a novel stagewise decomposition algorithm for large-scale multistage stochastic convex programs that efficiently learns the value function gradient without scenario tree approximation.

Contribution

VFGL is a new method that uses stochastic gradient descent to learn value function parameters with fixed subproblem sizes, improving scalability and parallelization.

Findings

VFGL produces high-quality solutions in tested examples.

VFGL is computationally efficient compared to traditional methods.

VFGL handles nonlinear problems without increasing subproblem size.

Abstract

A stagewise decomposition algorithm called value function gradient learning (VFGL) is proposed for large-scale multistage stochastic convex programs. VFGL finds the parameter values that best fit the gradient of the value function within a given parametric family. Widely used decomposition algorithms for multistage stochastic programming, such as stochastic dual dynamic programming (SDDP), approximate the value function by adding linear subgradient cuts at each iteration. Although this approach has been successful for linear problems, nonlinear problems may suffer from the increasing size of each subproblem as the iteration proceeds. On the other hand, VFGL has a fixed number of parameters; thus, the size of the subproblems remains constant throughout the iteration. Furthermore, VFGL can learn the parameters by means of stochastic gradient descent, which means that it can be easily parallelized and does not require a scenario tree approximation of the underlying uncertainties. VFGL was compared with a deterministic equivalent formulation of the multistage stochastic programming problem and SDDP approaches for three illustrative examples: production planning, hydrothermal generation, and the lifetime financial planning problem. Numerical examples show that VFGL generates high-quality solutions and is computationally efficient.