Efficient Online Mirror Descent Stochastic Approximation for Multi-Stage Stochastic Programming

TL;DR

This paper introduces an efficient online mirror descent stochastic approximation method for multi-stage stochastic programming, achieving linear gradient complexity in the number of stages, significantly improving over previous algorithms.

Contribution

The paper proposes a novel MDSA algorithm for multi-stage stochastic programming with linear gradient complexity, leveraging decomposability and stochastic conditional gradients.

Findings

Achieves gradient complexity linear in the number of stages T.

Provides an online implementation suitable for high-dimensional problems.

Improves exponentially over existing algorithms in efficiency.

Abstract

We study the unconstrained and the minimax saddle point variants of the convex multi-stage stochastic programming problem, where consecutive decisions are coupled through the objective functions, rather than through the constraints. We approach the problems from the infinite-dimensional policy perspective, but consider an online setting where only the policies corresponding to the actual realization of the underlying stochastic process is needed. This leads to a trackable formulation, where the dimension of the output is linear in the number of stages $T$. We propose hypothetical Mirror Descent Stochastic Approximation (MDSA) for the infinite dimensional policies using stochastic conditional gradients. By taking advantage of the decomposability of the updates across stages and realizations of the underlying stochastic process, we show that the proposed MDSA algorithms admit efficient online implementation, which achieves overall gradient complexity linear in $T$, improving exponentially over all existing algorithms.