Certainty Equivalence Control-Based Heuristics in Multi-Stage Convex Stochastic Optimization Problems

TL;DR

This paper introduces heuristics based on Certainty Equivalence Control for multi-stage convex stochastic optimization, providing bounds on their performance gap and demonstrating their effectiveness on network utility maximization.

Contribution

It develops universal re-solving and projection-based heuristics, combines them into a hybrid policy, and derives performance bounds under regularity conditions, advancing stochastic optimization methods.

Findings

Performance gap bounds are proportional to the square root of noise variance.

Additional smoothness assumptions yield bounds proportional to noise variance.

Numerical experiments demonstrate the effectiveness of the proposed methods.

Abstract

We examine a multi-stage stochastic optimization problem characterized by stagewise-independent, decision-dependent noises with strict constraints. The problem assumes convexity in that, following a specific relaxation, it transforms into a deterministic convex program. The relaxation process is inspired by the principle of Certainty Equivalence Control, which substitutes uncertainties with their nominal values and requires the hard constraints to be satisfied only in an expected sense. Utilizing the solutions obtained from these convex programs, we propose two universal methodologies -- re-solving-based and projection-based -- to formulate feasible policies relevant to the original problem. These methodologies are subsequently amalgamated to develop a hybrid policy, equipped with a tuning parameter that manages the frequency of re-solving. We derive upper bounds on the gap between the performance of these heuristic policies and the optimal one. Under the Lipschitz-type regularity of the model, these bounds are proportional to the square root of the stochastic noise variance. Assuming additional $C^2$-smoothness regularity, an alternative bound, proportional to the variance of the stochastic noise, can be established -- providing a refinement when variances are small. We also delve into the issues concerning the exponential growth of a multiplicative constant in these bounds with respect to the stage number. Our model strives to lay a comprehensive framework for dynamic sequential decision-making in the presence of uncertainties, incorporating classic problems from inventory management and Markovian bandits, while also accommodating a wider spectrum of stochastic optimization challenges. Specifically, we illustrate our methods on a network utility maximization problem through numerical experiments.