Multi-cut stochastic approximation methods for solving stochastic convex composite optimization

TL;DR

This paper introduces multi-cut stochastic approximation methods for stochastic convex composite optimization, using convex combinations of linearizations to achieve near-optimal convergence with improved computational performance.

Contribution

It proposes a novel SA approach that overcomes bias issues in multi-cut models by using convex combinations of linearizations, enhancing convergence and efficiency.

Findings

Achieves nearly optimal convergence rate.

Computational performance is comparable or superior to existing methods.

Effectively handles bias in multi-cut models.

Abstract

This paper considers the stochastic convex composite optimization problem and presents multi-cut stochastic approximation (SA) methods for solving it, whose models in expectation overestimate its objective function. The multi-cut model obtained by taking the maximum of a finite number of linearizations of the stochastic objective function provides a biased estimate of the objective function, with the error being uncontrollable. Instead, our proposed SA method uses models obtained by taking the maximum of a finite number of one-cut models, i.e., suitable convex combinations of linearizations of the stochastic objective function. It is shown that the proposed methods achieve nearly optimal convergence rate and have computational performance comparable, and sometimes superior, to other SA-type methods.