Vaidya's method for convex stochastic optimization in small dimension

TL;DR

This paper adapts Vaidya's cutting plane method for convex stochastic optimization in low-dimensional spaces by replacing subgradients with their stochastic averages, resulting in an effective solution approach.

Contribution

It introduces a stochastic version of Vaidya's method that maintains convergence properties by controlling inaccuracy accumulation in stochastic subgradients.

Findings

The method effectively solves convex stochastic problems in low dimensions.

It does not accumulate errors over iterations, ensuring convergence.

Parallel batching enhances computational efficiency.

Abstract

This paper considers a general problem of convex stochastic optimization in a relatively low-dimensional space (e.g., 100 variables). It is known that for deterministic convex optimization problems of small dimensions, the fastest convergence is achieved by the center of gravity type methods (e.g., Vaidya's cutting plane method). For stochastic optimization problems, the question of whether Vaidya's method can be used comes down to the question of how it accumulates inaccuracy in the subgradient. The recent result of the authors states that the errors do not accumulate on iterations of Vaidya's method, which allows proposing its analog for stochastic optimization problems. The primary technique is to replace the subgradient in Vaidya's method with its probabilistic counterpart (the arithmetic mean of the stochastic subgradients). The present paper implements the described plan, which ultimately leads to an effective (if parallel computations for batching are possible) method for solving convex stochastic optimization problems in relatively low-dimensional spaces.