Methods for Optimization Problems with Markovian Stochasticity and Non-Euclidean Geometry

TL;DR

This paper introduces four stochastic optimization methods tailored for problems with Markovian noise and non-Euclidean geometry, providing optimal convergence guarantees for smooth convex and variational inequality problems.

Contribution

It extends stochastic optimization techniques to handle Markovian noise and arbitrary geometries, with theoretical analysis and optimal convergence guarantees.

Findings

Proposed four Mirror Descent-based methods for complex stochastic problems.

Established optimal convergence rates for the proposed methods.

Provided lower bounds confirming the optimality of the results.

Abstract

This paper examines a variety of classical optimization problems, including well-known minimization tasks and more general variational inequalities. We consider a stochastic formulation of these problems, and unlike most previous work, we take into account the complex Markov nature of the noise. We also consider the geometry of the problem in an arbitrary non-Euclidean setting, and propose four methods based on the Mirror Descent iteration technique. Theoretical analysis is provided for smooth and convex minimization problems and variational inequalities with Lipschitz and monotone operators. The convergence guarantees obtained are optimal for first-order stochastic methods, as evidenced by the lower bound estimates provided in this paper.