Convergence guarantees for stochastic algorithms solving non-unique problems in metric spaces

TL;DR

This paper establishes explicit convergence rates for stochastic algorithms solving non-unique problems in metric spaces, unifying various regularity conditions and applying to multiple stochastic approximation methods.

Contribution

It provides a general quantitative convergence theorem for stochastic quasi-Fejér sequences in metric spaces, extending and unifying many existing regularity conditions.

Findings

Explicit convergence rates in mean and almost surely for stochastic processes.

Unified framework covering contractivity, error bounds, and metric subregularity.

Application to classical stochastic methods like proximal point and Krasnoselskii-Mann schemes.

Abstract

We prove a general quantitative theorem on the asymptotic behavior of stochastic quasi-Fej\'er monotone sequences in a broad metric context. Concretely, our result explicitly constructs a rate of convergence for such process, both in mean and almost surely, under an abstract stochastic regularity assumption, derived from previous work of Kohlenbach, L\'opez-Acedo and Nicolae [Isr. J. Math. 232(1), pp. 261-297, 2019] on such notions in a deterministic context. Our notion of regularity extends and unifies many common conditions from the literature, such as generalized contractivity for self maps, weak sharp minima and error bounds for real-valued functions, uniform monotonicity and global metric subregularity for set-valued operators, related Polyak-{\L}ojasiewicz or Kurdyka-{\L}ojasiewicz conditions, as well as expected sharp growth as e.g. studied by Asi and Duchi [SIAM J. Optim. 29(3), pp. 2257-2290, 2019]. The rate is moreover highly uniform, depending only on very few data of the surrounding objects. We also discuss special cases which allow for the construction of fast rates in the form of linear non-asymptotic guarantees. We conclude by presenting three concrete methods from stochastic approximation where our results yield new rates of convergence, including the classical example of the stochastic proximal point method, a randomized variant of the Krasnoselskii-Mann scheme for solving stochastic fixed-point equations, and a Busemann subgradient method recently introduced by Goodwin, Lewis, L\'opez-Acedo and Nicolae [Math. Program., to appear], all of which make use of our metric generality by being formulated over complete geodesic metric spaces of nonpositive curvature.