An error bound-based convergence analysis framework for a class of randomized algorithms

TL;DR

This paper introduces a unified error bound framework for analyzing the convergence of various randomized algorithms, providing less conservative guarantees and broad applicability to optimization and fixed point problems.

Contribution

The paper develops a new abstract UEB framework that generalizes existing conditions, enabling comprehensive convergence analysis for a wide class of stochastic algorithms.

Findings

Established non-asymptotic and asymptotic convergence rates under global UEB conditions.

Proved convergence guarantees for the randomized alternating Krasnoselskii-Mann method.

Provided novel convergence results for the randomized subspace descent method.

Abstract

Existing error-bound-based analyses for stochastic algorithms that exhibit certain descent properties, such as randomized coordinate descent and randomized projection methods, are often limited in scope and typically lead to overly conservative convergence guarantees. To address this gap, we develop an abstract framework for analyzing such stochastic algorithms based on new unified error bound (UEB) conditions. The proposed UEB conditions subsume many common error bound- and Kurdyka--{\L}ojasiewicz-type conditions used in existing studies of algorithms for optimization, convex feasibility, and common fixed point problems. Under the global UEB condition, we establish non-asymptotic in-expectation and asymptotic almost-sure convergence rates for the stochastic algorithms in our framework. Under the local UEB condition, we also show asymptotic almost sure convergence rates. We demonstrate the strength and versatility of our framework through two applications. For the common fixed point problem, we provide comprehensive convergence guarantees for the randomized alternating Krasnoselskii-Mann method under H\"olderian error bound conditions. Furthermore, for unconstrained minimization of smooth definable functions, we establish novel convergence guarantees for the randomized subspace descent method, an algorithm subsuming both randomized coordinate and block coordinate descent.