Convergence of an Asynchronous Block-Coordinate Forward-Backward Algorithm for Convex Composite Optimization

TL;DR

This paper analyzes the convergence of an asynchronous randomized block-coordinate algorithm for convex composite optimization, proving almost sure convergence and establishing sublinear and linear convergence rates under certain conditions.

Contribution

It provides the first convergence analysis of an asynchronous block-coordinate forward-backward algorithm with arbitrary probability distributions.

Findings

Almost sure convergence of the iterates to a minimizer.

Sublinear convergence rate in expectation for function values.

Linear convergence rate under an error bound condition.

Abstract

In this paper, we study the convergence properties of a randomized block-coordinate descent algorithm for the minimization of a composite convex objective function, where the block-coordinates are updated asynchronously and randomly according to an arbitrary probability distribution. We prove that the iterates generated by the algorithm form a stochastic quasi-Fej\'er sequence and thus converge almost surely to a minimizer of the objective function. Moreover, we prove a general sublinear rate of convergence in expectation for the function values and a linear rate of convergence in expectation under an error bound condition of Tseng type.