Asynchronous Parallel Nonconvex Optimization Under the Polyak-Lojasiewicz Condition

TL;DR

This paper introduces an asynchronous block coordinate descent algorithm for nonconvex optimization problems satisfying the Polyak-Lojasiewicz condition, achieving linear convergence despite delays and without requiring convexity.

Contribution

It presents a novel asynchronous algorithm for nonconvex problems under the Polyak-Lojasiewicz condition, with proven linear convergence and practical validation.

Findings

Linear convergence rate achieved under bounded delays

Algorithm effective for logistic regression problems

Asynchrony impacts convergence speed

Abstract

Communication delays and synchronization are major bottlenecks for parallel computing, and tolerating asynchrony is therefore crucial for accelerating parallel computation. Motivated by optimization problems that do not satisfy convexity assumptions, we present an asynchronous block coordinate descent algorithm for nonconvex optimization problems whose objective functions satisfy the Polyak-Lojasiewicz condition. This condition is a generalization of strong convexity to nonconvex problems and requires neither convexity nor uniqueness of minimizers. Under only assumptions of mild smoothness of objective functions and bounded delays, we prove that a linear convergence rate is obtained. Numerical experiments for logistic regression problems are presented to illustrate the impact of asynchrony upon convergence.