The Asynchronous PALM Algorithm for Nonsmooth Nonconvex Problems

TL;DR

This paper presents the Asynchronous PALM algorithm, an extension of PALM that allows parallel, asynchronous updates of coordinate blocks in nonsmooth nonconvex optimization, leading to faster computations without sacrificing convergence guarantees.

Contribution

The paper introduces the Asynchronous PALM algorithm, enabling parallel, asynchronous updates for nonsmooth nonconvex problems with proven convergence properties.

Findings

Asynchronous PALM achieves linear speedup with more cores.

Cluster points of the algorithm are stationary points.

Global convergence is established under the Kurdyka-Łojasiewicz property.

Abstract

We introduce the Asynchronous PALM algorithm, a new extension of the Proximal Alternating Linearized Minimization (PALM) algorithm for solving nonsmooth, nonconvex optimization problems. Like the PALM algorithm, each step of the Asynchronous PALM algorithm updates a single block of coordinates; but unlike the PALM algorithm, the Asynchronous PALM algorithm eliminates the need for sequential updates that occur one after the other. Instead, our new algorithm allows each of the coordinate blocks to be updated asynchronously and in any order, which means that any number of computing cores can compute updates in parallel without synchronizing their computations. In practice, this asynchronization strategy often leads to speedups that increase linearly with the number of computing cores. We introduce two variants of the Asynchronous PALM algorithm, one stochastic and one deterministic. In the stochastic \textit{and} deterministic cases, we show that cluster points of the algorithm are stationary points. In the deterministic case, we show that the algorithm converges globally whenever the Kurdyka-{\L}ojasiewicz property holds for a function closely related to the objective function, and we derive its convergence rate in a common special case. Finally, we provide a concrete case in which our assumptions hold.