The proximal alternating direction method of multipliers in the nonconvex setting: convergence analysis and rates

TL;DR

This paper introduces two new algorithms based on the proximal alternating direction method of multipliers for nonconvex optimization problems, providing convergence analysis and rates under mild conditions.

Contribution

It develops novel proximal splitting algorithms with variable metrics for nonconvex problems and establishes convergence and rate results assuming Kurdyka-Lojasiewicz properties.

Findings

Algorithms converge to KKT points under mild conditions.

Convergence rates are derived assuming Lojasiewicz property.

New proximal splitting schemes extend existing methods to nonconvex settings.

Abstract

We propose two numerical algorithms in the fully nonconvex setting for the minimization of the sum of a smooth function and the composition of a nonsmooth function with a linear operator. The iterative schemes are formulated in the spirit of the proximal alternating direction method of multipliers and its linearized variant, respectively. The proximal terms are introduced via variable metrics, a fact which allows us to derive new proximal splitting algorithms for nonconvex structured optimization problems, as particular instances of the general schemes. Under mild conditions on the sequence of variable metrics and by assuming that a regularization of the associated augmented Lagrangian has the Kurdyka-Lojasiewicz property, we prove that the iterates converge to a KKT point of the objective function. By assuming that the augmented Lagrangian has the Lojasiewicz property, we also derive convergence rates for both the augmented Lagrangian and the iterates.