Local Convergence Analysis of ADMM for Nonconvex Composite Optimization

TL;DR

This paper analyzes the local convergence of ADMM for nonconvex composite problems common in imaging and machine learning, establishing conditions for convergence and linear rates under certain assumptions.

Contribution

It provides the first local convergence analysis of ADMM for nonconvex problems with polyhedral constraints, including a new strong convexity property of the Moreau envelope.

Findings

ADMM converges locally to primal-dual solutions under specific conditions.

A local linear convergence rate is established for polyhedral convex constraints.

Three examples demonstrate the applicability and necessity of the assumptions.

Abstract

In this paper, we study the local convergence of the standard ADMM scheme for a class of nonconvex composite problems arising from modern imaging and machine learning models. This problem is constrained by a closed convex set, while its objective is the sum of a continuously differentiable (possibly nonconvex) smooth term and a polyhedral convex nonsmooth term composed with a linear mapping. Our analysis is mainly motivated by the recent works of Rockafellar [29,30]. We begin with an elementary proof of a key local strong convexity property of the Moreau envelope of polyhedral convex functions. Building on this property, we show that the strong variational sufficiency condition holds for the considered problem under appropriate assumptions. Using the strong variational sufficiency condition, we further derive a descent inequality for the ADMM iterates, in a form analogous to the classical descent analysis of ADMM for convex problems. As a consequence, for a suitable choice of the penalty parameter, we establish local convergence of the ADMM scheme to a primal-dual solution, and a local linear convergence rate for the case where the constraint set is polyhedral convex. Finally, we present three analytic examples to illustrate the applicability of our local convergence result and the necessity of the local assumptions.