Convergence Analysis of Alternating Direction Method of Multipliers for a Family of Nonconvex Problems

TL;DR

This paper provides a theoretical convergence analysis of the ADMM algorithm for certain nonconvex optimization problems, showing it converges to stationary points under specific conditions without restrictive assumptions.

Contribution

It establishes convergence of ADMM for nonconvex problems with large penalty parameters and proves convergence regardless of variable block number, broadening theoretical understanding.

Findings

ADMM converges to stationary solutions for nonconvex problems with sufficiently large penalty.

Convergence holds regardless of the number of variable blocks in sharing problems.

The analysis applies broadly to various ADMM variants without assumptions on iterates.

Abstract

The alternating direction method of multipliers (ADMM) is widely used to solve large-scale linearly constrained optimization problems, convex or nonconvex, in many engineering fields. However there is a general lack of theoretical understanding of the algorithm when the objective function is nonconvex. In this paper we analyze the convergence of the ADMM for solving certain nonconvex consensus and sharing problems, and show that the classical ADMM converges to the set of stationary solutions, provided that the penalty parameter in the augmented Lagrangian is chosen to be sufficiently large. For the sharing problems, we show that the ADMM is convergent regardless of the number of variable blocks. Our analysis does not impose any assumptions on the iterates generated by the algorithm, and is broadly applicable to many ADMM variants involving proximal update rules and various flexible block selection rules.