A Framework of Inertial Alternating Direction Method of Multipliers for Non-Convex Non-Smooth Optimization

TL;DR

This paper introduces an inertial ADMM framework for nonconvex nonsmooth optimization, unifying and extending previous methods with convergence guarantees and demonstrating effectiveness on low-rank problems.

Contribution

It proposes a novel inertial ADMM scheme based on the MM principle for nonconvex nonsmooth problems, with convergence analysis and practical validation.

Findings

Proved subsequential and global convergence of iADMM.

Demonstrated effectiveness on nonconvex low-rank problems.

Unified analysis for ADMM with surrogate functions and inertial terms.

Abstract

In this paper, we propose an algorithmic framework, dubbed inertial alternating direction methods of multipliers (iADMM), for solving a class of nonconvex nonsmooth multiblock composite optimization problems with linear constraints. Our framework employs the general minimization-majorization (MM) principle to update each block of variables so as to not only unify the convergence analysis of previous ADMM that use specific surrogate functions in the MM step, but also lead to new efficient ADMM schemes. To the best of our knowledge, in the nonconvex nonsmooth setting, ADMM used in combination with the MM principle to update each block of variables, and ADMM combined with \emph{inertial terms for the primal variables} have not been studied in the literature. Under standard assumptions, we prove the subsequential convergence and global convergence for the generated sequence of iterates. We illustrate the effectiveness of iADMM on a class of nonconvex low-rank representation problems.