An Adaptive Alternating Direction Method of Multipliers

TL;DR

This paper introduces an adaptive version of the ADMM algorithm tailored for nonconvex optimization problems involving a sum of strongly and weakly convex functions, with proven convergence and practical demonstrations.

Contribution

It develops and analyzes an adaptive ADMM that handles nonconvex functions with generalized convexity, extending existing methods and establishing convergence under natural conditions.

Findings

Convergence of the adaptive ADMM is proven under natural assumptions.

The method effectively handles nonconvex optimization with mixed convexity properties.

Numerical experiments demonstrate the approach's effectiveness on signal denoising.

Abstract

The alternating direction method of multipliers (ADMM) is a powerful splitting algorithm for linearly constrained convex optimization problems. In view of its popularity and applicability, a growing attention is drawn towards the ADMM in nonconvex settings. Recent studies of minimization problems for noncovex functions include various combinations of assumptions on the objective function including, in particular, a Lipschitz gradient assumption. We consider the case where the objective is the sum of a strongly convex function and a weakly convex function. To this end we present and study an adaptive version of the ADMM which incorporates generalized notions of convexity and varying penalty parameters adapted to the convexity constants of the functions. We prove convergence of the scheme under natural assumptions. To this end we employ the recent adaptive Douglas--Rachford algorithm by revisiting the well known duality relation between the classical ADMM and the Douglas--Rachford splitting algorithm, generalizing this connection to our setting. We illustrate our approach by relating and comparing to alternatives, and by numerical experiments on a signal denoising problem.