Moreau Envelope ADMM for Decentralized Weakly Convex Optimization

TL;DR

This paper introduces a proximal ADMM variant for decentralized weakly convex optimization, demonstrating convergence to stationary points and superior performance in numerical experiments.

Contribution

It develops a novel proximal ADMM method using Moreau envelope analysis to ensure convergence for weakly convex functions in distributed settings.

Findings

Converges to stationary points under mild conditions

Faster and more robust than existing methods

Provides bounds on dual variable updates

Abstract

This paper proposes a proximal variant of the alternating direction method of multipliers (ADMM) for distributed optimization. Although the current versions of ADMM algorithm provide promising numerical results in producing solutions that are close to optimal for many convex and non-convex optimization problems, it remains unclear if they can converge to a stationary point for weakly convex and locally non-smooth functions. Through our analysis using the Moreau envelope function, we demonstrate that MADM can indeed converge to a stationary point under mild conditions. Our analysis also includes computing the bounds on the amount of change in the dual variable update step by relating the gradient of the Moreau envelope function to the proximal function. Furthermore, the results of our numerical experiments indicate that our method is faster and more robust than widely-used approaches.