Alternating Direction Method of Multipliers for nonlinear constrained convex problems and applications to distributed resource allocation and constrained machine learning

TL;DR

This paper introduces a nonlinear ADMM method for structured convex problems with nonlinear constraints, improving communication efficiency in distributed resource allocation and machine learning applications.

Contribution

The paper proposes a novel NL-ADMM algorithm that handles convex problems with nonlinear constraints without requiring smoothness, extending ADMM's applicability and convergence guarantees.

Findings

NL-ADMM achieves up to 100x higher communication efficiency than classical methods.

The method converges globally with an ergodic rate of O(1/k).

Numerical experiments demonstrate significant performance improvements in distributed tasks.

Abstract

We study a class of structured convex optimization problems, which have a two-block separable objective and nonlinear functional constraints as well as affine constraints that couple the two block variables. Such problems naturally arise from distributed resource allocation and constrained machine learning. To achieve high communication efficiency for the distributed applications, we propose a nonlinear alternating direction method of multipliers (NL-ADMM) that preserves the classical splitting structure while accommodating general convex functional constraints. Unlike existing ADMM variants for nonconvex constrained problems, the proposed method does not require smoothness of the objective functions or differentiability of the constraint mapping, by leveraging convexity of the considered problem. We establish global convergence and an ergodic $O(1/k)$ convergence rate of NL-ADMM by assuming the existence of a KKT solution. The results extend those of ADMM for linearly constrained convex problems. Numerical experiments are conducted on two representative distributed tasks. The results on numerous instances demonstrate that NL-ADMM can achieve (in many cases) 100x higher communication efficiency than the classic augmented Lagrangian method and nearly 2x higher than the Douglas-Rachford operator splitting method, making the new method well suited for large-scale distributed learning systems.