Extensions of ADMM for Separable Convex Optimization Problems with Linear Equality or Inequality Constraints

TL;DR

This paper develops a unified framework to extend the ADMM algorithm for separable convex optimization problems that include linear inequality constraints, broadening its applicability and providing convergence guarantees.

Contribution

It introduces a general algorithmic framework and convergence analysis for ADMM extensions to problems with linear inequality constraints, applicable to multiple blocks.

Findings

Proposes a unified ADMM-based algorithmic framework.

Provides a convergence roadmap for these algorithms.

Applicable to various convex optimization problems with different constraints.

Abstract

The alternating direction method of multipliers (ADMM) proposed by Glowinski and Marrocco is a benchmark algorithm for two-block separable convex optimization problems with linear equality constraints. It has been modified, specified, and generalized from various perspectives to tackle more concrete or complicated application problems. Despite its versatility and phenomenal popularity, it remains unknown whether or not the ADMM can be extended to separable convex optimization problems with linear inequality constraints. In this paper, we lay down the foundation of how to extend the ADMM to two-block and multiple-block (more than two blocks) separable convex optimization problems with linear inequality constraints. From a high-level and methodological perspective, we propose a unified framework of algorithmic design and a roadmap for convergence analysis in the context of variational inequalities, based on which it is possible to design a series of concrete ADMM-based algorithms with provable convergence in the prediction-correction structure. The proposed algorithmic framework and roadmap for convergence analysis are eligible to various convex optimization problems with different degrees of separability, in which both linear equality and linear inequality constraints can be included. The analysis is comprehensive yet can be presented by elementary mathematics, and hence generically understandable.