Improved pointwise iteration-complexity of a regularized ADMM and of a regularized non-Euclidean HPE framework

TL;DR

This paper introduces a regularized ADMM variant with improved pointwise iteration-complexity, matching the ergodic complexity up to a logarithmic factor, by linking it to a non-Euclidean HPE framework.

Contribution

It develops a regularized ADMM with enhanced pointwise complexity and connects it to a novel non-Euclidean HPE framework for better analysis.

Findings

Pointwise iteration-complexity is improved over standard ADMM.

Complexity matches ergodic bounds up to a logarithmic factor.

The framework handles both relative and summable errors.

Abstract

This paper describes a regularized variant of the alternating direction method of multipliers (ADMM) for solving linearly constrained convex programs. It is shown that the pointwise iteration-complexity of the new method is better than the corresponding one for the standard ADMM method and that, up to a logarithmic term, is identical to the ergodic iteration-complexity of the latter method. Our analysis is based on first presenting and establishing the pointwise iteration-complexity of a regularized non-Euclidean hybrid proximal extragradient framework whose error condition at each iteration includes both a relative error and a summable error. It is then shown that the new method is a special instance of the latter framework where the sequence of summable errors is identically zero when the ADMM stepsize is less than one or a nontrivial sequence when the stepsize is in the interval [1, (1 +\sqrt{5})/2).