On the dual step length of the alternating direction method of multipliers

TL;DR

This paper investigates the limits of the dual step length in the ADMM algorithm, showing through analysis and counterexamples that exceeding the golden ratio may compromise convergence guarantees.

Contribution

It provides a rigorous analysis indicating the golden ratio as an upper bound for the dual step length in general convex settings within the conventional framework.

Findings

Counterexample shows loss of monotonicity beyond the golden ratio

Analysis confirms the golden ratio as a critical threshold

Performance estimation framework used for convergence analysis

Abstract

The alternating direction method of multipliers (ADMM) is a most widely used optimization scheme for solving linearly constrained separable convex optimization problems. The convergence of the ADMM can be guaranteed when the dual step length is less than the golden ratio, while plenty of numerical evidence suggests that even larger dual step length often accelerates the convergence. It has also been proved that the dual step length can be enlarged to less than 2 in some special cases, namely, one of the separable functions in the objective function is linear, or both are quadratic plus some additional assumptions. However, it remains unclear whether the golden ratio can be exceeded in the general convex setting. In this paper, the performance estimation framework is used to analyze the convergence of the ADMM, and assisted by numerical and symbolic computations, a counter example is constructed, which indicates that the conventional measure may lose monotonicity as the dual step length exceeds the golden ratio, ruling out the possibility of breaking the golden ratio within this conventional analytic framework.