A Generic Closed-form Optimal Step-size for ADMM

TL;DR

This paper introduces a theoretically optimal, closed-form step-size for ADMM algorithms based on the ratio of dual and primal solutions, with practical strategies for estimation and a new proximal operator definition.

Contribution

It proposes a novel, closed-form step-size for ADMM, addressing a scaling issue in classical proximal operators and providing strategies for practical implementation.

Findings

The step-size is optimal in worst-case convergence bounds.

Numerical tests show near-optimal practical performance.

A new proximal operator definition avoids scaling issues.

Abstract

In this work, we present a generic step-size choice for the ADMM type proximal algorithms. It admits a closed-form expression and is theoretically optimal with respect to a worst-case convergence rate bound. It is simply given by the ratio of Euclidean norms of the dual and primal solutions, i.e., $ ||{\lambda}^\star|| / ||{x}^\star||$. Numerical tests show that its practical performance is near-optimal in general. The only challenge is that such a ratio is not known a priori and we provide two strategies to address it. The derivation of our step-size choice is based on studying the fixed-point structure of ADMM using the proximal operator. However, we demonstrate that the classical proximal operator definition contains an input scaling issue. This leads to a scaled step-size optimization problem which would yield a false solution. Such an issue is naturally avoided by our proposed new definition of the proximal operator. A series of its properties is established.