ADMM Penalty Parameter Selection by Residual Balancing

TL;DR

This paper critically examines a common heuristic for selecting the penalty parameter in ADMM, reveals a significant flaw, and proposes a modification to improve its effectiveness.

Contribution

The paper identifies a flaw in the existing residual balancing heuristic for ADMM penalty parameter selection and proposes a modified approach to address it.

Findings

Identifies a flaw in the residual balancing heuristic.

Proposes a modified penalty parameter selection method.

Improves the reliability of ADMM performance.

Abstract

Appropriate selection of the penalty parameter is crucial to obtaining good performance from the Alternating Direction Method of Multipliers (ADMM). While analytic results for optimal selection of this parameter are very limited, there is a heuristic method that appears to be relatively successful in a number of different problems. The contribution of this paper is to demonstrate that their is a potentially serious flaw in this heuristic approach, and to propose a modification that at least partially addresses it.