Adaptive ADMM with Spectral Penalty Parameter Selection

TL;DR

This paper introduces an adaptive version of ADMM that automatically tunes penalty parameters using a spectral method, leading to faster convergence and reduced sensitivity to initial settings.

Contribution

The paper proposes AADMM, an adaptive ADMM algorithm that employs spectral penalty parameter selection inspired by Barzilai-Borwein, improving convergence speed and robustness.

Findings

AADMM converges faster than traditional ADMM.

AADMM is less sensitive to initial parameter choices.

The spectral penalty method enhances robustness across problems.

Abstract

The alternating direction method of multipliers (ADMM) is a versatile tool for solving a wide range of constrained optimization problems, with differentiable or non-differentiable objective functions. Unfortunately, its performance is highly sensitive to a penalty parameter, which makes ADMM often unreliable and hard to automate for a non-expert user. We tackle this weakness of ADMM by proposing a method to adaptively tune the penalty parameters to achieve fast convergence. The resulting adaptive ADMM (AADMM) algorithm, inspired by the successful Barzilai-Borwein spectral method for gradient descent, yields fast convergence and relative insensitivity to the initial stepsize and problem scaling.