An Adaptive Multiparameter Penalty Selection Method for Multiconstraint and Multiblock ADMM

TL;DR

This paper introduces an adaptive method for selecting multiple penalty parameters in ADMM to improve convergence in constrained optimization problems with varying scales.

Contribution

The proposed method adaptively adjusts multiple penalty parameters in ADMM, enhancing robustness and convergence speed for multi-constraint problems.

Findings

Outperforms existing penalty selection methods in numerical experiments.

Robust to problem transformations and initial parameter choices.

Effectively handles scale differences between constraints.

Abstract

This work presents a new method for online selection of multiple penalty parameters for the alternating direction method of multipliers (ADMM) algorithm applied to optimization problems with multiple constraints or functionals with block matrix components. ADMM is widely used for solving constrained optimization problems in a variety of fields, including signal and image processing. Implementations of ADMM often utilize a single hyperparameter, referred to as the penalty parameter, which needs to be tuned to control the rate of convergence. However, in problems with multiple constraints, ADMM may demonstrate slow convergence regardless of penalty parameter selection due to scale differences between constraints. Accounting for scale differences between constraints to improve convergence in these cases requires introducing a penalty parameter for each constraint. The proposed method is able to adaptively account for differences in scale between constraints, providing robustness with respect to problem transformations and initial selection of penalty parameters. It is also simple to understand and implement. Our numerical experiments demonstrate that the proposed method performs favorably compared to a variety of existing penalty parameter selection methods.