An Empirical Study of ADMM for Nonconvex Problems

TL;DR

This paper empirically evaluates ADMM's effectiveness on various nonconvex problems, demonstrating its practical performance and the benefits of adaptive penalty tuning for improved efficiency and solutions.

Contribution

It provides the first comprehensive empirical analysis of ADMM on multiple nonconvex applications, highlighting the advantages of adaptive penalty methods.

Findings

ADMM performs well on diverse nonconvex problems.

Adaptive ADMM improves efficiency and solution quality.

Empirical evidence supports ADMM's practical utility in nonconvex optimization.

Abstract

The alternating direction method of multipliers (ADMM) is a common optimization tool for solving constrained and non-differentiable problems. We provide an empirical study of the practical performance of ADMM on several nonconvex applications, including l0 regularized linear regression, l0 regularized image denoising, phase retrieval, and eigenvector computation. Our experiments suggest that ADMM performs well on a broad class of non-convex problems. Moreover, recently proposed adaptive ADMM methods, which automatically tune penalty parameters as the method runs, can improve algorithm efficiency and solution quality compared to ADMM with a non-tuned penalty.