Investigating Customization Strategies and Convergence Behaviors of Task-specific ADMM

TL;DR

This paper introduces GO-ADMM, a flexible framework that incorporates task-specific modules into ADMM using an optimality-guided strategy, ensuring convergence and improving performance in real-world applications.

Contribution

The paper proposes GO-ADMM, a novel customization strategy for ADMM that guarantees convergence without restrictions on embedded modules, enhancing its adaptability and effectiveness.

Findings

Proves convergence of GO-ADMM with arbitrary modules

Derives worst-case iteration complexity for GO-ADMM

Demonstrates improved performance through extensive experiments

Abstract

Alternating Direction Method of Multiplier (ADMM) has been a popular algorithmic framework for separable optimization problems with linear constraints. For numerical ADMM fail to exploit the particular structure of the problem at hand nor the input data information, leveraging task-specific modules (e.g., neural networks and other data-driven architectures) to extend ADMM is a significant but challenging task. This work focuses on designing a flexible algorithmic framework to incorporate various task-specific modules (with no additional constraints) to improve the performance of ADMM in real-world applications. Specifically, we propose Guidance from Optimality (GO), a new customization strategy, to embed task-specific modules into ADMM (GO-ADMM). By introducing an optimality-based criterion to guide the propagation, GO-ADMM establishes an updating scheme agnostic to the choice of additional modules. The existing task-specific methods just plug their task-specific modules into the numerical iterations in a straightforward manner. Even with some restrictive constraints on the plug-in modules, they can only obtain some relatively weaker convergence properties for the resulted ADMM iterations. Fortunately, without any restrictions on the embedded modules, we prove the convergence of GO-ADMM regarding objective values and constraint violations, and derive the worst-case convergence rate measured by iteration complexity. Extensive experiments are conducted to verify the theoretical results and demonstrate the efficiency of GO-ADMM.