Local Second-Order Limit Dynamics of the Alternating Direction Method of Multipliers for Semidefinite Programming

TL;DR

This paper develops a second-order dynamical framework to analyze why ADMM exhibits slow convergence in semidefinite programming, explaining phenomena like persistent drift and iterate trapping.

Contribution

It introduces a local second-order limit dynamics model for ADMM near KKT points, revealing new insights into its slow convergence behavior.

Findings

Identifies directions where ADMM updates stagnate, causing slow convergence.

Explains why primal-dual infeasibilities are insensitive to penalty changes.

Predicts iterate trapping in low-dimensional subspaces during slow convergence.

Abstract

The alternating direction method of multipliers (ADMM) is widely used for solving large-scale semidefinite programs (SDPs), yet on instances with multiple primal-dual optimal solution pairs, it often enters prolonged slow-convergence regions where the Karush-Kuhn-Tucker (KKT) residuals nearly stall. To explain and predict the fine-grained dynamical behavior inside these regions, we develop a local second-order limit dynamics framework for ADMM near an arbitrary KKT point -- not necessarily the eventual limit point of the iterates. Assuming the existence of a strictly complementary primal-dual solution pair, we derive a second-order local expansion of the ADMM dynamics by leveraging a refined and simplified variational characterization of the (parabolic) second-order directional derivative of the PSD projection operator. This expansion reveals a closed convex cone of directions along which the local first-order update vanishes, and it induces a second-order limit map that governs the persistent drift after transient effects are filtered out. We characterize fundamental properties of this mapping, including its kernel, range, and continuity. A primal-dual decoupling further yields a clean scaling law for the effect of the penalty parameter in ADMM. We connect these properties to second-order dynamical features of ADMM, including fixed points, almost-invariant sets, and microscopic phases. Three empirical phenomena in slow-convergence regions are then explained or predicted: (i) angles between consecutive iterate differences are small yet nonzero, except for sparse spikes; (ii) primal and dual infeasibilities are insensitive to penalty-parameter updates; and (iii) iterates can be transiently trapped in a low-dimensional subspace for an extended period. Extensive numerical experiments on the Mittelmann dataset corroborate our theoretical predictions.