GMRES-Accelerated ADMM for Quadratic Objectives

TL;DR

This paper introduces a GMRES-accelerated ADMM method for quadratic problems, significantly reducing iteration counts and improving convergence speed in solving large-scale conic optimization problems.

Contribution

It provides a theoretical and practical framework for accelerating ADMM with GMRES, achieving faster convergence for strongly convex quadratic objectives.

Findings

GMRES-accelerated ADMM reduces iteration complexity from O(√κ) to O(κ^{1/4})

The method outperforms standard preconditioned Krylov methods on saddle-point problems

Achieves O(1/k^{2}) error decay in large-scale semidefinite programming

Abstract

We consider the sequence acceleration problem for the alternating direction method-of-multipliers (ADMM) applied to a class of equality-constrained problems with strongly convex quadratic objectives, which frequently arise as the Newton subproblem of interior-point methods. Within this context, the ADMM update equations are linear, the iterates are confined within a Krylov subspace, and the General Minimum RESidual (GMRES) algorithm is optimal in its ability to accelerate convergence. The basic ADMM method solves a $\kappa$-conditioned problem in $O(\sqrt{\kappa})$ iterations. We give theoretical justification and numerical evidence that the GMRES-accelerated variant consistently solves the same problem in $O(\kappa^{1/4})$ iterations for an order-of-magnitude reduction in iterations, despite a worst-case bound of $O(\sqrt{\kappa})$ iterations. The method is shown to be competitive against standard preconditioned Krylov subspace methods for saddle-point problems. The method is embedded within SeDuMi, a popular open-source solver for conic optimization written in MATLAB, and used to solve many large-scale semidefinite programs with error that decreases like $O(1/k^{2})$, instead of $O(1/k)$, where $k$ is the iteration index.