Krylov Subspace Acceleration for First-Order Splitting Methods in Convex Quadratic Programming

TL;DR

This paper introduces a Krylov subspace acceleration method for first-order algorithms in convex quadratic programming, improving convergence speed and stability over existing Anderson acceleration techniques.

Contribution

The authors develop a novel Krylov subspace acceleration scheme tailored for convex QPs, addressing ill-conditioning issues and outperforming Anderson acceleration in various applications.

Findings

Our method reduces iteration count compared to Anderson acceleration.

It demonstrates faster convergence in model predictive control problems.

The scheme maintains stability and efficiency at high accuracy levels.

Abstract

We propose an acceleration scheme for first-order methods (FOMs) for convex quadratic programs (QPs) that is analogous to Anderson acceleration and the Generalized Minimal Residual algorithm for linear systems. We motivate our proposed method from the observation that FOMs applied to QPs typically consist of piecewise-affine operators. We describe our Krylov subspace acceleration scheme, contrasting it with existing Anderson acceleration schemes and showing that it largely avoids the latter's well-known ill-conditioning issues in regions of slow convergence. We demonstrate the performance of our scheme relative to Anderson acceleration using standard collections of problems from model predictive control and statistical learning applications. We show that our method is faster than Anderson acceleration across the board in terms of iteration count, and in many cases in computation time, particularly for optimal control and for problems solved to high accuracy.