Constraint-Preconditioned Krylov Solvers for Regularized Saddle-Point Systems

TL;DR

This paper develops and analyzes constraint-preconditioned Krylov subspace methods, including variants of Lanczos and Arnoldi algorithms, for efficiently solving regularized saddle-point systems in optimization.

Contribution

It generalizes the application of constraint preconditioning to a broad class of Krylov methods, providing theoretical guidelines and MATLAB implementations.

Findings

Constraint-preconditioned Krylov methods improve convergence for saddle-point systems.

New variants of Lanczos and Arnoldi methods are tailored for regularized saddle-point problems.

Numerical experiments demonstrate effectiveness on optimization-related systems.

Abstract

We consider the iterative solution of regularized saddle-point systems. When the leading block is symmetric and positive semi-definite on an appropriate subspace, Dollar, Gould, Schilders, and Wathen (2006) describe how to apply the conjugate gradient (CG) method coupled with a constraint preconditioner, a choice that has proved to be effective in optimization applications. We investigate the design of constraint-preconditioned variants of other Krylov methods for regularized systems by focusing on the underlying basis-generation process. We build upon principles laid out by Gould, Orban, and Rees (2014) to provide general guidelines that allow us to specialize any Krylov method to regularized saddle-point systems. In particular, we obtain constraint-preconditioned variants of Lanczos and Arnoldi-based methods, including the Lanczos version of CG, MINRES, SYMMLQ, GMRES(m) and DQGMRES. We also provide MATLAB implementations in hopes that they are useful as a basis for the development of more sophisticated software. Finally, we illustrate the numerical behavior of constraint-preconditioned Krylov solvers using symmetric and nonsymmetric systems arising from constrained optimization.