Convergence theory for preconditioned eigenvalue solvers in a nutshell

TL;DR

This paper presents a simplified and novel proof of convergence bounds for preconditioned eigenvalue solvers, crucial for ensuring their near-optimal complexity in large-scale problems across various scientific fields.

Contribution

It introduces a new, succinct proof technique for convergence bounds using Karush-Kuhn-Tucker theory and nonlinear programming, advancing theoretical understanding of preconditioned eigenvalue methods.

Findings

Provides a more concise proof of convergence bounds.

Uses novel ideas from nonlinear programming.

Supports near-optimal complexity claims for eigenvalue solvers.

Abstract

Preconditioned iterative methods for numerical solution of large matrix eigenvalue problems are increasingly gaining importance in various application areas, ranging from material sciences to data mining. Some of them, e.g., those using multilevel preconditioning for elliptic differential operators or graph Laplacian eigenvalue problems, exhibit almost optimal complexity in practice, i.e., their computational costs to calculate a fixed number of eigenvalues and eigenvectors grow linearly with the matrix problem size. Theoretical justification of their optimality requires convergence rate bounds that do not deteriorate with the increase of the problem size. Such bounds were pioneered by E. D'yakonov over three decades ago, but to date only a handful have been derived, mostly for symmetric eigenvalue problems. Just a few of known bounds are sharp. One of them is proved in [doi:10.1016/S0024-3795(01)00461-X] for the simplest preconditioned eigensolver with a fixed step size. The original proof has been greatly simplified and shortened in [doi:10.1137/080727567] by using a gradient flow integration approach. In the present work, we give an even more succinct proof, using novel ideas based on Karush-Kuhn-Tucker theory and nonlinear programming.