Applying acceleration to Krylov subspace eigenvalue solvers

TL;DR

This paper enhances Krylov subspace eigenvalue solvers by integrating acceleration techniques like Nesterov and Polyak's heavy-ball, leading to faster convergence especially for clustered eigenvalues, with theoretical proofs and numerical validation.

Contribution

It introduces accelerated versions of inverse-free preconditioned Krylov methods, extending acceleration to block methods and providing convergence proofs and empirical results.

Findings

Accelerated methods converge in fewer iterations than base methods.

Fixed and adaptive momentum parameters improve convergence speed.

Theoretical justification for Polyak's heavy-ball acceleration range.

Abstract

In this paper, we apply acceleration to the inverse-free preconditioned Krylov subspace method introduced by Golub and Ye, which solves the symmetric generalized eigenvalue problem for the algebraically smallest eigenvalue. As the method is an improvement on steepest descent, we consider acceleration based on Nesterov accelerated steepest descent and Polyak's heavy-ball method. We extend acceleration to the block version of the Krylov subspace method and prove convergence for a more generalized choice of subspace. We present numerical results demonstrating the effect of fixed and safeguarded-adaptive choice of the momentum parameter, which show convergence in fewer outer iterations compared with LOBPCG with the same subspace size and generally fewer iterations than the base method when solving for multiple clustered eigenvalues with small dimension size. We also provide an explanation for the acceleration seen from implementing Polyak's heavy-ball method, including justifying the given parameter range.