Minimizing Communication for Eigenproblems and the Singular Value Decomposition

TL;DR

This paper develops new algorithms for eigenproblems and SVD that minimize data movement, achieving theoretical lower bounds on communication costs in both parallel and sequential settings.

Contribution

It introduces algorithms that attain the fundamental communication lower bounds for eigenvalue and SVD computations, improving over conventional methods.

Findings

Algorithms match the theoretical communication lower bounds.

Parallel and sequential algorithms are developed and analyzed.

Convergence and communication costs are thoroughly examined.

Abstract

Algorithms have two costs: arithmetic and communication. The latter represents the cost of moving data, either between levels of a memory hierarchy, or between processors over a network. Communication often dominates arithmetic and represents a rapidly increasing proportion of the total cost, so we seek algorithms that minimize communication. In \cite{BDHS10} lower bounds were presented on the amount of communication required for essentially all $O(n^3)$-like algorithms for linear algebra, including eigenvalue problems and the SVD. Conventional algorithms, including those currently implemented in (Sca)LAPACK, perform asymptotically more communication than these lower bounds require. In this paper we present parallel and sequential eigenvalue algorithms (for pencils, nonsymmetric matrices, and symmetric matrices) and SVD algorithms that do attain these lower bounds, and analyze their convergence and communication costs.