Minimizing the Arithmetic and Communication Complexity of Jacobi's Method for Eigenvalues and Singular Values: Part One -- Serial Algorithms

TL;DR

This paper rigorously analyzes the complexity of serial Jacobi algorithms for eigenvalues and SVD, achieving near-optimal arithmetic and communication costs using blocking and recursive strategies.

Contribution

It provides the first detailed complexity bounds for serial Jacobi methods with classical and fast matrix multiplication, optimizing both arithmetic and communication costs.

Findings

Blocked Jacobi attains the communication lower bound for classical matrix multiplication.

Recursive Jacobi achieves essentially optimal complexity in both arithmetic and communication.

Complexity bounds are also derived for Jacobi SVD algorithms.

Abstract

We analyze several versions of Jacobi's method for the symmetric eigenvalue problem. Our goal is to reduce the asymptotic cost of the algorithm as much as possible, as measured by the number of arithmetic operations performed and associated (serial or parallel) communication, i.e., the amount of data moved between slow and fast memory or between processors in a network. The first half of this effort, which considers the serial setting, is presented here; this paper contains rigorous complexity bounds for a variety of serial Jacobi algorithms, built on both classic $O(n^3)$ matrix multiplication and fast, Strassen-like $O(n^{\omega_0})$ alternatives. In the classical case, we show that a blocked implementation of Jacobi's method attains the communication lower bound for $O(n^3)$ matrix multiplication (and is therefore expected to be communication optimal among $O(n^3)$ eigensolvers). In the fast setting, we demonstrate that a recursive version of blocked Jacobi can go further, reaching essentially optimal complexity in both measures. We also derive analogous complexity bounds for (one-sided) Jacobi SVD algorithms. A forthcoming sequel to this paper will extend our complexity analysis to the parallel case.