Vectorization of a thread-parallel Jacobi singular value decomposition method

TL;DR

This paper introduces a vectorized, thread-parallel Jacobi SVD method that leverages AVX-512 instructions for improved performance and reproducibility, with enhanced numerical accuracy over traditional LAPACK routines.

Contribution

The paper presents a novel vectorized implementation of batched eigenvalue decomposition and a parallel Jacobi SVD method that guarantees reproducibility and improves accuracy.

Findings

Batched EVD is vectorized using AVX-512 instructions.

The proposed EVD often surpasses LAPACK's xLAEV2 in accuracy.

Speedup is modest but beneficial with sufficient threading.

Abstract

The eigenvalue decomposition (EVD) of (a batch of) Hermitian matrices of order two has a role in many numerical algorithms, of which the one-sided Jacobi method for the singular value decomposition (SVD) is the prime example. In this paper the batched EVD is vectorized, with a vector-friendly data layout and the AVX-512 SIMD instructions of Intel CPUs, alongside other key components of a real and a complex OpenMP-parallel Jacobi-type SVD method, inspired by the sequential xGESVJ routines from LAPACK. These vectorized building blocks should be portable to other platforms that support similar vector operations. Unconditional numerical reproducibility is guaranteed for the batched EVD, sequential or threaded, and for the column transformations, that are, like the scaled dot-products, presently sequential but can be threaded if nested parallelism is desired. No avoidable overflow of the results can occur with the proposed EVD or the whole SVD. The measured accuracy of the proposed EVD often surpasses that of the xLAEV2 routines from LAPACK. While the batched EVD outperforms the matching sequence of xLAEV2 calls, speedup of the parallel SVD is modest but can be improved and is already beneficial with enough threads. Regardless of their number, the proposed SVD method gives identical results, but of somewhat lower accuracy than xGESVJ.