Extracting the Potential of Emerging Hardware Accelerators for Symmetric Eigenvalue Decomposition

TL;DR

This paper analyzes the bottlenecks of symmetric eigenvalue decomposition on emerging hardware accelerators and proposes algorithmic optimizations that significantly improve performance, especially on GPUs like H100, A100, and RTX 4090.

Contribution

It identifies the memory-bound bottlenecks in conventional EVD algorithms and introduces optimized methods that leverage hardware features for better utilization and speedup.

Findings

Up to 10.1x speedup on H100 GPU

Up to 7.5x speedup on A100 GPU

Up to 4.1x overall EVD performance improvement

Abstract

Benefiting from the advancement of hardware accelerators such as GPUs, deep neural networks and scientific computing applications can achieve superior performance. Recently, the computing capacity of emerging hardware accelerators has increased rapidly, while memory bandwidth has not kept pace with this growth. This disparity exacerbates the gap between computing and memory, leading to inefficiencies on conventional algorithms, as they're likely to be converted from compute-bound to memory-bound. Symmetric eigenvalue decomposition (EVD), a critical operation in various research domains including scientific computing, deep learning training, and inference algorithms, exhibits suboptimal performance due to achieving less than 3\% hardware computing utilization on the H100 GPU. In this paper, we analyze the features of emerging hardware accelerators to identify the bottlenecks inherent in conventional EVD algorithms. To improve EVD performance, we propose several algorithmic optimizations aimed at solving the memory-bound problem and providing a better utilization of the rich computing capacity and parallelism on the emerging hardware accelerators. Experimentally, our proposed method demonstrates significant speedups on tridiagonalization, which is the main workload that takes over 90\% elapsed time of EVD, compared to the SOTA cuSOLVER tridiagonalization, achieving up to 10.1x, 7.5x, and 2.3x improvements on H100, A100, and RTX 4090 GPUs, respectively. And the end-to-end the performance of EVD solver is also up to 4.1x faster than cuSOVLER.