A Systematic Literature Survey of Sparse Matrix-Vector Multiplication

TL;DR

This paper provides a comprehensive systematic survey of recent advancements in sparse matrix-vector multiplication (SpMV), analyzing various optimization techniques across modern hardware architectures and identifying future research challenges.

Contribution

It offers a detailed classification and comparison of SpMV optimization methods, filling a gap in the literature with an extensive experimental evaluation and analysis.

Findings

Performance varies significantly across different optimization techniques.

Auto-tuning and machine learning approaches show promising results.

Several hardware-specific challenges remain unresolved.

Abstract

Sparse matrix-vector multiplication (SpMV) is a crucial computing kernel with widespread applications in iterative algorithms. Over the past decades, research on SpMV optimization has made remarkable strides, giving rise to various optimization contributions. However, the comprehensive and systematic literature survey that introduces, analyzes, discusses, and summarizes the advancements of SpMV in recent years is currently lacking. Aiming to fill this gap, this paper compares existing techniques and analyzes their strengths and weaknesses. We begin by highlighting two representative applications of SpMV, then conduct an in-depth overview of the important techniques that optimize SpMV on modern architectures, which we specifically classify as classic, auto-tuning, machine learning, and mixed-precision-based optimization. We also elaborate on the hardware-based architectures, including CPU, GPU, FPGA, processing in Memory, heterogeneous, and distributed platforms. We present a comprehensive experimental evaluation that compares the performance of state-of-the-art SpMV implementations. Based on our findings, we identify several challenges and point out future research directions. This survey is intended to provide researchers with a comprehensive understanding of SpMV optimization on modern architectures and provide guidance for future work.