Explicit caching HYB: a new high-performance SpMV framework on GPGPU

TL;DR

This paper introduces EHYB, a new GPU-based SpMV framework that explicitly caches input vectors and optimizes data movement, significantly improving performance over existing methods in finite element method applications.

Contribution

The paper presents a novel explicit caching framework for GPU SpMV that reduces data movement and enhances performance beyond current state-of-the-art approaches.

Findings

EHYB outperforms existing GPU SpMV implementations.

Significant speedup achieved through explicit input vector caching.

Higher FLOPs than theoretical performance bounds.

Abstract

Sparse Matrix-Vector Multiplication (SpMV) is a critical operation for the iterative solver of Finite Element Methods on computer simulation. Since the SpMV operation is a memory-bound algorithm, the efficiency of data movements heavily influenced the performance of the SpMV on GPU. In recent years, many research is conducted in accelerating the performance of SpMV on the graphic processing units (GPU). The performance optimization methods used in existing studies focus on the following areas: improve the load balancing between GPU processors, and reduce the execution divergence between GPU threads. Although some studies have made preliminary optimization on the input vector fetching, the effect of explicitly caching the input vector on GPU base SpMV has not been studied in depth yet. In this study, we are trying to minimize the data movements cost for GPU-based SpMV using a new framework named "explicit caching Hybrid (EHYB)". The EHYB framework achieved significant performance improvement by using the following methods: 1. Improve the speed of data movements by partitioning and explicitly caching the input vector to the shared memory of the CUDA kernel. 2. Reduce the volume of data movements by storing the major part of the column index with a compact format. We tested our implementation with sparse matrices derived from FEM applications in different areas. The experiment results show that our implementation can overperform the state-of-the-arts implementation with significant speedup, and leads to higher FLOPs than the theoryperformance up-boundary of the existing GPU-based SpMV implementations.