Accelerating Sparse Matrix-Matrix Multiplication on GPUs with Processing Near HBMs

TL;DR

This paper introduces a hardware-software co-designed framework with near-memory processing techniques to accelerate sparse matrix-matrix multiplication on GPUs, significantly improving performance for graph analytics and GNN workloads.

Contribution

It presents a novel Hash based Multi-phase SpGEMM and AIA technique, optimizing irregular memory access patterns on GPU HBM, outperforming existing methods.

Findings

Up to 17.3% time reduction in graph analytics applications

76.5% and 58.4% speedups for Graph Contraction and Markov Clustering

Average 1.43x speedup for GNN training across multiple datasets

Abstract

Sparse General Matrix-Matrix Multiplication (SpGEMM) is a fundamental operation in numerous scientific computing and data analytics applications, often bottlenecked by irregular memory access patterns. This paper presents Hash based Multi-phase SpGEMM on GPU and the Acceleration of Indirect Memory Access (AIA) technique, a novel custom near-memory processing approach to optimizing SpGEMM on GPU HBM. Our hardware-software co-designed framework for SpGEMM demonstrates significant performance improvements over state-of-the-art methods, particularly in handling complex, application-specific workloads. We evaluate our approach on various graph workloads, including graph contraction, Markov clustering, and Graph Neural Networks (GNNs), showcasing its practical applicability. For graph analytics applications, AIA demonstrates up to 17.3% time reduction from the software-only implementation, while achieving time reduction of 76.5% for Graph Contraction and 58.4% for Markov Clustering compared to cuSPARSE. For GNN training applications with structured global pruning, our hybrid approach delivers an average of 1.43x speedup over software-only implementation across six benchmark datasets and three architectures (GCN, GIN, GraphSAGE), and shows 1.95x speedup for GNN workloads when compared to cuSPARSE, with up to 4.18x gains on large-scale datasets.