SparseZipper: Enhancing Matrix Extensions to Accelerate SpGEMM on CPUs

TL;DR

SparseZipper is a novel approach that modifies existing matrix extensions to efficiently accelerate sparse matrix multiplication on CPUs, significantly reducing wasted work and maintaining low area overhead.

Contribution

It introduces SparseZipper, a minimal modification to existing architectures that improves sparse-sparse GEMM performance on CPUs with minimal area increase.

Findings

Achieves nearly 6x speedup over scalar hash-based SpGEMM

More than 2.6x faster than state-of-the-art vectorized SpGEMM

Increases systolic array area by only 12.7%

Abstract

The importance of general matrix multiplication (GEMM) is motivating new instruction set extensions for multiplying dense matrices in almost all contemporary ISAs, and these extensions are often implemented using high-performance systolic arrays. However, matrices in emerging workloads are not always dense, and sparse matrices where the vast majority of values are zeros are becoming more common. Existing matrix extensions and micro-architectures cannot efficiently process highly sparse matrices due to two reasons: (1) wasted work when one or both input values are zero; and (2) incompatibility with sparse matrix formats. This work proposes SparseZipper that minimally modifies existing matrix extensions and systolic-array-based micro-architectures specialized for dense-dense GEMM to accelerate sparse-sparse GEMM operating on highly sparse matrices with unstructured sparsity structures. Our performance evaluation shows SparseZipper achieves 5.98x and 2.61x speedup over a scalar hash-based implementation of SpGEMM and a state-of-the-art vectorized SpGEMM version, respectively. Our component-level area evaluation shows SparseZipper increases the area of a baseline 16x16 systolic array by only 12.7% resulting in an area overhead for an entire system-on-chip of just a few percent.