Fast Entropy Decoding for Sparse MVM on GPUs

TL;DR

This paper introduces dtANS, a lossless entropy coding method that compresses sparse matrices more efficiently on GPUs, leading to significant speedups in sparse matrix-vector multiplication compared to existing libraries.

Contribution

The paper presents dtANS, a novel lossless compression technique optimized for GPU decoding, significantly improving sparse matrix storage and computation speed.

Findings

Compression reduces matrix size up to 11.77 times.

Achieves up to 3.48x speedup in SpMVM.

Outperforms state-of-the-art cuSPARSE and AlphaSparse in benchmarks.

Abstract

We present a novel, practical approach to speed up sparse matrix-vector multiplication (SpMVM) on GPUs. The novel key idea is to apply lossless entropy coding to further compress the sparse matrix when stored in one of the commonly supported formats. Our method is based on dtANS, our new lossless compression method that improves the entropy coding technique of asymmetric numeral systems (ANS) specifically for fast parallel GPU decoding when used in tandem with SpMVM. We apply dtANS on the widely used CSR format and present extensive benchmarks on the SuiteSparse collection of matrices against the state-of-the-art cuSPARSE library. On matrices with at least 2^(15) entries and at least 10 entries per row on average, our compression reduces the matrix size over the smallest cuSPARSE format (CSR, COO and SELL) in almost all cases and up to 11.77 times. Further, we achieve an SpMVM speedup for the majority of matrices with at least 2^(25) nonzero entries. The best speedup is 3.48x. We also show that we can improve over the AI-based multi-format AlphaSparse in an experiment that is limited due to its extreme computation overhead. We provide our code as an open source C++/CUDA header library, which includes both compression and multiplication kernels.