A sparsity-aware distributed-memory algorithm for sparse-sparse matrix multiplication

TL;DR

This paper introduces a sparsity-aware distributed-memory 1D algorithm for sparse-sparse matrix multiplication that reduces communication costs and outperforms existing 2D and 3D methods in various scenarios.

Contribution

It presents a novel sparsity-aware 1D distributed-memory SpGEMM algorithm utilizing MPI RDMA and block fetching, improving efficiency over traditional methods.

Findings

Outperforms state-of-the-art 2D and 3D implementations in many configurations

Reduces communication by exploiting sparsity in matrices

Simpler conceptual design compared to existing approaches

Abstract

Multiplying two sparse matrices (SpGEMM) is a common computational primitive used in many areas including graph algorithms, bioinformatics, algebraic multigrid solvers, and randomized sketching. Distributed-memory parallel algorithms for SpGEMM have mainly focused on sparsity-oblivious approaches that use 2D and 3D partitioning. Sparsity-aware 1D algorithms can theoretically reduce communication by not fetching nonzeros of the sparse matrices that do not participate in the multiplication. Here, we present a distributed-memory 1D SpGEMM algorithm and implementation. It uses MPI RDMA operations to mitigate the cost of packing/unpacking submatrices for communication, and it uses a block fetching strategy to avoid excessive fine-grained messaging. Our results show that our 1D implementation outperforms state-of-the-art 2D and 3D implementations within CombBLAS for many configurations, inputs, and use cases, while remaining conceptually simpler.