The Ubiquitous Sparse Matrix-Matrix Products

TL;DR

This paper provides a comprehensive overview of sparse matrix-matrix multiplication, emphasizing its importance across various applications and its operation over diverse algebraic structures, unifying many computational patterns.

Contribution

It offers a unifying framework for sparse matrix multiplication across different algebraic semirings and heterogeneous domains, covering broad application areas.

Findings

Applicable to graph algorithms, neural networks, and biological data analysis.

Supports arbitrary algebraic semirings and heterogeneous domains.

Provides a unified treatment of sparse matrix-matrix operations.

Abstract

Multiplication of a sparse matrix with another (dense or sparse) matrix is a fundamental operation that captures the computational patterns of many data science applications, including but not limited to graph algorithms, sparsely connected neural networks, graph neural networks, clustering, and many-to-many comparisons of biological sequencing data. In many application scenarios, the matrix multiplication takes places on an arbitrary algebraic semiring where the scalar operations are overloaded with user-defined functions with certain properties or a more general heterogenous algebra where even the domains of the input matrices can be different. Here, we provide a unifying treatment of the sparse matrix-matrix operation and its rich application space including machine learning, computational biology and chemistry, graph algorithms, and scientific computing.