Misam: Using ML in Dataflow Selection of Sparse-Sparse Matrix Multiplication

TL;DR

This paper introduces a machine learning approach to dynamically select optimal dataflow schemes for sparse matrix multiplication, significantly improving performance over traditional heuristics in diverse sparsity scenarios.

Contribution

It proposes ML-based methods, including decision trees and deep reinforcement learning, for adaptive dataflow selection in SpGEMM hardware accelerators, outperforming heuristic approaches.

Findings

ML methods achieve up to 28x performance gains.

Decision trees and reinforcement learning effectively adapt to different sparsity patterns.

ML-based selection surpasses traditional heuristic methods.

Abstract

Sparse matrix-matrix multiplication (SpGEMM) is a critical operation in numerous fields, including scientific computing, graph analytics, and deep learning. These applications exploit the sparsity of matrices to reduce storage and computational demands. However, the irregular structure of sparse matrices poses significant challenges for performance optimization. Traditional hardware accelerators are tailored for specific sparsity patterns with fixed dataflow schemes - inner, outer, and row-wise but often perform suboptimally when the actual sparsity deviates from these predetermined patterns. As the use of SpGEMM expands across various domains, each with distinct sparsity characteristics, the demand for hardware accelerators that can efficiently handle a range of sparsity patterns is increasing. This paper presents a machine learning based approach for adaptively selecting the most appropriate dataflow scheme for SpGEMM tasks with diverse sparsity patterns. By employing decision trees and deep reinforcement learning, we explore the potential of these techniques to surpass heuristic-based methods in identifying optimal dataflow schemes. We evaluate our models by comparing their performance with that of a heuristic, highlighting the strengths and weaknesses of each approach. Our findings suggest that using machine learning for dynamic dataflow selection in hardware accelerators can provide upto 28 times gains.