Bridging the Gap between Sparse Matrix Reordering and Factorization: A Deep Learning Framework for Fill-in Reduction

TL;DR

This paper introduces a deep learning framework utilizing graph neural networks to improve sparse matrix reordering, aiming to reduce fill-in during matrix factorization more effectively than traditional methods.

Contribution

It proposes a novel GNN-based approach that captures global matrix structure via spectral embedding to minimize fill-in, bridging the gap between reordering and factorization.

Findings

Achieves competitive performance with traditional algorithms.

Uses spectral embedding to capture global structural information.

Employs multi-grid-like GNN architectures for optimization.

Abstract

Sparse matrix reordering can significantly reduce the fill-in during matrix factorization, thereby decreasing the computational and storage requirements in sparse matrix computations. Finding a minimal fill-in ordering is known to be an NP-hard problem. Moreover, there is a paradox: matrix reordering is applied before matrix factorization, but fill-ins that matrix reordering methods aim at are generated from matrix factorization. To bridge the gap between reordering and factorization, we propose a deep learning framework to minimize a fill-in surrogate function based on spectral embedding. First, we employ a multi-grid-like GNN architecture to learn to approximate the smallest eigenvectors of its graph Laplacian matrix, i.e. spectral embedding, and capture the global structural information of the matrix. Then, another multi-grid-like GNN architecture is used to minimize the potential space where fill-in can occur based on the rank distribution. Experimental results indicate that our approach achieves competitive performance compared with traditional graph-theoretic algorithms and deep learning methods.