Self-Supervised Learning for Sparse Matrix Reordering

TL;DR

This paper introduces a novel self-supervised learning approach using graph neural networks and triplet loss to optimize sparse matrix reordering, effectively reducing fill-ins and accelerating LU factorization.

Contribution

It presents a new self-supervised method leveraging graph neural networks and triplet inequalities for sparse matrix reordering, with theoretical and empirical advantages.

Findings

Outperforms existing methods in fill-in reduction.

Achieves significant speedup in LU factorization.

Validated on SuiteSparse collection with superior results.

Abstract

Rearranging the rows or columns of a sparse matrix using an appropriate ordering can significantly reduce fill-ins, i.e., new nonzeros introduced during matrix factorization, decreasing memory usage and runtime. However, finding an ordering that minimizes fill-ins is NP-complete. Existing approaches, including graph-theoretic and deep learning methods, rely on surrogate objectives without theoretical guarantees. The Fill-Path Theorem reveals a direct and intrinsic relationship between fill-in generation and the sparse structure of the matrix as path triplet inequalities. Here we first employ a multigrid graph network to capture structural information for each vertex. We then derive a triplet sampling strategy based on inequalities. Finally, we introduce an end-max chain loss function to reduce the number of triplets whose predicted scores satisfy these inequalities. Experimental evaluations on the publicly available SuiteSparse matrix collection demonstrate the superiority of the proposed method in terms of both fill-in reduction and speedup in LU factorization time.