Factorization-in-Loop: Proximal Fill-in Minimization for Sparse Matrix Reordering

TL;DR

This paper introduces a novel learning-based approach for sparse matrix reordering that minimizes fill-ins during LU factorization, leading to significant reductions in fill-in count and computational time.

Contribution

It proposes a reordering network with a graph encoder and a new optimization framework incorporating factorization into the objective, providing theoretical guarantees and improved performance.

Findings

Achieves 20% reduction in fill-ins

Reduces LU factorization time by 17.8%

Outperforms state-of-the-art baselines

Abstract

Fill-ins are new nonzero elements in the summation of the upper and lower triangular factors generated during LU factorization. For large sparse matrices, they will increase the memory usage and computational time, and be reduced through proper row or column arrangement, namely matrix reordering. Finding a row or column permutation with the minimal fill-ins is NP-hard, and surrogate objectives are designed to derive fill-in reduction permutations or learn a reordering function. However, there is no theoretical guarantee between the golden criterion and these surrogate objectives. Here we propose to learn a reordering network by minimizing \(l_1\) norm of triangular factors of the reordered matrix to approximate the exact number of fill-ins. The reordering network utilizes a graph encoder to predict row or column node scores. For inference, it is easy and fast to derive the permutation from sorting algorithms for matrices. For gradient based optimization, there is a large gap between the predicted node scores and resultant triangular factors in the optimization objective. To bridge the gap, we first design two reparameterization techniques to obtain the permutation matrix from node scores. The matrix is reordered by multiplying the permutation matrix. Then we introduce the factorization process into the objective function to arrive at target triangular factors. The overall objective function is optimized with the alternating direction method of multipliers and proximal gradient descent. Experimental results on benchmark sparse matrix collection SuiteSparse show the fill-in number and LU factorization time reduction of our proposed method is 20% and 17.8% compared with state-of-the-art baselines.