NeuraLSP: An Efficient and Rigorous Neural Left Singular Subspace Preconditioner for Conjugate Gradient Methods

TL;DR

NeuraLSP introduces a neural preconditioner leveraging the left singular subspace of system matrices, offering theoretical guarantees and empirical robustness, leading to significant speedups in solving PDE-related linear systems.

Contribution

The paper presents NeuraLSP, a novel neural preconditioner that uses spectral information and a new loss metric to improve convergence in solving large sparse linear systems from PDEs.

Findings

Achieves up to 53% speedup in convergence.

Provides theoretical guarantees for the new loss function.

Demonstrates robustness across diverse PDE families.

Abstract

Numerical techniques for solving partial differential equations (PDEs) are integral for many fields across science and engineering. Such techniques usually involve solving large, sparse linear systems, where preconditioning methods are critical. In recent years, neural methods, particularly graph neural networks (GNNs), have demonstrated their potential through accelerated convergence. Nonetheless, to extract connective structures, existing techniques aggregate discretized system matrices into graphs, and suffer from rank inflation and a suboptimal convergence rate. In this paper, we articulate NeuraLSP, a novel neural preconditioner combined with a novel loss metric that leverages the left singular subspace of the system matrix's near-nullspace vectors. By compressing spectral information into a fixed low-rank operator, our method exhibits both theoretical guarantees and empirical robustness to rank inflation, affording up to a 53% speedup. Besides the theoretical guarantees for our newly-formulated loss function, our comprehensive experimental results across diverse families of PDEs also substantiate the aforementioned theoretical advances.