Higher order Jacobi method for solving system of linear equations

TL;DR

This paper introduces a higher-order Jacobi method inspired by neural networks, enabling efficient, training-free solutions to linear systems with improved computational complexity and interpretability.

Contribution

It presents the higher order Jacobi method (HOJM), a novel iterative approach that resembles a shallow neural network and allows direct inverse computation without retraining.

Findings

Significant reduction in computational complexity on GPU

Explicit, physics-based network parameters without training

Efficient resolution of system variations without recomputing coefficients

Abstract

This work proposes a higher-order iterative framework for solving matrix equations, inspired by the structure and functionality of neural networks. A modification of the classical Jacobi iterative method is introduced to compute higher-order coefficient matrices through matrix-matrix multiplications. The resulting method, termed the higher order Jacobi method (HOJM), structurally resembles a shallow linear network and allows direct computation of the inverse of the coefficient matrix. Building on this, an iterative scheme is developed that allows efficient resolution of system variations without recomputing the coefficients, once the network parameters are trained for a known system. This iterative process naturally assumes the form of a deep recurrent neural network. The proposed approach goes beyond conventional physics-informed neural networks (PINNs) by providing an explicit, training-free definition of network parameters rooted in physical and mathematical formulations. Computational analysis on GPU reveals significant enhancement in the order of complexity, highlighting a compelling and transformative direction for advancing algorithmic efficiency in solving linear systems. This methodology opens avenues for interpretable and scalable solutions to physically motivated problems in computational science.