Complexity bounds on neural networks for the solution of structured linear systems of equations

TL;DR

This paper establishes explicit complexity bounds for ReLU neural networks approximating solutions to structured linear systems, especially symmetric positive definite and sparse matrices, extending classical iterative methods.

Contribution

It provides new upper bounds on neural network size for solving structured linear systems, incorporating matrix properties like condition number and sparsity.

Findings

Bounds depend explicitly on matrix size and condition number

Extends classical iterative methods to neural network approximation

Applicable to finite difference and finite element matrices

Abstract

We derive upper bounds on the complexity of ReLU neural networks approximating the solution of a linear system given the matrix and the right-hand side. We focus on matrices which are symmetric positive definite and sparse, as they appear in the context of finite difference and finite element methods. For such matrices, we extend available results for the matrix inversion to the task of solving a linear system, where we leverage favorable properties of classical methods such as the modified Richardson and the conjugate gradient method. Our bounds on the number of layers and neurons are not only explicit with respect to the size of the matrices, but also with respect to their condition numbers.