Rank Inspired Neural Network for solving linear partial differential equations

TL;DR

This paper introduces RINN, a neural network approach that improves the stability and accuracy of solving PDEs by orthogonalizing basis functions, addressing initialization sensitivity issues in previous methods.

Contribution

The paper presents a novel rank-inspired neural network with a covariance-driven regularization for orthogonal basis functions, enhancing stability and performance in PDE solutions.

Findings

RINN reduces variability caused by parameter initialization.

Incorporating orthogonality improves numerical stability.

Early stopping based on PDE loss enhances accuracy.

Abstract

This paper proposes a rank inspired neural network (RINN) to tackle the initialization sensitivity issue of physics informed extreme learning machines (PIELM) when numerically solving partial differential equations (PDEs). Unlike PIELM which randomly initializes the parameters of its hidden layers, RINN incorporates a preconditioning stage. In this stage, covariance-driven regularization is employed to optimize the orthogonality of the basis functions generated by the last hidden layer. The key innovation lies in minimizing the off-diagonal elements of the covariance matrix derived from the hidden-layer output. By doing so, pairwise orthogonality constraints across collocation points are enforced which effectively enhances both the numerical stability and the approximation ability of the optimized function space.The RINN algorithm unfolds in two sequential stages. First, it conducts a non-linear optimization process to orthogonalize the basis functions. Subsequently, it solves the PDE constraints using linear least-squares method. Extensive numerical experiments demonstrate that RINN significantly reduces performance variability due to parameter initialization compared to PIELM. Incorporating an early stopping mechanism based on PDE loss further improves stability, ensuring consistently high accuracy across diverse initialization settings.