SPINN: Sparse, Physics-based, and partially Interpretable Neural Networks for PDEs

TL;DR

SPINN introduces a sparse, interpretable neural network framework that bridges traditional PDE solvers and neural methods, capable of handling discontinuities and incorporating Fourier representations.

Contribution

The paper presents SPINN, a novel sparse and partially interpretable neural network architecture for PDEs, unifying neural and classical approaches with enhanced interpretability and adaptivity.

Findings

SPINN effectively solves various PDEs including elliptic, parabolic, hyperbolic, and nonlinear types.

The model can encode mesh adaptivity and handle solution discontinuities.

Fourier series can be represented as a special case of SPINN.

Abstract

We introduce a class of Sparse, Physics-based, and partially Interpretable Neural Networks (SPINN) for solving ordinary and partial differential equations (PDEs). By reinterpreting a traditional meshless representation of solutions of PDEs we develop a class of sparse neural network architectures that are partially interpretable. The SPINN model we propose here serves as a seamless bridge between two extreme modeling tools for PDEs, namely dense neural network based methods like Physics Informed Neural Networks (PINNs) and traditional mesh-free numerical methods, thereby providing a novel means to develop a new class of hybrid algorithms that build on the best of both these viewpoints. A unique feature of the SPINN model that distinguishes it from other neural network based approximations proposed earlier is that it is (i) interpretable, in a particular sense made precise in the work, and (ii) sparse in the sense that it has much fewer connections than typical dense neural networks used for PDEs. Further, the SPINN algorithm implicitly encodes mesh adaptivity and is able to handle discontinuities in the solutions. In addition, we demonstrate that Fourier series representations can also be expressed as a special class of SPINN and propose generalized neural network analogues of Fourier representations. We illustrate the utility of the proposed method with a variety of examples involving ordinary differential equations, elliptic, parabolic, hyperbolic and nonlinear partial differential equations, and an example in fluid dynamics.