A Discussion on Solving Partial Differential Equations using Neural Networks

TL;DR

This paper explores the capability of neural networks to solve PDEs like the Poisson and Navier-Stokes equations, demonstrating their effectiveness, analyzing initialization effects, and comparing with traditional methods.

Contribution

It provides empirical evidence that small neural networks can accurately solve complex PDEs and investigates factors affecting solution quality, including initialization and loss functions.

Findings

Small neural networks can learn complex PDE solutions.

Ensemble learning improves solution accuracy.

Neural network methods have both advantages and limitations compared to classical approaches.

Abstract

Can neural networks learn to solve partial differential equations (PDEs)? We investigate this question for two (systems of) PDEs, namely, the Poisson equation and the steady Navier--Stokes equations. The contributions of this paper are five-fold. (1) Numerical experiments show that small neural networks (< 500 learnable parameters) are able to accurately learn complex solutions for systems of partial differential equations. (2) It investigates the influence of random weight initialization on the quality of the neural network approximate solution and demonstrates how one can take advantage of this non-determinism using ensemble learning. (3) It investigates the suitability of the loss function used in this work. (4) It studies the benefits and drawbacks of solving (systems of) PDEs with neural networks compared to classical numerical methods. (5) It proposes an exhaustive list of possible directions of future work.