Neural networks catching up with finite differences in solving partial differential equations in higher dimensions

TL;DR

This paper demonstrates that neural networks can efficiently solve high-dimensional partial differential equations with high accuracy using sparse grids, outperforming traditional finite difference methods in computational time.

Contribution

It introduces a neural network approach that achieves high-precision solutions for high-dimensional PDEs on sparse grids, reducing computational resources compared to classical methods.

Findings

Neural networks attain near machine precision solutions with minimal grid points.

The method efficiently solves PDEs up to 5 dimensions within minutes.

Neural approach outperforms finite difference methods in speed for high-dimensional problems.

Abstract

Fully connected multilayer perceptrons are used for obtaining numerical solutions of partial differential equations in various dimensions. Independent variables are fed into the input layer, and the output is considered as solution's value. To train such a network one can use square of equation's residual as a cost function and minimize it with respect to weights by gradient descent. Following previously developed method, derivatives of the equation's residual along random directions in space of independent variables are also added to cost function. Similar procedure is known to produce nearly machine precision results using less than 8 grid points per dimension for 2D case. The same effect is observed here for higher dimensions: solutions are obtained on low density grids, but maintain their precision in the entire region. Boundary value problems for linear and nonlinear Poisson equations are solved inside 2, 3, 4, and 5 dimensional balls. Grids for linear cases have 40, 159, 512 and 1536 points and for nonlinear 64, 350, 1536 and 6528 points respectively. In all cases maximum error is less than $8.8\cdot10^{-6}$, and median error is less than $2.4\cdot10^{-6}$. Very weak grid requirements enable neural networks to obtain solution of 5D linear problem within 22 minutes, whereas projected solving time for finite differences on the same hardware is 50 minutes. Method is applied to second order equation, but requires little to none modifications to solve systems or higher order PDEs.