Neural Networks to solve Partial Differential Equations: a Comparison with Finite Elements

TL;DR

This paper compares neural network and finite element methods for solving a 2D heat equation, finding FEM generally more accurate and faster, with neural networks requiring careful tuning for high precision.

Contribution

It provides a detailed empirical comparison between FEM and neural networks for PDE solutions, highlighting the strengths and limitations of each approach.

Findings

FEM outperforms NN in accuracy and computational time for the tested PDE.

NN accuracy improves with optimizer refinement but remains less precise than FEM.

FEM achieves high accuracy (~10^{-5}) faster than NN in the studied case.

Abstract

We compare the Finite Element Method (FEM) simulation of a standard Partial Differential Equation thermal problem of a plate with a hole with a Neural Network (NN) simulation. The largest deviation from the true solution obtained from FEM ($0.015$ for a solution on the order of unity) is easily achieved with NN too without much tuning of the hyperparameters. Accuracies below $0.01$ instead require refinement with an alternative optimizer to reach a similar performance with NN. A rough comparison between the Floating Point Operations values, as a machine-independent quantification of the computational performance, suggests a significant difference between FEM and NN in favour of the former. This also strongly holds for computation time: for an accuracy on the order of $10^{-5}$, FEM and NN require $54$ and $1100$ seconds, respectively. A detailed analysis of the effect of varying different hyperparameters shows that accuracy and computational time only weakly depend on the major part of them. Accuracies below $0.01$ cannot be achieved with the "adam'' optimizers and it looks as though accuracies below $10^{-5}$ cannot be achieved at all. In conclusion, the present work shows that for the concrete case of solving a steady-state 2D heat equation, the performance of a FEM algorithm is significantly better than the solution via networks.