Unveiling the optimization process of Physics Informed Neural Networks: How accurate and competitive can PINNs be?

TL;DR

This paper explores the accuracy limits of physics-informed neural networks (PINNs), demonstrating that improved optimization algorithms and loss function modifications can significantly enhance their precision, making them competitive with traditional numerical methods.

Contribution

It reveals how optimization improvements and loss function tweaks can substantially boost PINN accuracy, providing practical guidance for their development.

Findings

Enhanced optimizers greatly improve convergence.

Simple loss function modifications can increase precision.

PINNs with small networks achieve accuracy comparable to large-grid finite difference methods.

Abstract

This study investigates the potential accuracy boundaries of physics-informed neural networks, contrasting their approach with previous similar works and traditional numerical methods. We find that selecting improved optimization algorithms significantly enhances the accuracy of the results. Simple modifications to the loss function may also improve precision, offering an additional avenue for enhancement. Despite optimization algorithms having a greater impact on convergence than adjustments to the loss function, practical considerations often favor tweaking the latter due to ease of implementation. On a global scale, the integration of an enhanced optimizer and a marginally adjusted loss function enables a reduction in the loss function by several orders of magnitude across diverse physical problems. Consequently, our results obtained using compact networks (typically comprising 2 or 3 layers of 20-30 neurons) achieve accuracies comparable to finite difference schemes employing thousands of grid points. This study encourages the continued advancement of PINNs and associated optimization techniques for broader applications across various fields.