Improved Physics-informed neural networks loss function regularization with a variance-based term

TL;DR

This paper introduces a new loss function for physics-informed neural networks that combines mean and variance of errors to improve solution accuracy and reduce localized high-error regions.

Contribution

The paper proposes a novel variance-based regularization term in the PINNs loss function, enhancing error uniformity and solution quality across complex physics problems.

Findings

Improved accuracy in solving PDEs with PINNs

Reduced maximum error in test problems

Minimal additional computational cost

Abstract

In machine learning and statistical modeling, the mean square or absolute error is commonly used as an error metric, also called a "loss function." While effective in reducing the average error, this approach may fail to address localized outliers, leading to significant inaccuracies in regions with sharp gradients or discontinuities. This issue is particularly evident in physics-informed neural networks (PINNs), where such localized errors are expected and affect the overall solution. To overcome this limitation, we propose a novel loss function that combines the mean and the standard deviation of the chosen error metric. By minimizing this combined loss function, the method ensures a more uniform error distribution and reduces the impact of localized high-error regions. The proposed loss function is easy to implement and tested on problems of varying complexity: the 1D Poisson equation, the unsteady Burgers' equation, 2D linear elastic solid mechanics, and 2D steady Navier-Stokes equations. Results demonstrate improved solution quality and lower maximum error compared to the standard mean-based loss, with minimal impact on computational time.