Strategies for training point distributions in physics-informed neural networks

TL;DR

This paper systematically evaluates how different training point distributions affect the accuracy of physics-informed neural networks in solving differential equations, highlighting the importance of data placement strategies.

Contribution

It introduces sine-based training points inspired by Chebyshev nodes and assesses their effectiveness compared to traditional distributions in physics-informed neural networks.

Findings

Training point distribution significantly impacts solution accuracy.

Sine-based points improve performance for certain differential equations.

Distribution effectiveness depends on the characteristics of the differential equation.

Abstract

Physics-informed neural networks approach the approximation of differential equations by directly incorporating their structure and given conditions in a loss function. This enables conditions like, e.g., invariants to be easily added during the modelling phase. In addition, the approach can be considered as mesh free and can be utilised to compute solutions on arbitrary grids after the training phase. Therefore, physics-informed neural networks are emerging as a promising alternative to solving differential equations with methods from numerical mathematics. However, their performance highly depends on a large variety of factors. In this paper, we systematically investigate and evaluate a core component of the approach, namely the training point distribution. We test two ordinary and two partial differential equations with five strategies for training data generation and shallow network architectures, with one and two hidden layers. In addition to common distributions, we introduce sine-based training points, which are motivated by the construction of Chebyshev nodes. The results are challenged by using certain parameter combinations like, e.g., random and fixed-seed weight initialisation for reproducibility. The results show the impact of the training point distributions on the solution accuracy and we find evidence that they are connected to the characteristics of the differential equation.