Optimal time sampling in physics-informed neural networks

TL;DR

This paper investigates optimal time sampling strategies in physics-informed neural networks (PINNs), demonstrating that an exponential distribution often yields better results than uniform sampling for time-dependent equations.

Contribution

It provides a rigorous explanation for optimal time sampling in PINNs, showing that it follows a truncated exponential distribution under standard neural network convergence assumptions.

Findings

Optimal sampling often follows a truncated exponential distribution.

Uniform sampling is optimal only in specific cases.

Numerical examples confirm the theoretical results.

Abstract

Physics-informed neural networks (PINN) is a extremely powerful paradigm used to solve equations encountered in scientific computing applications. An important part of the procedure is the minimization of the equation residual which includes, when the equation is time-dependent, a time sampling. It was argued in the literature that the sampling need not be uniform but should overweight initial time instants, but no rigorous explanation was provided for this choice. In the present work we take some prototypical examples and, under standard hypothesis concerning the neural network convergence, we show that the optimal time sampling follows a (truncated) exponential distribution. In particular we explain when is best to use uniform time sampling and when one should not. The findings are illustrated with numerical examples on linear equation, Burgers' equation and the Lorenz system.