Learning the Exact Time Integration Algorithm for Initial Value Problems by Randomized Neural Networks

TL;DR

This paper introduces a novel method using randomized neural networks to learn exact time integration algorithms for initial value problems, achieving high accuracy and computational efficiency across various system types.

Contribution

The paper presents a physics-informed neural network approach to learn exact time integration algorithms, including for non-autonomous systems with periodicity properties, which is a new contribution.

Findings

The learned neural network algorithms achieve nearly exponential error decay with increased network complexity.

The method performs competitively with traditional algorithms in accuracy and computational cost.

Numerical experiments demonstrate effectiveness on stiff, non-stiff, and chaotic systems.

Abstract

We present a method leveraging extreme learning machine (ELM) type randomized neural networks (NNs) for learning the exact time integration algorithm for initial value problems (IVPs). The exact time integration algorithm for non-autonomous systems can be represented by an algorithmic function in higher dimensions, which satisfies an associated system of partial differential equations with corresponding boundary conditions. Our method learns the algorithmic function by solving this associated system using ELM with a physics informed approach. The trained ELM network serves as the learned algorithm and can be used to solve the IVP with arbitrary initial data or step sizes from some domain. When the right hand side of the non-autonomous system exhibits a periodicity with respect to any of its arguments, while the solution itself to the problem is not periodic, we show that the algorithmic function is either periodic, or when it is not, satisfies a well-defined relation for different periods. This property can greatly simplify the algorithm learning in many problems. We consider explicit and implicit NN formulations, leading to explicit or implicit time integration algorithms, and discuss how to train the ELM network by the nonlinear least squares method. Extensive numerical experiments with benchmark problems, including non-stiff, stiff and chaotic systems, show that the learned NN algorithm produces highly accurate solutions in long-time simulations, with its time-marching errors decreasing nearly exponentially with increasing degrees of freedom in the neural network. We compare extensively the computational performance (accuracy vs.~cost) between the current NN algorithm and the leading traditional time integration algorithms. The learned NN algorithm is computationally competitive, markedly outperforming the traditional algorithms in many problems.