Arbitrarily high order implicit ODE integration by correcting a neural network approximation with Newton's method

TL;DR

This paper introduces a hybrid approach combining neural network approximations with Newton's method to perform high-order implicit ODE integration efficiently, enabling large time-steps with high accuracy.

Contribution

It develops a novel method that uses DNNs to provide initial guesses for high-order IRK methods, reducing computational effort in solving nonlinear ODEs.

Findings

Enables large implicit time-steps with high accuracy

Reduces computational effort in IRK methods

Provides a general formula for IRK matrix elements

Abstract

As a method of universal approximation deep neural networks (DNNs) are capable of finding approximate solutions to problems posed with little more constraints than a suitably-posed mathematical system and an objective function. Consequently, DNNs have considerably more flexibility in applications than classical numerical methods. On the other hand they offer an uncontrolled approximation to the sought-after mathematical solution. This suggests that hybridization of classical numerical methods with DNN-based approximations may be a desirable approach. In this work a DNN-based approximator inspired by the physics-informed neural networks (PINNs) methodology is used to provide an initial guess to a Newton's method iteration of a very-high order implicit Runge-Kutta (IRK) integration of a nonlinear system of ODEs, namely the Lorenz system. In the usual approach many explicit timesteps are needed to provide a guess to the implicit system's nonlinear solver, requiring enough work to make the IRK method infeasible. The DNN-based approach described in this work enables large implicit time-steps to be taken to any desired degree of accuracy for as much effort as it takes to converge the DNN solution to within a few percent accuracy. This work also develops a general formula for the matrix elements of the IRK method for an arbitrary quadrature order.