Synthesis of recurrent neural networks for dynamical system simulation

TL;DR

This paper introduces a novel algorithm for training recurrent neural networks to accurately simulate continuous-time dynamical systems, leveraging theoretical guarantees and matrix manipulations.

Contribution

It presents a new method to convert trained feedforward networks into recurrent networks for dynamical system simulation, with demonstrated numerical examples.

Findings

The algorithm guarantees the quality of network approximation.

Recurrent networks can replicate continuous-time dynamical systems.

Numerical examples validate the approach.

Abstract

We review several of the most widely used techniques for training recurrent neural networks to approximate dynamical systems, then describe a novel algorithm for this task. The algorithm is based on an earlier theoretical result that guarantees the quality of the network approximation. We show that a feedforward neural network can be trained on the vector field representation of a given dynamical system using backpropagation, then recast, using matrix manipulations, as a recurrent network that replicates the original system's dynamics. After detailing this algorithm and its relation to earlier approaches, we present numerical examples that demonstrate its capabilities. One of the distinguishing features of our approach is that both the original dynamical systems and the recurrent networks that simulate them operate in continuous time.