Accelerating Simulation of Stiff Nonlinear Systems using Continuous-Time Echo State Networks

TL;DR

This paper introduces the continuous-time echo state network (CTESN), a data-driven surrogate modeling approach that significantly accelerates the simulation of stiff nonlinear systems with minimal loss of accuracy.

Contribution

The paper presents the CTESN method, a novel approach for efficiently approximating complex nonlinear ODEs, outperforming existing techniques like physics-informed neural networks in handling stiff dynamics.

Findings

Near-constant evaluation time with CTESNs on complex models

Maintains relative error within 0.2%

Effectively captures both fast transients and slow dynamics

Abstract

Modern design, control, and optimization often requires simulation of highly nonlinear models, leading to prohibitive computational costs. These costs can be amortized by evaluating a cheap surrogate of the full model. Here we present a general data-driven method, the continuous-time echo state network (CTESN), for generating surrogates of nonlinear ordinary differential equations with dynamics at widely separated timescales. We empirically demonstrate near-constant time performance using our CTESNs on a physically motivated scalable model of a heating system whose full execution time increases exponentially, while maintaining relative error of within 0.2 %. We also show that our model captures fast transients as well as slow dynamics effectively, while other techniques such as physics informed neural networks have difficulties trying to train and predict the highly nonlinear behavior of these models.