Generalised Synchronisations, Embeddings, and Approximations for Continuous Time Reservoir Computers

TL;DR

This paper investigates the conditions under which continuous time reservoir computers can achieve generalized synchronisation and embeddings, enabling accurate forecasting and topological replication of source dynamics, including in noisy environments.

Contribution

It provides theoretical conditions for generalized synchronisation, embeddings, and approximations in continuous time reservoir systems, extending Takens' theorem and analyzing linear reservoirs.

Findings

Existence of multiple generalized synchronisations in reservoir systems

Closed-form expression for synchronisation in linear reservoirs

Embedding of fixed points occurs almost surely in random reservoirs

Abstract

We establish conditions under which a continuous time reservoir computer, such as a leaky integrator echo state network, admits a generalised synchronisation $f$ between between the source dynamics and reservoir dynamics. We show that multiple generalised synchronisations can exist simultaneously, and connect this to the multi-Echo-State-Property (multi-ESP). In the special case of a linear reservoir computer, we derive a closed form expression for the generalised synchronisation $f$. Furthermore, we establish conditions under which $f$ is of class $C^1$, and conditions under which $f$ is a topological embedding on the fixed points of the source system. This embedding result is closely related to Takens' embedding Theorem. We also prove that the embedding of fixed points occurs almost surely for randomly generated linear reservoir systems. With an embedding achieved, we discuss how the universal approximation theorem makes it possible to forecast the future dynamics of the source system and replicate its topological properties. We illustrate the theory by embedding a fixed point of the Lorenz-63 system into the reservoir space using numerical methods. Finally, we show that if the observations are perturbed by white noise, the GS is preserved up to a perturbation by an Ornstein-Uhlenbeck process.