Minimal Deterministic Echo State Networks Outperform Random Reservoirs in Learning Chaotic Dynamics

TL;DR

This paper shows that minimal deterministic echo state networks (MESNs) outperform traditional random ESNs in learning chaotic systems, offering better accuracy and robustness with simpler, structured designs.

Contribution

The study introduces minimal deterministic reservoir initializations for ESNs, demonstrating their superior performance and robustness over standard random reservoirs in chaotic dynamics modeling.

Findings

MESNs reduce error by up to 41% compared to standard ESNs.

MESNs exhibit less variation across runs, indicating higher robustness.

MESNs can reuse hyperparameters across different chaotic systems.

Abstract

Machine learning (ML) is widely used to model chaotic systems. Among ML approaches, echo state networks (ESNs) have received considerable attention due to their simple construction and fast training. However, ESN performance is highly sensitive to hyperparameter choices and to its random initialization. In this work, we demonstrate that ESNs constructed using deterministic rules and simple topologies (MESNs) outperform standard ESNs in the task of chaotic attractor reconstruction. We use a dataset of more than 90 chaotic systems to benchmark 10 different minimal deterministic reservoir initializations. We find that MESNs obtain up to a 41% reduction in error compared to standard ESNs. Furthermore, we show that the MESNs are more robust, exhibiting less inter-run variation, and have the ability to reuse hyperparameters across different systems. Our results illustrate how structured simplicity in ESN design can outperform stochastic complexity in learning chaotic dynamics.