A mathematical framework for time-delay reservoir computing analysis

TL;DR

This paper develops a rigorous mathematical framework for analyzing time-delay reservoir computing, connecting core properties to control theory, and providing practical design criteria validated on benchmark tasks.

Contribution

It introduces a control-theoretic approach to reservoir computing, formalizes separation and fading memory, and derives explicit bounds for linear reservoirs.

Findings

Established a connection between reservoir properties and stability concepts.

Derived a lower bound for separation distance in linear reservoirs.

Validated the framework with numerical experiments on benchmarks.

Abstract

Reservoir computing is a well-established approach for processing data with a much lower complexity compared to traditional neural networks. Despite two decades of experimental progress, the core properties of reservoir computing (namely separation, robustness, and fading memory) still lack rigorous mathematical foundations. This paper addresses this gap by providing a control-theoretic framework for the analysis of time-delay-based reservoir computers. We introduce formal definitions of the separation property and fading memory in terms of functional norms, and establish their connection to well-known stability notions for time-delay systems as incremental input-to-state stability. For a class of linear reservoirs, we derive an explicit lower bound for the separation distance via Fourier analysis, offering a computable criterion for reservoir design. Numerical results on the NARMA10 benchmark and continuous-time system prediction validate the approach with a minimal digital implementation.