Stochastic dynamics learning with state-space systems

TL;DR

This paper provides a unified theoretical framework for reservoir computing, analyzing fading memory and stability in deterministic and stochastic state-space systems, explaining empirical success without strict contractivity.

Contribution

It introduces a novel distributional perspective on stochastic echo states and generalizes existing dynamical systems theory for reservoir computing.

Findings

Fading memory and solution stability are shown to be generic properties in state-space systems.

The stochastic echo state concept is extended via attractor dynamics on probability distributions.

The theory supports reliable generative modeling of temporal data in stochastic and deterministic regimes.

Abstract

This work advances the theoretical foundations of reservoir computing (RC) by providing a unified treatment of fading memory and the echo state property (ESP) in both deterministic and stochastic settings. We investigate state-space systems, a central model class in time series learning, and establish that fading memory and solution stability hold generically -- even in the absence of the ESP -- offering a robust explanation for the empirical success of RC models without strict contractivity conditions. In the stochastic case, we critically assess stochastic echo states, proposing a novel distributional perspective rooted in attractor dynamics on the space of probability distributions, which leads to a rich and coherent theory. Our results extend and generalize previous work on non-autonomous dynamical systems, offering new insights into causality, stability, and memory in RC models. This lays the groundwork for reliable generative modeling of temporal data in both deterministic and stochastic regimes.