On Dominant Manifolds in Reservoir Computing Networks

TL;DR

This paper investigates how training influences the formation of low-dimensional dominant manifolds in reservoir computing networks, linking their structure to data properties and dynamical systems theory.

Contribution

It establishes a theoretical connection between reservoir dominant modes, data intrinsic dimensionality, and Koopman eigenfunctions, extending to nonlinear cases.

Findings

Dominant modes relate to data's intrinsic dimensionality.

Trained reservoir modes approximate Koopman eigenfunctions.

Eigenvalue dynamics during training illustrate manifold formation.

Abstract

Understanding how training shapes the geometry of recurrent network dynamics is a central problem in time-series modeling. We study the emergence of low-dimensional dominant manifolds in the training of Reservoir Computing (RC) networks for temporal forecasting tasks. For a simplified linear and continuous-time reservoir model, we link the dimensionality and structure of the dominant modes directly to the intrinsic dimensionality and information content of the training data. In particular, for training data generated by an autonomous dynamical system, we relate the dominant modes of the trained reservoir to approximations of the Koopman eigenfunctions of the original system, illuminating an explicit connection between reservoir computing and the Dynamic Mode Decomposition algorithm. We illustrate the eigenvalue motion that generates the dominant manifolds during training in simulation, and discuss generalization to nonlinear RC via tangent dynamics and differential p-dominance.