Fundamental performance bounds on time-series generation using reservoir computing

TL;DR

This paper establishes theoretical performance bounds for reservoir computing in time-series generation, analyzing stability and reachability factors, and providing insights to improve reservoir network design.

Contribution

It formulates existence conditions for target sequences, analyzes stability and reach limitations, and uses dynamical mean field theory to derive amplitude-period bounds for RC networks.

Findings

Successful training requires stability and reachability of the target orbit.

Failures are due to either stability issues or reach limitations, depending on the training algorithm.

Dynamical mean field theory provides analytical bounds on achievable outputs.

Abstract

Reservoir computing (RC) harnesses the intrinsic dynamics of a chaotic system, called the reservoir, to perform various time-varying functions. An important use-case of RC is the generation of target temporal sequences via a trainable output-to-reservoir feedback loop. Despite the promise of RC in various domains, we lack a theory of performance bounds on RC systems. Here, we formulate an existence condition for a feedback loop that produces the target sequence. We next demonstrate that, given a sufficiently chaotic neural network reservoir, two separate factors are needed for successful training: global network stability of the target orbit, and the ability of the training algorithm to drive the system close enough to the target, which we term `reach'. By computing the training phase diagram over a range of target output amplitudes and periods, we verify that reach-limited failures depend on the training algorithm while stability-limited failures are invariant across different algorithms. We leverage dynamical mean field theory (DMFT) to provide an analytical amplitude-period bound on achievable outputs by RC networks and propose a way of enhancing algorithm reach via forgetting. The resulting mechanistic understanding of RC performance can guide the future design and deployment of reservoir networks.