Restrictions on Physical Stochastic Reservoir Computers

TL;DR

This paper investigates how noise in physical reservoir computers limits their information processing capacity and the complexity of tasks they can perform, revealing fundamental restrictions on their learning capabilities.

Contribution

It extends the analysis of noise effects on reservoir computing by linking IPC degradation to quantum complexity and fat-shattering dimension, establishing fundamental limits.

Findings

Noise causes exponential reduction in reservoir configuration space.

IPC degradation limits the features that can be constructed.

Physical reservoirs can only perform polynomial learning despite large latent spaces.

Abstract

Reservoir computation is a recurrent framework for learning and predicting time series data, that benefits from extremely simple training and interpretability, often as the the dynamics of a physical system. In this paper, we will study the impact of noise on the learning capabilities of analog reservoir computers. Recent work on reservoir computation has shown that the information processing capacity (IPC) is a useful metric for quantifying the degradation of the performance due to noise. We further this analysis and demonstrate that this degradation of the IPC limits the possible features that can be meaningfully constructed in an analog reservoir computing setting. We borrow a result from quantum complexity theory that relates the circuit model of computation to a continuous time model, and demonstrate an exponential reduction in the accessible volume of reservoir configurations. We conclude by relating this degradation in the IPC to the fat-shattering dimension of a family of functions describing the reservoir dynamics, which allows us to express our result in terms of a classification task. We conclude that any physical, analog reservoir computer that is exposed to noise can only be used to perform a polynomial amount of learning, despite the exponentially large latent space, even with an exponential amount of post-processing.