Learning Reservoir Dynamics with Temporal Self-Modulation

TL;DR

This paper introduces self-modulated reservoir computing (SM-RC), enhancing traditional RC by adding gating mechanisms that improve learning performance and enable complex information processing, demonstrated through attention and prediction tasks.

Contribution

The paper proposes a novel self-modulation mechanism for RC using input and reservoir gates, significantly improving its learning capabilities and physical implementability.

Findings

SM-RC outperforms traditional RC in NARMA and Lorentz tasks.

SM-RC achieves higher accuracy with smaller reservoirs.

Chaotic states emerge as a result of learning in SM-RC.

Abstract

Reservoir computing (RC) can efficiently process time-series data by transferring the input signal to randomly connected recurrent neural networks (RNNs), which are referred to as a reservoir. The high-dimensional representation of time-series data in the reservoir significantly simplifies subsequent learning tasks. Although this simple architecture allows fast learning and facile physical implementation, the learning performance is inferior to that of other state-of-the-art RNN models. In this paper, to improve the learning ability of RC, we propose self-modulated RC (SM-RC), which extends RC by adding a self-modulation mechanism. The self-modulation mechanism is realized with two gating variables: an input gate and a reservoir gate. The input gate modulates the input signal, and the reservoir gate modulates the dynamical properties of the reservoir. We demonstrated that SM-RC can perform attention tasks where input information is retained or discarded depending on the input signal. We also found that a chaotic state emerged as a result of learning in SM-RC. This indicates that self-modulation mechanisms provide RC with qualitatively different information-processing capabilities. Furthermore, SM-RC outperformed RC in NARMA and Lorentz model tasks. In particular, SM-RC achieved a higher prediction accuracy than RC with a reservoir 10 times larger in the Lorentz model tasks. Because the SM-RC architecture only requires two additional gates, it is physically implementable as RC, providing a new direction for realizing edge AI.