A Comparative Study of Reservoir Computing for Temporal Signal Processing

TL;DR

This paper compares different reservoir computing architectures, specifically echo state networks, tapped-delay lines, and NARX networks, in their ability to process and predict benchmark time series, revealing the reservoir's role beyond simple memory.

Contribution

It provides a comparative analysis of RC architectures, highlighting the unique generalization capabilities of echo state networks over other models with limited memory.

Findings

ESNs outperform DL and NARX in generalization.

DL and NARX have higher memorization but less generalization.

Reservoirs extend beyond memory to enable better prediction.

Abstract

Reservoir computing (RC) is a novel approach to time series prediction using recurrent neural networks. In RC, an input signal perturbs the intrinsic dynamics of a medium called a reservoir. A readout layer is then trained to reconstruct a target output from the reservoir's state. The multitude of RC architectures and evaluation metrics poses a challenge to both practitioners and theorists who study the task-solving performance and computational power of RC. In addition, in contrast to traditional computation models, the reservoir is a dynamical system in which computation and memory are inseparable, and therefore hard to analyze. Here, we compare echo state networks (ESN), a popular RC architecture, with tapped-delay lines (DL) and nonlinear autoregressive exogenous (NARX) networks, which we use to model systems with limited computation and limited memory respectively. We compare the performance of the three systems while computing three common benchmark time series: H{\'e}non Map, NARMA10, and NARMA20. We find that the role of the reservoir in the reservoir computing paradigm goes beyond providing a memory of the past inputs. The DL and the NARX network have higher memorization capability, but fall short of the generalization power of the ESN.