Reservoir Topology in Deep Echo State Networks

TL;DR

This paper investigates how structured reservoir topologies in Deep Echo State Networks enhance predictive performance, highlighting the benefits of deep architectures combined with permutation recurrent matrices.

Contribution

It introduces the study of constrained reservoir topologies in DeepESNs and demonstrates their significant impact on performance through numerical experiments.

Findings

Structured topologies improve prediction accuracy

Permutation recurrent matrices offer notable advantages

Deep architectures with structured reservoirs outperform unstructured ones

Abstract

Deep Echo State Networks (DeepESNs) recently extended the applicability of Reservoir Computing (RC) methods towards the field of deep learning. In this paper we study the impact of constrained reservoir topologies in the architectural design of deep reservoirs, through numerical experiments on several RC benchmarks. The major outcome of our investigation is to show the remarkable effect, in terms of predictive performance gain, achieved by the synergy between a deep reservoir construction and a structured organization of the recurrent units in each layer. Our results also indicate that a particularly advantageous architectural setting is obtained in correspondence of DeepESNs where reservoir units are structured according to a permutation recurrent matrix.