Residual Reservoir Memory Networks

TL;DR

This paper introduces Residual Reservoir Memory Networks, a new untrained RNN model within Reservoir Computing that enhances long-term memory through residual orthogonal connections, showing improved performance on time-series and pixel classification tasks.

Contribution

The paper proposes a novel untrained RNN architecture combining linear and non-linear reservoirs with residual connections, advancing reservoir computing methods.

Findings

ResRMN outperforms conventional RC models on benchmark tasks.

Residual orthogonal connections improve long-term memory propagation.

Linear stability analysis explains the dynamics of the proposed model.

Abstract

We introduce a novel class of untrained Recurrent Neural Networks (RNNs) within the Reservoir Computing (RC) paradigm, called Residual Reservoir Memory Networks (ResRMNs). ResRMN combines a linear memory reservoir with a non-linear reservoir, where the latter is based on residual orthogonal connections along the temporal dimension for enhanced long-term propagation of the input. The resulting reservoir state dynamics are studied through the lens of linear stability analysis, and we investigate diverse configurations for the temporal residual connections. The proposed approach is empirically assessed on time-series and pixel-level 1-D classification tasks. Our experimental results highlight the advantages of the proposed approach over other conventional RC models.