Comparison between DeepESNs and gated RNNs on multivariate time-series prediction

TL;DR

This paper compares Deep Echo State Networks and gated RNNs for multivariate time-series prediction, showing DeepESNs are more efficient and accurate on most tasks, with GRUs outperforming LSTMs and simple RNNs.

Contribution

It provides an experimental comparison highlighting the efficiency and accuracy of DeepESNs versus fully-trained gated RNNs on multivariate time-series tasks.

Findings

DeepESN outperforms ESN in accuracy and efficiency.

GRU generally outperforms LSTM and RNN.

DeepESN is the most efficient and accurate on 3 out of 4 tasks.

Abstract

We propose an experimental comparison between Deep Echo State Networks (DeepESNs) and gated Recurrent Neural Networks (RNNs) on multivariate time-series prediction tasks. In particular, we compare reservoir and fully-trained RNNs able to represent signals featured by multiple time-scales dynamics. The analysis is performed in terms of efficiency and prediction accuracy on 4 polyphonic music tasks. Our results show that DeepESN is able to outperform ESN in terms of prediction accuracy and efficiency. Whereas, between fully-trained approaches, Gated Recurrent Units (GRU) outperforms Long Short-Term Memory (LSTM) and simple RNN models in most cases. Overall, DeepESN turned out to be extremely more efficient than others RNN approaches and the best solution in terms of prediction accuracy on 3 out of 4 tasks.