Multivariate time series prediction using clustered echo state network

TL;DR

This paper introduces a clustered echo state network architecture that improves multivariate time series prediction by organizing reservoir nodes into variable-specific clusters, enhancing accuracy and efficiency across diverse datasets.

Contribution

The paper proposes a novel clustered ESN architecture with modular reservoirs and evaluates different topologies, demonstrating improved performance over traditional ESNs in multivariate forecasting.

Findings

CESNs outperform conventional ESNs in accuracy and robustness.

ER and SF topologies yield the best predictive performance.

CESNs effectively model diverse real-world multivariate datasets.

Abstract

Many natural and physical processes can be understood by analyzing multiple system variables evolving, forming a multivariate time series. Predicting such time series is challenging due to the inherent noise and interdependencies among variables. Echo state networks (ESNs), a class of Reservoir Computing (RC) models, offer an efficient alternative to conventional recurrent neural networks by training only the output weights while keeping the reservoir dynamics fixed, reducing computational complexity. We propose a clustered ESNs (CESNs) that enhances the ability to model and predict multivariate time series by organizing the reservoir nodes into clusters, each corresponding to a distinct input variable. Input signals are directly mapped to their associated clusters, and intra-cluster connections remain dense while inter-cluster connections are sparse, mimicking the modular architecture of biological neural networks. This architecture improves information processing by limiting cross-variable interference and enhances computational efficiency through independent cluster-wise training via ridge regression. We further explore different reservoir topologies, including ring, Erd\H{o}s-R\'enyi (ER), and scale-free (SF) networks, to evaluate their impact predictive performance. Our algorithm works well across diverse real-world datasets such as the stock market, solar wind, and chaotic R\"ossler system, demonstrating that CESNs consistently outperform conventional ESNs in terms of predictive accuracy and robustness to noise, particularly when using ER and SF topologies. These findings highlight the adaptability of CESNs for complex, multivariate time series forecasting.