An Experimental Analysis of the Echo State Network Initialization Using the Particle Swarm Optimization

TL;DR

This paper explores a hybrid approach combining Echo State Networks and Particle Swarm Optimization to improve the initialization of recurrent weights, enhancing performance on supervised learning tasks.

Contribution

It introduces a novel hybrid method using PSO for ESN weight initialization, demonstrating improved results over canonical ESN models.

Findings

Hybrid PSO-ESN outperforms canonical ESN on benchmark problems

PSO effectively optimizes initial hidden-hidden weights

Improved initialization leads to better learning performance

Abstract

This article introduces a robust hybrid method for solving supervised learning tasks, which uses the Echo State Network (ESN) model and the Particle Swarm Optimization (PSO) algorithm. An ESN is a Recurrent Neural Network with the hidden-hidden weights fixed in the learning process. The recurrent part of the network stores the input information in internal states of the network. Another structure forms a free-memory method used as supervised learning tool. The setting procedure for initializing the recurrent structure of the ESN model can impact on the model performance. On the other hand, the PSO has been shown to be a successful technique for finding optimal points in complex spaces. Here, we present an approach to use the PSO for finding some initial hidden-hidden weights of the ESN model. We present empirical results that compare the canonical ESN model with this hybrid method on a wide range of benchmark problems.