Re-visiting Reservoir Computing architectures optimized by Evolutionary Algorithms

TL;DR

This paper surveys how Evolutionary Algorithms have been used to optimize Reservoir Computing architectures, highlighting recent advances, challenges, and future directions in this niche of neural network design.

Contribution

It provides a systematic overview of the application of EAs to optimize RC architectures, including recent progress and open research questions.

Findings

EAs effectively optimize RC hyperparameters and architectures.

Recent advances improve RC performance and robustness.

Open questions remain on scalability and generalization.

Abstract

For many years, Evolutionary Algorithms (EAs) have been applied to improve Neural Networks (NNs) architectures. They have been used for solving different problems, such as training the networks (adjusting the weights), designing network topology, optimizing global parameters, and selecting features. Here, we provide a systematic brief survey about applications of the EAs on the specific domain of the recurrent NNs named Reservoir Computing (RC). At the beginning of the 2000s, the RC paradigm appeared as a good option for employing recurrent NNs without dealing with the inconveniences of the training algorithms. RC models use a nonlinear dynamic system, with fixed recurrent neural network named the \textit{reservoir}, and learning process is restricted to adjusting a linear parametric function. %so the performance of learning is fast and precise. However, an RC model has several hyper-parameters, therefore EAs are helpful tools to figure out optimal RC architectures. We provide an overview of the results on the area, discuss novel advances, and we present our vision regarding the new trends and still open questions.