Effect of shapes of activation functions on predictability in the echo state network

TL;DR

This paper examines how different activation function shapes affect the prediction accuracy of echo state networks, highlighting that certain nonlinear functions outperform traditional sigmoid functions.

Contribution

It introduces an analysis of various activation functions in echo state networks, identifying those that enhance predictability.

Findings

Some nonlinear activation functions improve prediction accuracy.

Certain activation functions outperform sigmoid in echo state networks.

Nonlinear functions with appropriate shapes lead to better performance.

Abstract

We investigate prediction accuracy for time series of Echo state networks with respect to several kinds of activation functions. As a result, we found that some kinds of activation functions with an appropriate nonlinearity show high performance compared to the conventional sigmoid function.