Fading memory echo state networks are universal

TL;DR

This paper demonstrates that for inputs with bounded amplitude, a family of echo state networks with fading memory can be constructed that are universal approximators, extending previous results to this specific case.

Contribution

It introduces a new construction of universal echo state networks with fading memory properties for bounded inputs, filling a gap in existing literature.

Findings

Universal ESNs with fading memory can be constructed for bounded inputs.

The new family of ESNs contains only elements with echo state and fading memory properties.

This approach extends the universality results to the case of bounded inputs.

Abstract

Echo state networks (ESNs) have been recently proved to be universal approximants for input/output systems with respect to various $L ^p$-type criteria. When $1\leq p< \infty$, only $p$-integrability hypotheses need to be imposed, while in the case $p=\infty$ a uniform boundedness hypotheses on the inputs is required. This note shows that, in the last case, a universal family of ESNs can be constructed that contains exclusively elements that have the echo state and the fading memory properties. This conclusion could not be drawn with the results and methods available so far in the literature.