Universality of reservoir systems with recurrent neural networks

TL;DR

This paper proves that reservoir systems based on recurrent neural networks can universally approximate a class of dynamical systems by adjusting linear readouts, demonstrating their broad approximation capabilities.

Contribution

It introduces the concept of uniform strong universality for RNN reservoir systems and constructs a parallel concatenation method to achieve error bounds independent of specific targets.

Findings

RNN reservoir systems can approximate a class of dynamical systems with arbitrary accuracy.

A construction method for RNN reservoirs with uniform approximation error bounds.

Theoretical proof of the universality of RNN reservoir systems.

Abstract

Approximation capability of reservoir systems whose reservoir is a recurrent neural network (RNN) is discussed. We show what we call uniform strong universality of RNN reservoir systems for a certain class of dynamical systems. This means that, given an approximation error to be achieved, one can construct an RNN reservoir system that approximates each target dynamical system in the class just via adjusting its linear readout. To show the universality, we construct an RNN reservoir system via parallel concatenation that has an upper bound of approximation error independent of each target in the class.