Universal discrete-time reservoir computers with stochastic inputs and linear readouts using non-homogeneous state-affine systems

TL;DR

This paper introduces a new class of non-homogeneous state-affine systems for reservoir computing, demonstrating their universality and properties with stochastic inputs and linear readouts.

Contribution

It defines sufficient conditions for these systems to be causal, time-invariant, and universal for fading memory filters with stochastic inputs.

Findings

Reservoir computers with these systems are causal, time-invariant, and satisfy fading memory.

A subset of these systems is universal for stochastic fading memory filters.

Any such filter can be approximated by the proposed reservoir computing models.

Abstract

A new class of non-homogeneous state-affine systems is introduced for use in reservoir computing. Sufficient conditions are identified that guarantee first, that the associated reservoir computers with linear readouts are causal, time-invariant, and satisfy the fading memory property and second, that a subset of this class is universal in the category of fading memory filters with stochastic almost surely uniformly bounded inputs. This means that any discrete-time filter that satisfies the fading memory property with random inputs of that type can be uniformly approximated by elements in the non-homogeneous state-affine family.