Deep Time-Delay Reservoir Computing: Dynamics and Memory Capacity

TL;DR

This paper explores how the dynamical properties of deep time-delay reservoir computing systems influence their memory capacity, providing insights for optimizing system design through bifurcation analysis and resonance effects.

Contribution

It introduces a detailed analysis of the dynamical regimes of deep Ikeda-based reservoirs and links these to memory capacity, enabling targeted optimization of reservoir performance.

Findings

Memory capacity relates to bifurcation proximity and Lyapunov exponents.

Resonances between clock cycles and delays can enhance memory performance.

Configurations can be designed for high nonlinear or long linear memory capacity.

Abstract

The Deep Time-Delay Reservoir Computing concept utilizes unidirectionally connected systems with time-delays for supervised learning. We present how the dynamical properties of a deep Ikeda-based reservoir are related to its memory capacity (MC) and how that can be used for optimization. In particular, we analyze bifurcations of the corresponding autonomous system and compute conditional Lyapunov exponents, which measure the generalized synchronization between the input and the layer dynamics. We show how the MC is related to the systems distance to bifurcations or magnitude of the conditional Lyapunov exponent. The interplay of different dynamical regimes leads to a adjustable distribution between linear and nonlinear MC. Furthermore, numerical simulations show resonances between clock cycle and delays of the layers in all degrees of the MC. Contrary to MC losses in a single-layer reservoirs, these resonances can boost separate degrees of the MC and can be used, e.g., to design a system with maximum linear MC. Accordingly, we present two configurations that empower either high nonlinear MC or long time linear MC.