Guided Self-Organization of Input-Driven Recurrent Neural Networks

TL;DR

This paper reviews the computational properties of input-driven recurrent neural networks, discusses methods to quantify and improve their performance through guided self-organization, and explores future directions for understanding their computational mechanisms.

Contribution

It introduces a framework for guiding self-organization in reservoirs using quantitative measures and proposes an information-theoretic approach to evaluate task performance.

Findings

Self-organization can enhance reservoir performance.

Quantitative measures help understand RNN computational properties.

Information-theoretic metrics can evaluate task success.

Abstract

We review attempts that have been made towards understanding the computational properties and mechanisms of input-driven dynamical systems like RNNs, and reservoir computing networks in particular. We provide details on methods that have been developed to give quantitative answers to the questions above. Following this, we show how self-organization may be used to improve reservoirs for better performance, in some cases guided by the measures presented before. We also present a possible way to quantify task performance using an information-theoretic approach, and finally discuss promising future directions aimed at a better understanding of how these systems perform their computations and how to best guide self-organized processes for their optimization.