Recurrence Resonance -- Noise-Enhanced Dynamics in Recurrent Neural Networks

TL;DR

This paper investigates how noise can enhance information flow in recurrent neural networks by inducing recurrence resonance, especially in systems with multiple attractors, with implications for neuroscience and reservoir computing.

Contribution

It demonstrates that recurrence resonance occurs in neural networks with multiple attractors and shows how noise can be used to switch between dynamical states, advancing understanding of noise's role in neural dynamics.

Findings

Recurrence resonance occurs in systems with multiple attractors.

Short noise pulses can switch networks between attractors.

Resonance can be observed in small neuron subsets.

Abstract

In specific motifs of three recurrently connected neurons with probabilistic response, the spontaneous information flux, defined as the mutual information between subsequent states, has been shown to increase by adding ongoing white noise of some optimal strength to each of the neurons \cite{krauss2019recurrence}. However, the precise conditions for and mechanisms of this phenomenon called 'recurrence resonance' (RR) remain largely unexplored. Using Boltzmann machines of different sizes and with various types of weight matrices, we show that RR can generally occur when a system has multiple dynamical attractors, but is trapped in one or a few of them. In probabilistic networks, the phenomenon is bound to a suitable observation time scale, as the system could autonomously access its entire attractor landscape even without the help of external noise, given enough time. Yet, even in large systems, where time scales for observing RR in the full network become too long, the resonance can still be detected in small subsets of neurons. Finally, we show that short noise pulses can be used to transfer recurrent neural networks, both probabilistic and deterministic, between their dynamical attractors. Our results are relevant to the fields of reservoir computing and neuroscience, where controlled noise may turn out a key factor for efficient information processing leading to more robust and adaptable systems.