How noise affects memory in linear recurrent networks

TL;DR

This paper theoretically investigates how noise impacts memory in linear recurrent networks, revealing that memory degradation depends on the noise's spectral properties and can be unaffected by noise intensity under certain conditions.

Contribution

It introduces a spectral-based framework for understanding noise effects on memory in linear recurrent networks, including conditions where noise does not impair memory.

Findings

Memory reduction is determined by the noise's PSD.

Memory remains unaffected by noise intensity for certain PSD classes.

Results align with human brain signal data.

Abstract

The effects of noise on memory in a linear recurrent network are theoretically investigated. Memory is characterized by its ability to store previous inputs in its instantaneous state of network, which receives a correlated or uncorrelated noise. Two major properties are revealed: First, the memory reduced by noise is uniquely determined by the noise's power spectral density (PSD). Second, the memory will not decrease regardless of noise intensity if the PSD is in a certain class of distribution (including power law). The results are verified using the human brain signals, showing good agreement.