Nearly extensive sequential memory lifetime achieved by coupled nonlinear neurons

TL;DR

This paper demonstrates that by leveraging nonlinear neuron dynamics, it is possible to achieve a nearly extensive memory lifetime proportional to network size, surpassing previous linear bounds.

Contribution

The work shows how nonlinear neuron responses enable near-linear scaling of memory lifetime, introducing an error-correction mechanism in neural networks.

Findings

Memory lifetime scales as N / log N with nonlinear neurons.

Nonlinear dynamics reduce noise accumulation in neural activity.

Achieves near-extensive memory capacity in theoretical neural models.

Abstract

Many cognitive processes rely on the ability of the brain to hold sequences of events in short-term memory. Recent studies have revealed that such memory can be read out from the transient dynamics of a network of neurons. However, the memory performance of such a network in buffering past information has only been rigorously estimated in networks of linear neurons. When signal gain is kept low, so that neurons operate primarily in the linear part of their response nonlinearity, the memory lifetime is bounded by the square root of the network size. In this work, I demonstrate that it is possible to achieve a memory lifetime almost proportional to the network size, "an extensive memory lifetime", when the nonlinearity of neurons is appropriately utilized. The analysis of neural activity revealed that nonlinear dynamics prevented the accumulation of noise by partially removing noise in each time step. With this error-correcting mechanism, I demonstrate that a memory lifetime of order $N/\log N$ can be achieved.