Dynamical stability for dense patterns in discrete attractor neural networks

TL;DR

This paper develops a new theory for the local stability of discrete attractor states in neural networks with graded activities, noise, and sparse patterns, revealing conditions for stability beyond classical capacity limits.

Contribution

It introduces a broad theoretical framework analyzing the stability of fixed points in neural networks with realistic features like noise and graded responses, extending previous restrictive models.

Findings

All fixed points are stable below a critical load dependent on neural activity statistics.

Stability depends on the activation function and pattern sparsity.

Threshold-linear activation and sparse patterns enhance computational stability.

Abstract

Neural networks storing multiple discrete attractors are canonical models of biological memory. Previously, the dynamical stability of such networks could only be guaranteed under highly restrictive conditions. Here, we derive a theory of the local stability of discrete fixed points in a broad class of networks with graded neural activities and in the presence of noise. By directly analyzing the bulk and the outliers of the Jacobian spectrum, we show that all fixed points are stable below a critical load that is distinct from the classical \textit{critical capacity} and depends on the statistics of neural activities in the fixed points as well as the single-neuron activation function. Our analysis highlights the computational benefits of threshold-linear activation and sparse-like patterns.