Eigenvalue spectrum of neural networks with arbitrary Hebbian length

TL;DR

This paper extends associative memory models to include long-range Hebbian interactions, deriving the spectral density of the coupling matrix and analyzing the eigen-spectrum to understand memory capacity.

Contribution

It introduces a generalized model with arbitrary Hebbian length and derives its spectral properties using replica and free probability methods.

Findings

Spectral density of the coupling matrix derived

Maximal eigenvalue characterized by an iterative equation

Connection established between Hebbian length and eigen-spectrum

Abstract

Associative memory is a fundamental function in the brain. Here, we generalize the standard associative memory model to include long-range Hebbian interactions at the learning stage, corresponding to a large synaptic integration window. In our model, the Hebbian length can be arbitrarily large. The spectral density of the coupling matrix is derived using the replica method, which is also shown to be consistent with the results obtained by applying the free probability method. The maximal eigenvalue is then obtained by an iterative equation, related to the paramagnetic to spin glass transition in the model. Altogether, this work establishes the connection between the associative memory with arbitrary Hebbian length and the asymptotic eigen-spectrum of the neural-coupling matrix.