The signal-to-noise analysis of the Little-Hopfield model revisited

TL;DR

This paper revisits the signal-to-noise analysis of the Little-Hopfield model, deriving exact dynamics using generating functional analysis and extending it to include all correlations, with implications for various neural network architectures.

Contribution

It provides an exact recursion relation for the model's dynamics and extends the signal-to-noise analysis to a full theory including all correlations.

Findings

Numerical simulations confirm the derived recursion relations.

Leaving out feedback correlations yields the usual signal-to-noise approximation.

The extended theory applies to diluted architectures and sequence processing networks.

Abstract

Using the generating functional analysis an exact recursion relation is derived for the time evolution of the effective local field of the fully connected Little-Hopfield model. It is shown that, by leaving out the feedback correlations arising from earlier times in this effective dynamics, one precisely finds the recursion relations usually employed in the signal-to-noise approach. The consequences of this approximation as well as the physics behind it are discussed. In particular, it is pointed out why it is hard to notice the effects, especially for model parameters corresponding to retrieval. Numerical simulations confirm these findings. The signal-to-noise analysis is then extended to include all correlations, making it a full theory for dynamics at the level of the generating functional analysis. The results are applied to the frequently employed extremely diluted (a)symmetric architectures and to sequence processing networks.