The Retrieval Phase of the Hopfield Model: A Rigorous Analysis of the Overlap Distribution

TL;DR

This paper rigorously analyzes the retrieval phase of the Hopfield model by studying the minima of a related random function, revealing conditions for the location and nature of minima that influence the model's storage capacity and retrieval performance.

Contribution

It provides a detailed, rigorous characterization of the minima structure of the overlap distribution in the Hopfield model, connecting minima locations to storage capacity thresholds.

Findings

Existence of positive thresholds g_a, 3g_c for minima localization

Minima are near 3b m^*e^b 03b vectors under certain conditions

Presence of extensive energy barriers around local minima

Abstract

Standard large deviation estimates or the use of the Hubbard-Stratonovich transformation reduce the analysis of the distribution of the overlap parameters essentially to that of an explicitly known random function $\Phi_{N,\b}$ on $\R^M$. In this article we present a rather careful study of the structure of the minima of this random function related to the retrieval of the stored patterns. We denote by $m^*(\b)$ the modulus of the spontaneous magnetization in the Curie-Weiss model and by $\a$ the ratio between the number of the stored patterns and the system size. We show that there exist strictly positive numbers $0<\g_a<\g_c$ such that 1) If $\sqrt\a\leq \g_a (m^*(\b))^2$, then the absolute minima of $\Phi$ are located within small balls around the points $\pm m^*e^\mu$, where $e^\mu$ denotes the $\mu$-th unit vector while 2) if $\sqrt\a\leq \g_c (m^*(\b))^2$ at least a local minimum surrounded by extensive energy barriers exists near these points. The random location of these minima is given within precise bounds. These are used to prove sharp estimates on the support of the Gibbs measures. KEYWORDS: Hopfield model, neural networks, storage capacity, Gibbs measures, self-averaging, random matrices