Replica Symmetry Breaking in Attractor Neural Network Models

TL;DR

This paper investigates replica symmetry breaking in Hopfield neural network models, finding that the actual storage capacity is slightly higher than the replica symmetric estimate and questioning the validity of one-step RSB results.

Contribution

The study provides a detailed analysis of replica symmetry breaking in generalized Hopfield models, challenging previous 1RSB results and exploring the effects of low neural activity.

Findings

1RSB and 2RSB capacities are only marginally higher than RS capacity.

The capacities are lower than simulation-based estimates.

Reentrant phase behavior may vanish in the infinite Parisi scheme.

Abstract

The phenomenon of replica symmetry breaking is investigated for the retrieval phases of Hopfield-type network models. The basic calculation is done for the generalized version of the standard model introduced by Horner [1] and by Perez-Vicente and Amit [2] which can exhibit low mean levels of neural activity. For a mean activity $\bar a =1/2$ the Hopfield model is recovered. In this case, surprisingly enough, we cannot confirm the well known one step replica symmetry breaking (1RSB) result for the storage capacity which was presented by Crisanti, Amit and Gutfreund [3] ($\alpha_c^{\hbox{\mf 1RSB}}\simeq 0.144$). Rather, we find that 1RSB- and 2RSB-Ans\"atze yield only slightly increased capacities as compared to the replica symmetric value ($\alpha_c^{\hbox{\mf 1RSB}}\simeq 0.138\,186$ and $\alpha_c^{\hbox{\mf 2RSB}}\simeq 0.138\,187 $ compared to $\alpha_c^{\hbox{\mf RS}}\simeq 0.137\,905$), significantly smaller also than the value $\alpha_c^{\hbox{\mf sim}} = 0.145\pm 0.009$ reported from simulation studies. These values still lie within the recently discovered reentrant phase [4]. We conjecture that in the infinite Parisi-scheme the reentrant behaviour disappears as is the case in the SK-spin-glass model (Parisi--Toulouse-hypothesis). The same qualitative results are obtained in the low activity range.