Mean-field theory for a spin-glass model of neural networks: TAP free energy and paramagnetic to spin-glass transition

TL;DR

This paper develops a mean-field approach for the Hopfield neural network model, deriving the TAP free energy explicitly and analyzing the spin-glass transition, comparing it with replica and cavity methods.

Contribution

It introduces a TAP-based mean-field treatment for the Hopfield model considering pattern correlations, providing explicit free energy expressions and transition analysis.

Findings

Explicit TAP free energy for the Hopfield model derived

Analysis of spin-glass transition compared with replica and cavity methods

Cluster effects incorporated into the free energy expression

Abstract

An approach is proposed to the Hopfield model where the mean-field treatment is made for a given set of stored patterns (sample) and then the statistical average over samples is taken. This corresponds to the approach made by Thouless, Anderson and Palmer (TAP) to the infinite-range model of spin glasses. Taking into account the fact that in the Hopfield model there exist correlations between different elements of the interaction matrix, we obtain its TAP free energy explicitly, which consists of a series of terms exhibiting the cluster effect. Nature of the spin-glass transition in the model is also examined and compared with those given by the replica method as well as the cavity method.