The Hopfield Model with Superlinearly Many Patterns

TL;DR

This paper analyzes the Hopfield model with a large number of patterns, revealing its free energy behavior and connecting it to the Sherrington-Kirkpatrick model, thus advancing understanding of high-capacity neural networks.

Contribution

It establishes the asymptotic free energy of the Hopfield model with superlinearly many patterns and links it to the SK model's free energy, providing new theoretical insights.

Findings

Free energy scales as +

Connection between Hopfield and SK models in high-pattern regime

Asymptotic behavior characterized for large 

Abstract

We study the Hopfield model where the ratio $\alpha$ of patterns to sites grows large. We prove that the free energy with inverse temperature $\beta$ and external field $B$ behaves like $\beta\sqrt\alpha+\gamma$, where $\gamma$ is the limiting free energy of the Sherrington-Kirkpatrick model with inverse temperature $\sqrt2\beta$ and external field $B$.