Effect of spatial correlations on Hopfield Neural Network and Dense Associative Memories

TL;DR

This paper investigates how spatial correlations in data, modeled by an Ising configuration, affect the retrieval capacity of Hopfield networks and Dense Associative Memories, revealing that increased correlations reduce their critical load.

Contribution

It provides an analytical study of the impact of spatial correlations on neural network retrieval capacity, extending results to Dense Associative Memories with arbitrary odd-body interactions.

Findings

Increased spatial correlations decrease the critical load of Hopfield networks.

A phase diagram shows the relationship between load, temperature, and retrieval ability.

Results confirmed by numerical simulations.

Abstract

Hopfield model is one of the few neural networks for which analytical results can be obtained. However, most of them are derived under the assumption of random uncorrelated patterns, while in real life applications data to be stored show non-trivial correlations. In the present paper we study how the retrieval capability of the Hopfield network at null temperature is affected by spatial correlations in the data we feed to it. In particular, we use as patterns to be stored the configurations of a linear Ising model at inverse temperature $\beta$. Exploiting the signal to noise technique we obtain a phase diagram in the load of the Hopfield network and the Ising temperature where a fuzzy phase and a retrieval region can be observed. Remarkably, as the spatial correlation inside patterns is increased, the critical load of the Hopfield network diminishes, a result also confirmed by numerical simulations. The analysis is then generalized to Dense Associative Memories with arbitrary odd-body interactions, for which we obtain analogous results.