Enhanced storage capacity with errors in scale-free Hopfield neural networks: an analytical study

TL;DR

This study analytically investigates how scale-free network structures influence the storage capacity and error rates of Hopfield neural networks, revealing enhanced capacity with some retrieval errors due to network heterogeneity.

Contribution

It introduces an analytical framework for Hopfield networks on scale-free graphs, showing increased storage capacity with errors, unlike traditional fully connected models.

Findings

Storage capacity is significantly increased in scale-free networks.

Memory retrieval errors increase with network heterogeneity.

Real neural networks exhibit similar error patterns as the scale-free model.

Abstract

The Hopfield model is a pioneering neural network model with associative memory retrieval. The analytical solution of the model in mean field limit revealed that memories can be retrieved without any error up to a finite storage capacity of $O(N)$, where $N$ is the system size. Beyond the threshold, they are completely lost. Since the introduction of the Hopfield model, the theory of neural networks has been further developed toward realistic neural networks using analog neurons, spiking neurons, etc. Nevertheless, those advances are based on fully connected networks, which are inconsistent with recent experimental discovery that the number of connections of each neuron seems to be heterogeneous, following a heavy-tailed distribution. Motivated by this observation, we consider the Hopfield model on scale-free networks and obtain a different pattern of associative memory retrieval from that obtained on the fully connected network: the storage capacity becomes tremendously enhanced but with some error in the memory retrieval, which appears as the heterogeneity of the connections is increased. Moreover, the error rates are also obtained on several real neural networks and are indeed similar to that on scale-free model networks.