Weighted Patterns as a Tool for Improving the Hopfield Model

TL;DR

This paper extends the Hopfield model by incorporating pattern weights, enabling online learning without catastrophic memory loss, and identifies conditions under which memory is preserved based on weight distributions.

Contribution

It introduces a weighted Hopfield model with a new analytical framework, allowing for online learning and improved memory retention compared to the standard model.

Findings

Memory is preserved for patterns with weights above a critical threshold.

The model allows online learning without catastrophic memory destruction.

Memory capacity decreases compared to the standard Hopfield model.

Abstract

We generalize the standard Hopfield model to the case when a weight is assigned to each input pattern. The weight can be interpreted as the frequency of the pattern occurrence at the input of the network. In the framework of the statistical physics approach we obtain the saddle-point equation allowing us to examine the memory of the network. In the case of unequal weights our model does not lead to the catastrophic destruction of the memory due to its overfilling (that is typical for the standard Hopfield model). The real memory consists only of the patterns with weights exceeding a critical value that is determined by the weights distribution. We obtain the algorithm allowing us to find this critical value for an arbitrary distribution of the weights, and analyze in detail some particular weights distributions. It is shown that the memory decreases as compared to the case of the standard Hopfield model. However, in our model the network can learn online without the catastrophic destruction of the memory.