Dreaming improves memorization in a Hopfield model with bounded synaptic strength

TL;DR

This paper explores how incorporating a dreaming phase into a biologically plausible clipped Hopfield model enhances its memorization capacity and mitigates the limitations caused by synaptic strength bounds, compared to traditional models.

Contribution

It introduces a dreaming phase into the clipped Hopfield model, demonstrating improved memory capacity and more realistic performance optimization.

Findings

Dreaming phase enhances memorization capacity.

Clipping prevents catastrophic forgetting but reduces capacity.

Alternating learning and dreaming improves performance.

Abstract

The Hopfield model provides a paradigmatic framework for associative memory. Its classical implementation, based on the Hebbian learning rule, suffers from catastrophic forgetting: when one attempts storing too many patterns, the network fails to retrieve any of them. Yet, the Hebbian rule does not take into account that synaptic strength is bounded. Introducing this biologically plausible modification, known as "clipping", eliminates catastrophic forgetting; the model is now able to retrieve the most recently seen memories, eliminating older ones. Yet, its memorization capacity is much reduced with respect to the unclipped case. Here, we investigate the effects of adding a "dreaming" phase on the capacity of a clipped Hopfield model. Following a proposal by Hopfield, Feinstein and Palmer, we assume that during the dreaming phase, the model generates random patterns that are then "unlearned". We show that while clipping still removes catastrophic forgetting, alternating learning and dreaming phases improves the memorization capacity and makes the search for optimal performance more realistic from an evolutionary perspective.