In search of dispersed memories: Generative diffusion models are associative memory networks

TL;DR

This paper demonstrates that generative diffusion models can function as associative memory networks, with energy-based interpretations linking them to Hopfield networks, offering a unified view of memory formation and recall.

Contribution

It reveals that diffusion models can be viewed as energy-based associative memory networks, bridging generative modeling and neural memory mechanisms.

Findings

Diffusion models' energy functions are equivalent to Hopfield networks.

Supervised training encodes associative memory dynamics.

Unified framework for memory formation and recall.

Abstract

Uncovering the mechanisms behind long-term memory is one of the most fascinating open problems in neuroscience and artificial intelligence. Artificial associative memory networks have been used to formalize important aspects of biological memory. Generative diffusion models are a type of generative machine learning techniques that have shown great performance in many tasks. Like associative memory systems, these networks define a dynamical system that converges to a set of target states. In this work we show that generative diffusion models can be interpreted as energy-based models and that, when trained on discrete patterns, their energy function is (asymptotically) identical to that of modern Hopfield networks. This equivalence allows us to interpret the supervised training of diffusion models as a synaptic learning process that encodes the associative dynamics of a modern Hopfield network in the weight structure of a deep neural network. Leveraging this connection, we formulate a generalized framework for understanding the formation of long-term memory, where creative generation and memory recall can be seen as parts of a unified continuum.