Do LLMs dream of elephants (when told not to)? Latent concept association and associative memory in transformers

TL;DR

This paper investigates how large language models can be manipulated to recall facts through context changes, revealing their behavior as associative memory systems and analyzing the underlying transformer mechanisms both theoretically and empirically.

Contribution

It demonstrates that transformers function as associative memory by using self-attention and value matrices, supported by theoretical analysis and experiments on a simple latent concept association task.

Findings

Transformers retrieve facts via self-attention mechanisms.

Context manipulation can alter LLMs' factual recall.

Transformers can be modeled as associative memory systems.

Abstract

Large Language Models (LLMs) have the capacity to store and recall facts. Through experimentation with open-source models, we observe that this ability to retrieve facts can be easily manipulated by changing contexts, even without altering their factual meanings. These findings highlight that LLMs might behave like an associative memory model where certain tokens in the contexts serve as clues to retrieving facts. We mathematically explore this property by studying how transformers, the building blocks of LLMs, can complete such memory tasks. We study a simple latent concept association problem with a one-layer transformer and we show theoretically and empirically that the transformer gathers information using self-attention and uses the value matrix for associative memory.