Large Language Model Recall Uncertainty is Modulated by the Fan Effect

TL;DR

This study investigates whether large language models exhibit fan effects similar to humans, finding that their recall uncertainty is influenced by the fan effect and related to typicality, with implications for understanding LLM cognition.

Contribution

First demonstration that LLM recall uncertainty is modulated by fan effects, linking cognitive phenomena in humans to artificial models.

Findings

LLMs exhibit fan effects in recall uncertainty.

Removing uncertainty disrupts the fan effect.

Fan effects are consistent in in-context and pre-training data.

Abstract

This paper evaluates whether large language models (LLMs) exhibit cognitive fan effects, similar to those discovered by Anderson in humans, after being pre-trained on human textual data. We conduct two sets of in-context recall experiments designed to elicit fan effects. Consistent with human results, we find that LLM recall uncertainty, measured via token probability, is influenced by the fan effect. Our results show that removing uncertainty disrupts the observed effect. The experiments suggest the fan effect is consistent whether the fan value is induced in-context or in the pre-training data. Finally, these findings provide in-silico evidence that fan effects and typicality are expressions of the same phenomena.