The production of meaning in the processing of natural language

TL;DR

This paper investigates the contextuality in natural language processing, revealing that quantum-like mechanisms better explain semantic production and highlighting implications for AI safety and manipulation.

Contribution

It demonstrates that models exhibit quantum-like contextuality in language understanding, with the interquartile range of the CHSH parameter distinguishing models independently of benchmarks.

Findings

Models show quantum contextuality violations similar to cognitive science findings.

The interquartile range of the CHSH parameter uniquely differentiates models.

Violation rate correlates weakly with external language benchmarks.

Abstract

Understanding the fundamental mechanisms governing the production of meaning in the processing of natural language is critical for designing safe, thoughtful, engaging, and empowering human-agent interactions. Experiments in cognitive science and social psychology have demonstrated that human semantic processing exhibits contextuality more consistent with quantum logical mechanisms than classical Boolean theories, and recent works have found similar results in large language models -- in particular, clear violations of the Bell inequality in experiments of contextuality during interpretation of ambiguous expressions. We explore the CHSH $|S|$ parameter -- the metric associated with the inequality -- across the inference parameter space of models spanning four orders of magnitude in scale, cross-referencing it with MMLU, hallucination rate, and nonsense detection benchmarks. We find that the interquartile range of the $|S|$ distribution -- the statistic that most sharply differentiates models from one another -- is completely orthogonal to all external benchmarks, while violation rate shows weak anticorrelation with all three benchmarks that does not reach significance. We investigate how $|S|$ varies with sampling parameters and word order, and discuss the information-theoretic constraints that genuine contextuality imposes on prompt injection defenses and its human analogue, whereby careful construction and maintenance of social contextuality can be carried out at scale -- manufacturing not consent but contextuality itself, a subtler and more fundamental form of manipulation that shapes the space of possible interpretations before any particular one is reached.