Scale-Dependent Semantic Dynamics Revealed by Allan Deviation

TL;DR

This paper applies Allan deviation to analyze the semantic stability of written text, revealing distinct dynamical regimes and differences between human language and language models in semantic coherence.

Contribution

It introduces a novel application of Allan deviation to quantify semantic dynamics and distinguishes human language from models based on stability horizons.

Findings

Two dynamical regimes identified: short-time power-law scaling and long-time stability limit.

Language models mimic local statistics but have reduced stability horizons.

Semantic coherence can be quantitatively measured as a physical property.

Abstract

While language progresses through a sequence of semantic states, the underlying dynamics of this progression remain elusive. Here, we treat the semantic progression of written text as a stochastic trajectory in a high-dimensional state space. We utilize Allan deviation, a tool from precision metrology, to analyze the stability of meaning by treating ordered sentence embeddings as a displacement signal. Our analysis reveals two distinct dynamical regimes: short-time power-law scaling, which differentiates creative literature from technical texts, and a long-time crossover to a stability-limited noise floor. We find that while large language models successfully mimic the local scaling statistics of human text, they exhibit a systematic reduction in their stability horizon. These results establish semantic coherence as a measurable physical property, offering a framework to differentiate the nuanced dynamics of human cognition from the patterns generated by algorithmic models.