Latent Trajectory Dynamics in Large Language Models: A Manifold Evolution Framework with Empirical Validation

TL;DR

This paper introduces DMET, a framework modeling LLM generation as a dynamical system on a semantic manifold, with metrics predicting text quality and enabling adaptive decoding control.

Contribution

The paper presents DMET, a novel dynamical systems framework for understanding and controlling LLM latent trajectory evolution during text generation.

Findings

Metrics C, Q, P predict text quality outcomes across models and tasks.

Trajectory structure effects are validated as genuine, not artifacts.

Online monitoring of C improves perplexity through adaptive decoding.

Abstract

Understanding how latent representations evolve during generation is a central open problem in large language model interpretability. We introduce \textbf{Dynamical Manifold Evolution Theory} (DMET), a phenomenological framework that models LLM generation as a controlled dynamical system evolving along a trajectory on a low-dimensional semantic manifold. DMET formalizes the structural correspondence between Transformer components and a first-order ODE governed by a semantic potential $V$, and characterizes trajectory geometry through three falsifiable proxy metrics: state continuity $C$, attractor clustering quality $Q$, and topological persistence $P$, targeting local smoothness, meso-scale basin structure, and global topological organization, respectively. Across six model architectures, four task types, and 1,080 experimental runs, all three metrics consistently predict text quality outcomes -- log-perplexity, grammaticality, and cross-sentence coherence -- after controlling for decoding parameters, with associations surviving Benjamini--Hochberg correction. Ablation and sanity-check experiments confirm that the effects arise from genuine trajectory structure rather than static distributional artefacts. Furthermore, online monitoring of $C$ drives an adaptive decoding controller that reduces perplexity from 48.5 to 14.6 relative to a fixed-parameter baseline, demonstrating that latent dynamics characterization translates directly into actionable generation control.