The Geometric Price of Discrete Logic: Context-driven Manifold Dynamics of Number Representations
Long Zhang, Dai-jun Lin, Wei-neng Chen

TL;DR
This paper reveals how large language models form discrete logical boundaries through topological distortions driven by task context, challenging the linear-isometric assumptions and linking geometry to logical reasoning.
Contribution
It introduces a dual-modulation geometric mechanism involving topological preservation and algebraic divergence, validated across various tasks, and demonstrates causal links via vector ablation.
Findings
Topological distortion enforces logical boundaries in LLMs.
Ablation of divergence collapses classification accuracy.
Models under social pressure exhibit manifold entanglement.
Abstract
Large language models (LLMs) generalize smoothly across continuous semantic spaces, yet strict logical reasoning demands the formation of discrete decision boundaries. Prevailing theories relying on linear isometric projections fail to resolve this fundamental tension. In this work, we argue that task context operates as a non-isometric dynamical operator that enforces a necessary "topological distortion." By applying Gram-Schmidt decomposition to residual-stream activations , we reveal a dual-modulation mechanism driving this process: a class-agnostic topological preservation that anchors global structure to prevent semantic collapse, and a specific algebraic divergence that directionally tears apart cross-class concepts to forge logical boundaries. We validate this geometric evolution across a gradient of tasks, from simple mapping to complex primality testing. Crucially, targeted…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Ferroelectric and Negative Capacitance Devices · Language and cultural evolution
