The Geometric Price of Discrete Logic: Context-driven Manifold Dynamics of Number Representations

TL;DR

This paper reveals how large language models form discrete logical boundaries through topological distortions driven by task context, challenging the linear-isometric assumptions and linking geometry to logical reasoning.

Contribution

It introduces a dual-modulation geometric mechanism involving topological preservation and algebraic divergence, validated across various tasks, and demonstrates causal links via vector ablation.

Findings

Topological distortion enforces logical boundaries in LLMs.

Ablation of divergence collapses classification accuracy.

Models under social pressure exhibit manifold entanglement.

Abstract

Large language models (LLMs) generalize smoothly across continuous semantic spaces, yet strict logical reasoning demands the formation of discrete decision boundaries. Prevailing theories relying on linear isometric projections fail to resolve this fundamental tension. In this work, we argue that task context operates as a non-isometric dynamical operator that enforces a necessary "topological distortion." By applying Gram-Schmidt decomposition to residual-stream activations , we reveal a dual-modulation mechanism driving this process: a class-agnostic topological preservation that anchors global structure to prevent semantic collapse, and a specific algebraic divergence that directionally tears apart cross-class concepts to forge logical boundaries. We validate this geometric evolution across a gradient of tasks, from simple mapping to complex primality testing. Crucially, targeted specific vector ablation establishes a strict causal binding between this topology and model function: algebraically erasing the divergence component collapses parity classification accuracy from 100% to chance levels (38.57%). Furthermore, we uncover a three-phase layer-wise geometric dynamic and demonstrate that under social pressure prompts, models fail to generate sufficient divergence. This results in a "manifold entanglement" that geometrically explains sycophancy and hallucination. Ultimately, our findings revise the linear-isometric presumption, demonstrating that the emergence of discrete logic in LLMs is purchased at an irreducible cost of topological deformation.