The Geometry of Thought: How Scale Restructures Reasoning In Large Language Models

TL;DR

This paper explores how scaling large language models affects reasoning, revealing domain-specific geometric phase transitions and introducing neural reasoning operators to predict reasoning outcomes.

Contribution

It uncovers domain-specific geometric restructuring in reasoning with scale and proposes neural reasoning operators for predicting reasoning endpoints.

Findings

Legal reasoning shows dimensionality collapse and untangling with scale.

Scientific and mathematical reasoning remain geometrically invariant despite scale increases.

Neural reasoning operators achieve 63.6% accuracy in predicting reasoning endpoints.

Abstract

Scale does not uniformly improve reasoning - it restructures it. Analyzing 25,000+ chain-of-thought trajectories across four domains (Law, Science, Code, Math) and two scales (8B, 70B parameters), we discover that neural scaling laws trigger domain-specific phase transitions rather than uniform capability gains. Legal reasoning undergoes Crystallization: 45% collapse in representational dimensionality (d95: 501 -> 274), 31% increase in trajectory alignment, and 10x manifold untangling. Scientific and mathematical reasoning remain Liquid - geometrically invariant despite 9x parameter increase. Code reasoning forms a discrete Lattice of strategic modes (silhouette: 0.13 -> 0.42). This geometry predicts learnability. We introduce Neural Reasoning Operators - learned mappings from initial to terminal hidden states. In crystalline legal reasoning, our operator achieves 63.6% accuracy on held-out tasks via probe decoding, predicting reasoning endpoints without traversing intermediate states. We further identify a universal oscillatory signature (coherence ~ -0.4) invariant across domains and scales, suggesting attention and feedforward layers drive reasoning through opposing dynamics. These findings establish that the cost of thought is determined not by task difficulty but by manifold geometry - offering a blueprint for inference acceleration where topology permits.