Curved Inference: Concern-Sensitive Geometry in Large Language Model Residual Streams

TL;DR

This paper introduces Curved Inference, a geometric interpretability framework that analyzes how large language models' internal trajectories bend in response to semantic concern shifts, revealing insights into their alignment and inference dynamics.

Contribution

It presents a novel geometric framework for analyzing LLM internal trajectories, focusing on curvature and salience, and demonstrates its effectiveness across multiple models and domains.

Findings

Concern shifts reliably alter activation trajectories.

LLaMA shows significant scaling in curvature and salience with concern intensity.

Gemma responds to concern but with weaker differentiation.

Abstract

We propose Curved Inference - a geometric Interpretability framework that tracks how the residual stream trajectory of a large language model bends in response to shifts in semantic concern. Across 20 matched prompts spanning emotional, moral, perspective, logical, identity, environmental, and nonsense domains, we analyse Gemma3-1b and LLaMA3.2-3b using five native-space metrics, with a primary focus on curvature (\k{appa}_i) and salience (S(t)). These metrics are computed under a pullback semantic metric derived from the unembedding matrix, ensuring that all measurements reflect token-aligned geometry rather than raw coordinate structure. We find that concern-shifted prompts reliably alter internal activation trajectories in both models - with LLaMA exhibiting consistent, statistically significant scaling in both curvature and salience as concern intensity increases. Gemma also responds to concern but shows weaker differentiation between moderate and strong variants. Our results support a two-layer view of LLM geometry - a latent conceptual structure encoded in the embedding space, and a contextual trajectory shaped by prompt-specific inference. Curved Inference reveals how models navigate, reorient, or reinforce semantic meaning over depth, offering a principled method for diagnosing alignment, abstraction, and emergent inference dynamics. These findings offer fresh insight into semantic abstraction and model alignment through the lens of Curved Inference.