The Geometry of Numerical Reasoning: Language Models Compare Numeric Properties in Linear Subspaces

TL;DR

This study reveals that large language models encode numerical attributes in low-dimensional subspaces within their embeddings and that manipulating these subspaces can alter their numerical reasoning outcomes.

Contribution

The paper identifies and manipulates low-dimensional subspaces encoding numerical attributes in LLMs, demonstrating their role in numerical reasoning.

Findings

LLMs encode numerical attributes in linear subspaces.

Intervening in these subspaces changes model outputs.

Results are consistent across different models and attributes.

Abstract

This paper investigates whether large language models (LLMs) utilize numerical attributes encoded in a low-dimensional subspace of the embedding space when answering questions involving numeric comparisons, e.g., Was Cristiano born before Messi? We first identified, using partial least squares regression, these subspaces, which effectively encode the numerical attributes associated with the entities in comparison prompts. Further, we demonstrate causality, by intervening in these subspaces to manipulate hidden states, thereby altering the LLM's comparison outcomes. Experiments conducted on three different LLMs showed that our results hold across different numerical attributes, indicating that LLMs utilize the linearly encoded information for numerical reasoning.