What is a Number, That a Large Language Model May Know It?

TL;DR

This paper investigates how large language models represent numbers, revealing an entangled space blending string and numerical features, with implications for their understanding and decision-making capabilities.

Contribution

It introduces a similarity-based prompting method to analyze LLM number representations, showing the duality and entanglement of string-like and numerical features in these models.

Findings

LLMs learn blended string and number representations.

Similarity judgments align with Levenshtein and Log-Linear distances.

Context influences but does not fully resolve the representation duality.

Abstract

Numbers are a basic part of how humans represent and describe the world around them. As a consequence, learning effective representations of numbers is critical for the success of large language models as they become more integrated into everyday decisions. However, these models face a challenge: depending on context, the same sequence of digit tokens, e.g., 911, can be treated as a number or as a string. What kind of representations arise from this duality, and what are its downstream implications? Using a similarity-based prompting technique from cognitive science, we show that LLMs learn representational spaces that blend string-like and numerical representations. In particular, we show that elicited similarity judgments from these models over integer pairs can be captured by a combination of Levenshtein edit distance and numerical Log-Linear distance, suggesting an entangled representation. In a series of experiments we show how this entanglement is reflected in the latent embeddings, how it can be reduced but not entirely eliminated by context, and how it can propagate into a realistic decision scenario. These results shed light on a representational tension in transformer models that must learn what a number is from text input.