Pre-trained Language Models Learn Remarkably Accurate Representations of Numbers

TL;DR

Pretrained language models encode numbers with high precision, and a new probing method reveals their accurate numeric representations, which can be leveraged to reduce arithmetic errors.

Contribution

A novel probing technique that accurately decodes numeric values from language model embeddings, demonstrating their precise numeric representations post pre-training.

Findings

Language models encode numbers with remarkable accuracy.

Embedding alignment can reduce arithmetic errors.

Proposed method outperforms previous probing techniques.

Abstract

Pretrained language models (LMs) are prone to arithmetic errors. Existing work showed limited success in probing numeric values from models' representations, indicating that these errors can be attributed to the inherent unreliability of distributionally learned embeddings in representing exact quantities. However, we observe that previous probing methods are inadequate for the emergent structure of learned number embeddings with sinusoidal patterns. In response, we propose a novel probing technique that decodes numeric values from input embeddings with near-perfect accuracy across a range of open-source LMs. This proves that after the sole pre-training, LMs represent numbers with remarkable precision. Finally, we find that the embeddings' precision, judged by our probe's accuracy, explains a large portion of LM's errors in elementary arithmetic, and show that aligning the embeddings with the pattern our probes discover can mitigate these errors.