Language Models Represent Space and Time
Wes Gurnee, Max Tegmark
TL;DR
This paper provides evidence that large language models learn coherent, grounded representations of space and time, reflecting real-world structures across multiple scales and entity types.
Contribution
The study demonstrates that LLMs develop linear, robust, and unified spatiotemporal representations, including identifiable neurons encoding coordinates, indicating they learn more than superficial statistics.
Findings
LLMs learn linear representations of space and time.
Representations are robust to prompting variations.
Identified neurons reliably encode spatial and temporal coordinates.
Abstract
The capabilities of large language models (LLMs) have sparked debate over whether such systems just learn an enormous collection of superficial statistics or a set of more coherent and grounded representations that reflect the real world. We find evidence for the latter by analyzing the learned representations of three spatial datasets (world, US, NYC places) and three temporal datasets (historical figures, artworks, news headlines) in the Llama-2 family of models. We discover that LLMs learn linear representations of space and time across multiple scales. These representations are robust to prompting variations and unified across different entity types (e.g. cities and landmarks). In addition, we identify individual "space neurons" and "time neurons" that reliably encode spatial and temporal coordinates. While further investigation is needed, our results suggest modern LLMs learn rich…
Peer Reviews
Decision·ICLR 2024 poster
- This paper presents important evidence towards a critical debate (what do LLMs encode). - This paper organizes the probing studies in a way that is more comprehensive and rigorous than most probing papers that I have seen. To illustrate that the features are encoded linearly, this paper compares linear (ridge) regression probes and nonlinear MLP probes, and found that the nonlinear probes show minimal improvement in performance in any dataset or model. To illustrate the sensitivity to prompts,
The experiments only involve Llama-2 series models, whereas various locations in the paper stretches the claim to be about all LLMs and modern LLMs. I recommend rephrasing some texts into e.g., “LLMs, with Llama as examples” to make the claims better supported by the scope of the experiments. Typo and comments: - There are some minor typos. In page 6, a punctuation is needed before the footnote. In the end of the next paragraph, the period should be before the footnote. - A related work should
- examine an interesting and timely question that will be relevant to the ICLR community - well-designed experiments to rule out confounding factors - investigate several different open source models
W1. This work uses established methods for probing and there is no methodological innovation, though this is on its own not a deal breaker W2. The research question is motivated as trying to study whether LLMs build a world model, but it's not clear to me why learning important properties of famous landmarks, such as location, and of famous events, such as years, are a sign of a "world model". My guess is that these properties co-occur with the names of the landmarks, events, and people in the
1. The paper presents an exploration into the internal workings of large language models (LLMs), particularly focusing on how these models internalize and represent continuous variables including spatial and temporal dimensions. 2. The investigation is well-conducted, employing rigorous experiments and analysis methods to support that the probing results are not merely superficial statistics. The identification of specific "space neurons" and "time neurons" within these models is an innovative
1. One of the paper's limitations is its potential overlap with findings already established in the word embedding literature. Earlier works in this area, such as those by Mikolov et al. (2013), have demonstrated that simpler word embedding models can capture relational information and regularities in a linear fashion. This raises the question of whether the findings in this paper are genuinely novel or simply an extension of what is already known about linear representations in language models.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsLanguage and cultural evolution · Topic Modeling · Natural Language Processing Techniques