Language Models Grow Less Humanlike beyond Phase Transition

TL;DR

This paper investigates why language models become less humanlike after a certain training point, identifying a phase transition involving attention heads that impacts their alignment with human reading behavior.

Contribution

It introduces the concept of a pretraining phase transition caused by specialized attention heads, explaining the PPP plateau and degradation in language models.

Findings

Phase transition correlates with the PPP tipping point.

Attention head specialization emerges rapidly during pretraining.

Further training after the phase transition worsens PPP.

Abstract

LMs' alignment with human reading behavior (i.e. psychometric predictive power; PPP) is known to improve during pretraining up to a tipping point, beyond which it either plateaus or degrades. Various factors, such as word frequency, recency bias in attention, and context size, have been theorized to affect PPP, yet there is no current account that explains why such a tipping point exists, and how it interacts with LMs' pretraining dynamics more generally. We hypothesize that the underlying factor is a pretraining phase transition, characterized by the rapid emergence of specialized attention heads. We conduct a series of correlational and causal experiments to show that such a phase transition is responsible for the tipping point in PPP. We then show that, rather than producing attention patterns that contribute to the degradation in PPP, phase transitions alter the subsequent learning dynamics of the model, such that further training keeps damaging PPP.