How Linguistics Learned to Stop Worrying and Love the Language Models

TL;DR

Language models, while not perfect replicas of human language learning, offer valuable insights and serve as useful tools for exploring linguistic structure, processing, and learning in a way that complements traditional linguistic theories.

Contribution

The paper argues that language models contribute meaningfully to linguistic research, challenging both dismissive and overly optimistic views, and highlight their role as models for usage-based language theories.

Findings

Language models can inform linguistic structure and processing.

They serve as models for gradient, usage-based language theories.

LMs do not replace but complement traditional linguistics.

Abstract

Language models can produce fluent, grammatical text. Nonetheless, some maintain that language models don't really learn language and also that, even if they did, that would not be informative for the study of human learning and processing. On the other side, there have been claims that the success of LMs obviates the need for studying linguistic theory and structure. We argue that both extremes are wrong. LMs can contribute to fundamental questions about linguistic structure, language processing, and learning. They force us to rethink arguments and ways of thinking that have been foundational in linguistics. While they do not replace linguistic structure and theory, they serve as model systems and working proofs of concept for gradient, usage-based approaches to language. We offer an optimistic take on the relationship between language models and linguistics.