What We Talk About When We Talk About LMs: Implicit Paradigm Shifts and the Ship of Language Models

TL;DR

This paper examines the evolving use and conceptual shifts of the term 'Language Models' in NLP literature, highlighting implicit paradigm shifts and the ongoing transformation of scientific discourse.

Contribution

It introduces a novel perspective on scientific progress by analyzing the implicit evolution of terminology in NLP through data-driven text analysis.

Findings

Language Models term undergoes continuous reinterpretation.

Scientific discourse shows interconnected influence between systems and theories.

The study provides a quantitative understanding of terminology evolution in NLP.

Abstract

The term Language Models (LMs) as a time-specific collection of models of interest is constantly reinvented, with its referents updated much like the $\textit{Ship of Theseus}$ replaces its parts but remains the same ship in essence. In this paper, we investigate this $\textit{Ship of Language Models}$ problem, wherein scientific evolution takes the form of continuous, implicit retrofits of key existing terms. We seek to initiate a novel perspective of scientific progress, in addition to the more well-studied emergence of new terms. To this end, we construct the data infrastructure based on recent NLP publications. Then, we perform a series of text-based analyses toward a detailed, quantitative understanding of the use of Language Models as a term of art. Our work highlights how systems and theories influence each other in scientific discourse, and we call for attention to the transformation of this Ship that we all are contributing to.