The Singleton Fallacy: Why Current Critiques of Language Models Miss the Point

TL;DR

This paper challenges the critique of language models by highlighting the fallacy of assuming a single, uniform notion of understanding, and argues that current models can acquire meaningful structural understanding across multiple modalities.

Contribution

It introduces the concept of the singleton fallacy and defends the idea that structural understanding in language models is a valid form of real understanding.

Findings

Current language models can acquire multiple types of structural understanding.

The singleton fallacy underpins many critiques of language models.

Structural understanding may encompass various modalities and meanings.

Abstract

This paper discusses the current critique against neural network-based Natural Language Understanding (NLU) solutions known as language models. We argue that much of the current debate rests on an argumentation error that we will refer to as the singleton fallacy: the assumption that language, meaning, and understanding are single and uniform phenomena that are unobtainable by (current) language models. By contrast, we will argue that there are many different types of language use, meaning and understanding, and that (current) language models are build with the explicit purpose of acquiring and representing one type of structural understanding of language. We will argue that such structural understanding may cover several different modalities, and as such can handle several different types of meaning. Our position is that we currently see no theoretical reason why such structural knowledge would be insufficient to count as "real" understanding.