Tokens, the oft-overlooked appetizer: Large language models, the distributional hypothesis, and meaning

TL;DR

This paper examines how tokenization influences large language models' understanding and cognition, highlighting the importance of linguistically meaningful units and the impact of tokenization algorithms on model bias and semantic access.

Contribution

It argues for revising tokenization techniques to better reflect linguistic units and explores how tokenization affects LLM cognition, bias, and meaning construction.

Findings

Tokenization impacts LLM's access to distributional patterns.

Current tokenization methods create suboptimal semantic units.

Tokenization algorithms influence model bias and cognition.

Abstract

Tokenization is a necessary component within the current architecture of many language mod-els, including the transformer-based large language models (LLMs) of Generative AI, yet its impact on the model's cognition is often overlooked. We argue that LLMs demonstrate that the Distributional Hypothesis (DH) is sufficient for reasonably human-like language performance (particularly with respect to inferential lexical competence), and that the emergence of human-meaningful linguistic units among tokens and current structural constraints motivate changes to existing, linguistically-agnostic tokenization techniques, particularly with respect to their roles as (1) vehicles for conveying salient distributional patterns from human language to the model and as (2) semantic primitives. We explore tokenizations from a BPE tokenizer; extant model vocabularies obtained from Hugging Face and tiktoken; and the information in exemplar token vectors as they move through the layers of a RoBERTa (large) model. Besides creating suboptimal semantic building blocks and obscuring the model's access to the necessary distributional patterns, we describe how tokens and pretraining can act as a backdoor for bias and other unwanted content, which current alignment practices may not remediate. Additionally, we relay evidence that the tokenization algorithm's objective function impacts the LLM's cognition, despite being arguably meaningfully insulated from the main system intelligence. Finally, we discuss implications for architectural choices, meaning construction, the primacy of language for thought, and LLM cognition. [First uploaded to arXiv in December, 2024.]