Revisiting subword tokenization: A case study on affixal negation in large language models

TL;DR

This paper investigates how different subword tokenization methods affect large language models' ability to understand affixal negation, revealing that models generally recognize negation despite tokenization challenges.

Contribution

It provides a comprehensive analysis of the impact of tokenization on negation understanding in LLMs, highlighting the interaction between tokenization accuracy and negation detection.

Findings

Models can reliably recognize affixal negation despite tokenization mismatches.

Tokenization performance does not always correlate with negation detection accuracy.

Different subword tokenization methods have varying effects on negation sensitivity.

Abstract

In this work, we measure the impact of affixal negation on modern English large language models (LLMs). In affixal negation, the negated meaning is expressed through a negative morpheme, which is potentially challenging for LLMs as their tokenizers are often not morphologically plausible. We conduct extensive experiments using LLMs with different subword tokenization methods, which lead to several insights on the interaction between tokenization performance and negation sensitivity. Despite some interesting mismatches between tokenization accuracy and negation detection performance, we show that models can, on the whole, reliably recognize the meaning of affixal negation.