The unreasonable effectiveness of pattern matching

TL;DR

Large language models demonstrate an astonishing ability to interpret nonsensical language by leveraging structural pattern matching, highlighting its crucial role in their apparent understanding.

Contribution

The paper reveals the significant role of pattern matching in LLMs' ability to interpret distorted language, challenging traditional views of their functioning.

Findings

LLMs can interpret nonsense language using pattern recognition.

Pattern matching is a key component of LLMs' capabilities.

This ability questions the nature of language understanding in AI.

Abstract

We report on an astonishing ability of large language models (LLMs) to make sense of "Jabberwocky" language in which most or all content words have been randomly replaced by nonsense strings, e.g., translating "He dwushed a ghanc zawk" to "He dragged a spare chair". This result addresses ongoing controversies regarding how to best think of what LLMs are doing: are they a language mimic, a database, a blurry version of the Web? The ability of LLMs to recover meaning from structural patterns speaks to the unreasonable effectiveness of pattern-matching. Pattern-matching is not an alternative to "real" intelligence, but rather a key ingredient.