Transparency Helps Reveal When Language Models Learn Meaning

TL;DR

This paper investigates how language models learn meaning, showing they succeed in transparent languages but struggle with natural language's context-dependent semantics, especially with phenomena like referential opacity.

Contribution

It demonstrates that language models can learn semantics in transparent languages but fail in natural language due to context-dependent meanings, highlighting limitations in current models.

Findings

Models learn semantics in transparent languages

Models struggle with context-dependent meanings

Natural language semantics are challenging for current models

Abstract

Many current NLP systems are built from language models trained to optimize unsupervised objectives on large amounts of raw text. Under what conditions might such a procedure acquire meaning? Our systematic experiments with synthetic data reveal that, with languages where all expressions have context-independent denotations (i.e., languages with strong transparency), both autoregressive and masked language models successfully learn to emulate semantic relations between expressions. However, when denotations are changed to be context-dependent with the language otherwise unmodified, this ability degrades. Turning to natural language, our experiments with a specific phenomenon -- referential opacity -- add to the growing body of evidence that current language models do not represent natural language semantics well. We show this failure relates to the context-dependent nature of natural language form-meaning mappings.