Language Models Understand Us, Poorly

TL;DR

This paper critically examines what it means for language models to understand us, arguing that internal representations are key, and highlights the limitations of current models and the need for deeper internal probing and understanding.

Contribution

It clarifies different views of language understanding, advocates for internal representations as sufficient, and discusses limitations and future directions for language models.

Findings

Behavioral reliability is necessary for understanding.

Internal representations are sufficient for understanding.

Scaling limits may hinder models from truly understanding.

Abstract

Some claim language models understand us. Others won't hear it. To clarify, I investigate three views of human language understanding: as-mapping, as-reliability and as-representation. I argue that while behavioral reliability is necessary for understanding, internal representations are sufficient; they climb the right hill. I review state-of-the-art language and multi-modal models: they are pragmatically challenged by under-specification of form. I question the Scaling Paradigm: limits on resources may prohibit scaled-up models from approaching understanding. Last, I describe how as-representation advances a science of understanding. We need work which probes model internals, adds more of human language, and measures what models can learn.