Are Language Models Models?

TL;DR

This paper critically evaluates the claim that language models serve as cognitive models, arguing that they fall short at multiple theoretical levels and are better viewed as tools rather than true models of cognition.

Contribution

The paper provides a multi-level assessment of language models, challenging their designation as cognitive models and clarifying their role as tools instead.

Findings

Language models do not accurately implement cognitive processes.

They are poorly motivated at the algorithmic-representational level.

Calling LMs cognitive models overstates their capabilities.

Abstract

Futrell and Mahowald claim LMs "serve as model systems", but an assessment at each of Marr's three levels suggests the claim is clearly not true at the implementation level, poorly motivated at the algorithmic-representational level, and problematic at the computational theory level. LMs are good candidates as tools; calling them cognitive models overstates the case and unnecessarily feeds LLM hype.