Neural reality of argument structure constructions

TL;DR

This paper investigates whether Transformer-based language models encode argument structure constructions (ASCs) similar to human psycholinguistic findings, providing evidence that LMs associate constructions with meaning beyond individual verbs.

Contribution

It introduces novel psycholinguistic-inspired probing methods to demonstrate that LMs encode argument structure constructions, aligning with empirical evidence from human language processing.

Findings

Sentences sharing the same construction are closer in embedding space than those sharing the same verb.

Language models increasingly group sentences by construction with more data, similar to non-native language learners.

LMs associate argument structure constructions with meaning even in nonsensical sentences.

Abstract

In lexicalist linguistic theories, argument structure is assumed to be predictable from the meaning of verbs. As a result, the verb is the primary determinant of the meaning of a clause. In contrast, construction grammarians propose that argument structure is encoded in constructions (or form-meaning pairs) that are distinct from verbs. Decades of psycholinguistic research have produced substantial empirical evidence in favor of the construction view. Here we adapt several psycholinguistic studies to probe for the existence of argument structure constructions (ASCs) in Transformer-based language models (LMs). First, using a sentence sorting experiment, we find that sentences sharing the same construction are closer in embedding space than sentences sharing the same verb. Furthermore, LMs increasingly prefer grouping by construction with more input data, mirroring the behaviour of non-native language learners. Second, in a "Jabberwocky" priming-based experiment, we find that LMs associate ASCs with meaning, even in semantically nonsensical sentences. Our work offers the first evidence for ASCs in LMs and highlights the potential to devise novel probing methods grounded in psycholinguistic research.