Language model acceptability judgements are not always robust to context

TL;DR

This paper investigates how the stability of language models' syntactic acceptability judgments varies with different contextual properties, revealing that models are sensitive to specific syntactic features in context, which impacts their judgment robustness.

Contribution

The study demonstrates that language models' syntactic judgments are highly sensitive to contextual syntactic structures, highlighting the role of implicit in-context learning in these judgments.

Findings

Models are robust in random contexts but unstable with matching syntactic structures.

Providing matching syntactic contexts improves judgments; unacceptable contexts worsen them.

Sensitivity to syntactic features is linked to implicit in-context learning abilities.

Abstract

Targeted syntactic evaluations of language models ask whether models show stable preferences for syntactically acceptable content over minimal-pair unacceptable inputs. Most targeted syntactic evaluation datasets ask models to make these judgements with just a single context-free sentence as input. This does not match language models' training regime, in which input sentences are always highly contextualized by the surrounding corpus. This mismatch raises an important question: how robust are models' syntactic judgements in different contexts? In this paper, we investigate the stability of language models' performance on targeted syntactic evaluations as we vary properties of the input context: the length of the context, the types of syntactic phenomena it contains, and whether or not there are violations of grammaticality. We find that model judgements are generally robust when placed in randomly sampled linguistic contexts. However, they are substantially unstable for contexts containing syntactic structures matching those in the critical test content. Among all tested models (GPT-2 and five variants of OPT), we significantly improve models' judgements by providing contexts with matching syntactic structures, and conversely significantly worsen them using unacceptable contexts with matching but violated syntactic structures. This effect is amplified by the length of the context, except for unrelated inputs. We show that these changes in model performance are not explainable by simple features matching the context and the test inputs, such as lexical overlap and dependency overlap. This sensitivity to highly specific syntactic features of the context can only be explained by the models' implicit in-context learning abilities.