Some of Them Can be Guessed! Exploring the Effect of Linguistic Context in Predicting Quantifiers

TL;DR

This paper investigates how linguistic context influences quantifier prediction, revealing that models excel in local contexts but struggle with global understanding, unlike humans who benefit from broader context.

Contribution

It provides a comprehensive comparison of human and model performance in quantifier prediction across different contextual scopes, highlighting strengths and limitations.

Findings

Models outperform humans in local context prediction.

Humans improve with global context, especially for proportional quantifiers.

Model performance declines with increased global context complexity.

Abstract

We study the role of linguistic context in predicting quantifiers (`few', `all'). We collect crowdsourced data from human participants and test various models in a local (single-sentence) and a global context (multi-sentence) condition. Models significantly out-perform humans in the former setting and are only slightly better in the latter. While human performance improves with more linguistic context (especially on proportional quantifiers), model performance suffers. Models are very effective in exploiting lexical and morpho-syntactic patterns; humans are better at genuinely understanding the meaning of the (global) context.