Superlatives in Context: Modeling the Implicit Semantics of Superlatives

TL;DR

This paper investigates the semantics of superlatives in context, proposing a unified semantic model, annotating a dataset, and evaluating model performance on interpreting implicit and ambiguous superlative expressions.

Contribution

It introduces a unified semantic schema for superlatives, creates a multi-domain annotated dataset, and analyzes model performance on contextual superlative interpretation.

Findings

Models struggle with fine-grained superlative semantics in context

Contextual information significantly affects superlative interpretation

Contemporary models, including GPT-4, face challenges in this task

Abstract

Superlatives are used to single out elements with a maximal/minimal property. Semantically, superlatives perform a set comparison: something (or some things) has the min/max property out of a set. As such, superlatives provide an ideal phenomenon for studying implicit phenomena and discourse restrictions. While this comparison set is often not explicitly defined, its (implicit) restrictions can be inferred from the discourse context the expression appears in. In this work we provide an extensive computational study on the semantics of superlatives. We propose a unified account of superlative semantics which allows us to derive a broad-coverage annotation schema. Using this unified schema we annotated a multi-domain dataset of superlatives and their semantic interpretations. We specifically focus on interpreting implicit or ambiguous superlative expressions, by analyzing how the discourse context restricts the set of interpretations. In a set of experiments we then analyze how well models perform at variations of predicting superlative semantics, with and without context. We show that the fine-grained semantics of superlatives in context can be challenging for contemporary models, including GPT-4.