From Disagreement to Understanding: The Case for Ambiguity Detection in NLI

TL;DR

This paper advocates for ambiguity detection in NLI to better reflect human interpretation, proposing a framework and taxonomy to identify and classify ambiguous cases before inference, aiming to improve model robustness and explainability.

Contribution

It introduces a new framework and taxonomy for ambiguity detection in NLI, emphasizing the importance of identifying ambiguity prior to inference to align models with human perspectives.

Findings

Content-based ambiguity signals meaningful human variation.

A unified taxonomy of ambiguity types is proposed.

Highlighting the need for annotated datasets for ambiguity detection.

Abstract

This position paper argues that annotation disagreement in Natural Language Inference (NLI) is not mere noise but often reflects meaningful variation, especially when triggered by ambiguity in the premise or hypothesis. While underspecified guidelines and annotator behavior contribute to variation, content-based ambiguity provides a process-independent signal of divergent human perspectives. We call for a shift toward ambiguity-aware NLI that first identifies ambiguous input pairs, classifies their types, and only then proceeds to inference. To support this shift, we present a framework that incorporates ambiguity detection and classification prior to inference. We also introduce a unified taxonomy that synthesizes existing taxonomies, illustrates key subtypes with examples, and motivates targeted detection methods that better align models with human interpretation. Although current resources lack datasets explicitly annotated for ambiguity and subtypes, this gap presents an opportunity: by developing new annotated resources and exploring unsupervised approaches to ambiguity detection, we enable more robust, explainable, and human-aligned NLI systems.