A Comprehensive Taxonomy of Negation for NLP and Neural Retrievers

TL;DR

This paper develops a detailed taxonomy of negation, creates benchmark datasets for evaluation, and introduces a logic-based classification method to improve neural retrieval models' handling of negation in NLP tasks.

Contribution

It introduces a comprehensive negation taxonomy, generates new datasets for evaluation and training, and proposes a classification mechanism to analyze and enhance model performance on negation.

Findings

Balanced negation data improves model convergence.

Taxonomy reveals coverage gaps in existing datasets.

Classification schema provides insights into negation handling.

Abstract

Understanding and solving complex reasoning tasks is vital for addressing the information needs of a user. Although dense neural models learn contextualised embeddings, they still underperform on queries containing negation. To understand this phenomenon, we study negation in both traditional neural information retrieval and LLM-based models. We (1) introduce a taxonomy of negation that derives from philosophical, linguistic, and logical definitions; (2) generate two benchmark datasets that can be used to evaluate the performance of neural information retrieval models and to fine-tune models for a more robust performance on negation; and (3) propose a logic-based classification mechanism that can be used to analyze the performance of retrieval models on existing datasets. Our taxonomy produces a balanced data distribution over negation types, providing a better training setup that leads to faster convergence on the NevIR dataset. Moreover, we propose a classification schema that reveals the coverage of negation types in existing datasets, offering insights into the factors that might affect the generalization of fine-tuned models on negation.