A Question-Answer Driven Approach to Reveal Affirmative Interpretations from Verbal Negations

TL;DR

This paper investigates how to identify and generate affirmative interpretations from verbal negations using a new corpus, question-answering, and transformer models, revealing current limitations and challenges in the task.

Contribution

It introduces a new corpus of verbal negations, formulates the problem as an NLI classification and generation task, and evaluates transformer models, highlighting the difficulty of accurately interpreting negations.

Findings

67.1% of negations convey actual event occurrence

State-of-the-art NLI models perform poorly on this task

Generation with T5 underperforms human benchmarks

Abstract

This paper explores a question-answer driven approach to reveal affirmative interpretations from verbal negations (i.e., when a negation cue grammatically modifies a verb). We create a new corpus consisting of 4,472 verbal negations and discover that 67.1% of them convey that an event actually occurred. Annotators generate and answer 7,277 questions for the 3,001 negations that convey an affirmative interpretation. We first cast the problem of revealing affirmative interpretations from negations as a natural language inference (NLI) classification task. Experimental results show that state-of-the-art transformers trained with existing NLI corpora are insufficient to reveal affirmative interpretations. We also observe, however, that fine-tuning brings small improvements. In addition to NLI classification, we also explore the more realistic task of generating affirmative interpretations directly from negations with the T5 transformer. We conclude that the generation task remains a challenge as T5 substantially underperforms humans.