Can NLP Models Correctly Reason Over Contexts that Break the Common Assumptions?

TL;DR

This paper evaluates whether state-of-the-art NLP models can accurately reason over scenarios that violate common assumptions, revealing significant performance gaps and highlighting the need for more robust reasoning capabilities.

Contribution

The study systematically creates evaluation data for reasoning over assumption-breaking contexts and analyzes model performance, exposing current limitations and guiding future improvements.

Findings

Models perform well on assumption-following contexts.

Models struggle with assumption-breaking contexts, with up to 20% performance gap.

Analysis reveals key challenges in reasoning over atypical scenarios.

Abstract

Pre-training on large corpora of text enables the language models to acquire a vast amount of factual and commonsense knowledge which allows them to achieve remarkable performance on a variety of language understanding tasks. They typically acquire this knowledge by learning from the pre-training text and capturing certain patterns from it. However, real-world settings often present scenarios that do not abide by these patterns i.e. scenarios that break the common assumptions. Can state-of-the-art NLP models correctly reason over the contexts of such scenarios? Addressing the above question, in this paper, we investigate the ability of models to correctly reason over contexts that break the common assumptions. To this end, we first systematically create evaluation data in which each data instance consists of (a) a common assumption, (b) a context that follows the assumption, (c) a context that breaks the assumption, and (d) questions based on the contexts. Then, through evaluations on multiple models including GPT-3 and Flan T5, we show that while doing fairly well on contexts that follow the common assumptions, the models struggle to correctly reason over contexts that break those assumptions. Specifically, the performance gap is as high as 20% absolute points. Furthermore, we thoroughly analyze these results revealing several interesting findings. We believe our work and findings will encourage and facilitate further research in developing more robust models that can also reliably reason over contexts that break the common assumptions. Data is available at \url{https://github.com/nrjvarshney/break_the_common_assumptions}.