Looking at the Overlooked: An Analysis on the Word-Overlap Bias in Natural Language Inference

TL;DR

This paper investigates the reverse word-overlap bias in NLI models, revealing that models are biased towards non-entailment with low overlap, and current debiasing methods are ineffective against this overlooked bias.

Contribution

It uncovers the overlooked reverse overlap bias in NLI models and analyzes why existing debiasing methods fail to address it.

Findings

Models are highly biased towards non-entailment on low-overlap instances.

Existing debiasing methods are ineffective against reverse overlap bias.

Eliminating minority examples does not improve debiasing effectiveness.

Abstract

It has been shown that NLI models are usually biased with respect to the word-overlap between premise and hypothesis; they take this feature as a primary cue for predicting the entailment label. In this paper, we focus on an overlooked aspect of the overlap bias in NLI models: the reverse word-overlap bias. Our experimental results demonstrate that current NLI models are highly biased towards the non-entailment label on instances with low overlap, and the existing debiasing methods, which are reportedly successful on existing challenge datasets, are generally ineffective in addressing this category of bias. We investigate the reasons for the emergence of the overlap bias and the role of minority examples in its mitigation. For the former, we find that the word-overlap bias does not stem from pre-training, and for the latter, we observe that in contrast to the accepted assumption, eliminating minority examples does not affect the generalizability of debiasing methods with respect to the overlap bias.