UnNatural Language Inference

TL;DR

This paper reveals that state-of-the-art NLI models are largely insensitive to word order, assigning the same labels to permuted sentences, which challenges assumptions about their understanding of syntax.

Contribution

The study provides empirical evidence that NLI models are invariant to word permutations, highlighting a significant limitation in their language understanding capabilities.

Findings

Models often assign the same label to permuted examples as to original sentences.

Almost 99% of MNLI examples have permutations that elicit the correct label.

The invariance issue is consistent across different model architectures and languages.

Abstract

Recent investigations into the inner-workings of state-of-the-art large-scale pre-trained Transformer-based Natural Language Understanding (NLU) models indicate that they appear to know humanlike syntax, at least to some extent. We provide novel evidence that complicates this claim: we find that state-of-the-art Natural Language Inference (NLI) models assign the same labels to permuted examples as they do to the original, i.e. they are largely invariant to random word-order permutations. This behavior notably differs from that of humans; we struggle with ungrammatical sentences. To measure the severity of this issue, we propose a suite of metrics and investigate which properties of particular permutations lead models to be word-order invariant. In the MNLI dataset, for example, we find almost all (98.7%) examples contain at least one permutation which elicits the gold label. Models are sometimes even able to assign gold labels to permutations that they originally failed to predict correctly. We provide a comprehensive empirical evaluation of this phenomenon, and further show that this issue exists for both Transformers and pre-Transformer RNN / ConvNet based encoders, as well as across multiple languages (English and Mandarin Chinese). Our code and data are available at https://github.com/facebookresearch/unlu.