Improving the Natural Language Inference robustness to hard dataset by data augmentation and preprocessing

TL;DR

This paper proposes data augmentation and preprocessing techniques to enhance the robustness of Natural Language Inference models against hard, out-of-distribution datasets, addressing issues like word overlap and numerical reasoning.

Contribution

It introduces general data augmentation and preprocessing methods that improve NLI model robustness without relying on test data distribution.

Findings

Enhanced model performance on hard datasets

Reduced reliance on spurious correlations

Improved handling of numerical reasoning and length mismatch

Abstract

Natural Language Inference (NLI) is the task of inferring whether the hypothesis can be justified by the given premise. Basically, we classify the hypothesis into three labels(entailment, neutrality and contradiction) given the premise. NLI was well studied by the previous researchers. A number of models, especially the transformer based ones, have achieved significant improvement on these tasks. However, it is reported that these models are suffering when they are dealing with hard datasets. Particularly, they perform much worse when dealing with unseen out-of-distribution premise and hypothesis. They may not understand the semantic content but learn the spurious correlations. In this work, we propose the data augmentation and preprocessing methods to solve the word overlap, numerical reasoning and length mismatch problems. These methods are general methods that do not rely on the distribution of the testing data and they help improve the robustness of the models.