Analyzing and Mitigating Negation Artifacts using Data Augmentation for Improving ELECTRA-Small Model Accuracy

TL;DR

This paper investigates how ELECTRA-small models struggle with negation in NLI tasks and proposes data augmentation techniques to improve their understanding and accuracy on negation cases.

Contribution

The study introduces targeted data augmentation methods to mitigate negation artifacts, enhancing ELECTRA-small model performance on negation in NLI tasks.

Findings

Improved accuracy on negation examples

Mitigated dataset artifacts related to negation

Maintained overall model performance

Abstract

Pre-trained models for natural language inference (NLI) often achieve high performance on benchmark datasets by using spurious correlations, or dataset artifacts, rather than understanding language touches such as negation. In this project, we investigate the performance of an ELECTRA-small model fine-tuned on the Stanford Natural Language Inference (SNLI) dataset, focusing on its handling of negation. Through analysis, we identify that the model struggles with correctly classifying examples containing negation. To address this, we augment the training data with contrast sets and adversarial examples emphasizing negation. Our results demonstrate that this targeted data augmentation improves the model's accuracy on negation-containing examples without adversely affecting overall performance, therefore mitigating the identified dataset artifact.