Relation-based Counterfactual Data Augmentation and Contrastive Learning for Robustifying Natural Language Inference Models

TL;DR

This paper introduces a novel approach combining relation-based counterfactual data augmentation with contrastive learning to enhance the robustness and performance of natural language inference models against non-causal biases.

Contribution

It proposes a new method that generates counterfactual sentence pairs and applies contrastive learning to improve NLI model robustness and understanding of semantic classes.

Findings

Improved accuracy on counterfactually-revised datasets

Enhanced model robustness to non-causal features

Better generalization on standard NLI datasets

Abstract

Although pre-trained language models show good performance on various natural language processing tasks, they often rely on non-causal features and patterns to determine the outcome. For natural language inference tasks, previous results have shown that even a model trained on a large number of data fails to perform well on counterfactually revised data, indicating that the model is not robustly learning the semantics of the classes. In this paper, we propose a method in which we use token-based and sentence-based augmentation methods to generate counterfactual sentence pairs that belong to each class, and apply contrastive learning to help the model learn the difference between sentence pairs of different classes with similar contexts. Evaluation results with counterfactually-revised dataset and general NLI datasets show that the proposed method can improve the performance and robustness of the NLI model.