Automatic Counterfactual Augmentation for Robust Text Classification Based on Word-Group Search

TL;DR

This paper introduces a novel word-group based counterfactual augmentation method to improve the robustness of text classification models by mitigating shortcut learning and emphasizing causal features.

Contribution

It proposes a new word-group mining approach using beam search and a counterfactual augmentation technique with adaptive voting, addressing limitations of previous single-word focus methods.

Findings

Enhanced robustness against shortcut learning in text classification

Improved performance on cross-domain and fairness tasks

Effective identification of causal word-groups

Abstract

Despite large-scale pre-trained language models have achieved striking results for text classificaion, recent work has raised concerns about the challenge of shortcut learning. In general, a keyword is regarded as a shortcut if it creates a superficial association with the label, resulting in a false prediction. Conversely, shortcut learning can be mitigated if the model relies on robust causal features that help produce sound predictions. To this end, many studies have explored post-hoc interpretable methods to mine shortcuts and causal features for robustness and generalization. However, most existing methods focus only on single word in a sentence and lack consideration of word-group, leading to wrong causal features. To solve this problem, we propose a new Word-Group mining approach, which captures the causal effect of any keyword combination and orders the combinations that most affect the prediction. Our approach bases on effective post-hoc analysis and beam search, which ensures the mining effect and reduces the complexity. Then, we build a counterfactual augmentation method based on the multiple word-groups, and use an adaptive voting mechanism to learn the influence of different augmentated samples on the prediction results, so as to force the model to pay attention to effective causal features. We demonstrate the effectiveness of the proposed method by several tasks on 8 affective review datasets and 4 toxic language datasets, including cross-domain text classificaion, text attack and gender fairness test.