Bias Challenges in Counterfactual Data Augmentation

TL;DR

This paper investigates the limitations of counterfactual data augmentation in achieving out-of-distribution robustness, revealing that context-guessing mechanisms may hinder the desired invariance and robustness in NLP tasks.

Contribution

It provides a theoretical analysis of counterfactual augmentation limitations and demonstrates a specific NLP example where such methods fail to improve robustness.

Findings

Counterfactual augmentation may not ensure invariance when using context-guessing machines.

Theoretical analysis highlights conditions under which augmentation fails.

Empirical example in NLP shows limited robustness gains.

Abstract

Deep learning models tend not to be out-of-distribution robust primarily due to their reliance on spurious features to solve the task. Counterfactual data augmentations provide a general way of (approximately) achieving representations that are counterfactual-invariant to spurious features, a requirement for out-of-distribution (OOD) robustness. In this work, we show that counterfactual data augmentations may not achieve the desired counterfactual-invariance if the augmentation is performed by a context-guessing machine, an abstract machine that guesses the most-likely context of a given input. We theoretically analyze the invariance imposed by such counterfactual data augmentations and describe an exemplar NLP task where counterfactual data augmentation by a context-guessing machine does not lead to robust OOD classifiers.