Counterfactual Augmentation for Multimodal Learning Under Presentation Bias

TL;DR

This paper introduces counterfactual augmentation, a causal approach to mitigate presentation bias in multimodal learning systems, improving model performance by generating and utilizing counterfactual labels.

Contribution

It presents a novel causal method for correcting presentation bias through counterfactual label generation, enhancing downstream task performance.

Findings

Counterfactual augmentation outperforms existing bias correction methods.

Generated counterfactuals closely match true counterfactuals in oracle settings.

Empirical results show improved downstream performance.

Abstract

In real-world machine learning systems, labels are often derived from user behaviors that the system wishes to encourage. Over time, new models must be trained as new training examples and features become available. However, feedback loops between users and models can bias future user behavior, inducing a presentation bias in the labels that compromises the ability to train new models. In this paper, we propose counterfactual augmentation, a novel causal method for correcting presentation bias using generated counterfactual labels. Our empirical evaluations demonstrate that counterfactual augmentation yields better downstream performance compared to both uncorrected models and existing bias-correction methods. Model analyses further indicate that the generated counterfactuals align closely with true counterfactuals in an oracle setting.