Mitigating Spurious Correlations with Causal Logit Perturbation

TL;DR

This paper introduces Causal Logit Perturbation (CLP), a novel framework that mitigates spurious correlations in deep learning models by generating sample-specific causal perturbations, improving robustness across various biased learning scenarios.

Contribution

The paper proposes a new CLP framework that uses a perturbation network and meta-learning to disentangle causal from spurious features, incorporating human causal knowledge for better robustness.

Findings

CLP achieves state-of-the-art results on biased learning benchmarks.

Visualization shows CLP redirects model focus to causal features.

Effective across multiple biased learning scenarios.

Abstract

Deep learning has seen widespread success in various domains such as science, industry, and society. However, it is acknowledged that certain approaches suffer from non-robustness, relying on spurious correlations for predictions. Addressing these limitations is of paramount importance, necessitating the development of methods that can disentangle spurious correlations. {This study attempts to implement causal models via logit perturbations and introduces a novel Causal Logit Perturbation (CLP) framework to train classifiers with generated causal logit perturbations for individual samples, thereby mitigating the spurious associations between non-causal attributes (i.e., image backgrounds) and classes.} {Our framework employs a} perturbation network to generate sample-wise logit perturbations using a series of training characteristics of samples as inputs. The whole framework is optimized by an online meta-learning-based learning algorithm and leverages human causal knowledge by augmenting metadata in both counterfactual and factual manners. Empirical evaluations on four typical biased learning scenarios, including long-tail learning, noisy label learning, generalized long-tail learning, and subpopulation shift learning, demonstrate that CLP consistently achieves state-of-the-art performance. Moreover, visualization results support the effectiveness of the generated causal perturbations in redirecting model attention towards causal image attributes and dismantling spurious associations.