Self-supervised debiasing using low rank regularization

TL;DR

This paper introduces a self-supervised debiasing method that uses low rank regularization to identify and mitigate spurious correlations in neural networks, improving generalization especially with limited annotations.

Contribution

It reveals that spurious attributes induce low rank representations and leverages this insight to develop a self-supervised framework for debiasing without extensive annotations.

Findings

Significantly improves generalization of self-supervised models.

Outperforms some state-of-the-art supervised debiasing methods.

Effectively identifies bias-conflicting samples using spectral analysis.

Abstract

Spurious correlations can cause strong biases in deep neural networks, impairing generalization ability. While most existing debiasing methods require full supervision on either spurious attributes or target labels, training a debiased model from a limited amount of both annotations is still an open question. To address this issue, we investigate an interesting phenomenon using the spectral analysis of latent representations: spuriously correlated attributes make neural networks inductively biased towards encoding lower effective rank representations. We also show that a rank regularization can amplify this bias in a way that encourages highly correlated features. Leveraging these findings, we propose a self-supervised debiasing framework potentially compatible with unlabeled samples. Specifically, we first pretrain a biased encoder in a self-supervised manner with the rank regularization, serving as a semantic bottleneck to enforce the encoder to learn the spuriously correlated attributes. This biased encoder is then used to discover and upweight bias-conflicting samples in a downstream task, serving as a boosting to effectively debias the main model. Remarkably, the proposed debiasing framework significantly improves the generalization performance of self-supervised learning baselines and, in some cases, even outperforms state-of-the-art supervised debiasing approaches.