Delving Deep into Simplicity Bias for Long-Tailed Image Recognition

TL;DR

This paper investigates how simplicity bias affects long-tailed image recognition, revealing that tail classes suffer more, and proposes a novel self-supervised learning method to mitigate this bias and improve performance.

Contribution

The paper introduces a new SSL approach tailored for imbalanced data that leverages triple diverse levels to better learn complex patterns in tail classes.

Findings

SSL can mitigate simplicity bias in long-tailed recognition

Proposed method outperforms state-of-the-art on five benchmarks

Enhances learning of complex features in tail classes

Abstract

Simplicity Bias (SB) is a phenomenon that deep neural networks tend to rely favorably on simpler predictive patterns but ignore some complex features when applied to supervised discriminative tasks. In this work, we investigate SB in long-tailed image recognition and find the tail classes suffer more severely from SB, which harms the generalization performance of such underrepresented classes. We empirically report that self-supervised learning (SSL) can mitigate SB and perform in complementary to the supervised counterpart by enriching the features extracted from tail samples and consequently taking better advantage of such rare samples. However, standard SSL methods are designed without explicitly considering the inherent data distribution in terms of classes and may not be optimal for long-tailed distributed data. To address this limitation, we propose a novel SSL method tailored to imbalanced data. It leverages SSL by triple diverse levels, i.e., holistic-, partial-, and augmented-level, to enhance the learning of predictive complex patterns, which provides the potential to overcome the severe SB on tail data. Both quantitative and qualitative experimental results on five long-tailed benchmark datasets show our method can effectively mitigate SB and significantly outperform the competing state-of-the-arts.