Combating Representation Learning Disparity with Geometric Harmonization

TL;DR

This paper introduces Geometric Harmonization, a novel method to improve representation learning in self-supervised learning by promoting category-level uniformity, especially under long-tailed distributions, without altering existing SSL frameworks.

Contribution

The paper proposes a new geometric calibration technique that enhances category-level uniformity in SSL, effectively addressing representation disparity in long-tailed data.

Findings

GH improves robustness on benchmark datasets

It maintains performance across skewed distributions

Easy to integrate into existing SSL methods

Abstract

Self-supervised learning (SSL) as an effective paradigm of representation learning has achieved tremendous success on various curated datasets in diverse scenarios. Nevertheless, when facing the long-tailed distribution in real-world applications, it is still hard for existing methods to capture transferable and robust representation. Conventional SSL methods, pursuing sample-level uniformity, easily leads to representation learning disparity where head classes dominate the feature regime but tail classes passively collapse. To address this problem, we propose a novel Geometric Harmonization (GH) method to encourage category-level uniformity in representation learning, which is more benign to the minority and almost does not hurt the majority under long-tailed distribution. Specially, GH measures the population statistics of the embedding space on top of self-supervised learning, and then infer an fine-grained instance-wise calibration to constrain the space expansion of head classes and avoid the passive collapse of tail classes. Our proposal does not alter the setting of SSL and can be easily integrated into existing methods in a low-cost manner. Extensive results on a range of benchmark datasets show the effectiveness of GH with high tolerance to the distribution skewness. Our code is available at https://github.com/MediaBrain-SJTU/Geometric-Harmonization.