An Asymmetric Contrastive Loss for Handling Imbalanced Datasets

TL;DR

This paper introduces an asymmetric contrastive loss (ACL) and a generalized asymmetric focal contrastive loss (AFCL) to improve representation learning on imbalanced datasets, demonstrating superior performance over existing contrastive methods.

Contribution

The paper proposes novel asymmetric contrastive loss functions specifically designed for imbalanced datasets, enhancing contrastive learning effectiveness.

Findings

AFCL outperforms CL and FCL on imbalanced datasets

AFCL improves both weighted and unweighted classification accuracy

Theoretical analysis of entropy and proofs provided in appendix

Abstract

Contrastive learning is a representation learning method performed by contrasting a sample to other similar samples so that they are brought closely together, forming clusters in the feature space. The learning process is typically conducted using a two-stage training architecture, and it utilizes the contrastive loss (CL) for its feature learning. Contrastive learning has been shown to be quite successful in handling imbalanced datasets, in which some classes are overrepresented while some others are underrepresented. However, previous studies have not specifically modified CL for imbalanced datasets. In this work, we introduce an asymmetric version of CL, referred to as ACL, in order to directly address the problem of class imbalance. In addition, we propose the asymmetric focal contrastive loss (AFCL) as a further generalization of both ACL and focal contrastive loss (FCL). Results on the FMNIST and ISIC 2018 imbalanced datasets show that AFCL is capable of outperforming CL and FCL in terms of both weighted and unweighted classification accuracies. In the appendix, we provide a full axiomatic treatment on entropy, along with complete proofs.