SCoRe: Submodular Combinatorial Representation Learning

TL;DR

SCoRe introduces a submodular combinatorial framework for representation learning that effectively reduces intra-class variance and inter-class bias, improving performance across various tasks and datasets.

Contribution

It presents a novel set-based submodular loss formulation that unifies and extends existing contrastive and metric learning methods, enhancing class-imbalance handling.

Findings

Up to 7.6% improvement on CIFAR-10-LT and CIFAR-100-LT

2.1% improvement on ImageNet-LT

19.4% improvement in object detection on IDD and LVIS

Abstract

In this paper we introduce the SCoRe (Submodular Combinatorial Representation Learning) framework, a novel approach in representation learning that addresses inter-class bias and intra-class variance. SCoRe provides a new combinatorial viewpoint to representation learning, by introducing a family of loss functions based on set-based submodular information measures. We develop two novel combinatorial formulations for loss functions, using the Total Information and Total Correlation, that naturally minimize intra-class variance and inter-class bias. Several commonly used metric/contrastive learning loss functions like supervised contrastive loss, orthogonal projection loss, and N-pairs loss, are all instances of SCoRe, thereby underlining the versatility and applicability of SCoRe in a broad spectrum of learning scenarios. Novel objectives in SCoRe naturally model class-imbalance with up to 7.6\% improvement in classification on CIFAR-10-LT, CIFAR-100-LT, MedMNIST, 2.1% on ImageNet-LT, and 19.4% in object detection on IDD and LVIS (v1.0), demonstrating its effectiveness over existing approaches.