Domain-Aware Contrastive Knowledge Transfer for Multi-domain Imbalanced Data

TL;DR

This paper introduces DCMI, a domain-aware contrastive transfer method that enhances multi-domain imbalanced learning by leveraging shared knowledge and isolating domain-specific information, leading to improved performance.

Contribution

The paper proposes a novel contrastive transfer approach that effectively handles domain imbalance by distinguishing shared and domain-specific knowledge.

Findings

Significant performance improvements on three datasets.

Effective transfer among similar domains reduces negative transfer.

Enhanced learning in scenarios with limited data.

Abstract

In many real-world machine learning applications, samples belong to a set of domains e.g., for product reviews each review belongs to a product category. In this paper, we study multi-domain imbalanced learning (MIL), the scenario that there is imbalance not only in classes but also in domains. In the MIL setting, different domains exhibit different patterns and there is a varying degree of similarity and divergence among domains posing opportunities and challenges for transfer learning especially when faced with limited or insufficient training data. We propose a novel domain-aware contrastive knowledge transfer method called DCMI to (1) identify the shared domain knowledge to encourage positive transfer among similar domains (in particular from head domains to tail domains); (2) isolate the domain-specific knowledge to minimize the negative transfer from dissimilar domains. We evaluated the performance of DCMI on three different datasets showing significant improvements in different MIL scenarios.