Balanced Distribution Adaptation for Transfer Learning

TL;DR

This paper introduces Balanced Distribution Adaptation (BDA) and Weighted BDA (W-BDA), innovative transfer learning methods that adaptively balance distribution discrepancies and handle class imbalance, outperforming existing approaches.

Contribution

The paper proposes BDA and W-BDA, novel transfer learning algorithms that adaptively balance marginal and conditional distribution differences and address class imbalance issues.

Findings

BDA outperforms several state-of-the-art methods.

W-BDA effectively handles class imbalance in transfer learning.

Extensive experiments validate the effectiveness of the proposed methods.

Abstract

Transfer learning has achieved promising results by leveraging knowledge from the source domain to annotate the target domain which has few or none labels. Existing methods often seek to minimize the distribution divergence between domains, such as the marginal distribution, the conditional distribution or both. However, these two distances are often treated equally in existing algorithms, which will result in poor performance in real applications. Moreover, existing methods usually assume that the dataset is balanced, which also limits their performances on imbalanced tasks that are quite common in real problems. To tackle the distribution adaptation problem, in this paper, we propose a novel transfer learning approach, named as Balanced Distribution \underline{A}daptation~(BDA), which can adaptively leverage the importance of the marginal and conditional distribution discrepancies, and several existing methods can be treated as special cases of BDA. Based on BDA, we also propose a novel Weighted Balanced Distribution Adaptation~(W-BDA) algorithm to tackle the class imbalance issue in transfer learning. W-BDA not only considers the distribution adaptation between domains but also adaptively changes the weight of each class. To evaluate the proposed methods, we conduct extensive experiments on several transfer learning tasks, which demonstrate the effectiveness of our proposed algorithms over several state-of-the-art methods.