IT-RUDA: Information Theory Assisted Robust Unsupervised Domain Adaptation

TL;DR

This paper introduces IT-RUDA, a robust unsupervised domain adaptation method using $oldsymbol{ extalpha}$-divergence, which effectively handles outliers and distribution shifts by providing a flexible, theoretically grounded approach that generalizes existing divergence-based UDA techniques.

Contribution

The paper proposes a novel $oldsymbol{ extalpha}$-divergence based UDA method that is robust to outliers and unifies several existing divergence-based techniques as special cases.

Findings

The method effectively handles outliers in open-set and partial UDA scenarios.

Theoretical upper bound relates target loss to source loss and initial divergence.

Validated on multiple benchmark datasets showing robustness and effectiveness.

Abstract

Distribution shift between train (source) and test (target) datasets is a common problem encountered in machine learning applications. One approach to resolve this issue is to use the Unsupervised Domain Adaptation (UDA) technique that carries out knowledge transfer from a label-rich source domain to an unlabeled target domain. Outliers that exist in either source or target datasets can introduce additional challenges when using UDA in practice. In this paper, $\alpha$-divergence is used as a measure to minimize the discrepancy between the source and target distributions while inheriting robustness, adjustable with a single parameter $\alpha$, as the prominent feature of this measure. Here, it is shown that the other well-known divergence-based UDA techniques can be derived as special cases of the proposed method. Furthermore, a theoretical upper bound is derived for the loss in the target domain in terms of the source loss and the initial $\alpha$-divergence between the two domains. The robustness of the proposed method is validated through testing on several benchmarked datasets in open-set and partial UDA setups where extra classes existing in target and source datasets are considered as outliers.