Open-Set Heterogeneous Domain Adaptation: Theoretical Analysis and Algorithm

TL;DR

This paper introduces a new open-set heterogeneous domain adaptation scenario, providing a theoretical framework and a novel algorithm that effectively transfers knowledge across heterogeneous data sources while identifying novel classes.

Contribution

It develops a theoretical framework for open-set heterogeneous domain adaptation and proposes the RL-OSHeDA algorithm to handle feature heterogeneity and novel class detection.

Findings

The proposed method outperforms existing approaches on text, image, and clinical datasets.

Theoretical bounds for prediction error in open-set heterogeneous DA are established.

RL-OSHeDA effectively transfers knowledge and detects novel classes in diverse data scenarios.

Abstract

Domain adaptation (DA) tackles the issue of distribution shift by learning a model from a source domain that generalizes to a target domain. However, most existing DA methods are designed for scenarios where the source and target domain data lie within the same feature space, which limits their applicability in real-world situations. Recently, heterogeneous DA (HeDA) methods have been introduced to address the challenges posed by heterogeneous feature space between source and target domains. Despite their successes, current HeDA techniques fall short when there is a mismatch in both feature and label spaces. To address this, this paper explores a new DA scenario called open-set HeDA (OSHeDA). In OSHeDA, the model must not only handle heterogeneity in feature space but also identify samples belonging to novel classes. To tackle this challenge, we first develop a novel theoretical framework that constructs learning bounds for prediction error on target domain. Guided by this framework, we propose a new DA method called Representation Learning for OSHeDA (RL-OSHeDA). This method is designed to simultaneously transfer knowledge between heterogeneous data sources and identify novel classes. Experiments across text, image, and clinical data demonstrate the effectiveness of our algorithm. Model implementation is available at \url{https://github.com/pth1993/OSHeDA}.