Multiple Source Domain Adaptation with Adversarial Training of Neural Networks

TL;DR

This paper introduces a new theoretical bound and neural network models for effective multiple source domain adaptation, leveraging adversarial training to learn invariant feature representations across domains.

Contribution

It proposes a novel generalization bound for multisource domain adaptation and develops two adversarial neural network models, MDANs, that optimize this bound for improved adaptation.

Findings

Superior performance on sentiment analysis, digit classification, and vehicle counting datasets.

Effective learning of domain-invariant features through adversarial training.

Models outperform existing methods in multisource domain adaptation tasks.

Abstract

While domain adaptation has been actively researched in recent years, most theoretical results and algorithms focus on the single-source-single-target adaptation setting. Naive application of such algorithms on multiple source domain adaptation problem may lead to suboptimal solutions. As a step toward bridging the gap, we propose a new generalization bound for domain adaptation when there are multiple source domains with labeled instances and one target domain with unlabeled instances. Compared with existing bounds, the new bound does not require expert knowledge about the target distribution, nor the optimal combination rule for multisource domains. Interestingly, our theory also leads to an efficient learning strategy using adversarial neural networks: we show how to interpret it as learning feature representations that are invariant to the multiple domain shifts while still being discriminative for the learning task. To this end, we propose two models, both of which we call multisource domain adversarial networks (MDANs): the first model optimizes directly our bound, while the second model is a smoothed approximation of the first one, leading to a more data-efficient and task-adaptive model. The optimization tasks of both models are minimax saddle point problems that can be optimized by adversarial training. To demonstrate the effectiveness of MDANs, we conduct extensive experiments showing superior adaptation performance on three real-world datasets: sentiment analysis, digit classification, and vehicle counting.