Multi-source Attention for Unsupervised Domain Adaptation

TL;DR

This paper introduces an attention-based method for multi-source unsupervised domain adaptation, learning to select relevant sources for each target instance, which improves classification performance across domains.

Contribution

It models source selection as an attention-learning problem, integrating source-specific models and relatedness maps to enhance adaptation.

Findings

Outperforms prior methods on sentiment classification benchmarks

Effectively reduces negative transfer from unrelated sources

Demonstrates improved accuracy in multi-source domain adaptation

Abstract

Domain adaptation considers the problem of generalising a model learnt using data from a particular source domain to a different target domain. Often it is difficult to find a suitable single source to adapt from, and one must consider multiple sources. Using an unrelated source can result in sub-optimal performance, known as the \emph{negative transfer}. However, it is challenging to select the appropriate source(s) for classifying a given target instance in multi-source unsupervised domain adaptation (UDA). We model source-selection as an attention-learning problem, where we learn attention over sources for a given target instance. For this purpose, we first independently learn source-specific classification models, and a relatedness map between sources and target domains using pseudo-labelled target domain instances. Next, we learn attention-weights over the sources for aggregating the predictions of the source-specific models. Experimental results on cross-domain sentiment classification benchmarks show that the proposed method outperforms prior proposals in multi-source UDA.