Understanding the Limits of Unsupervised Domain Adaptation via Data Poisoning

TL;DR

This paper establishes theoretical limits of unsupervised domain adaptation (UDA), introduces data poisoning attacks to exploit these limits, and demonstrates significant accuracy drops in UDA methods under attack, highlighting their robustness issues.

Contribution

The paper provides a lower bound on target error in UDA, proposes data poisoning attacks to challenge UDA methods, and empirically shows their vulnerability to such attacks.

Findings

Lower bound on target domain error established

Poisoning attacks can reduce UDA accuracy to near zero

UDA methods are vulnerable to data poisoning in benchmark datasets

Abstract

Unsupervised domain adaptation (UDA) enables cross-domain learning without target domain labels by transferring knowledge from a labeled source domain whose distribution differs from that of the target. However, UDA is not always successful and several accounts of `negative transfer' have been reported in the literature. In this work, we prove a simple lower bound on the target domain error that complements the existing upper bound. Our bound shows the insufficiency of minimizing source domain error and marginal distribution mismatch for a guaranteed reduction in the target domain error, due to the possible increase of induced labeling function mismatch. This insufficiency is further illustrated through simple distributions for which the same UDA approach succeeds, fails, and may succeed or fail with an equal chance. Motivated from this, we propose novel data poisoning attacks to fool UDA methods into learning representations that produce large target domain errors. We evaluate the effect of these attacks on popular UDA methods using benchmark datasets where they have been previously shown to be successful. Our results show that poisoning can significantly decrease the target domain accuracy, dropping it to almost 0% in some cases, with the addition of only 10% poisoned data in the source domain. The failure of these UDA methods demonstrates their limitations at guaranteeing cross-domain generalization consistent with our lower bound. Thus, evaluating UDA methods in adversarial settings such as data poisoning provides a better sense of their robustness to data distributions unfavorable for UDA.