Virtual Mixup Training for Unsupervised Domain Adaptation

TL;DR

This paper introduces Virtual Mixup Training (VMT), a novel regularization technique for unsupervised domain adaptation that enforces linear behavior between training points, significantly enhancing model performance.

Contribution

VMT extends mixup regularization to unsupervised domain adaptation by constructing label-free combination samples, improving the locally-Lipschitz constraint enforcement in-between data points.

Findings

VMT improves VADA accuracy by over 30% on MNIST to SVHN.

VMT enhances performance across six domain adaptation benchmarks.

The method is compatible with existing models like VADA.

Abstract

We study the problem of unsupervised domain adaptation which aims to adapt models trained on a labeled source domain to a completely unlabeled target domain. Recently, the cluster assumption has been applied to unsupervised domain adaptation and achieved strong performance. One critical factor in successful training of the cluster assumption is to impose the locally-Lipschitz constraint to the model. Existing methods only impose the locally-Lipschitz constraint around the training points while miss the other areas, such as the points in-between training data. In this paper, we address this issue by encouraging the model to behave linearly in-between training points. We propose a new regularization method called Virtual Mixup Training (VMT), which is able to incorporate the locally-Lipschitz constraint to the areas in-between training data. Unlike the traditional mixup model, our method constructs the combination samples without using the label information, allowing it to apply to unsupervised domain adaptation. The proposed method is generic and can be combined with most existing models such as the recent state-of-the-art model called VADA. Extensive experiments demonstrate that VMT significantly improves the performance of VADA on six domain adaptation benchmark datasets. For the challenging task of adapting MNIST to SVHN, VMT can improve the accuracy of VADA by over 30\%. Code is available at \url{https://github.com/xudonmao/VMT}.