SRoUDA: Meta Self-training for Robust Unsupervised Domain Adaptation

TL;DR

SRoUDA introduces a meta self-training approach to enhance adversarial robustness in unsupervised domain adaptation, effectively mitigating error propagation from pseudo labels and achieving state-of-the-art results without sacrificing accuracy.

Contribution

It proposes a novel meta self-training pipeline, SRoUDA, that integrates adversarial training into UDA and mitigates pseudo-label noise through a meta learning step.

Findings

Achieves significant robustness improvements on benchmark datasets.

Maintains high accuracy on clean data.

Outperforms existing UDA methods in robustness metrics.

Abstract

As acquiring manual labels on data could be costly, unsupervised domain adaptation (UDA), which transfers knowledge learned from a rich-label dataset to the unlabeled target dataset, is gaining increasing popularity. While extensive studies have been devoted to improving the model accuracy on target domain, an important issue of model robustness is neglected. To make things worse, conventional adversarial training (AT) methods for improving model robustness are inapplicable under UDA scenario since they train models on adversarial examples that are generated by supervised loss function. In this paper, we present a new meta self-training pipeline, named SRoUDA, for improving adversarial robustness of UDA models. Based on self-training paradigm, SRoUDA starts with pre-training a source model by applying UDA baseline on source labeled data and taraget unlabeled data with a developed random masked augmentation (RMA), and then alternates between adversarial target model training on pseudo-labeled target data and finetuning source model by a meta step. While self-training allows the direct incorporation of AT in UDA, the meta step in SRoUDA further helps in mitigating error propagation from noisy pseudo labels. Extensive experiments on various benchmark datasets demonstrate the state-of-the-art performance of SRoUDA where it achieves significant model robustness improvement without harming clean accuracy. Code is available at https://github.com/Vision.