Semi-supervised Domain Adaptation via Sample-to-Sample Self-Distillation

TL;DR

This paper introduces a novel semi-supervised domain adaptation method that uses sample-to-sample self-distillation with assistant features to effectively bridge domain gaps and improve model performance.

Contribution

It proposes a pair-based self-distillation approach that generates assistant features to adapt models to target domains with limited labeled data.

Findings

Significant performance improvements on standard benchmarks.

Effective reduction of inter-domain and intra-domain discrepancies.

Outperforms recent state-of-the-art methods.

Abstract

Semi-supervised domain adaptation (SSDA) is to adapt a learner to a new domain with only a small set of labeled samples when a large labeled dataset is given on a source domain. In this paper, we propose a pair-based SSDA method that adapts a model to the target domain using self-distillation with sample pairs. Each sample pair is composed of a teacher sample from a labeled dataset (i.e., source or labeled target) and its student sample from an unlabeled dataset (i.e., unlabeled target). Our method generates an assistant feature by transferring an intermediate style between the teacher and the student, and then train the model by minimizing the output discrepancy between the student and the assistant. During training, the assistants gradually bridge the discrepancy between the two domains, thus allowing the student to easily learn from the teacher. Experimental evaluation on standard benchmarks shows that our method effectively minimizes both the inter-domain and intra-domain discrepancies, thus achieving significant improvements over recent methods.