On-target Adaptation

TL;DR

This paper proposes on-target adaptation for domain adaptation tasks, focusing on optimizing model representations directly on target data using source predictions, leading to improved accuracy especially in long-tailed classification scenarios.

Contribution

It introduces on-target adaptation, a method that optimizes model representations solely on target data with source predictions for supervision, and extends it with class distribution learning for long-tailed data.

Findings

Significant accuracy improvements with on-target adaptation.

Enhanced performance in long-tailed classification through class distribution learning.

Outperforms traditional source-dependent domain adaptation methods.

Abstract

Domain adaptation seeks to mitigate the shift between training on the \emph{source} domain and testing on the \emph{target} domain. Most adaptation methods rely on the source data by joint optimization over source data and target data. Source-free methods replace the source data with a source model by fine-tuning it on target. Either way, the majority of the parameter updates for the model representation and the classifier are derived from the source, and not the target. However, target accuracy is the goal, and so we argue for optimizing as much as possible on the target data. We show significant improvement by on-target adaptation, which learns the representation purely from target data while taking only the source predictions for supervision. In the long-tailed classification setting, we show further improvement by on-target class distribution learning, which learns the (im)balance of classes from target data.