PAS: Estimating the target accuracy before domain adaptation

TL;DR

PAS is a novel scoring method that predicts the effectiveness of source domains and pre-trained models for domain adaptation tasks, aiding in better selection before actual adaptation.

Contribution

The paper introduces PAS, a new score that estimates transferability based on pre-trained features, improving source and model selection for domain adaptation.

Findings

PAS correlates strongly with actual target accuracy.

Using PAS improves selection of pre-trained models and source domains.

Experiments show PAS reduces computational overhead while enhancing performance.

Abstract

The goal of domain adaptation is to make predictions for unlabeled samples from a target domain with the help of labeled samples from a different but related source domain. The performance of domain adaptation methods is highly influenced by the choice of source domain and pre-trained feature extractor. However, the selection of source data and pre-trained model is not trivial due to the absence of a labeled validation set for the target domain and the large number of available pre-trained models. In this work, we propose PAS, a novel score designed to estimate the transferability of a source domain set and a pre-trained feature extractor to a target classification task before actually performing domain adaptation. PAS leverages the generalization power of pre-trained models and assesses source-target compatibility based on the pre-trained feature embeddings. We integrate PAS into a framework that indicates the most relevant pre-trained model and source domain among multiple candidates, thus improving target accuracy while reducing the computational overhead. Extensive experiments on image classification benchmarks demonstrate that PAS correlates strongly with actual target accuracy and consistently guides the selection of the best-performing pre-trained model and source domain for adaptation.