Domain Adaptive Transfer Learning with Specialist Models

TL;DR

This paper investigates how the choice of pre-training data affects transfer learning performance in computer vision, proposing a domain adaptive method that uses importance weights for improved results.

Contribution

It introduces a novel domain adaptive transfer learning approach using importance weights derived from domain adaptation principles, enhancing transfer learning effectiveness.

Findings

More pre-training data does not always improve performance.

The proposed method achieves state-of-the-art results on fine-grained datasets.

Importance weighting improves transfer learning by selecting relevant pre-training data.

Abstract

Transfer learning is a widely used method to build high performing computer vision models. In this paper, we study the efficacy of transfer learning by examining how the choice of data impacts performance. We find that more pre-training data does not always help, and transfer performance depends on a judicious choice of pre-training data. These findings are important given the continued increase in dataset sizes. We further propose domain adaptive transfer learning, a simple and effective pre-training method using importance weights computed based on the target dataset. Our method to compute importance weights follow from ideas in domain adaptation, and we show a novel application to transfer learning. Our methods achieve state-of-the-art results on multiple fine-grained classification datasets and are well-suited for use in practice.