Transfer Adaptation Learning: A Decade Survey

TL;DR

This survey reviews a decade of transfer adaptation learning (TAL), highlighting its methodologies, challenges, and future directions for building models that adapt across different domains with distribution shifts.

Contribution

It provides a comprehensive overview of TAL advancements, categorizes key techniques, and discusses theoretical limitations and future research challenges.

Findings

Identified main TAL methodologies: instance re-weighting, feature, classifier, deep network, and adversarial adaptation.

Discussed the evolution from semi-supervised and unsupervised approaches to broader solutions.

Highlighted open issues like universality, interpretability, and credibility in TAL.

Abstract

The world we see is ever-changing and it always changes with people, things, and the environment. Domain is referred to as the state of the world at a certain moment. A research problem is characterized as transfer adaptation learning (TAL) when it needs knowledge correspondence between different moments/domains. Conventional machine learning aims to find a model with the minimum expected risk on test data by minimizing the regularized empirical risk on the training data, which, however, supposes that the training and test data share similar joint probability distribution. TAL aims to build models that can perform tasks of target domain by learning knowledge from a semantic related but distribution different source domain. It is an energetic research filed of increasing influence and importance, which is presenting a blowout publication trend. This paper surveys the advances of TAL methodologies in the past decade, and the technical challenges and essential problems of TAL have been observed and discussed with deep insights and new perspectives. Broader solutions of transfer adaptation learning being created by researchers are identified, i.e., instance re-weighting adaptation, feature adaptation, classifier adaptation, deep network adaptation and adversarial adaptation, which are beyond the early semi-supervised and unsupervised split. The survey helps researchers rapidly but comprehensively understand and identify the research foundation, research status, theoretical limitations, future challenges and under-studied issues (universality, interpretability, and credibility) to be broken in the field toward universal representation and safe applications in open-world scenarios.