A Survey on Deep Transfer Learning

TL;DR

This survey reviews recent research on deep transfer learning, highlighting its role in overcoming data scarcity in domains like bioinformatics and robotics by leveraging pre-trained neural networks.

Contribution

It provides a comprehensive categorization and review of deep transfer learning techniques and applications, clarifying its definitions and recent advancements.

Findings

Deep transfer learning effectively addresses data scarcity issues.

Various techniques and applications of deep transfer learning are systematically categorized.

The survey identifies key challenges and future directions in the field.

Abstract

As a new classification platform, deep learning has recently received increasing attention from researchers and has been successfully applied to many domains. In some domains, like bioinformatics and robotics, it is very difficult to construct a large-scale well-annotated dataset due to the expense of data acquisition and costly annotation, which limits its development. Transfer learning relaxes the hypothesis that the training data must be independent and identically distributed (i.i.d.) with the test data, which motivates us to use transfer learning to solve the problem of insufficient training data. This survey focuses on reviewing the current researches of transfer learning by using deep neural network and its applications. We defined deep transfer learning, category and review the recent research works based on the techniques used in deep transfer learning.