Gated Transfer Network for Transfer Learning

TL;DR

This paper introduces a Gated Transfer Network that enhances transfer learning by selectively weighting features from pre-trained models, leading to improved performance across multiple tasks with minimal additional computation.

Contribution

The paper proposes a novel transfer module for feature selection in fine-tuning, significantly boosting transfer learning effectiveness and introducing the Gated Transfer Network architecture.

Findings

Achieved state-of-the-art results on six tasks.

Feature selection via the transfer module improves transfer learning.

Incorporating an auxiliary classifier reduces overfitting.

Abstract

Deep neural networks have led to a series of breakthroughs in computer vision given sufficient annotated training datasets. For novel tasks with limited labeled data, the prevalent approach is to transfer the knowledge learned in the pre-trained models to the new tasks by fine-tuning. Classic model fine-tuning utilizes the fact that well trained neural networks appear to learn cross domain features. These features are treated equally during transfer learning. In this paper, we explore the impact of feature selection in model fine-tuning by introducing a transfer module, which assigns weights to features extracted from pre-trained models. The proposed transfer module proves the importance of feature selection for transferring models from source to target domains. It is shown to significantly improve upon fine-tuning results with only marginal extra computational cost. We also incorporate an auxiliary classifier as an extra regularizer to avoid over-fitting. Finally, we build a Gated Transfer Network (GTN) based on our transfer module and achieve state-of-the-art results on six different tasks.