Efficient Transfer Learning via Joint Adaptation of Network Architecture and Weight

TL;DR

This paper introduces a novel transfer learning framework that combines reduced super-network architecture search and weight reuse, enabling efficient adaptation to target tasks with minimal source training.

Contribution

It proposes a joint architecture and weight transfer method using a reduced super-network, reducing training redundancy and improving transfer efficiency.

Findings

Achieves promising results on MS-COCO and CUB-200 datasets.

Reduces super-network complexity to O(CN), enhancing efficiency.

Demonstrates effective transfer learning with minimal source task training.

Abstract

Transfer learning can boost the performance on the targettask by leveraging the knowledge of the source domain. Recent worksin neural architecture search (NAS), especially one-shot NAS, can aidtransfer learning by establishing sufficient network search space. How-ever, existing NAS methods tend to approximate huge search spaces byexplicitly building giant super-networks with multiple sub-paths, anddiscard super-network weights after a child structure is found. Both thecharacteristics of existing approaches causes repetitive network trainingon source tasks in transfer learning. To remedy the above issues, we re-duce the super-network size by randomly dropping connection betweennetwork blocks while embedding a larger search space. Moreover, wereuse super-network weights to avoid redundant training by proposinga novel framework consisting of two modules, the neural architecturesearch module for architecture transfer and the neural weight searchmodule for weight transfer. These two modules conduct search on thetarget task based on a reduced super-networks, so we only need to trainonce on the source task. We experiment our framework on both MS-COCO and CUB-200 for the object detection and fine-grained imageclassification tasks, and show promising improvements with onlyO(CN)super-network complexity.