Improved Automated Machine Learning from Transfer Learning

TL;DR

This paper introduces a neural architecture search method leveraging task similarity based on Fisher Information to efficiently find optimal models for new tasks without extensive training or human bias.

Contribution

It presents a novel task similarity measure and a transfer learning-based NAS framework that reduces search space and computational cost.

Findings

Significantly reduces GPU days for NAS.

Eliminates need for training from scratch.

Avoids human bias in search space initialization.

Abstract

In this paper, we propose a neural architecture search framework based on a similarity measure between some baseline tasks and a target task. We first define the notion of the task similarity based on the log-determinant of the Fisher Information matrix. Next, we compute the task similarity from each of the baseline tasks to the target task. By utilizing the relation between a target and a set of learned baseline tasks, the search space of architectures for the target task can be significantly reduced, making the discovery of the best candidates in the set of possible architectures tractable and efficient, in terms of GPU days. This method eliminates the requirement for training the networks from scratch for a given target task as well as introducing the bias in the initialization of the search space from the human domain.