# XferNAS: Transfer Neural Architecture Search

## TL;DR

XferNAS introduces a transfer learning framework for neural architecture search that significantly reduces search time by reusing knowledge from previous tasks, achieving state-of-the-art results on CIFAR datasets.

## Contribution

The paper presents a flexible transfer learning framework for NAS that can be integrated with existing optimizers to drastically cut down search time and improve performance.

## Key findings

- Search time reduced from 200 to 6 GPU days
- Achieved new NAS records on CIFAR-10 and CIFAR-100
- Framework improves results or matches unmodified optimizers

## Abstract

The term Neural Architecture Search (NAS) refers to the automatic optimization of network architectures for a new, previously unknown task. Since testing an architecture is computationally very expensive, many optimizers need days or even weeks to find suitable architectures. However, this search time can be significantly reduced if knowledge from previous searches on different tasks is reused. In this work, we propose a generally applicable framework that introduces only minor changes to existing optimizers to leverage this feature. As an example, we select an existing optimizer and demonstrate the complexity of the integration of the framework as well as its impact. In experiments on CIFAR-10 and CIFAR-100, we observe a reduction in the search time from 200 to only 6 GPU days, a speed up by a factor of 33. In addition, we observe new records of 1.99 and 14.06 for NAS optimizers on the CIFAR benchmarks, respectively. In a separate study, we analyze the impact of the amount of source and target data. Empirically, we demonstrate that the proposed framework generally gives better results and, in the worst case, is just as good as the unmodified optimizer.