# EAT-NAS: Elastic Architecture Transfer for Accelerating Large-scale Neural Architecture Search

## TL;DR

EAT-NAS introduces an elastic transfer mechanism that accelerates large-scale neural architecture search by leveraging architectures found on smaller datasets, achieving competitive accuracy with reduced computational cost.

## Contribution

The paper presents a novel architecture transfer mechanism that significantly speeds up large-scale NAS by initializing search with architectures from small datasets.

## Key findings

- Achieved 74.7% and 74.2% accuracy on ImageNet with EATNet-A and EATNet-B.
- Reduced NAS computational time to less than 5 days on 8 GPUs.
- Surpassed models searched directly on ImageNet in both accuracy and efficiency.

## Abstract

Neural architecture search (NAS) methods have been proposed to release human experts from tedious architecture engineering. However, most current methods are constrained in small-scale search due to the issue of computational resources. Meanwhile, directly applying architectures searched on small datasets to large datasets often bears no performance guarantee. This limitation impedes the wide use of NAS on large-scale tasks. To overcome this obstacle, we propose an elastic architecture transfer mechanism for accelerating large-scale neural architecture search (EAT-NAS). In our implementations, architectures are first searched on a small dataset, e.g., CIFAR-10. The best one is chosen as the basic architecture. The search process on the large dataset, e.g., ImageNet, is initialized with the basic architecture as the seed. The large-scale search process is accelerated with the help of the basic architecture. What we propose is not only a NAS method but a mechanism for architecture-level transfer. In our experiments, we obtain two final models EATNet-A and EATNet-B that achieve competitive accuracies, 74.7% and 74.2% on ImageNet, respectively, which also surpass the models searched from scratch on ImageNet under the same settings. For the computational cost, EAT-NAS takes only less than 5 days on 8 TITAN X GPUs, which is significantly less than the computational consumption of the state-of-the-art large-scale NAS methods.