Across-Task Neural Architecture Search via Meta Learning

TL;DR

This paper introduces AT-NAS, a meta-learning based neural architecture search method that efficiently finds task-sensitive architectures, improving few-shot classification with minimal search time and resource use.

Contribution

It combines gradient-based meta-learning with evolutionary NAS to learn a versatile supernet across tasks, enabling rapid adaptation to new tasks.

Findings

AT-NAS outperforms related methods in few-shot classification accuracy.

Achieves comparable performance to models trained from scratch with significantly less search time.

Adapts architectures in less than an hour from a pretrained meta-network.

Abstract

Adequate labeled data and expensive compute resources are the prerequisites for the success of neural architecture search(NAS). It is challenging to apply NAS in meta-learning scenarios with limited compute resources and data. In this paper, an across-task neural architecture search (AT-NAS) is proposed to address the problem through combining gradient-based meta-learning with EA-based NAS to learn over the distribution of tasks. The supernet is learned over an entire set of tasks by meta-learning its weights. Architecture encodes of subnets sampled from the supernet are iteratively adapted by evolutionary algorithms while simultaneously searching for a task-sensitive meta-network. Searched meta-network can be adapted to a novel task via a few learning steps and only costs a little search time. Empirical results show that AT-NAS surpasses the related approaches on few-shot classification accuracy. The performance of AT-NAS on classification benchmarks is comparable to that of models searched from scratch, by adapting the architecture in less than an hour from a 5-GPU-day pretrained meta-network.