Ranking architectures using meta-learning

TL;DR

This paper introduces a pairwise ranking loss for a neural architecture ranking network that improves search efficiency by better predicting architecture performance across tasks using meta-features.

Contribution

It proposes a novel ranking-based training method for architecture performance prediction that outperforms previous predictors in multi-task neural architecture search.

Findings

Ranking network outperforms previous performance predictor

Effective in zero-shot architecture ranking for new tasks

Improves efficiency of neural architecture search

Abstract

Neural architecture search has recently attracted lots of research efforts as it promises to automate the manual design of neural networks. However, it requires a large amount of computing resources and in order to alleviate this, a performance prediction network has been recently proposed that enables efficient architecture search by forecasting the performance of candidate architectures, instead of relying on actual model training. The performance predictor is task-aware taking as input not only the candidate architecture but also task meta-features and it has been designed to collectively learn from several tasks. In this work, we introduce a pairwise ranking loss for training a network able to rank candidate architectures for a new unseen task conditioning on its task meta-features. We present experimental results, showing that the ranking network is more effective in architecture search than the previously proposed performance predictor.