Rapid Model Architecture Adaption for Meta-Learning

TL;DR

This paper introduces H-Meta-NAS, a hardware-aware NAS method integrated with MAML, enabling rapid adaptation of model architectures across multiple tasks and hardware platforms, significantly improving efficiency and accuracy in few-shot learning.

Contribution

It presents the first method to combine NAS with MAML for fast, hardware-aware architecture adaptation in many-task, many-hardware few-shot learning scenarios.

Findings

Outperforms manual baselines by 5.21% accuracy on Mini-ImageNet 5-way 1-shot.

Reduces computation by 40% compared to manual methods.

Demonstrates Pareto dominance over existing NAS and manual baselines.

Abstract

Network Architecture Search (NAS) methods have recently gathered much attention. They design networks with better performance and use a much shorter search time compared to traditional manual tuning. Despite their efficiency in model deployments, most NAS algorithms target a single task on a fixed hardware system. However, real-life few-shot learning environments often cover a great number of tasks (T ) and deployments on a wide variety of hardware platforms (H ). The combinatorial search complexity T times H creates a fundamental search efficiency challenge if one naively applies existing NAS methods to these scenarios. To overcome this issue, we show, for the first time, how to rapidly adapt model architectures to new tasks in a many-task many-hardware few-shot learning setup by integrating Model Agnostic Meta Learning (MAML) into the NAS flow. The proposed NAS method (H-Meta-NAS) is hardware-aware and performs optimisation in the MAML framework. H-Meta-NAS shows a Pareto dominance compared to a variety of NAS and manual baselines in popular few-shot learning benchmarks with various hardware platforms and constraints. In particular, on the 5-way 1-shot Mini-ImageNet classification task, the proposed method outperforms the best manual baseline by a large margin (5.21% in accuracy) using 60% less computation.