Learn to Explore: Meta NAS via Bayesian Optimization Guided Graph Generation

TL;DR

This paper introduces GraB-NAS, a Meta-NAS framework that models neural architectures as graphs and employs a hybrid Bayesian Optimization and gradient ascent search strategy to efficiently discover high-performing, task-adaptive neural network architectures.

Contribution

GraB-NAS is the first to combine graph-based architecture modeling with a hybrid search strategy for improved Meta-NAS performance and generalization.

Findings

Outperforms state-of-the-art Meta-NAS methods in experiments.

Achieves better generalization to new tasks.

Effectively explores architectures beyond predefined search spaces.

Abstract

Neural Architecture Search (NAS) automates the design of high-performing neural networks but typically targets a single predefined task, thereby restricting its real-world applicability. To address this, Meta Neural Architecture Search (Meta-NAS) has emerged as a promising paradigm that leverages prior knowledge across tasks to enable rapid adaptation to new ones. Nevertheless, existing Meta-NAS methods often struggle with poor generalization, limited search spaces, or high computational costs. In this paper, we propose a novel Meta-NAS framework, GraB-NAS. Specifically, GraB-NAS first models neural architectures as graphs, and then a hybrid search strategy is developed to find and generate new graphs that lead to promising neural architectures. The search strategy combines global architecture search via Bayesian Optimization in the search space with local exploration for novel neural networks via gradient ascent in the latent space. Such a hybrid search strategy allows GraB-NAS to discover task-aware architectures with strong performance, even beyond the predefined search space. Extensive experiments demonstrate that GraB-NAS outperforms state-of-the-art Meta-NAS baselines, achieving better generalization and search effectiveness.