Population-based guiding for evolutionary neural architecture search

TL;DR

This paper introduces a new method for efficiently searching neural network architectures using evolutionary techniques guided by population data.

Contribution

The novel PBG framework combines greedy selection and guided mutation to improve the efficiency of evolutionary NAS.

Findings

PBG outperforms baseline methods like regularized evolution by up to three times on NAS-Bench-101.

The framework effectively balances exploration and exploitation in neural architecture search.

PBG achieves competitive performance across multiple NAS benchmarks.

Abstract

Neural Architecture Search (NAS)—combined with biology-inspired evolutionary methods—can help discover suitable architectures tailored to a given objective. A guided evolutionary approach can enhance efficiency, aiming to accelerate the discovery of top-performing architectures within a given search space. We propose a novel algorithmic framework that implements selection, crossover, and mutation operations to generate new candidate architectures during an evolutionary Neural Architecture Search: A greedy selection operator, relying solely on model accuracy data, promotes exploitation. Incorporating architecture embeddings to further refine the mutation process enhances exploration. We introduce a guided mutation approach to steer the search toward unexplored regions of the current population. The proposed strategy, PBG (Population-Based Guiding), synergizes both explorative and exploitative methods. It substantially outperforms baseline methods such as regularized evolution by being up to three times faster on NAS-Bench-101. This combined approach not only leverages the strengths of both explorative guided mutation and exploitative greedy selection strategies, but also provides a robust and efficient framework reaching competitive performance for evolutionary Neural Architecture Search across benchmarks.

Figures