Mutation is all you need

TL;DR

This paper investigates the factors influencing the performance of the BANANAS neural architecture search method, revealing that its success depends on the mutation strategy rather than the surrogate model or acquisition function.

Contribution

The study provides experimental evidence that the effectiveness of BANANAS is primarily driven by its mutation approach, challenging previous assumptions about its components.

Findings

BANANAS performance on NAS-Bench-101 is influenced by path encoding.

On NAS-Bench-301, performance depends on the mutation-based acquisition function optimizer.

Mutations play a crucial role in the success of the NAS method.

Abstract

Neural architecture search (NAS) promises to make deep learning accessible to non-experts by automating architecture engineering of deep neural networks. BANANAS is one state-of-the-art NAS method that is embedded within the Bayesian optimization framework. Recent experimental findings have demonstrated the strong performance of BANANAS on the NAS-Bench-101 benchmark being determined by its path encoding and not its choice of surrogate model. We present experimental results suggesting that the performance of BANANAS on the NAS-Bench-301 benchmark is determined by its acquisition function optimizer, which minimally mutates the incumbent.