Search Space Adaptation for Differentiable Neural Architecture Search in Image Classification

TL;DR

This paper introduces a search space adaptation method for differentiable neural architecture search (NAS) in image classification, aiming to improve architecture performance by dynamically adjusting the search scope.

Contribution

It proposes a novel search space adaptation scheme using a search scope, enhancing differentiable NAS effectiveness in image classification tasks.

Findings

Improved architecture performance with the proposed search space adaptation.

Visualization of architecture parameter updates offers insights into the search process.

Demonstrated effectiveness on ProxylessNAS for image classification.

Abstract

As deep neural networks achieve unprecedented performance in various tasks, neural architecture search (NAS), a research field for designing neural network architectures with automated processes, is actively underway. More recently, differentiable NAS has a great impact by reducing the search cost to the level of training a single network. Besides, the search space that defines candidate architectures to be searched directly affects the performance of the final architecture. In this paper, we propose an adaptation scheme of the search space by introducing a search scope. The effectiveness of proposed method is demonstrated with ProxylessNAS for the image classification task. Furthermore, we visualize the trajectory of architecture parameter updates and provide insights to improve the architecture search.