Improving Differentiable Architecture Search with a Generative Model

TL;DR

This paper proposes DASGM, a novel NAS method that incorporates synthesized datasets generated by a generative model to improve architecture search by emphasizing uncommon features, leading to better recognition accuracy.

Contribution

Introducing DASGM, a new NAS approach that uses synthesized datasets to enhance architecture search and address data distribution limitations in differentiable NAS methods.

Findings

DASGM improves classification accuracy on CIFAR-10, CIFAR-100, and ImageNet.

Synthesized datasets help models learn better features.

The method effectively addresses data distribution issues in NAS.

Abstract

In differentiable neural architecture search (NAS) algorithms like DARTS, the training set used to update model weight and the validation set used to update model architectures are sampled from the same data distribution. Thus, the uncommon features in the dataset fail to receive enough attention during training. In this paper, instead of introducing more complex NAS algorithms, we explore the idea that adding quality synthesized datasets into training can help the classification model identify its weakness and improve recognition accuracy. We introduce a training strategy called ``Differentiable Architecture Search with a Generative Model(DASGM)." In DASGM, the training set is used to update the classification model weight, while a synthesized dataset is used to train its architecture. The generated images have different distributions from the training set, which can help the classification model learn better features to identify its weakness. We formulate DASGM into a multi-level optimization framework and develop an effective algorithm to solve it. Experiments on CIFAR-10, CIFAR-100, and ImageNet have demonstrated the effectiveness of DASGM. Code will be made available.