UnrealNAS: Can We Search Neural Architectures with Unreal Data?

TL;DR

This paper investigates whether neural architecture search (NAS) can be effectively performed using unreal data instead of real labeled datasets, potentially broadening NAS applicability in data-limited scenarios.

Contribution

The study demonstrates that NAS can produce competitive architectures using various types of unreal data, challenging the necessity of real data for effective NAS.

Findings

Architectures searched on unreal data achieve promising results.

Unreal datasets can facilitate NAS without real labels.

NAS with unreal data is feasible and effective.

Abstract

Neural architecture search (NAS) has shown great success in the automatic design of deep neural networks (DNNs). However, the best way to use data to search network architectures is still unclear and under exploration. Previous work has analyzed the necessity of having ground-truth labels in NAS and inspired broad interest. In this work, we take a further step to question whether real data is necessary for NAS to be effective. The answer to this question is important for applications with limited amount of accessible data, and can help people improve NAS by leveraging the extra flexibility of data generation. To explore if NAS needs real data, we construct three types of unreal datasets using: 1) randomly labeled real images; 2) generated images and labels; and 3) generated Gaussian noise with random labels. These datasets facilitate to analyze the generalization and expressivity of the searched architectures. We study the performance of architectures searched on these constructed datasets using popular differentiable NAS methods. Extensive experiments on CIFAR, ImageNet and CheXpert show that the searched architectures can achieve promising results compared with those derived from the conventional NAS pipeline with real labeled data, suggesting the feasibility of performing NAS with unreal data.