Conditional Neural Architecture Search

TL;DR

This paper introduces a novel conditional neural architecture search method using GANs to generate resource-efficient DNNs tailored for different edge platforms, reducing the need for repeated retraining and model adjustments.

Contribution

It presents the first integration of conditional and adversarial techniques into Neural Architecture Search to produce platform-specific, resource-optimized neural networks.

Findings

Successfully generates resource-optimized MLP and CNN models

Effective on regression and classification tasks including CIFAR-10

Reduces iteration in model deployment for edge platforms

Abstract

Designing resource-efficient Deep Neural Networks (DNNs) is critical to deploy deep learning solutions over edge platforms due to diverse performance, power, and memory budgets. Unfortunately, it is often the case a well-trained ML model does not fit to the constraint of deploying edge platforms, causing a long iteration of model reduction and retraining process. Moreover, a ML model optimized for platform-A often may not be suitable when we deploy it on another platform-B, causing another iteration of model retraining. We propose a conditional neural architecture search method using GAN, which produces feasible ML models for different platforms. We present a new workflow to generate constraint-optimized DNN models. This is the first work of bringing in condition and adversarial technique into Neural Architecture Search domain. We verify the method with regression problems and classification on CIFAR-10. The proposed workflow can successfully generate resource-optimized MLP or CNN-based networks.