Enhanced Gradient for Differentiable Architecture Search
Haichao Zhang, Kuangrong Hao, Lei Gao, Xuesong Tang, and Bing Wei

TL;DR
This paper introduces an enhanced gradient-based neural architecture search method that optimizes for both accuracy and efficiency, producing compact networks that outperform hand-crafted models in image classification tasks.
Contribution
The paper presents a novel two-stage NAS framework using an enhanced gradient relaxation and evolutionary multi-objective optimization for efficient, high-performance network design.
Findings
Outperforms hand-crafted networks on CIFAR10 and CIFAR100
Achieves high accuracy with networks under one megabit
Significantly reduces network parameter count compared to other NAS methods
Abstract
In recent years, neural architecture search (NAS) methods have been proposed for the automatic generation of task-oriented network architecture in image classification. However, the architectures obtained by existing NAS approaches are optimized only for classification performance and do not adapt to devices with limited computational resources. To address this challenge, we propose a neural network architecture search algorithm aiming to simultaneously improve network performance (e.g., classification accuracy) and reduce network complexity. The proposed framework automatically builds the network architecture at two stages: block-level search and network-level search. At the stage of block-level search, a relaxation method based on the gradient is proposed, using an enhanced gradient to design high-performance and low-complexity blocks. At the stage of network-level search, we apply an…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Neural Network Applications · Machine Learning and Data Classification · Machine Learning and ELM
