NAAS: Neural Accelerator Architecture Search
Yujun Lin, Mengtian Yang, Song Han

TL;DR
NAAS introduces a comprehensive neural accelerator architecture search method that optimizes neural network design, hardware architecture, and compiler mappings simultaneously, significantly improving energy efficiency and accuracy over existing approaches.
Contribution
It presents a holistic search framework for neural accelerators that considers architecture, connectivity, and compiler mappings in one unified optimization process.
Findings
NAAS achieves 4.4x EDP reduction over human-designed Eyeriss.
NAAS improves accuracy by 2.7% on ImageNet.
NAAS outperforms hyper-parameter sizing methods with 1.4x to 3.5x EDP reduction.
Abstract
Data-driven, automatic design space exploration of neural accelerator architecture is desirable for specialization and productivity. Previous frameworks focus on sizing the numerical architectural hyper-parameters while neglect searching the PE connectivities and compiler mappings. To tackle this challenge, we propose Neural Accelerator Architecture Search (NAAS) which holistically searches the neural network architecture, accelerator architecture, and compiler mapping in one optimization loop. NAAS composes highly matched architectures together with efficient mapping. As a data-driven approach, NAAS rivals the human design Eyeriss by 4.4x EDP reduction with 2.7% accuracy improvement on ImageNet under the same computation resource, and offers 1.4x to 3.5x EDP reduction than only sizing the architectural hyper-parameters.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
DAC'21 NAAS: Neural Accelerator Architecture Search· youtube
Taxonomy
TopicsAdvanced Neural Network Applications · Machine Learning in Materials Science · CCD and CMOS Imaging Sensors
