ESM: A Framework for Building Effective Surrogate Models for Hardware-Aware Neural Architecture Search

TL;DR

This paper presents a comprehensive framework for developing accurate surrogate models to predict latency in hardware-aware neural architecture search, focusing on GPU devices and optimizing the model creation process.

Contribution

It introduces a systematic analysis of surrogate models for latency prediction and proposes a holistic framework for dataset and model generation in hardware-aware NAS.

Findings

Analyzed strengths and weaknesses of various surrogate models.

Identified key factors influencing prediction accuracy.

Proposed a cost-effective, reliable model generation pipeline.

Abstract

Hardware-aware Neural Architecture Search (NAS) is one of the most promising techniques for designing efficient Deep Neural Networks (DNNs) for resource-constrained devices. Surrogate models play a crucial role in hardware-aware NAS as they enable efficient prediction of performance characteristics (e.g., inference latency and energy consumption) of different candidate models on the target hardware device. In this paper, we focus on building hardware-aware latency prediction models. We study different types of surrogate models and highlight their strengths and weaknesses. We perform a systematic analysis to understand the impact of different factors that can influence the prediction accuracy of these models, aiming to assess the importance of each stage involved in the model designing process and identify methods and policies necessary for designing/training an effective estimation model, specifically for GPU-powered devices. Based on the insights gained from the analysis, we present a holistic framework that enables reliable dataset generation and efficient model generation, considering the overall costs of different stages of the model generation pipeline.