MAPLE-X: Latency Prediction with Explicit Microprocessor Prior Knowledge

TL;DR

MAPLE-X enhances neural network latency prediction by incorporating explicit hardware and architecture priors, leading to more accurate and robust estimations across diverse embedded devices, thus improving neural architecture search efficiency.

Contribution

It introduces MAPLE-X, which extends MAPLE by integrating explicit hardware and architecture priors, improving latency prediction accuracy and robustness in diverse hardware environments.

Findings

Achieves 5% better accuracy than MAPLE in latency prediction.

Demonstrates 9% improvement over HELP in experimental benchmarks.

Uses virtual examples and hardware similarity to enhance model robustness.

Abstract

Deep neural network (DNN) latency characterization is a time-consuming process and adds significant cost to Neural Architecture Search (NAS) processes when searching for efficient convolutional neural networks for embedded vision applications. DNN Latency is a hardware dependent metric and requires direct measurement or inference on target hardware. A recently introduced latency estimation technique known as MAPLE predicts DNN execution time on previously unseen hardware devices by using hardware performance counters. Leveraging these hardware counters in the form of an implicit prior, MAPLE achieves state-of-the-art performance in latency prediction. Here, we propose MAPLE-X which extends MAPLE by incorporating explicit prior knowledge of hardware devices and DNN architecture latency to better account for model stability and robustness. First, by identifying DNN architectures that exhibit a similar latency to each other, we can generate multiple virtual examples to significantly improve the accuracy over MAPLE. Secondly, the hardware specifications are used to determine the similarity between training and test hardware to emphasize training samples captured from comparable devices (domains) and encourages improved domain alignment. Experimental results using a convolution neural network NAS benchmark across different types of devices, including an Intel processor that is now used for embedded vision applications, demonstrate a 5% improvement over MAPLE and 9% over HELP. Furthermore, we include ablation studies to independently assess the benefits of virtual examples and hardware-based sample importance.