Hardware-Aware DNN Compression for Homogeneous Edge Devices

TL;DR

This paper introduces HDAP, a hardware-aware DNN compression framework tailored for homogeneous edge devices, addressing device performance variability over time to optimize average model performance.

Contribution

HDAP is a novel device clustering and surrogate evaluation-based method that improves DNN compression for homogeneous edge devices with performance variability.

Findings

HDAP reduces average latency significantly, e.g., 2.86× on ResNet50.

It maintains competitive accuracy across device clusters.

Demonstrates effectiveness on multiple device types and tasks.

Abstract

Deploying deep neural networks (DNNs) across homogeneous edge devices (the devices with the same SKU labeled by the manufacturer) often assumes identical performance among them. However, once a device model is widely deployed, the performance of each device becomes different after a period of running. This is caused by the differences in user configurations, environmental conditions, manufacturing variances, battery degradation, etc. Existing DNN compression methods have not taken this scenario into consideration and can not guarantee good compression results in all homogeneous edge devices. To address this, we propose Homogeneous-Device Aware Pruning (HDAP), a hardware-aware DNN compression framework explicitly designed for homogeneous edge devices, aiming to achieve optimal average performance of the compressed model across all devices. To deal with the difficulty of time-consuming hardware-aware evaluations for thousands or millions of homogeneous edge devices, HDAP partitions all the devices into several device clusters, which can dramatically reduce the number of devices to evaluate and use the surrogate-based evaluation instead of hardware evaluation in real-time. Extensive experiments on multiple device types (Jetson Xavier NX and Jetson Nano) and task types (image classification with ResNet50, MobileNetV1, ResNet56, VGG16; object detection with YOLOv8n) demonstrate that HDAP consistently achieves lower average latency and competitive accuracy compared to state-of-the-art methods, with significant speedups (e.g., 2.86$\times$ on ResNet50 at 1.0G FLOPs). HDAP offers an effective solution for scalable, high-performance DNN deployment methods for homogeneous edge devices.