AdaScale: Dynamic Context-aware DNN Scaling via Automated Adaptation Loop on Mobile Devices

TL;DR

AdaScale is an automated framework that dynamically adapts deep neural networks to mobile device contexts, improving performance, accuracy, and energy efficiency through resource-aware optimization.

Contribution

It introduces a self-evolutionary model and automated adaptation loop for efficient, context-aware DNN scaling on mobile devices.

Findings

Increases accuracy by 5.09%

Reduces training overhead by 66.89%

Speeds up inference latency by 1.51 to 6.2 times

Abstract

Deep learning is reshaping mobile applications, with a growing trend of deploying deep neural networks (DNNs) directly to mobile and embedded devices to address real-time performance and privacy. To accommodate local resource limitations, techniques like weight compression, convolution decomposition, and specialized layer architectures have been developed. However, the \textit{dynamic} and \textit{diverse} deployment contexts of mobile devices pose significant challenges. Adapting deep models to meet varied device-specific requirements for latency, accuracy, memory, and energy is labor-intensive. Additionally, changing processor states, fluctuating memory availability, and competing processes frequently necessitate model re-compression to preserve user experience. To address these issues, we introduce AdaScale, an elastic inference framework that automates the adaptation of deep models to dynamic contexts. AdaScale leverages a self-evolutionary model to streamline network creation, employs diverse compression operator combinations to reduce the search space and improve outcomes, and integrates a resource availability awareness block and performance profilers to establish an automated adaptation loop. Our experiments demonstrate that AdaScale significantly enhances accuracy by 5.09%, reduces training overhead by 66.89%, speeds up inference latency by 1.51 to 6.2 times, and lowers energy costs by 4.69 times.