AutoTailor: Automatic and Efficient Adaptive Model Deployment for Diverse Edge Devices

TL;DR

AutoTailor is an automated framework that simplifies and accelerates the deployment of adaptive neural network models on diverse edge devices, improving efficiency and accuracy.

Contribution

AutoTailor introduces an end-to-end automated SuperNet-based deployment framework with graph-guided compilation and learning-free predictors, reducing development complexity and profiling costs.

Findings

Reduces SuperNet construction code by 11-27 times

Decreases profiling costs by at least 11 times

Achieves up to 15.60% accuracy improvement and 60.03% latency reduction

Abstract

On-device machine learning (ML) has become a fundamental component of emerging mobile applications. Adaptive model deployment delivers efficient inference for heterogeneous device capabilities and performance requirements through customizing neural architectures. SuperNet-based approaches offer a promising solution by generating a large number of model variants from a pre-trained ML model. However, applying SuperNet in existing frameworks suffers from tedious model-aware development and time-consuming hardware-aware profiling, which limits their practical adoption. We present AutoTailor, the first framework to enable automated, end-to-end SuperNet-based adaptive model deployment for edge devices. Unlike manual SuperNet construction, AutoTailor employs a computation graph-guided compilation approach to automatically transform user-provided ML models into SuperNets. To support efficient specialization, AutoTailor incorporates learning-free latency and accuracy predictors, enabling low-cost yet accurate performance prediction. Our extended evaluations demonstrate that AutoTailor reduces the lines of code for SuperNet construction by 11--27$\times$, decreases hardware-aware profiling costs by at least 11$\times$, and achieves up to 15.60\% absolute accuracy improvement and 60.03\% latency reduction compared to state-of-the-art approaches across diverse models and devices.