Play It Cool: Dynamic Shifting Prevents Thermal Throttling

TL;DR

This paper introduces a dynamic model shifting technique to prevent thermal throttling on mobile devices, maintaining consistent performance and balancing accuracy and latency.

Contribution

It proposes a novel dynamic shifting method between large and small ML models based on thermal profiles to avoid overheating and performance degradation.

Findings

Prevents CPU frequency reduction during thermal throttling

Maintains consistent inference latency on edge devices

Balances model accuracy with latency through dynamic shifting

Abstract

Machine learning (ML) has entered the mobile era where an enormous number of ML models are deployed on edge devices. However, running common ML models on edge devices continuously may generate excessive heat from the computation, forcing the device to "slow down" to prevent overheating, a phenomenon called thermal throttling. This paper studies the impact of thermal throttling on mobile phones: when it occurs, the CPU clock frequency is reduced, and the model inference latency may increase dramatically. This unpleasant inconsistent behavior has a substantial negative effect on user experience, but it has been overlooked for a long time. To counter thermal throttling, we propose to utilize dynamic networks with shared weights and dynamically shift between large and small ML models seamlessly according to their thermal profile, i.e., shifting to a small model when the system is about to throttle. With the proposed dynamic shifting, the application runs consistently without experiencing CPU clock frequency degradation and latency increase. In addition, we also study the resulting accuracy when dynamic shifting is deployed and show that our approach provides a reasonable trade-off between model latency and model accuracy.