AdaOper: Energy-efficient and Responsive Concurrent DNN Inference on Mobile Devices

TL;DR

AdaOper is a system that improves energy efficiency for concurrent DNN inference on mobile devices by dynamically optimizing operator partitioning across heterogeneous processors, reducing energy use while maintaining responsiveness.

Contribution

We introduce AdaOper, a novel runtime system that adaptively optimizes DNN operator partitioning for energy efficiency on mobile heterogeneous processors.

Findings

Reduces energy consumption by 16.88% compared to existing methods.

Maintains real-time responsiveness during DNN inference.

Effectively adapts to dynamic device conditions.

Abstract

Deep neural network (DNN) has driven extensive applications in mobile technology. However, for long-running mobile apps like voice assistants or video applications on smartphones, energy efficiency is critical for battery-powered devices. The rise of heterogeneous processors in mobile devices today has introduced new challenges for optimizing energy efficiency. Our key insight is that partitioning computations across different processors for parallelism and speedup doesn't necessarily correlate with energy consumption optimization and may even increase it. To address this, we present AdaOper, an energy-efficient concurrent DNN inference system. It optimizes energy efficiency on mobile heterogeneous processors while maintaining responsiveness. AdaOper includes a runtime energy profiler that dynamically adjusts operator partitioning to optimize energy efficiency based on dynamic device conditions. We conduct preliminary experiments, which show that AdaOper reduces energy consumption by 16.88% compared to the existing concurrent method while ensuring real-time performance.