Dissecting the Impact of Mobile DVFS Governors on LLM Inference Performance and Energy Efficiency

TL;DR

This paper investigates how independent DVFS governors for CPU, GPU, and memory in mobile devices affect LLM inference efficiency, and proposes FUSE, a unified governor that improves latency without increasing energy consumption.

Contribution

It provides a detailed measurement of the impact of separate governors on LLM inference and introduces FUSE, a novel unified energy-aware governor for mobile LLM deployment.

Findings

Triplet mobile governors cause up to 40.4% longer latency compared to optimal frequency combinations.

FUSE reduces inference latency by up to 36.8% while maintaining the same energy per token.

Unified governor improves energy efficiency and inference speed for mobile LLMs.

Abstract

Large Language Models (LLMs) are increasingly being integrated into various applications and services running on billions of mobile devices. However, deploying LLMs on resource-limited mobile devices faces a significant challenge due to their high demand for computation, memory, and ultimately energy. While current LLM frameworks for mobile use three power-hungry components-CPU, GPU, and Memory-even when running primarily-GPU LLM models, optimized DVFS governors for CPU, GPU, and memory featured in modern mobile devices operate independently and are oblivious of each other. Motivated by the above observation, in this work, we first measure the energy-efficiency of a SOTA LLM framework consisting of various LLM models on mobile phones which showed the triplet mobile governors result in up to 40.4% longer prefilling and decoding latency compared to optimal combinations of CPU, GPU, and memory frequencies with the same energy consumption for sampled prefill and decode lengths. Second, we conduct an in-depth measurement study to uncover how the intricate interplay (or lack of) among the mobile governors cause the above inefficiency in LLM inference. Finally, based on these insights, we design FUSE - a unified energy-aware governor for optimizing the energy efficiency of LLM inference on mobile devices. Our evaluation using a ShareGPT dataset shows FUSE reduces the time-to-first-token and time-per-output-token latencies by 7.0%-16.9% and 25.4%-36.8% on average with the same energy-per-token for various mobile LLM models.