CPU frequency scheduling of real-time applications on embedded devices with temporal encoding-based deep reinforcement learning

TL;DR

This paper presents a deep reinforcement learning-based method using temporal encoding to optimize CPU frequency scheduling for real-time tasks on embedded devices, outperforming Linux's built-in solutions in energy efficiency.

Contribution

It introduces a novel RL-based DVFS governor that adapts to workload patterns without explicit task models, using only one performance counter, and demonstrates its effectiveness on embedded hardware.

Findings

Energy savings of 3%-11% on Mibench workloads

5%-14% energy reduction on specific applications

Outperforms Linux's built-in DVFS in energy efficiency

Abstract

Small devices are frequently used in IoT and smart-city applications to perform periodic dedicated tasks with soft deadlines. This work focuses on developing methods to derive efficient power-management methods for periodic tasks on small devices. We first study the limitations of the existing Linux built-in methods used in small devices. We illustrate three typical workload/system patterns that are challenging to manage with Linux's built-in solutions. We develop a reinforcement-learning-based technique with temporal encoding to derive an effective DVFS governor even with the presence of the three system patterns. The derived governor uses only one performance counter, the same as the built-in Linux mechanism, and does not require an explicit task model for the workload. We implemented a prototype system on the Nvidia Jetson Nano Board and experimented with it with six applications, including two self-designed and four benchmark applications. Under different deadline constraints, our approach can quickly derive a DVFS governor that can adapt to performance requirements and outperform the built-in Linux approach in energy saving. On Mibench workloads, with performance slack ranging from 0.04 s to 0.4 s, the proposed method can save 3% - 11% more energy compared to Ondemand. AudioReg and FaceReg applications tested have 5%- 14% energy-saving improvement. We have open-sourced the implementation of our in-kernel quantized neural network engine. The codebase can be found at: https://github.com/coladog/tinyagent.