# Q-Learning Inspired Self-Tuning for Energy Efficiency in HPC

## TL;DR

This paper presents a novel self-tuning method for HPC energy efficiency that leverages application regions and Q-Learning to adapt processor power settings, achieving significant energy savings with minimal overhead.

## Contribution

It introduces a new self-tuning approach combining application regions with Q-Learning for energy optimization in HPC, exploiting processor feedback for improved results.

## Key findings

- Up to 15% energy savings achieved.
- Minor runtime overhead introduced.
- Effective exploitation of processor power interface.

## Abstract

System self-tuning is a crucial task to lower the energy consumption of computers. Traditional approaches decrease the processor frequency in idle or synchronisation periods. However, in High-Performance Computing (HPC) this is not sufficient: if the executed code is load balanced, there are neither idle nor synchronisation phases that can be exploited. Therefore, alternative self-tuning approaches are needed, which allow exploiting different compute characteristics of HPC programs. The novel notion of application regions based on function call stacks, introduced in the Horizon 2020 Project READEX, allows us to define such a self-tuning approach. In this paper, we combine these regions with the Q-Learning typical state-action maps, which save information about available states, possible actions to take, and the expected rewards. By exploiting the existing processor power interface, we are able to provide direct feedback to the learning process. This approach allows us to save up to 15% energy, while only adding a minor runtime overhead.