Online GPU Energy Optimization with Switching-Aware Bandits

TL;DR

This paper presents EnergyUCB, an online bandit-based controller that optimizes GPU energy consumption in HPC systems by balancing energy savings, performance, and switching overhead, demonstrated on real supercomputing workloads.

Contribution

It introduces a practical online GPU energy optimization method using a multi-armed bandit approach with switching-aware and QoS-constrained features, addressing real-world challenges.

Findings

EnergyUCB achieves significant energy savings in supercomputing workloads.

The QoS-constrained variant reliably maintains user-specified performance budgets.

The method effectively balances exploration, exploitation, and switching overhead in real-time.

Abstract

Energy consumption has become a bottleneck for future computing architectures, from wearable devices to leadership-class supercomputers. Existing energy management techniques largely target CPUs, even though GPUs now dominate power draw in heterogeneous high performance computing (HPC) systems. Moreover, many prior methods rely on either purely offline or hybrid offline and online training, which is impractical and results in energy inefficiencies during data collection. In this paper, we introduce a practical online GPU energy optimization problem in a HPC scenarios. The problem is challenging because (1) GPU frequency scaling exhibits performance-energy trade-offs, (2) online control must balance exploration and exploitation, and (3) frequent frequency switching incurs non-trivial overhead and degrades quality of service (QoS). To address the challenges, we formulate online GPU energy optimization as a multi-armed bandit problem and propose EnergyUCB, a lightweight UCB-based controller that dynamically adjusts GPU core frequency in real time to save energy. Specifically, EnergyUCB (1) defines a reward that jointly captures energy and performance using a core-to-uncore utilization ratio as a proxy for GPU throughput, (2) employs optimistic initialization and UCB-style confidence bonuses to accelerate learning from scratch, and (3) incorporates a switching-aware UCB index and a QoS-constrained variant that enforce explicit slowdown budgets while discouraging unnecessary frequency oscillations. Extensive experiments on real-world workloads from the world's third fastest supercomputer Aurora show that EnergyUCB achieves substantial energy savings with modest slowdown and that the QoS-constrained variant reliably respects user-specified performance budgets.