Input-Dependent Power Usage in GPUs

TL;DR

This paper investigates how input data variations in GEMMs significantly affect GPU power consumption, revealing potential for power management through compiler and scheduler optimizations.

Contribution

It demonstrates that input data modifications can alter GPU power usage by up to 40%, and proposes leveraging this for energy-efficient GPU operation.

Findings

Input variations can change power usage by nearly 40%.

Power fluctuations are linked to bit flip changes in GPU hardware.

Input-dependent power management strategies can reduce energy consumption.

Abstract

GPUs are known to be power-hungry, and due to the boom in artificial intelligence, they are currently the major contributors to the high power demands of upcoming datacenters. Most GPU usage in these popular workloads consist of large general matrix-matrix multiplications (GEMMs), which have therefore been optimized to achieve high utilization of hardware resources. In this work, we show that modifying the input data to GEMMs, while maintaining the matrix shapes and sizes can notably change the power consumption of these kernels. We experiment with four kinds of input variations: value distribution, bit similarity, placement, and sparsity, across different data types. Our findings indicate that these variations can change the GPU power usage during GEMM by almost 40%. We hypothesize that input-dependent power usage variations occur due to changes in the number of bit flips in the GPUs. We propose leveraging this property through compiler and scheduler optimizations to manage power and reduce energy consumption.