EnergAIzer: Fast and Accurate GPU Power Estimation Framework for AI Workloads

TL;DR

EnergAIzer is a fast, accurate GPU power estimation framework for AI workloads that uses structured kernel patterns to predict utilization and power consumption efficiently.

Contribution

It introduces a lightweight, analytical approach to predict GPU utilization and power, significantly reducing prediction time compared to traditional methods.

Findings

Achieves 8% power estimation error on NVIDIA Ampere GPUs.

Forecasts power of NVIDIA H100 with only 7% error.

Reduces estimation walltime from hours to seconds.

Abstract

As AI workloads drive increases in datacenter power consumption, accurate GPU power estimation is critical for proactive power management. However, existing power models face a scalability bottleneck not in the modeling techniques themselves, but in obtaining the hardware utilization inputs they require. Conventional approaches rely on either costly simulation or hardware profiling, which makes them impractical when rapid predictions are required. This work presents EnergAIzer, which addresses this scalability bottleneck by developing a lightweight solution to predict utilization inputs, reducing the estimation walltime from hours to seconds. Our key insight is that kernels in AI workloads commonly employ optimizations that create structured patterns, which analytically determine memory traffic and execution timeline. We construct a performance model using these patterns as an analytical scaffold for empirical data fitting, which also naturally exposes module-level utilization. This predicted utilization is then fed into our power model to estimate dynamic power consumption. EnergAIzer achieves 8% power errors on NVIDIA Ampere GPUs, competitive with traditional power models with elaborate cycle-level simulation or hardware profiling. We demonstrate EnergAIzer's exploration capabilities for frequency scaling and architectural configurations, including forecasting the power of NVIDIA H100 with just 7% error. In summary, EnergAIzer provides fast and accurate power prediction for AI workloads, paving the way for power-aware design explorations.