Workload composition smooths aggregate power demand while sustaining short-horizon ramps in AI data centers

TL;DR

This paper reveals how workload composition in AI data centers influences power demand variability and ramping, highlighting the importance of workload mix in grid impact management.

Contribution

It introduces a trace-calibrated framework to analyze how batch and inference workloads decouple power variability and ramping in shared-GPU AI data centers.

Findings

Increasing inference share causes U-shaped power variability and hump-shaped ramping.

Queued batch jobs fill capacity at intermediate workloads, reducing variability.

Inference fluctuations directly influence short-term power ramping.

Abstract

Artificial intelligence (AI) is driving rapid growth in electricity demand, yet the grid-facing power dynamics of AI data centers remain poorly understood. Here we show that, in shared-GPU systems, the composition of batch and inference workloads decouples aggregate power variability from short-horizon ramping. As the inference share rises, variability becomes U-shaped, whereas ramping becomes hump-shaped, particularly under higher loading. The magnitude and turning points of these patterns also depend on system loading. Using a trace-calibrated framework linking workload arrivals, queueing, scheduling, and GPU power, we show that the underlying mechanism is asymmetric. At intermediate workload mixes, queued batch jobs fill capacity left idle by fluctuating inference demand, reducing aggregate power variability. However, short-horizon ramping remains elevated because inference-side fluctuations propagate more directly into realized power. AI data centers should therefore be understood as dynamic systems whose workload composition shapes their grid impact.