The Shift from Processor Power Consumption to Performance Variations: Fundamental Implications at Scale

TL;DR

This paper investigates how recent processor power management features, especially in Intel Haswell-EP, shift focus from power consumption to performance variations, affecting performance analysis and application behavior at scale.

Contribution

It provides empirical measurements showing the impact of new power control mechanisms on performance homogeneity and variability across CPU generations.

Findings

Haswell-EP exhibits inhomogeneous performance under full load.

Power consumption is capped by TDP and RAPL, leading to performance variability.

Previous generations maintained homogeneous performance across CPUs.

Abstract

The Intel Haswell-EP processor generation introduces several major advancements of power control and energy-efficiency features. For computationally intense applications using advanced vector extension (AVX) instructions, the processor cannot continuously operate at full speed but instead reduces its frequency below the nominal frequency to maintain operations within thermal design power (TDP) limitations. Moreover, the running average power limitation (RAPL) mechanism to enforce the TDP limitation has changed from a modeling to a measurement approach. The combination of these two novelties have significant implications. Through measurements on an Intel Sandy Bridge-EP cluster, we show that previous generations have sustained homogeneous performance across multiple CPUs and compensated for hardware manufacturing variability through varying power consumption. In contrast, our measurements on a Petaflop Haswell system show that this generation exhibits rather homogeneous power consumption limited by the TDP and capped by the improved RAPL while providing inhomogeneous performance under full load. Since all of these controls are transparent to the user, this behavior is likely to complicate performance analysis tasks and impact tightly coupled parallel applications.