CompPow: A Case for Component-level GPU Power Management

TL;DR

This paper advocates for component-level GPU power management, demonstrating that component-aware strategies can enhance energy efficiency by 10% and performance by 5% in ML workloads.

Contribution

It introduces CompPow, a novel approach for component-aware GPU power management, highlighting its potential for improved energy efficiency and performance.

Findings

Component-aware power management can improve energy efficiency by 10%.

Component-aware strategies can enhance GPU performance by 5%.

Recommendations for software-hardware co-design to optimize GPU energy use.

Abstract

The ever increasing demand for ML-driven intelligence in a wide spectrum of domains has led to ubiquity of GPUs. At the same time, GPUs are notorious for their power consumption needs and often dominate power allocation in a typical ML datacenter. While datacenter-level power optimizations which focus on collection of GPUs are promising, in this work, we take a different tack -- namely, we take a closer look at power consumption inside a GPU. Specifically, as modern GPUs are comprised of integrated components, we make a case for component-awareness, termed CompPow in this work, for improved power management in modern GPUs. We demonstrate for a variety of ML operations and execution patterns, CompPow has the potential to deliver higher energy efficiency (10%) and even improved performance (5%). We conclude with recommendations on how component-aware software-hardware co-design can extract additional energy efficiency from modern GPUs.