Orchestrated Co-scheduling, Resource Partitioning, and Power Capping on CPU-GPU Heterogeneous Systems via Machine Learning

TL;DR

This paper presents a machine learning-based approach to optimize co-scheduling, resource partitioning, and power capping in CPU-GPU systems to maximize throughput while respecting power constraints.

Contribution

It introduces a predictive performance modeling framework that jointly optimizes scheduling, resource allocation, and power capping on heterogeneous systems.

Findings

Achieves up to 67% speedup over naive scheduling.

Effectively balances power and performance in CPU-GPU systems.

Demonstrates the benefits of ML-driven optimization in real hardware.

Abstract

CPU-GPU heterogeneous architectures are now commonly used in a wide variety of computing systems from mobile devices to supercomputers. Maximizing the throughput for multi-programmed workloads on such systems is indispensable as one single program typically cannot fully exploit all available resources. At the same time, power consumption is a key issue and often requires optimizing power allocations to the CPU and GPU while enforcing a total power constraint, in particular when the power/thermal requirements are strict. The result is a system-wide optimization problem with several knobs. In particular we focus on (1) co-scheduling decisions, i.e., selecting programs to co-locate in a space sharing manner; (2) resource partitioning on both CPUs and GPUs; and (3) power capping on both CPUs and GPUs. We solve this problem using predictive performance modeling using machine learning in order to coordinately optimize the above knob setups. Our experiential results using a real system show that our approach achieves up to 67% of speedup compared to a time-sharing-based scheduling with a naive power capping that evenly distributes power budgets across components.