Decoupling GPU Programming Models from Resource Management for Enhanced Programming Ease, Portability, and Performance

TL;DR

Zorua introduces a resource virtualization framework that decouples GPU resource specifications from hardware allocation, enhancing programming ease, portability, and performance through transparent virtualization and dynamic resource management.

Contribution

This paper presents Zorua, a novel framework that virtualizes GPU resources to decouple software specifications from hardware management, improving ease of programming, portability, and performance.

Findings

Zorua enables resource virtualization, providing the illusion of more resources than physically available.

It improves programming ease by reducing the need for resource tuning.

It enhances portability by minimizing re-tuning across GPU generations.

Abstract

The application resource specification--a static specification of several parameters such as the number of threads and the scratchpad memory usage per thread block--forms a critical component of modern GPU programming models. This specification determines the parallelism, and hence performance, of the application during execution because the corresponding on-chip hardware resources are allocated and managed based on this specification. This tight-coupling between the software-provided resource specification and resource management in hardware leads to significant challenges in programming ease, portability, and performance. Zorua is a new resource virtualization framework, that decouples the programmer-specified resource usage of a GPU application from the actual allocation in the on-chip hardware resources. Zorua enables this decoupling by virtualizing each resource transparently to the programmer. We demonstrate that by providing the illusion of more resources than physically available via controlled and coordinated virtualization, Zorua offers several important benefits: (i) Programming Ease. Zorua eases the burden on the programmer to provide code that is tuned to efficiently utilize the physically available on-chip resources. (ii) Portability. Zorua alleviates the necessity of re-tuning an application's resource usage when porting the application across GPU generations. (iii) Performance. By dynamically allocating resources and carefully oversubscribing them when necessary, Zorua improves or retains the performance of applications that are already highly tuned to best utilize the resources.