Shared Virtual Memory: Its Design and Performance Implications for Diverse Applications

TL;DR

This paper investigates Shared Virtual Memory (SVM) in GPUs, analyzing its design, performance impacts, and proposing mitigation strategies to improve application efficiency in oversubscribed scenarios.

Contribution

It provides the first comprehensive analysis of SVM design, its interaction with applications, and strategies to mitigate performance degradation due to prefetching and eviction policies.

Findings

SVM employs aggressive prefetching for demand paging.

Prefetching is efficient without oversubscription, but causes thrashing under oversubscription.

Proposed algorithms and design changes can mitigate performance issues.

Abstract

Discrete GPU accelerators, while providing massive computing power for supercomputers and data centers, have their separate memory domain. Explicit memory management across device and host domains in programming is tedious and error-prone. To improve programming portability and productivity, Unified Memory (UM) integrates GPU memory into the host virtual memory systems, and provides transparent data migration between them and GPU memory oversubscription. Nevertheless, current UM technologies cause significant performance loss for applications. With AMD GPUs increasingly being integrated into the world's leading supercomputers, it is necessary to understand their Shared Virtual Memory (SVM) and mitigate the performance impacts. In this work, we delve into the SVM design, examine its interactions with applications' data accesses at fine granularity, and quantitatively analyze its performance effects on various applications and identify the performance bottlenecks. Our research reveals that SVM employs an aggressive prefetching strategy for demand paging. This prefetching is efficient when GPU memory is not oversubscribed. However, in tandem with the eviction policy, it causes excessive thrashing and performance degradation for certain applications under oversubscription. We discuss SVM-aware algorithms and SVM design changes to mitigate the performance impacts. To the best of our knowledge, this work is the first in-depth and comprehensive study for SVM technologies.