Ohm-GPU: Integrating New Optical Network and Heterogeneous Memory into GPU Multi-Processors

TL;DR

Ohm-GPU introduces an optical network-based heterogeneous memory system for GPUs, significantly increasing memory capacity and bandwidth while managing data migrations efficiently to boost performance.

Contribution

This paper presents a novel optical network integrated with heterogeneous memory modules in GPUs, addressing capacity and bandwidth limitations with a new memory controller and infrastructure.

Findings

Performance improved by 181% over DRAM-based GPU memory.

Performance increased by 27% compared to baseline optical network system.

Efficient handling of data migrations reduces memory channel blocking.

Abstract

Traditional graphics processing units (GPUs) suffer from the low memory capacity and demand for high memory bandwidth. To address these challenges, we propose Ohm-GPU, a new optical network based heterogeneous memory design for GPUs. Specifically, Ohm-GPU can expand the memory capacity by combing a set of high-density 3D XPoint and DRAM modules as heterogeneous memory. To prevent memory channels from throttling throughput of GPU memory system, Ohm-GPU replaces the electrical lanes in the traditional memory channel with a high-performance optical network. However, the hybrid memory can introduce frequent data migrations between DRAM and 3D XPoint, which can unfortunately occupy the memory channel and increase the optical network traffic. To prevent the intensive data migrations from blocking normal memory services, Ohm-GPU revises the existing memory controller and designs a new optical network infrastructure, which enables the memory channel to serve the data migrations and memory requests, in parallel. Our evaluation results reveal that Ohm-GPU can improve the performance by 181% and 27%, compared to a DRAM-based GPU memory system and the baseline optical network based heterogeneous memory system, respectively.