MARS: Memory Aware Reordered Source

TL;DR

MARS is a memory reordering architecture that enhances memory efficiency and bandwidth utilization in GPUs by reordering data streams based on row-buffer addresses, addressing interleaving issues.

Contribution

MARS introduces a lookahead-based reordering method to improve memory locality and bandwidth in GPU workloads without requiring detailed memory configuration knowledge.

Findings

Achieves 11% increase in memory bandwidth on microbenchmarks.

Effectively restores locality lost due to interleaved data streams.

Operates without specific knowledge of memory system details.

Abstract

Memory bandwidth is critical in today's high performance computing systems. The bandwidth is particularly paramount for GPU workloads such as 3D Gaming, Imaging and Perceptual Computing, GPGPU due to their data-intensive nature. As the number of threads and data streams in the GPUs increases with each generation, along with a high available memory bandwidth, memory efficiency is also crucial in order to achieve desired performance. In presence of multiple concurrent data streams, the inherent locality in a single data stream is often lost as these streams are interleaved while moving through multiple levels of memory system. In DRAM based main memory, the poor request locality reduces row-buffer reuse resulting in underutilized and inefficient memory bandwidth. In this paper we propose Memory-Aware Reordered Source (\textit{MARS}) architecture to address memory inefficiency arising from highly interleaved data streams. The key idea of \textit{MARS} is that with a sufficiently large lookahead before the main memory, data streams can be reordered based on their row-buffer address to regain the lost locality and improve memory efficiency. We show that \textit{MARS} improves achieved memory bandwidth by 11\% for a set of synthetic microbenchmarks. Moreover, MARS does so without any specific knowledge of the memory configuration.