Performance Implications of NoCs on 3D-Stacked Memories: Insights from the Hybrid Memory Cube

TL;DR

This paper investigates how network-on-chip (NoC) designs in 3D-stacked memories like Hybrid Memory Cube affect access latency and bandwidth, providing insights through prototype characterization and system analysis.

Contribution

It offers the first detailed analysis of NoC impacts on 3D-stacked memory performance, including latency behaviors and system implications.

Findings

Characterized HMC prototype access latency behaviors

Revealed implications of NoC on system design

Provided insights for optimizing 3D-stacked memory systems

Abstract

Memories that exploit three-dimensional (3D)-stacking technology, which integrate memory and logic dies in a single stack, are becoming popular. These memories, such as Hybrid Memory Cube (HMC), utilize a network-on-chip (NoC) design for connecting their internal structural organizations. This novel usage of NoC, in addition to aiding processing-in-memory capabilities, enables numerous benefits such as high bandwidth and memory-level parallelism. However, the implications of NoCs on the characteristics of 3D-stacked memories in terms of memory access latency and bandwidth have not been fully explored. This paper addresses this knowledge gap by (i) characterizing an HMC prototype on the AC-510 accelerator board and revealing its access latency behaviors, and (ii) by investigating the implications of such behaviors on system and software designs.