Towards 3D AI Hardware: Fine-Grain Hardware Characterization of 3D Stacks for Heterogeneous System Integration & AI Systems

TL;DR

This paper discusses the importance of 3D integration in AI hardware, highlighting the need for detailed hardware characterization to address challenges like temperature and reliability in heterogeneous 3D stacks.

Contribution

It introduces a comprehensive hardware profiling framework for 3D stacks, enabling emulation and analysis of power, temperature, noise, and reliability for heterogeneous AI systems.

Findings

Framework effectively characterizes heat propagation and inter-layer noise.

Enables control of activity levels and customization of sensor infrastructure.

Supports evaluation of various stacking alternatives for AI hardware.

Abstract

3D integration offers key advantages in improving system performance and efficiency for the End-of-Scaling era. It enables the incorporation of heterogeneous system components and disparate technologies, eliminates off-chip communication constraints, reduces on-chip latency and total power dissipation. Moreover, AIs demand for increased computational power, larger GPU cache capacity, energy efficiency and low power custom AI hardware integration all serve as drivers for 3D integration. Although 3D advantages such as enhanced interconnectivity and increased performance have been demonstrated through numerous technology sites, heterogeneous 3D system design raises numerous unanswered questions. Among the primary challenges are the temperature and lifetime reliability issues caused by the complex interaction patterns among system components. Such interactions are harder to model with current modeling tools and require detailed hardware characterization. This study presents the latest drivers for 3D integration and the resulting need for hardware emulation frameworks. It then presents a design to profile power, temperature, noise, inter-layer bandwidth and lifetime reliability characterization that can emulate a wide range of stacking alternatives. This framework allows for controlling activity levels at the macro-level, along with customized sensor infrastructure to characterize heat propagation, inter-layer noise, power delivery, reliability and inter-connectivity as well as the interactions among critical design objectives.