Gem5-AcceSys: Enabling System-Level Exploration of Standard Interconnects for Novel Accelerators

TL;DR

This paper presents Gem5-AcceSys, a framework for exploring standard interconnects in accelerators, demonstrating how optimized interconnects can significantly improve data transfer performance for machine learning workloads.

Contribution

The paper introduces Gem5-AcceSys, enabling detailed system-level exploration of interconnects and memory configurations for accelerators, with a case study on transformer workloads.

Findings

Optimized interconnects can reach up to 80% of device-side memory performance.

In some cases, interconnect performance can surpass device-side memory performance.

The framework provides actionable insights for designing balanced accelerator systems.

Abstract

The growing demand for efficient, high-performance processing in machine learning (ML) and image processing has made hardware accelerators, such as GPUs and Data Streaming Accelerators (DSAs), increasingly essential. These accelerators enhance ML and image processing tasks by offloading computation from the CPU to dedicated hardware. These accelerators rely on interconnects for efficient data transfer, making interconnect design crucial for system-level performance. This paper introduces Gem5-AcceSys, an innovative framework for system-level exploration of standard interconnects and configurable memory hierarchies. Using a matrix multiplication accelerator tailored for transformer workloads as a case study, we evaluate PCIe performance across diverse memory types (DDR4, DDR5, GDDR6, HBM2) and configurations, including host-side and device-side memory. Our findings demonstrate that optimized interconnects can achieve up to 80% of device-side memory performance and, in some scenarios, even surpass it. These results offer actionable insights for system architects, enabling a balanced approach to performance and cost in next-generation accelerator design.