An Open-Source Platform for High-Performance Non-Coherent On-Chip Communication

TL;DR

This paper introduces a modular, high-performance on-chip communication platform designed for heterogeneous many-core systems, enabling scalable, high-bandwidth, low-latency data transfer across thousands of cores.

Contribution

It presents a novel, topology-agnostic communication platform with customizable modules that support high bandwidth and concurrency for heterogeneous SoCs.

Findings

Supports 2.5 GHz, 1024-bit data width communication fabrics

Scales to 1024 cores with 32 TB/s bandwidth

Achieves 24 ns round-trip latency between cores

Abstract

On-chip communication infrastructure is a central component of modern systems-on-chip (SoCs), and it continues to gain importance as the number of cores, the heterogeneity of components, and the on-chip and off-chip bandwidth continue to grow. Decades of research on on-chip networks enabled cache-coherent shared-memory multiprocessors. However, communication fabrics that meet the needs of heterogeneous many-cores and accelerator-rich SoCs, which are not, or only partially, coherent, are a much less mature research area. In this work, we present a modular, topology-agnostic, high-performance on-chip communication platform. The platform includes components to build and link subnetworks with customizable bandwidth and concurrency properties and adheres to a state-of-the-art, industry-standard protocol. We discuss microarchitectural trade-offs and timing/area characteristics of our modules and show that they can be composed to build high-bandwidth (e.g., 2.5 GHz and 1024 bit data width) end-to-end on-chip communication fabrics (not only network switches but also DMA engines and memory controllers) with high degrees of concurrency. We design and implement a state-of-the-art ML training accelerator, where our communication fabric scales to 1024 cores on a die, providing 32 TB/s cross-sectional bandwidth at only 24 ns round-trip latency between any two cores.