FractalSync: Lightweight Scalable Global Synchronization of Massive Bulk Synchronous Parallel AI Accelerators

TL;DR

FractalSync is a hardware-based synchronization mechanism designed for massive AI accelerators, significantly improving synchronization speed and scalability while maintaining low area overhead, enabling efficient parallel AI processing.

Contribution

We introduce FractalSync, a novel hardware acceleration technique for synchronization in large-scale BSP AI accelerators, demonstrating substantial speedup and scalability improvements.

Findings

Achieves up to 43x faster synchronization than software-based schemes.

Maintains negligible area overhead (<0.01%).

Supports scalability from 2x2 to 16x16 PEs.

Abstract

The slow-down of technology scaling and the emergence of Artificial Intelligence (AI) workloads have led computer architects to increasingly exploit parallelization coupled with hardware acceleration to keep pushing the performance envelope. However, this solution comes with the challenge of synchronization of processing elements (PEs) in massive heterogeneous many-core platforms. To address this challenge, we propose FractalSync, a hardware accelerated synchronization mechanism for Bulk Synchronous Parallel (BSP) systems. We integrate FractalSync in MAGIA, a scalable tile-based AI accelerator, with each tile featuring a RISC-V-coupled matrix-multiplication (MatMul) accelerator, scratchpad memory (SPM), and a DMA connected to a global mesh Network-on-Chip (NoC). We study the scalability of the proposed barrier synchronization scheme on tile meshes ranging from 2x2 PEs to 16x16 PEs to evaluate its design boundaries. Compared to a synchronization scheme based on software atomic memory operations (AMOs), the proposed solution achieves up to 43x speedup on synchronization, introducing a negligible area overhead (<0.01%). FractalSync closes timing at MAGIA's target 1GHz frequency.