XDMA: A Distributed, Extensible DMA Architecture for Layout-Flexible Data Movements in Heterogeneous Multi-Accelerator SoCs

TL;DR

XDMA introduces a distributed, extensible DMA architecture that enables high-efficiency, layout-flexible data movements in heterogeneous multi-accelerator SoCs, significantly improving link utilization and performance.

Contribution

The paper presents XDMA, a novel distributed DMA architecture with hardware address generation, on-the-fly data manipulation, and separation of configuration and data transfer, enhancing data movement efficiency.

Findings

Up to 151.2x higher link utilization compared to software-based methods.

Achieves 2.3x average speedup over state-of-the-art DMA in real applications.

Consumes less than 2% additional area and 17% system power.

Abstract

As modern AI workloads increasingly rely on heterogeneous accelerators, ensuring high-bandwidth and layout-flexible data movements between accelerator memories has become a pressing challenge. Direct Memory Access (DMA) engines promise high bandwidth utilization for data movements but are typically optimal only for contiguous memory access, thus requiring additional software loops for data layout transformations. This, in turn, leads to excessive control overhead and underutilized on-chip interconnects. To overcome this inefficiency, we present XDMA, a distributed and extensible DMA architecture that enables layout-flexible data movements with high link utilization. We introduce three key innovations: (1) a data streaming engine as XDMA Frontend, replacing software address generators with hardware ones; (2) a distributed DMA architecture that maximizes link utilization and separates configuration from data transfer; (3) flexible plugins for XDMA enabling on-the-fly data manipulation during data transfers. XDMA demonstrates up to 151.2x/8.2x higher link utilization than software-based implementations in synthetic workloads and achieves 2.3x average speedup over accelerators with SoTA DMA in real-world applications. Our design incurs <2% area overhead over SoTA DMA solutions while consuming 17% of system power. XDMA proves that co-optimizing memory access, layout transformation, and interconnect protocols is key to unlocking heterogeneous multi-accelerator SoC performance.