Torrent: A Distributed DMA for Efficient and Flexible Point-to-Multipoint Data Movement

TL;DR

Torrent introduces a distributed DMA architecture that efficiently supports point-to-multipoint data transfers in SoCs without modifying existing NoC hardware, significantly improving performance and scalability for data-parallel workloads.

Contribution

It presents a novel Chainwrite mechanism for P2MP data movement that avoids hardware modifications and optimizes transfer chains with new scheduling algorithms.

Findings

Achieves up to 7.88x speedup over unicast baseline.

Minimal area (1.2%) and power (2.3%) overheads in ASIC implementation.

Demonstrates scalability and flexibility over network-layer multicast.

Abstract

The growing disparity between computational power and on-chip communication bandwidth is a critical bottleneck in modern Systems-on-Chip (SoCs), especially for data-parallel workloads like AI. Efficient point-to-multipoint (P2MP) data movement, such as multicast, is essential for high performance. However, native multicast support is lacking in standard interconnect protocols. Existing P2MP solutions, such as multicast-capable Network-on-Chip (NoC), impose additional overhead to the network hardware and require modifications to the interconnect protocol, compromising scalability and compatibility. This paper introduces Torrent, a novel distributed DMA architecture that enables efficient P2MP data transfers without modifying NoC hardware and interconnect protocol. Torrent conducts P2MP data transfers by forming logical chains over the NoC, where the data traverses through targeted destinations resembling a linked list. This Chainwrite mechanism preserves the P2P nature of every data transfer while enabling flexible data transfers to an unlimited number of destinations. To optimize the performance and energy consumption of Chainwrite, two scheduling algorithms are developed to determine the optimal chain order based on NoC topology. Our RTL and FPGA prototype evaluations using both synthetic and real workloads demonstrate significant advantages in performance, flexibility, and scalability over network-layer multicast. Compared to the unicast baseline, Torrent achieves up to a 7.88x speedup. ASIC synthesis on 16nm technology confirms the architecture's minimal footprint in area (1.2%) and power (2.3%). Thanks to the Chainwrite, Torrent delivers scalable P2MP data transfers with a small cycle overhead of 82CC and area overhead of 207um2 per destination.