Tascade: Hardware Support for Atomic-free, Asynchronous and Efficient Reduction Trees

TL;DR

Tascade is a hardware-software co-design that enables scalable, asynchronous reduction trees for large manycore servers, significantly reducing communication overhead and improving performance for irregular graph workloads.

Contribution

It introduces a novel execution model and hardware support for efficient, atomic-free reductions, scaling up to a million PUs in large-scale parallel systems.

Findings

Achieves over 7600 GTEPS in BFS on RMAT-26 with a million PUs

Reduces communication and power consumption compared to prior approaches

Scales efficiently for irregular workloads on large manycore architectures

Abstract

Graph search and sparse data-structure traversal workloads contain challenging irregular memory patterns on global data structures that need to be modified atomically. Distributed processing of these workloads has relied on server threads operating on their own data copies that are merged upon global synchronization. As parallelism increases within each server, the communication challenges that arose in distributed systems a decade ago are now being encountered within large manycore servers. Prior work has achieved scalability for sparse applications up to thousands of PUs on-chip, but does not scale further due to increasing communication distances and load-imbalance across PUs. To address these challenges we propose Tascade, a hardware-software co-design that offers support for storage-efficient data-private reductions as well as asynchronous and opportunistic reduction trees. Tascade introduces an execution model along with supporting hardware design that allows coalescing of data updates regionally and merges the data from these regions through cascaded updates. Together, Tascade innovations minimize communication and increase work balance in task-based parallelization schemes and scales up to a million PUs. We evaluate six applications and four datasets to provide a detailed analysis of Tascade's performance, power, and traffic-reduction gains over prior work. Our parallelization of Breadth-First-Search with RMAT-26 across a million PUs -- the largest of the literature -- reaches over 7600 GTEPS.