Dynamic Hierarchical Birkhoff-von Neumann Decomposition for All-to-All GPU Communication

TL;DR

This paper introduces a dynamic hierarchical Birkhoff-von Neumann decomposition method for optimizing all-to-all GPU communication, effectively reducing bottlenecks caused by traffic skew and hierarchical network structures in large-scale GPU clusters.

Contribution

It presents a novel hierarchical BvN decomposition framework tailored for two-tier GPU fabrics, improving scheduling efficiency and stability in all-to-all GPU communication.

Findings

Significant reduction in mean frame length during simulations.

Enhanced stability under Poisson traffic arrivals.

Effective mitigation of traffic skew within servers.

Abstract

All-to-all GPU communication is a critical bottleneck in large-scale training clusters, where completion time is constrained by per-port bandwidth and can be severely impacted by traffic skew across GPUs and network interface cards (NICs). This issue is amplified by the two-tier structure of modern GPU systems, which combine fast intra-server links with much slower inter-server networks. Motivated by recent system observations that highlight the importance of traffic reshaping and hierarchy awareness, we study all-to-all scheduling from an online switching and queueing-theoretic perspective. We propose a dynamic hierarchical Birkhoff--von Neumann (BvN) decomposition framework tailored to two-tier GPU fabrics. At each frame boundary, traffic is first balanced within each server using simple local operations to mitigate micro-level GPU/NIC skew while preserving aggregate server-to-server demand. A hierarchical BvN decomposition is then applied at the server level and refined into GPU-level matchings, significantly reducing decomposition complexity relative to a flat GPU-level approach. By integrating this construction with the dynamic frame sizing (DFS) principle, we obtain an online scheduler with provable stability under admissible Poisson arrivals. Simulations demonstrate substantial reductions in mean frame length, particularly under server-localized hotspot traffic.