Enabling Reconfiguration-Communication Overlap for Collective Communication in Optical Networks

TL;DR

SWOT is a demand-aware optical network framework that overlaps reconfiguration with data transmission, significantly reducing collective communication time in distributed machine learning.

Contribution

It introduces intra-collective reconfiguration techniques to dynamically align optical network resources with communication patterns, overcoming static topology limitations.

Findings

Reduces communication time by up to 89.7% in simulations.

Effectively overlaps reconfiguration latency with data transmission.

Demonstrates robustness to varying optical resources and delays.

Abstract

Collective communication (CC) is critical for scaling distributed machine learning (DML). The predictable traffic patterns of DML present a great opportunity for applying optical network technologies. Optical networks with reconfigurable topologies promise high bandwidth and low latency for collective communications. However, existing approaches face inherent limitations: static topologies are inefficient for dynamic communication patterns within CC algorithm, while frequent topology reconfiguration matching every step of the algorithm incurs significant overhead. In this paper, we propose SWOT, a demand-aware optical network framework that employs ``intra-collective reconfiguration'' to dynamically align network resources with CC traffic patterns. SWOT hides reconfiguration latency by overlapping it with data transmission through three key techniques: \textit{Heterogeneous Message Splitting}, \textit{Asynchronous Overlapping}, and \textit{Topology Bypassing}. Extensive simulations demonstrate that SWOT reduces communication completion time up to 89.7% across diverse CC algorithm compared to static baselines, demonstrating strong robustness to varying optical resources and reconfiguration delay.