Photonic Rails in ML Datacenters with Opus

TL;DR

This paper introduces Opus, a photonic rail-based network fabric for ML datacenters that significantly reduces power and cost by leveraging optical switches and parallelism-driven reconfiguration, with minimal training overhead.

Contribution

It proposes a novel optical circuit switch-based rail abstraction with a reconfiguration method tailored for ML workloads, and demonstrates its effectiveness on real hardware and simulations.

Findings

Over 23x power reduction in network fabric

Up to 4x cost savings compared to electrical switches

Less than 6% training overhead due to reconfiguration latency

Abstract

Rail-optimized network fabrics have become the de facto datacenter scale-out fabric for large-scale ML training. However, the use of high-radix electrical switches to provide all-to-all connectivity in rails imposes massive power and cost. We propose a rethinking of the rail abstraction by retaining its communication semantics, but realizing it using optical circuit switches. The key challenge is that optical switches support one-to-one connectivity at a time, limiting the fan-out of traffic in ML workloads using hybrid parallelisms. We overcome this through \emph{parallelism-driven rail reconfiguration}, which exploits the non-overlapping communication phases of different parallelism dimensions. This time-multiplexes a single set of physical ports across circuit configurations tailored to each phase within a training iteration. We design and implement Opus, a control plane that orchestrates this in-job reconfiguration of photonic rails at parallelism phase boundaries, and evaluate it on a physical OCS testbed, the Perlmutter supercomputer, and in simulation at up to 2,048 GPUs. Our results show that photonic rails can achieve over $23\times$ network power reduction and $4\times$ cost savings while incurring less than $6\%$ training overhead at production-relevant OCS reconfiguration latencies.