ACOS: Arrays of Cheap Optical Switches

TL;DR

ACOS introduces a scalable, cost-effective optical switch architecture using low-radix switches, enabling efficient training of large language models with performance comparable to traditional networks but at lower costs.

Contribution

It presents a novel architecture that leverages low-radix optical switches for scalable, adaptable, and cost-efficient training cluster networks, overcoming limitations of high-radix switches.

Findings

ACOS matches performance of packet-switched networks for LLM training.

ACOS achieves significant cost savings with existing off-the-shelf OCSes.

The architecture supports topology adaptation and failure resilience effectively.

Abstract

Machine learning training places immense demands on cluster networks, motivating specialized architectures and co-design with parallelization strategies. Recent designs incorporating optical circuit switches (OCSes) are promising, offering improved cost, power efficiency, and long-term bandwidth scaling than packet switches. However, most existing approaches rely on costly high-radix OCSes and/or combine them with packet switches to achieve competitive performance at scale. Unfortunately, high-radix OCSes are both expensive and slow to reconfigure, limiting both scalability and performance. We propose Arrays of Cheap Optical Switches (ACOS), which bring application co-design directly to the structure of the reconfigurable fabric. Using low-radix OCSes as building blocks, ACOS supports the forms of reconfiguration needed in training clusters including topology selection, workload adaptation, and failure resilience. The cost of ACOS scales with supported topologies and adaptations rather than with port count, breaking past the scalability barriers of current specialized ML networks. We show through simulation that ACOS-based deployments match the performance of fully provisioned packet-switched networks when training state-of-the-art LLMs at scale, while delivering significant cost savings using existing off-the-shelf OCSes, with strong bandwidth scaling and higher cost savings in the future.