Chopin: Combining Distributed and Centralized Schedulers for Self-Adjusting Datacenter Networks

TL;DR

Chopin is a hybrid scheduling system for reconfigurable datacenter networks that combines centralized and distributed methods to efficiently allocate optical circuits based on demand, improving network adaptability and scalability.

Contribution

It introduces a novel hybrid scheduler that balances global demand-awareness with fast local reactions in reconfigurable datacenter networks.

Findings

Chopin effectively allocates optical circuits to high-demand flows.

The distributed scheduler reacts swiftly to traffic changes.

Chopin reduces overhead while maintaining demand-awareness.

Abstract

The performance of distributed and data-centric applications often critically depends on the interconnecting network. Emerging reconfigurable datacenter networks (RDCNs) are a particularly innovative approach to improve datacenter throughput. Relying on a dynamic optical topology which can be adjusted towards the workload in a demand-aware manner, RDCNs allow to exploit temporal and spatial locality in the communication pattern, and to provide topological shortcuts for frequently communicating racks. The key challenge, however, concerns how to realize demand-awareness in RDCNs in a scalable fashion. This paper presents and evaluates Chopin, a hybrid scheduler for self-adjusting networks that provides demand-awareness at low overhead, by combining centralized and distributed approaches. Chopin allocates optical circuits to elephant flows, through its slower centralized scheduler, utilizing global information. Chopin's distributed scheduler is orders of magnitude faster and can swiftly react to changes in the traffic and adjust the optical circuits accordingly, by using only local information and running at each rack separately.